Logo image
Back
Regularized Step Directions in Nonlinear Conjugate Gradient Methods
Journal article   Open access   Peer reviewed

Regularized Step Directions in Nonlinear Conjugate Gradient Methods

Cassidy K Buhler, Hande Y Benson and David Shanno
Mathematical programming computation, v 16, pp 629-664
16 Sep 2024
DOI: https://doi.org/10.1007/s12532-024-00265-9
Featured in Collection :   Research Supported by Drexel Libraries' OA Programs
url
https://doi.org/10.1007/s12532-024-00265-9View
Published, Version of Record (VoR)Open Access via Drexel Libraries Read and Publish Program 2024CC BY V4.0,  Open

Abstract

Nonlinear programming Cubic regularization Optimization
Conjugate gradient minimization methods (CGM) and their accelerated variants are widely used. We focus on the use of cubic regularization to improve the CGM direction independent of the step length computation. In this paper, we propose the Hybrid Cubic Regularization of CGM, where regularized steps are used selectively. Using Shanno’s reformulation of CGM as a memoryless BFGS method, we derive new formulas for the regularized step direction. We show that the regularized step direction uses the same order of computational burden per iteration as its non-regularized version. Moreover, the Hybrid Cubic Regularization of CGM exhibits global convergence with fewer assumptions. In numerical experiments, the new step directions are shown to require fewer iteration counts, improve runtime, and reduce the need to reset the step direction. Overall, the Hybrid Cubic Regularization of CGM exhibits the same memoryless and matrix-free properties, while outperforming CGM as a memoryless BFGS method in iterations and runtime.

Metrics

9 Record Views

Details

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Collaboration types
Domestic collaboration
Web of Science research areas
Computer Science, Software Engineering
Mathematics, Applied
Operations Research & Management Science
Logo image