Abstract
We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This not only provides an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but it also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze and extend the adaptive Barzilai–Borwein method to a new family of stepsizes. While this family exploits negative values for the target, we also consider positive targets. We present a convergence analysis for quadratic problems extending results by Dai and Liao (IMA J Numer Anal 22(1):1–10, 2002), and carry out experiments outlining the potential of the approaches.
| Original language | English |
|---|---|
| Pages (from-to) | 75-106 |
| Number of pages | 32 |
| Journal | Computational Optimization and Applications |
| Volume | 85 |
| DOIs | |
| Publication status | Published - May 2023 |
Funding
The authors thank the referees and editor for their very useful comments. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 812912.
Keywords
- ABB method
- Framework for steplength selection
- Gradient methods
- Harmonic Rayleigh quotient
- Hessian spectral properties
- Unconstrained optimization