Abstract
In this paper, we analyze the convergence rate of the Heavy-ball algorithm applied to optimize a class of continuously differentiable functions. The analysis is performed with the Heavy-ball tuned to achieve the best convergence rate on the sub-class of quadratic functions. We review recent work to characterize convergence rate upper bounds for optimization algorithms using integral quadratic constraints (IQC). This yields a linear matrix inequality (LMI) condition which is typically solved numerically to obtain convergence rate bounds. We construct an analytical solution for this LMI condition using a specific 'weighted off-by-one' IQC. We also construct a specific objective function such that the Heavy-ball algorithm enters a limit cycle. These results demonstrate that IQC condition is tight for the analysis of the tuned Heavy-ball, i.e. it yields the exact condition ratio that separates global convergence from non-global convergence for the algorithm.
| Original language | English (US) |
|---|---|
| Title of host publication | 2019 American Control Conference, ACC 2019 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 4081-4085 |
| Number of pages | 5 |
| ISBN (Electronic) | 9781538679265 |
| DOIs | |
| State | Published - Jul 2019 |
| Event | 2019 American Control Conference, ACC 2019 - Philadelphia, United States Duration: Jul 10 2019 → Jul 12 2019 |
Publication series
| Name | Proceedings of the American Control Conference |
|---|---|
| Volume | 2019-July |
| ISSN (Print) | 0743-1619 |
Conference
| Conference | 2019 American Control Conference, ACC 2019 |
|---|---|
| Country/Territory | United States |
| City | Philadelphia |
| Period | 7/10/19 → 7/12/19 |
Bibliographical note
Funding Information:This work was supported by the National Science Foundation under Grant No. NSF-CMMI-1254129 entitled CAREER: Probabilistic Tools for High Reliability and Monitoring and Control of Wind Farms 1A. Badithela is a graduate student at California Institute of Technology, Pasadena, USA apurva@caltech.edu 2P. Seiler is with Faculty of Aerospace Engineering and Mechanics, University of Minnesota, Twin-Cities, USA seile017@umn.edu
Funding Information:
This work was supported by the National Science Foundation under Grant No. NSF-CMMI-1254129 entitled CAREER: Probabilistic Tools for High Reliability and Monitoring and Control of Wind Farms
Publisher Copyright:
© 2019 American Automatic Control Council.