TY - GEN
T1 - PA-GD
T2 - 36th International Conference on Machine Learning, ICML 2019
AU - Lu, Songtao
AU - Hong, Mingyi
AU - Wang, Zhengdao
PY - 2019/1/1
Y1 - 2019/1/1
N2 - Alternating gradient descent (A-GD) is a simple but popular algorithm in machine learning, which updates two blocks of variables in an alternating manner using gradient descent steps. In this paper, we consider a smooth unconstrained nonconvex optimization problem, and propose a perturbed A-GD (PA-GD) which is able to converge (with high probability) to the second-order stationary points (SOSPs) with a global sublinear rate. Existing analysis on A-GD type algorithm either only guarantees convergence to first-order solutions, or converges to second-order solutions asymptotically (without rates). To the best of our knowledge, this is the first alternating type algorithm that takes O(polylog(d)/∈#p2##p) iterations to achieve an (∈,√∈-SOSP with high probability, where polylog(d) denotes the polynomial of the logarithm with respect to problem dimension d.
AB - Alternating gradient descent (A-GD) is a simple but popular algorithm in machine learning, which updates two blocks of variables in an alternating manner using gradient descent steps. In this paper, we consider a smooth unconstrained nonconvex optimization problem, and propose a perturbed A-GD (PA-GD) which is able to converge (with high probability) to the second-order stationary points (SOSPs) with a global sublinear rate. Existing analysis on A-GD type algorithm either only guarantees convergence to first-order solutions, or converges to second-order solutions asymptotically (without rates). To the best of our knowledge, this is the first alternating type algorithm that takes O(polylog(d)/∈#p2##p) iterations to achieve an (∈,√∈-SOSP with high probability, where polylog(d) denotes the polynomial of the logarithm with respect to problem dimension d.
UR - http://www.scopus.com/inward/record.url?scp=85078010035&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85078010035&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85078010035
T3 - 36th International Conference on Machine Learning, ICML 2019
SP - 7301
EP - 7327
BT - 36th International Conference on Machine Learning, ICML 2019
PB - International Machine Learning Society (IMLS)
Y2 - 9 June 2019 through 15 June 2019
ER -