Abstract
The alternating direction method of multipliers (ADMM) is widely used in solving structured convex optimization problems due to its superior practical performance. On the theoretical side however, a counterexample was shown in Chen et al. (Math Program 155(1):57–79, 2016.) indicating that the multi-block ADMM for minimizing the sum of N(N≥ 3) convex functions with N block variables linked by linear constraints may diverge. It is therefore of great interest to investigate further sufficient conditions on the input side which can guarantee convergence for the multi-block ADMM. The existing results typically require the strong convexity on parts of the objective. In this paper, we provide two different ways related to multi-block ADMM that can find an ϵ-optimal solution and do not require strong convexity of the objective function. Specifically, we prove the following two results: (1) the multi-block ADMM returns an ϵ-optimal solution within O(1/ϵ2) iterations by solving an associated perturbation to the original problem; this case can be seen as using multi-block ADMM to solve a modified problem; (2) the multi-block ADMM returns an ϵ-optimal solution within O(1/ϵ) iterations when it is applied to solve a certain sharing problem, under the condition that the augmented Lagrangian function satisfies the Kurdyka–Łojasiewicz property, which essentially covers most convex optimization models except for some pathological cases; this case can be seen as applying multi-block ADMM to solving a special class of problems.
Original language | English (US) |
---|---|
Pages (from-to) | 52-81 |
Number of pages | 30 |
Journal | Journal of Scientific Computing |
Volume | 69 |
Issue number | 1 |
DOIs | |
State | Published - Oct 1 2016 |
Bibliographical note
Funding Information:Shiqian Ma: Research of this author was supported in part by the Hong Kong Research Grants Council General Research Fund Early Career Scheme (Project ID: CUHK 439513). Shuzhong Zhang: Research of this author was supported in part by the National Science Foundation under Grant Number CMMI-1462408.
Publisher Copyright:
© 2016, Springer Science+Business Media New York.
Keywords
- Alternating direction method of multipliers (ADMM)
- Convergence rate
- Convex optimization
- Kurdyka–Łojasiewicz property
- Regularization