Consider the problem of minimizing the sum of a smooth convex function and a separable nonsmooth convex function subject to linear coupling constraints. Problems of this form arise in many contemporary applications, including signal processing, wireless networking, and smart grid provisioning. Motivated by the huge size of these applications, we propose a new class of first-order primal dual algorithms called the block successive upperboundminimizationmethod ofmultipliers (BSUM-M) to solve this family of problems. The BSUMMupdates the primal variable blocks successively by minimizing locally tight upper bounds of the augmented Lagrangian of the original problem, followed by a gradient-Type update for the dual variable in closed form.We show that under certain regularity conditions, and when the primal block variables are updated in either a deterministic or a random fashion, the BSUM-M converges to a point in the set of optimal solutions. Moreover, in the absence of linear constraints and under similar conditions as in the previous result, we show that the randomized BSUM-M(which reduces to the randomized block successive upper-bound minimization method) converges at an asymptotically linear rate without relying on strong convexity.
Bibliographical noteFunding Information:
Funding: M. Hong is supported in part by the National Science Foundation [Grants CMMI-1727757, AFOSR 15RT0767] and in part by the Shenzhen Fundamental Research Fund [Grant ZDSYS201707251409055]. T.-H. Chang was supported in part by the National Natural Science Foundation of China (NSFC) [Grants 61571385 and 61731018] and in part by the Shenzhen Fundamental Research Fund [Grants ZDSYS201707251409055 and KQTD2015033114415450]. X. Wang is supported by the NSFC [Grants 11871279, U1509219, and 61672231]. S. Ma is supported in part by a startup package from the Department of Mathematics at University of California, Davis. Z.-Q. Luo is supported by the Leading Talents of Guangdong Province Program [Grant 00201501], the NSFC [Grant 61731018], the Development and Reform Commission of Shenzhen Municipality, and the Shenzhen Funda-mental Research Fund [Grant KQTD201503311441545].
- alternating direction method of multipliers
- block successive upper-bound minimization
- randomized block coordinate descent