Hybrid ADMM: a unifying and fast approach to decentralized optimization

Meng Ma, Athanasios N. Nikolakopoulos, Georgios B. Giannakis

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

The present work introduces the hybrid consensus alternating direction method of multipliers (H-CADMM), a novel framework for optimization over networks which unifies existing distributed optimization approaches, including the centralized and the decentralized consensus ADMM. H-CADMM provides a flexible tool that leverages the underlying graph topology in order to achieve a desirable sweet spot between node-to-node communication overhead and rate of convergence—thereby alleviating known limitations of both C-CADMM and D-CADMM. A rigorous analysis of the novel method establishes linear convergence rate and also guides the choice of parameters to optimize this rate. The novel hybrid update rules of H-CADMM lend themselves to “in-network acceleration” that is shown to effect considerable—and essentially “free-of-charge”—performance boost over the fully decentralized ADMM. Comprehensive numerical tests validate the analysis and showcase the potential of the method in tackling efficiently, widely useful learning tasks.

Original languageEnglish (US)
Article number73
JournalEurasip Journal on Advances in Signal Processing
Volume2018
Issue number1
DOIs
StatePublished - Dec 1 2018

Bibliographical note

Funding Information:
The authors wish to thank Prof. M. Hong from University of Minnesota for insightful discussions on related subjects. Part of this work was supported by NSF grants 1442686, 1500713, 1509040, and 171141.

Publisher Copyright:
© 2018, The Author(s).

Keywords

  • ADMM
  • Consensus
  • Decentralized learning
  • Distributed optimization
  • Hybrid

Fingerprint

Dive into the research topics of 'Hybrid ADMM: a unifying and fast approach to decentralized optimization'. Together they form a unique fingerprint.

Cite this