Preconditioning ADMM for Fast Decentralized Optimization

In this work, we consider the distributed optimization problem using networked computing machines. Specifically, we are interested in solving this problem using the alternating direction method of multipliers (ADMM) while accounting for edge weights. Existing works focus on star graphs and use simple heuristics for other types of graphs. The present work shows that the optimal edge weights design is equivalent to the preconditioning matrix of ADMM that leads to the fastest convergence speed. Based on a tight convergence rate of ADMM, we show that the preconditioning matrix of general graphs can be found by minimizing the ratio of the largest and smallest nonzero eigenvalue of the graph Laplacian. Numerical experiments show that preconditioned ADMM converges much faster to a certain accuracy with less communication rounds, and exemplify the robustness to topology changes of the underlying network.