Sparse approximation via penalty decomposition methods

Zhaosong Lu, Yong Zhang

Research output: Contribution to journalArticlepeer-review

87 Scopus citations

Abstract

In this paper we consider sparse approximation problems, that is, general l0 minimization problems with the l0-"norm" of a vector being a part of constraints or objective function. In particular, we first study the first-order optimality conditions for these problems. We then propose penalty decomposition (PD) methods for solving them in which a sequence of penalty subproblems are solved by a block coordinate descent (BCD) method. Under some suitable assumptions, we establish that any accumulation point of the sequence generated by the PD methods satisfies the first-order optimality conditions of the problems. Furthermore, for the problems in which the l 0 part is the only nonconvex part, we show that such an accumulation point is a local minimizer of the problems. In addition, we show that any accumulation point of the sequence generated by the BCD method is a block coordinate minimizer of the penalty subproblem. Moreover, for the problems in which the l0 part is the only nonconvex part, we establish that such an accumulation point is a local minimizer of the penalty subproblem. Finally, we test the performance of our PD methods by applying them to sparse logistic regression, sparse inverse covariance selection, and compressed sensing problems. The computational results demonstrate that when solutions of same cardinality are sought, our approach applied to the l0-based models generally has better solution quality and/or speed than the existing approaches that are applied to the corresponding l1-based models.

Original languageEnglish (US)
Pages (from-to)2448-2478
Number of pages31
JournalSIAM Journal on Optimization
Volume23
Issue number4
DOIs
StatePublished - 2013
Externally publishedYes

Keywords

  • Block coordinate descent method
  • Compressed sensing
  • L minimization
  • Penalty decomposition methods
  • Sparse inverse covariance selection
  • Sparse logistic regression

Fingerprint

Dive into the research topics of 'Sparse approximation via penalty decomposition methods'. Together they form a unique fingerprint.

Cite this