A coordinate majorization descent algorithm for ℓ1 penalized learning

Yi Yang; Hui Zou

doi:10.1080/00949655.2012.695374

A coordinate majorization descent algorithm for ℓ₁ penalized learning

Yi Yang, Hui Zou

Statistics (Twin Cities)

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ₁ penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ₁ penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.

Original language	English (US)
Pages (from-to)	84-95
Number of pages	12
Journal	Journal of Statistical Computation and Simulation
Volume	84
Issue number	1
DOIs	https://doi.org/10.1080/00949655.2012.695374
State	Published - Jan 2014

Bibliographical note

Funding Information:
The authors thank the editor, an associate editor and referee for their helpful comments and suggestions. This work is supported in part by NSF Grant DMS-08-46068.

Keywords

coordinate decent
glmnet
lasso
majorization-minimization

Access

10.1080/00949655.2012.695374

OpenUrl availability

Full text

Cite this

@article{260fc136debd4617a750e684871450e9,

title = "A coordinate majorization descent algorithm for ℓ1 penalized learning",

abstract = "The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ1 penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ1 penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.",

keywords = "coordinate decent, glmnet, lasso, majorization-minimization",

author = "Yi Yang and Hui Zou",

note = "Funding Information: The authors thank the editor, an associate editor and referee for their helpful comments and suggestions. This work is supported in part by NSF Grant DMS-08-46068.",

year = "2014",

month = jan,

doi = "10.1080/00949655.2012.695374",

language = "English (US)",

volume = "84",

pages = "84--95",

journal = "Journal of Statistical Computation and Simulation",

issn = "0094-9655",

publisher = "Taylor and Francis Ltd.",

number = "1",

}

TY - JOUR

T1 - A coordinate majorization descent algorithm for ℓ1 penalized learning

AU - Yang, Yi

AU - Zou, Hui

N1 - Funding Information: The authors thank the editor, an associate editor and referee for their helpful comments and suggestions. This work is supported in part by NSF Grant DMS-08-46068.

PY - 2014/1

Y1 - 2014/1

N2 - The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ1 penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ1 penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.

AB - The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ1 penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ1 penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.

KW - coordinate decent

KW - glmnet

KW - lasso

KW - majorization-minimization

UR - http://www.scopus.com/inward/record.url?scp=84887949355&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887949355&partnerID=8YFLogxK

U2 - 10.1080/00949655.2012.695374

DO - 10.1080/00949655.2012.695374

M3 - Article

AN - SCOPUS:84887949355

SN - 0094-9655

VL - 84

SP - 84

EP - 95

JO - Journal of Statistical Computation and Simulation

JF - Journal of Statistical Computation and Simulation

IS - 1

ER -