We consider efficient construction of nonlinear solution paths for general ℓ1-regularization. Unlike the existing methods that incrementally build the solution path through a combination of local linear approximation and recalibration, we propose an efficient global approximation to the whole solution path. With the loss function approximated by a quadratic spline, we show that the solution path can be computed using a generalized Lars algorithm. The proposed methodology avoids high-dimensional numerical optimization and thus provides faster and more stable computation. The methodology also can be easily extended to more general regularization framework. We illustrate such flexibility with several examples, including a generalization of the elastic net and a new method that effectively exploits the so-called "support vectors" in kernel logistic regression.
Bibliographical noteFunding Information:
Ming Yuan is Associate Professor, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0205 (E-mail: firstname.lastname@example.org). Hui Zou is Associate Professor, School of Statistics, University of Minnesota, Minneapolis, MN 55455 (E-mail: email@example.com. edu). Yuan’s research was supported in part by National Science Foundation (NSF) grant DMS-0706724. Zou’s research was supported in part by NSF grant DMS-0706733.
- Solution path
- Support vector pursuit