Beyond sub-Gaussian measurements: High-dimensional structured estimation with sub-exponential designs

Vidyashankar Sivakumar, Arindam Banerjee, Pradeep Ravikumar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

We consider the problem of high-dimensional structured estimation with normregularized estimators, such as Lasso, when the design matrix and noise are drawn from sub-exponential distributions. Existing results only consider sub-Gaussian designs and noise, and both the sample complexity and non-asymptotic estimation error have been shown to depend on the Gaussian width of suitable sets. In contrast, for the sub-exponential setting, we show that the sample complexity and the estimation error will depend on the exponential width of the corresponding sets, and the analysis holds for any norm. Further, using generic chaining, we show that the exponential width for any set will be at most √log p times the Gaussian width of the set, yielding Gaussian width based results even for the sub-exponential case. Further, for certain popular estimators, viz Lasso and Group Lasso, using a VC-dimension based analysis, we show that the sample complexity will in fact be the same order as Gaussian designs. Our general analysis and results are the first in the sub-exponential setting, and are readily applicable to special sub-exponential families such as log-concave and extreme-value distributions.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems
PublisherNeural information processing systems foundation
Pages2206-2214
Number of pages9
Volume2015-January
StatePublished - 2015
Event29th Annual Conference on Neural Information Processing Systems, NIPS 2015 - Montreal, Canada
Duration: Dec 7 2015Dec 12 2015

Other

Other29th Annual Conference on Neural Information Processing Systems, NIPS 2015
CountryCanada
CityMontreal
Period12/7/1512/12/15

Fingerprint

Dive into the research topics of 'Beyond sub-Gaussian measurements: High-dimensional structured estimation with sub-exponential designs'. Together they form a unique fingerprint.

Cite this