Fractional ridge regression: A fast, interpretable reparameterization of ridge regression

Ariel Rokem, Kendrick Kay

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Ridge regression is a regularization technique that penalizes the L2-norm of the coefficients in linear regression. One of the challenges of using ridge regression is the need to set a hyperparameter (α) that controls the amount of regularization. Cross-validation is typically used to select the best α from a set of candidates. However, efficient and appropriate selection of α can be challenging. This becomes prohibitive when large amounts of data are analyzed. Because the selected α depends on the scale of the data and correlations across predictors, it is also not straightforwardly interpretable. Results: The present work addresses these challenges through a novel approach to ridge regression. We propose to reparameterize ridge regression in terms of the ratio γ between the L2-norms of the regularized and unregularized coefficients. We provide an algorithm that efficiently implements this approach, called fractional ridge regression, as well as open-source software implementations in Python and MATLAB (https://github.com/nrdg/fracridge). We show that the proposed method is fast and scalable for large-scale data problems. In brain imaging data, we demonstrate that this approach delivers results that are straightforward to interpret and compare across models and datasets. Conclusion: Fractional ridge regression has several benefits: the solutions obtained for different γ are guaranteed to vary, guarding against wasted calculations; and automatically span the relevant range of regularization, avoiding the need for arduous manual exploration. These properties make fractional ridge regression particularly suitable for analysis of large complex datasets.

Original languageEnglish (US)
Article numbergiaa133
JournalGigaScience
Volume9
Issue number12
DOIs
StatePublished - Dec 1 2021

Bibliographical note

Publisher Copyright:
© 2020 The Author(s).

Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.

Keywords

  • Brain imaging
  • General linear model
  • Hyperparameters
  • Open-source software

PubMed: MeSH publication types

  • Journal Article

Fingerprint Dive into the research topics of 'Fractional ridge regression: A fast, interpretable reparameterization of ridge regression'. Together they form a unique fingerprint.

Cite this