Comment: Ridge Regression—Still Inspiring After 50 Years

It is very interesting to learn the history of ridge analysis/ridge regression as well as stories of its inventors from Professor Hoerl’s article. The overview article has covered many important aspects of ridge regression, regularization more generally, and their modern applications. Ridge has indeed become an essential concept in data science. My comments will focus on two new results related to ridge regularization: response guided principal component regression and leave-one-out analysis in kernel machines. The rst is motivated by the invariance property of ridge regression, and the second is a generalization of the leave-one-out analysis for ridge regression.