Comparison of loss functions for linear regression

Research output: Contribution to journalConference articlepeer-review

12 Scopus citations

Abstract

This paper addresses selection of the loss function for regression problems with finite data. It is well-known (under standard regression formulation) that for a known noise density there exist an optimal loss function under an asymptotic setting (large number of samples), i.e. squared loss is optimal for Gaussian noise density. However, in real-life applications the noise density is unknown and the number of training samples is finite. For such practical situations, we suggest using Vapnik's ε-insensitive loss function. We use practical method for setting the value of ε as a function of known number of samples and (known or estimated) noise variance [1,2]. We consider commonly used noise densities (such as Gaussian, Uniform and Laplacian noise). Empirical comparisons for several representative linear regression problems indicate that Vapnik's ε-insensitive loss yields more robust performance and improved prediction accuracy, in comparison with squared loss and least-modulus loss, especially for noisy high-dimensional data sets.

Original languageEnglish (US)
Pages (from-to)395-400
Number of pages6
JournalIEEE International Conference on Neural Networks - Conference Proceedings
Volume1
StatePublished - Dec 1 2004
Event2004 IEEE International Joint Conference on Neural Networks - Proceedings - Budapest, Hungary
Duration: Jul 25 2004Jul 29 2004

Fingerprint

Dive into the research topics of 'Comparison of loss functions for linear regression'. Together they form a unique fingerprint.

Cite this