Trace optimization and eigenproblems in dimension reduction methods

E. Kokiopoulou, J. Chen, Y. Saad

Research output: Contribution to journalArticlepeer-review

96 Scopus citations

Abstract

This paper gives an overview of the eigenvalue problems encountered in areas of data mining that are related to dimension reduction. Given some input high-dimensional data, the goal of dimension reduction is to map them to a low-dimensional space such that certain properties of the original data are preserved. Optimizing these properties among the reduced data can be typically posed as a trace optimization problem that leads to an eigenvalue problem. There is a rich variety of such problems and the goal of this paper is to unravel relationships between them as well as to discuss effective solution techniques. First, we make a distinction between projective methods that determine an explicit linear mapping from the high-dimensional space to the low-dimensional space, and nonlinear methods where the mapping between the two is nonlinear and implicit. Then, we show that all the eigenvalue problems solved in the context of explicit linear projections can be viewed as the projected analogues of the nonlinear or implicit projections. We also discuss kernels as a means of unifying linear and nonlinear methods and revisit some of the equivalences between methods established in this way. Finally, we provide some illustrative examples to showcase the behavior and the particular characteristics of the various dimension reduction techniques on real-world data sets.

Original languageEnglish (US)
Pages (from-to)565-602
Number of pages38
JournalNumerical Linear Algebra with Applications
Volume18
Issue number3
DOIs
StatePublished - May 1 2011

Keywords

  • Kernel methods
  • Laplacean eigenmaps
  • Linear dimension reduction
  • Locality preserving projections (LPP)
  • Locally linear embedding (LLE)
  • Nonlinear dimension reduction
  • Principal component analysis
  • Projection methods

Fingerprint Dive into the research topics of 'Trace optimization and eigenproblems in dimension reduction methods'. Together they form a unique fingerprint.

Cite this