Feature reduction via generalized uncorrelated linear discriminant analysis

Jieping Ye; Ravi Janardan; Qi Li; Haesun Park

doi:10.1109/TKDE.2006.160

Feature reduction via generalized uncorrelated linear discriminant analysis

Jieping Ye, Ravi Janardan, Qi Li, Haesun Park

Research output: Contribution to journal › Article › peer-review

121 Scopus citations

Abstract

High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.

Original language	English (US)
Article number	1683768
Pages (from-to)	1312-1322
Number of pages	11
Journal	IEEE Transactions on Knowledge and Data Engineering
Volume	18
Issue number	10
DOIs	https://doi.org/10.1109/TKDE.2006.160
State	Published - Oct 2006
Externally published	Yes

Bibliographical note

Funding Information:
endorsement should be inferred. Fellowships from Guidant Corporation and from the Department of Computer Science and Engineering, at the University of Minnesota, Twin Cities are gratefully acknowledged. The work of H. Park has been performed while serving as a program director at the US National Science Foundation (NSF) and was partly supported by IR/D from the NSF. Her work was also supported in part by the US National Science Foundation Grants CCR-0204109 and ACI-0305543. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US National Science Foundation.

Funding Information:
The authors would like to thank the four reviewers and the associate editor for their comments, which helped improve the paper significantly. The research of J. Ye and R. Janardan was sponsored, in part, by the Army High Performance Computing Research Center under the auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAD19-01-2-0014, the content of which does not necessarily reflect the position or the policy of the government, and no official

Keywords

Feature reduction
Generalized singular value decomposition
QR-decomposition
Uncorrelated linear discriminant analysis

Access

10.1109/TKDE.2006.160

OpenUrl availability

Full text

Cite this

@article{f9ce467c6fc7415f9c7300cae4a73f01,

title = "Feature reduction via generalized uncorrelated linear discriminant analysis",

abstract = "High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.",

keywords = "Feature reduction, Generalized singular value decomposition, QR-decomposition, Uncorrelated linear discriminant analysis",

author = "Jieping Ye and Ravi Janardan and Qi Li and Haesun Park",

note = "Funding Information: endorsement should be inferred. Fellowships from Guidant Corporation and from the Department of Computer Science and Engineering, at the University of Minnesota, Twin Cities are gratefully acknowledged. The work of H. Park has been performed while serving as a program director at the US National Science Foundation (NSF) and was partly supported by IR/D from the NSF. Her work was also supported in part by the US National Science Foundation Grants CCR-0204109 and ACI-0305543. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US National Science Foundation. Funding Information: The authors would like to thank the four reviewers and the associate editor for their comments, which helped improve the paper significantly. The research of J. Ye and R. Janardan was sponsored, in part, by the Army High Performance Computing Research Center under the auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAD19-01-2-0014, the content of which does not necessarily reflect the position or the policy of the government, and no official",

year = "2006",

month = oct,

doi = "10.1109/TKDE.2006.160",

language = "English (US)",

volume = "18",

pages = "1312--1322",

journal = "IEEE Transactions on Knowledge and Data Engineering",

issn = "1041-4347",

publisher = "IEEE Computer Society",

number = "10",

}

TY - JOUR

T1 - Feature reduction via generalized uncorrelated linear discriminant analysis

AU - Ye, Jieping

AU - Janardan, Ravi

AU - Li, Qi

AU - Park, Haesun

N1 - Funding Information: endorsement should be inferred. Fellowships from Guidant Corporation and from the Department of Computer Science and Engineering, at the University of Minnesota, Twin Cities are gratefully acknowledged. The work of H. Park has been performed while serving as a program director at the US National Science Foundation (NSF) and was partly supported by IR/D from the NSF. Her work was also supported in part by the US National Science Foundation Grants CCR-0204109 and ACI-0305543. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US National Science Foundation. Funding Information: The authors would like to thank the four reviewers and the associate editor for their comments, which helped improve the paper significantly. The research of J. Ye and R. Janardan was sponsored, in part, by the Army High Performance Computing Research Center under the auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAD19-01-2-0014, the content of which does not necessarily reflect the position or the policy of the government, and no official

PY - 2006/10

Y1 - 2006/10

N2 - High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.

AB - High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.

KW - Feature reduction

KW - Generalized singular value decomposition

KW - QR-decomposition

KW - Uncorrelated linear discriminant analysis

UR - http://www.scopus.com/inward/record.url?scp=33748336501&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748336501&partnerID=8YFLogxK

U2 - 10.1109/TKDE.2006.160

DO - 10.1109/TKDE.2006.160

M3 - Article

AN - SCOPUS:33748336501

SN - 1041-4347

VL - 18

SP - 1312

EP - 1322

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

IS - 10

M1 - 1683768

ER -

Feature reduction via generalized uncorrelated linear discriminant analysis

Abstract

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this