An information theoretic analysis of maximum likelihood mixture estimation for exponential families

Arindam Banerjee; Inderjit Dhillon; Joydeep Ghosh; Srujana Merugu

An information theoretic analysis of maximum likelihood mixture estimation for exponential families

Arindam Banerjee, Inderjit Dhillon, Joydeep Ghosh, Srujana Merugu

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

16 Scopus citations

Abstract

An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.

Original language	English (US)
Title of host publication	Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004
Editors	R. Greiner, D. Schuurmans
Pages	57-64
Number of pages	8
State	Published - Dec 1 2004
Event	Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004 - Banff, Alta, Canada Duration: Jul 4 2004 → Jul 8 2004

Publication series

Name	Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

Other

Other	Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004
Country/Territory	Canada
City	Banff, Alta
Period	7/4/04 → 7/8/04

OpenUrl availability

Full text

Cite this

Banerjee, A., Dhillon, I., Ghosh, J., & Merugu, S. (2004). An information theoretic analysis of maximum likelihood mixture estimation for exponential families. In R. Greiner, & D. Schuurmans (Eds.), Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004 (pp. 57-64). (Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004).

An information theoretic analysis of maximum likelihood mixture estimation for exponential families. / Banerjee, Arindam; Dhillon, Inderjit; Ghosh, Joydeep et al.
Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. ed. / R. Greiner; D. Schuurmans. 2004. p. 57-64 (Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Banerjee, A, Dhillon, I, Ghosh, J & Merugu, S 2004, An information theoretic analysis of maximum likelihood mixture estimation for exponential families. in R Greiner & D Schuurmans (eds), Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004, pp. 57-64, Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004, Banff, Alta, Canada, 7/4/04.

Banerjee, Arindam ; Dhillon, Inderjit ; Ghosh, Joydeep et al. / An information theoretic analysis of maximum likelihood mixture estimation for exponential families. Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. editor / R. Greiner ; D. Schuurmans. 2004. pp. 57-64 (Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004).

@inproceedings{203bed3dec194771854dd293847e8465,

title = "An information theoretic analysis of maximum likelihood mixture estimation for exponential families",

abstract = "An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.",

author = "Arindam Banerjee and Inderjit Dhillon and Joydeep Ghosh and Srujana Merugu",

year = "2004",

month = dec,

day = "1",

language = "English (US)",

isbn = "1581138385",

series = "Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004",

pages = "57--64",

editor = "R. Greiner and D. Schuurmans",

booktitle = "Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004",

note = "Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004 ; Conference date: 04-07-2004 Through 08-07-2004",

}

TY - GEN

T1 - An information theoretic analysis of maximum likelihood mixture estimation for exponential families

AU - Banerjee, Arindam

AU - Dhillon, Inderjit

AU - Ghosh, Joydeep

AU - Merugu, Srujana

PY - 2004/12/1

Y1 - 2004/12/1

N2 - An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.

AB - An important task in unsupervised learning is maximum likelihood mixture estimation (MLME) for exponential families. In this paper, we prove a mathematical equivalence between this MLME problem and the rate distortion problem for Bregman divergences. We also present new theoretical results in rate distortion theory for Bregman divergences. Further, an analysis of the problems as a trade-off between compression and preservation of information is presented that yields the information bottleneck method as an interesting special case.

UR - http://www.scopus.com/inward/record.url?scp=14344265819&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=14344265819&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:14344265819

SN - 1581138385

SN - 9781581138382

T3 - Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

SP - 57

EP - 64

BT - Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

A2 - Greiner, R.

A2 - Schuurmans, D.

T2 - Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

Y2 - 4 July 2004 through 8 July 2004

ER -

An information theoretic analysis of maximum likelihood mixture estimation for exponential families

Abstract

Publication series

Other

OpenUrl availability

Other files and links

Fingerprint

Cite this