Computationally efficient likelihood inference in exponential families when the maximum likelihood estimator does not exist

Daniel J. Eck, Charles J. Geyer

Research output: Contribution to journalArticlepeer-review

Abstract

In a regular full exponential family, the maximum likelihood estimator (MLE) need not exist in the traditional sense. However, the MLE may exist in the completion of the exponential family. Existing algorithms for finding the MLE in the completion solve many linear programs; they are slow in small problems and too slow for large problems. We provide new, fast, and scalable methodology for finding the MLE in the completion of the exponential family. This methodology is based on conventional maximum likelihood computations which come close, in a sense, to finding the MLE in the completion of the exponential family. These conventional computations construct a likelihood maximizing sequence of canonical parameter values which goes uphill on the likelihood function until they meet a convergence criteria. Nonexistence of the MLE in this context results from a degeneracy of the canonical statistic of the exponential family, the canonical statistic is on the boundary of its support. There is a correspondance between this boundary and the null eigenvectors of the Fisher information matrix. Convergence of Fisher information along a likelihood maximizing sequence follows from cumulant generating function (CGF) convergence along a likelihood maximizing sequence, conditions for which are given. This allows for the construction of necessarily one-sided confidence intervals for mean value parameters when the MLE exists in the completion. We demonstrate our methodology on three examples in the main text and three additional examples in an accompanying technical report. We show that when the MLE exists in the completion of the exponential family, our methodology provides statistical inference that is much faster than existing techniques.

Original languageEnglish (US)
Pages (from-to)2105-2156
Number of pages52
JournalElectronic Journal of Statistics
Volume15
Issue number1
DOIs
StatePublished - 2021

Bibliographical note

Publisher Copyright:
© 2021, Institute of Mathematical Statistics. All rights reserved.

Keywords

  • Complete separation
  • Completion of exponential families
  • Generalized linear models
  • Logistic regression

Fingerprint

Dive into the research topics of 'Computationally efficient likelihood inference in exponential families when the maximum likelihood estimator does not exist'. Together they form a unique fingerprint.

Cite this