TY - JOUR
T1 - Model selection and psychological theory
T2 - A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC)
AU - Vrieze, Scott I.
N1 - Copyright:
Copyright 2013 Elsevier B.V., All rights reserved.
PY - 2012/6
Y1 - 2012/6
N2 - This article reviews the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) in model selection and the appraisal of psychological theory. The focus is on latent variable models, given their growing use in theory testing and construction. Theoretical statistical results in regression are discussed, and more important issues are illustrated with novel simulations involving latent variable models including factor analysis, latent profile analysis, and factor mixture models. Asymptotically, the BIC is consistent, in that it will select the true model if, among other assumptions, the true model is among the candidate models considered. The AIC is not consistent under these circumstances. When the true model is not in the candidate model set the AIC is efficient, in that it will asymptotically choose whichever model minimizes the mean squared error of prediction/estimation. The BIC is not efficient under these circumstances. Unlike the BIC, the AIC also has a minimax property, in that it can minimize the maximum possible risk in finite sample sizes. In sum, the AIC and BIC have quite different properties that require different assumptions, and applied researchers and methodologists alike will benefit from improved understanding of the asymptotic and finite-sample behavior of these criteria. The ultimate decision to use the AIC or BIC depends on many factors, including the loss function employed, the study's methodological design, the substantive research question, and the notion of a true model and its applicability to the study at hand.
AB - This article reviews the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) in model selection and the appraisal of psychological theory. The focus is on latent variable models, given their growing use in theory testing and construction. Theoretical statistical results in regression are discussed, and more important issues are illustrated with novel simulations involving latent variable models including factor analysis, latent profile analysis, and factor mixture models. Asymptotically, the BIC is consistent, in that it will select the true model if, among other assumptions, the true model is among the candidate models considered. The AIC is not consistent under these circumstances. When the true model is not in the candidate model set the AIC is efficient, in that it will asymptotically choose whichever model minimizes the mean squared error of prediction/estimation. The BIC is not efficient under these circumstances. Unlike the BIC, the AIC also has a minimax property, in that it can minimize the maximum possible risk in finite sample sizes. In sum, the AIC and BIC have quite different properties that require different assumptions, and applied researchers and methodologists alike will benefit from improved understanding of the asymptotic and finite-sample behavior of these criteria. The ultimate decision to use the AIC or BIC depends on many factors, including the loss function employed, the study's methodological design, the substantive research question, and the notion of a true model and its applicability to the study at hand.
KW - Akaike information criterion
KW - Bayesian information criterion
KW - Factor analysis
KW - Model selection
KW - Theory testing
UR - http://www.scopus.com/inward/record.url?scp=84866456803&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84866456803&partnerID=8YFLogxK
U2 - 10.1037/a0027127
DO - 10.1037/a0027127
M3 - Article
C2 - 22309957
AN - SCOPUS:84866456803
SN - 1082-989X
VL - 17
SP - 228
EP - 243
JO - Psychological Methods
JF - Psychological Methods
IS - 2
ER -