Bias and Information of Bayesian Adaptive Testing

David J. Weiss, James R. McBride

Research output: Contribution to journalArticlepeer-review

21 Scopus citations

Abstract

Monte carlo simulation was used to investigate score bias and information characteristics of Owen's Bayesian adaptive testing strategy and to examine pos sible causes of score bias. Factors investigated in three related studies included effects of an accurate prior θ estimate, effects of item discrimination, and effects of fixed versus variable test length. Data were generated from a three-parameter logistic model for 3,100 simu lees in each of eight data sets, and Bayesian adaptive tests were administered, drawing items from a “per fect” item pool. Results showed that the Bayesian adaptive test yielded unbiased θ estimates and rela tively flat information functions only in the situation in which an accurate prior θ estimate was used. When a constant prior θ estimate was used with a fixed test length, severe bias was observed that varied with item discrimination. A different pattern of bias was ob served with variable test length and a constant prior. Information curves for the constant prior conditions generally became more peaked and asymmetric with increasing item discrimination. In the variable test length condition, the test length required to achieve a specified level of the posterior variance of θ estimates was an increasing function of θ level. These results indicate that θ estimates from Owen's Bayesian adap tive testing method are affected by the prior θ estimate used and that the method does not provide measure ments that are unbiased and equiprecise except when an accurate prior θ estimate is used.

Original languageEnglish (US)
Pages (from-to)273-285
Number of pages13
JournalApplied Psychological Measurement
Volume8
Issue number3
DOIs
StatePublished - Jul 1984

Bibliographical note

Copyright:
Copyright 2016 Elsevier B.V., All rights reserved.

Fingerprint Dive into the research topics of 'Bias and Information of Bayesian Adaptive Testing'. Together they form a unique fingerprint.

Cite this