Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing

Chun Wang

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

Many latent traits in social sciences display a hierarchical structure, such as intelligence, cognitive ability, or personality. Usually a second-order factor is linearly related to a group of first-order factors (also called domain abilities in cognitive ability measures), and the first-order factors directly govern the actual item responses. Because only a subtest of items is used to measure each domain, the lack of sufficient reliability becomes the primary impediment for generating and reporting domain abilities. In recent years, several item response theory (IRT) models have been proposed to account for hierarchical factor structures, and these models are also shown to alleviate the low reliability issue by using in-test collateral information to improve measurement precision. This article advocates using adaptive item selection together with a higher order IRT model to further increase the reliability of hierarchical latent trait estimation. Two item selection algorithms are proposed—the constrained D-optimal method and the sequencing domain method. Both are shown to yield improved measurement precision as compared to the unidimensional item selection (by treating each dimension separately). The improvement is more prominent when the test length is short and when the correlation between dimensions is high (e.g., higher than.64). Moreover, two reliability indices for hierarchical latent traits are discussed and their use for quantifying the reliability of hierarchical traits measured by adaptive testing is demonstrated.

Original languageEnglish (US)
Pages (from-to)452-477
Number of pages26
JournalJournal of Educational and Behavioral Statistics
Volume39
Issue number6
DOIs
StatePublished - Dec 20 2014

Bibliographical note

Publisher Copyright:
© 2014 AERA.

Keywords

  • computerized adaptive testing
  • domain ability
  • higher order IRT model
  • overall ability
  • reliability

Fingerprint

Dive into the research topics of 'Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing'. Together they form a unique fingerprint.

Cite this