Can Culture Be a Salient Predictor of Test-Taking Engagement? An Analysis of Differential Noneffortful Responding on an International College-Level Assessment of Critical Thinking

Joseph A. Rios, Hongwen Guo

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

The objective of this study was to evaluate whether differential noneffortful responding (identified via response latencies) was present in four countries administered a low-stakes college-level critical thinking assessment. Results indicated significant differences (as large as.90 SD) between nearly all country pairings in the average number of noneffortful responses per test taker. Furthermore, noneffortful responding was found to be associated with a number of individual-level predictors, such as demographics (both gender and academic year), prior ability, and perceived difficulty of the test, though, these predictors were found to differ across countries. Ignoring the presence of noneffortful responses was associated with: (a) model fit deterioration as well as inflation of reliability, and (b) the inclusion of non-invariant items in the score linking anchor set. However, no meaningful differences in relative performance were noted once accounting for noneffortful responses. Implications for test development and improving the validity of score-based inferences from international assessments are discussed.

Original languageEnglish (US)
Pages (from-to)263-279
Number of pages17
JournalApplied Measurement in Education
Volume33
Issue number4
DOIs
StatePublished - Oct 1 2020

Bibliographical note

Publisher Copyright:
© 2020 Taylor & Francis Group, LLC.

Fingerprint

Dive into the research topics of 'Can Culture Be a Salient Predictor of Test-Taking Engagement? An Analysis of Differential Noneffortful Responding on an International College-Level Assessment of Critical Thinking'. Together they form a unique fingerprint.

Cite this