Changing abilities vs. changing tasks: Examining validity degradation with test scores and college performance criteria both assessed longitudinally

Jeffrey A. Dahlke, Jack W. Kostal, Paul R Sackett, Nathan R Kuncel

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

We explore potential explanations for validity degradation using a unique predictive validation data set containing up to four consecutive years of high school students' cognitive test scores and four complete years of those students' college grades. This data set permits analyses that disentangle the effects of predictor-score age and timing of criterion measurements on validity degradation. We investigate the extent to which validity degradation is explained by criterion dynamism versus the limited shelf-life of ability scores. We also explore whether validity degradation is attributable to fluctuations in criterion variability over time and/or GPA contamination from individual differences in course-taking patterns. Analyses of multiyear predictor data suggest that changes to the determinants of performance over time have much stronger effects on validity degradation than does the shelf-life of cognitive test scores. The age of predictor scores had only a modest relationship with criterion-related validity when the criterion measurement occasion was held constant. Practical implications and recommendations for future research are discussed.

Original languageEnglish (US)
Pages (from-to)980-1000
Number of pages21
JournalJournal of Applied Psychology
Volume103
Issue number9
DOIs
StatePublished - Sep 2018

Keywords

  • Cognitive ability
  • College performance
  • Dynamic criteria
  • Validity
  • Validity degradation

Fingerprint Dive into the research topics of 'Changing abilities vs. changing tasks: Examining validity degradation with test scores and college performance criteria both assessed longitudinally'. Together they form a unique fingerprint.

Cite this