TY - JOUR
T1 - Situational Judgment Tests in High-Stakes Settings
T2 - Issues and Strategies With Generating Alternate Forms
AU - Lievens, Filip
AU - Sackett, Paul R.
PY - 2007/7
Y1 - 2007/7
N2 - This study used principles underlying item generation theory to posit competing perspectives about which features of situational judgment tests might enhance or impede consistent measurement across repeat test administrations. This led to 3 alternate-form development approaches (random assignment, incident isomorphism, and item isomorphism). The effects of these approaches on alternate-form consistency, mean score changes, and criterion-related validity were examined in a high-stakes context (N = 3,361). Generally, results revealed that even small changes in the context of the situations presented resulted in significantly lower alternate-form consistency. Conversely, placing more constraints on the alternate-form development process proved beneficial. The contributions, implications, and limitations of these results for the development of situational judgment tests and high-stakes testing are discussed.
AB - This study used principles underlying item generation theory to posit competing perspectives about which features of situational judgment tests might enhance or impede consistent measurement across repeat test administrations. This led to 3 alternate-form development approaches (random assignment, incident isomorphism, and item isomorphism). The effects of these approaches on alternate-form consistency, mean score changes, and criterion-related validity were examined in a high-stakes context (N = 3,361). Generally, results revealed that even small changes in the context of the situations presented resulted in significantly lower alternate-form consistency. Conversely, placing more constraints on the alternate-form development process proved beneficial. The contributions, implications, and limitations of these results for the development of situational judgment tests and high-stakes testing are discussed.
KW - alternate forms
KW - high stakes testing
KW - item generation
KW - situational judgment tests
UR - http://www.scopus.com/inward/record.url?scp=34548827973&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34548827973&partnerID=8YFLogxK
U2 - 10.1037/0021-9010.92.4.1043
DO - 10.1037/0021-9010.92.4.1043
M3 - Article
C2 - 17638464
AN - SCOPUS:34548827973
SN - 0021-9010
VL - 92
SP - 1043
EP - 1055
JO - Journal of Applied Psychology
JF - Journal of Applied Psychology
IS - 4
ER -