How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules

Ethan R. Van Norman, Theodore J. Christ

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Curriculum based measurement of oral reading (CBM-R) is used to monitor the effects of academic interventions for individual students. Decisions to continue, modify, or terminate these interventions are made by interpreting time series CBM-R data. Such interpretation is founded upon visual analysis or the application of decision rules. The purpose of this study was to compare the accuracy of visual analysis and decision rules. Visual analysts interpreted 108 CBM-R progress monitoring graphs one of three ways: (a) without graphic aids, (b) with a goal line, or (c) with a goal line and a trend line. Graphs differed along three dimensions, including trend magnitude, variability of observations, and duration of data collection. Automated trend line and data point decision rules were also applied to each graph. Inferential analyses permitted the estimation of the probability of a correct decision (i.e., the student is improving – continue the intervention, or the student is not improving – discontinue the intervention) for each evaluation method as a function of trend magnitude, variability of observations, and duration of data collection. All evaluation methods performed better when students made adequate progress. Visual analysis and decision rules performed similarly when observations were less variable. Results suggest that educators should collect data for more than six weeks, take steps to control measurement error, and visually analyze graphs when data are variable. Implications for practice and research are discussed.

Original languageEnglish (US)
Pages (from-to)41-55
Number of pages15
JournalJournal of school psychology
Volume58
DOIs
StatePublished - Oct 1 2016

Bibliographical note

Publisher Copyright:
© 2016 Society for the Study of School Psychology

Copyright:
Copyright 2016 Elsevier B.V., All rights reserved.

Keywords

  • Assessment
  • Curriculum based measurement
  • Progress monitoring

Fingerprint Dive into the research topics of 'How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules'. Together they form a unique fingerprint.

Cite this