An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students

Sandra M. Chafouleas, Amy M. Briesch, T. Chris Riley-Tillman, Theodore J Christ, Anne C. Black, Stephen P. Kilgus

Research output: Contribution to journalArticlepeer-review

50 Scopus citations

Abstract

A total of 4 raters, including 2 teachers and 2 research assistants, used Direct Behavior Rating Single Item Scales (DBR-SIS) to measure the academic engagement and disruptive behavior of 7 middle school students across multiple occasions. Generalizability study results for the full model revealed modest to large magnitudes of variance associated with persons (students), occasions of measurement (day), and associated interactions. However, an unexpectedly low proportion of the variance in DBR data was attributable to the facet of rater, as well as a negligible variance component for the facet of rating occasion nested within day (10-min interval within a class period). Results of a reduced model and subsequent decision studies specific to individual rater and rater type (research assistant and teacher) suggested degree of reliability-like estimates differed substantially depending on rater. Overall, findings supported previous recommendations that in the absence of estimates of rater reliability and firm recommendations regarding rater training, ratings obtained from DBR-SIS, and subsequent analyses, be conducted within rater. Additionally, results suggested that when selecting a teacher rater, the person most likely to substantially interact with target students during the specified observation period may be the best choice.

Original languageEnglish (US)
Pages (from-to)219-246
Number of pages28
JournalJournal of school psychology
Volume48
Issue number3
DOIs
StatePublished - Jun 2010

Bibliographical note

Funding Information:
Preparation of this article was supported by a grant from the Institute for Education Sciences, U.S. Department of Education ( R324B060014 ). Opinions expressed herein do not necessarily reflect the position of the U.S. Department of Education, and such endorsements should not be inferred.

Copyright:
Copyright 2010 Elsevier B.V., All rights reserved.

Keywords

  • Behavior assessment
  • DBR
  • Dependability
  • Direct Behavior Rating
  • Generalizability

Fingerprint

Dive into the research topics of 'An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students'. Together they form a unique fingerprint.

Cite this