Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers

One potentially feasible tool for use in the formative assessment of social behavior is the direct behavior rating, yet empirical support for the reliability of its use is limited. In this study, generalizability theory was used to provide preliminary psychometric data regarding the generalizability and depend-ability of the direct behavior rating to measure the social behavior of preschoolers. Two typical preschool behaviors (works to resolve conflicts, interacts cooperatively with peers) were selected for investigation within the direct behavior rating created for this study. Overall, results varied depending on which behavior was rated and the number of raters whose ratings were considered. The results suggested that a fairly substantial proportion of measurement variance was attributable to the different raters, and that the four raters varied in their mean level of ratings within and across the 15 students. In addition, although the actual number of days was dependent on the number of ratings collected per day, results suggested direct behavior ratings are likely to approximate or exceed reliability-like coefficients of .70 after 7 ratings are collected across 4-7 days, and .90 after 10 ratings. Limitations, future directions, and implications are discussed.