TY - JOUR
T1 - Reliability Assessment of an Observation Tool to Measure Praise Characteristics
AU - Markelz, Andrew M.
AU - Riden, Benjamin S.
AU - Zoder-Martell, Kimberly A.
AU - Miller, Joseph E.
AU - Bolinger, Sarah J.
N1 - Publisher Copyright:
© Hammill Institute on Disabilities 2020.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2021/1
Y1 - 2021/1
N2 - Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of.80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p <.001, and mean kappa score of.91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.
AB - Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of.80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p <.001, and mean kappa score of.91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.
KW - children with or at risk of EBD
KW - classroom intervention(s)
KW - intervention(s)
KW - positive behavior support(s)
KW - prevention of challenging behavior(s)
UR - http://www.scopus.com/inward/record.url?scp=85081631109&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85081631109&partnerID=8YFLogxK
U2 - 10.1177/1098300720907988
DO - 10.1177/1098300720907988
M3 - Article
AN - SCOPUS:85081631109
VL - 23
SP - 17
EP - 29
JO - Journal of Positive Behavior Interventions
JF - Journal of Positive Behavior Interventions
SN - 1098-3007
IS - 1
ER -