TY - JOUR
T1 - Narrative Performance Level Assignments at Initial Entrustment and Graduation
T2 - Integrating EPAs and Milestones to Improve Learner Assessment
AU - Schumacher, Daniel J.
AU - Schwartz, Alan
AU - Zenel, Joseph A.
AU - Paradise Black, Nicole
AU - Ponitz, Keith
AU - Blair, Robyn
AU - Traba, Christin M.
AU - Poynter, Sue
AU - King, Beth
AU - Englander, Robert
AU - Rosenberg, Adam
AU - Patel, Dilipkumar
AU - Smith-King, Candace
AU - O'Connor, Meghan
AU - Gonzalez Del Rey, Javier
AU - Lavoie, Suzanne
AU - Borman-Shoap, Emily
AU - Carraccio, Carol
N1 - Publisher Copyright:
© 2020 Lippincott Williams and Wilkins. All rights reserved.
PY - 2020/11/1
Y1 - 2020/11/1
N2 - Purpose To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. Method A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015-2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015-2016 to 2017-2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. Results ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home-well child (OR = 0.39; 95% CI: 0.26-0.57), transition to adult care (OR = 0.43; 95% CI: 0.19-0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18-0.71), make referrals (OR = 0.31; 95% CI: 0.17-0.55), lead a team (OR = 0.34; 95% CI: 0.22-0.52), and handovers (OR = 0.18; 95% CI: 0.09-0.36) EPAs. Conclusions CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.
AB - Purpose To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. Method A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015-2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015-2016 to 2017-2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. Results ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home-well child (OR = 0.39; 95% CI: 0.26-0.57), transition to adult care (OR = 0.43; 95% CI: 0.19-0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18-0.71), make referrals (OR = 0.31; 95% CI: 0.17-0.55), lead a team (OR = 0.34; 95% CI: 0.22-0.52), and handovers (OR = 0.18; 95% CI: 0.09-0.36) EPAs. Conclusions CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.
UR - http://www.scopus.com/inward/record.url?scp=85094933161&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094933161&partnerID=8YFLogxK
U2 - 10.1097/ACM.0000000000003300
DO - 10.1097/ACM.0000000000003300
M3 - Article
C2 - 32195689
AN - SCOPUS:85094933161
SN - 1040-2446
VL - 95
SP - 1736
EP - 1744
JO - Academic Medicine
JF - Academic Medicine
IS - 11
ER -
