Examination of Skill Acquisition and Grader Bias in a Distal Radius Fracture Fixation Model

Matthew D. Putnam, Julie E. Adams, Paul Lender, Ann E. Van Heest, Janet R. Shanedling, David J. Nuckley, Joan E. Bechtold

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Objectives: Primary: Assess the ability of faculty graders to predict the objectively measured strength of distal radius fracture fixation. Secondary: Compare resident skill variation and retention related to other knowable training data. Design: Residents were allowed 60 minutes to stabilize a standardized distal radius fracture using an assigned fixed-angle volar plate. Faculty observed and subjectively graded the residents without providing real-time feedback. Objective biomechanical evaluation (construct strength and stiffness) was compared to subjective grades. Resident-specific characteristics (sex, PGY, and ACGME case log) were also used to compare the objective data. Setting: A simulated operating room in our laboratory. Participants: Post-graduate year 2, 3, 4, and 5 orthopedic residents. Results: Primary: Faculty were not successful at predicting objectively measured fixation, and their subjective scoring suggests confirmation bias as PGY increased. Secondary: Resident year-in-training alone did not predict objective measures (p = 0.53), but was predictive of subjective scores (p < 0.001). Skills learned were not always retained, as 29% of residents objectively failed subsequent to passing. Notably, resident-reported case-specific experience alone was inversely correlated with objective fixation strength. Conclusions: This testing model enabled the collection of objective and subjective resident skill scores. Faculty graders did not routinely predict objective measures, and their subjective assessment appears biased related to PGY. Also, in vivo case volume alone does not predict objective results. Familiar faculty teaching consistency, and resident grading by external faculty unfamiliar with tested residents, might alter these results.

Original languageEnglish (US)
Pages (from-to)1299-1308
Number of pages10
JournalJournal of surgical education
Volume75
Issue number5
DOIs
StatePublished - Sep 1 2018

Bibliographical note

Funding Information:
Special thanks to University of Minnesota staff Andrea Chatfield and Cris Hansen (Department of Orthopaedic Surgery) for manuscript preparation; Conrad Lindquist (Department of Orthopaedic Surgery) for laboratory management and testing station design; Stanley Weisberg, PhD (Department of Statistics) for statistics design; Bradley Kuzel, MD (Essentia Health Org, Orthopedic Surgery); Brian Beaubien, PhD (PSoup, Inc), Arin Ellingson, PhD (Phys Med Rehab), Andrew L. Freeman, MS (Sci Engineer), Amy Claeson, PhD (University of Delaware), Hitesh Mehta (Sci Engineer), Tina M. Nagel (Sci Engineer), Patrick Truchinski (PA), and Daniel Wheeler, MD (Department of Internal Med) for specimen preparation and mechanical testing; and Jeffrey B. Husband, MD, Christina Ward, MD, Thomas P Varecka, MD, Jacqueline Geisler, MD, James Fletcher, MD, and Nicholas J Meyer, MD for grading. Also, thank you to Don Anderson, PhD (University of Iowa, Department of Orthopaedics) for initial review of the manuscript.

Publisher Copyright:
© 2018 Association of Program Directors in Surgery

Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.

Keywords

  • Medical Knowledge
  • Patient Care
  • Practice-Based Learning and Improvement
  • bias
  • competence
  • fracture
  • resident
  • skill assessment
  • training

Fingerprint Dive into the research topics of 'Examination of Skill Acquisition and Grader Bias in a Distal Radius Fracture Fixation Model'. Together they form a unique fingerprint.

Cite this