Purpose:: Finding effective methods of discriminating surgeon technical skill has proved a complex problem to solve computationally. Previous research has shown that obtaining non-expert crowd evaluations of surgical performances is as accurate as the gold standard, expert surgeon review. The aim of this research is: (1) to learn whether crowdsourced evaluators give higher ratings of technical skill to video of performances with increased playback speed, (2) its effect in discriminating skill levels, and (3) whether this increase is related to the evaluator consciously being aware that the video is manually manipulated. Methods:: A set of ten peg transfer videos (five novices, five experts) were used to evaluate the perceived technical skill of the performers at each video playback speed used (0.4 × - 3.6 ×). Objective metrics used for measuring technical skill were also computed for comparison by manipulating the corresponding kinematic data of each performance. Two videos of an expert and novice performing dry laboratory laparoscopic trials of peg transfer tasks were used to obtain evaluations at each playback speed (0.2 × - 3.0 ×) of perception of whether a video is played at real-time playback speed or not. Results:: We found that while both novices and experts had increased perceived technical skill as the video playback was increased, the amount of increase was significantly greater for experts. Each increase in the playback speed by 0.4 × was associated with, on average, a 0.72-point increase in the GOALS score (95% CI 0.60–0.84 point increase; p< 0.001) for expert videos and only a 0.24-point increase in the GOALS score (95% CI 0.13–0.36 point increase; p< 0.001) for novice videos. Conclusion:: Due to the differential increase in perceived technical skill due to increased playback speed for experts, the difference between novice and expert skill levels of surgical performances may be more easily discerned by manually increasing the video playback speed.
|Original language||English (US)|
|Number of pages||9|
|Journal||International Journal of Computer Assisted Radiology and Surgery|
|State||Published - May 1 2020|
Bibliographical noteFunding Information:
This work was supported, in part, by the Office of the Assistant Secretary of Defense for Health Affairs under Award No. W81XWH-15-2-0030, the National Science Foundation CAREER Grant under Award No. 1847610, as well as the National Institutes of Health’s National Center for Advancing Translational Sciences, Grant UL1TR002494. Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense, the National Science Foundation, or the National Institutes of Healths’s National Center for Advancing Translational Sciences.
© 2020, CARS.
- Crowd sourcing
- Speed perception
- Surgical technical skill
- Video playback
PubMed: MeSH publication types
- Journal Article