Validating pose-derived entropy metrics for video-based surgical skill assessment: a pilot study in orthopaedic surgery
Description
Introduction: A previous study demonstrated that automated hand motion analysis can aid in evaluating surgical performance. We hypothesized that single-camera video capture of surgical tasks, processed with OpenPose, a 2D pose estimation algorithm, could identify upper extremity movement patterns that correlate with Global Rating Scale (GRS) scores.
Methods: In a cadaveric laboratory, medical students (n = 7), residents (n = 12), fellows (n = 2), and attending surgeons (n = 3) were filmed performing standardized surgical tasks: incision, suturing, one- and two-handed ties, reduction clamp application, and K-wire and lag screw placement. Videos were captured on an iPhone and analyzed with OpenPose. Joint coordinates were processed in Python to compute kinematic and entropy-based metrics. Expert evaluators, blinded by face occlusion, scored each task recording with a validated GRS.
Results: Across all tasks, multiple upper extremity metrics demonstrated moderate to strong correlations with GRS scores (r = +0.43 to −0.72, p < 0.05). Time to task completion showed the strongest correlation (r = −0.62 to −0.84, p < 0.001). Reduced directional joint entropy between paired joints consistently correlated negatively with GRS, observed in five of eight tasks (r = −0.42 to −0.72, p < 0.05).
Discussion: Features extracted from single-camera 2D video strongly correlated with expert-evaluated scores, showing that pose estimation algorithms have the potential to provide an unbiased framework for evaluating orthopaedic surgical skills. By utilizing an objective evaluation method, it aims to reduce evaluator burden, standardize assessments, provide real-time intraoperative feedback, and ultimately enhance surgical training to improve patient outcomes.
Citation Information
Zelenski, Kyle N.; Nyul, Thomas; Chakraborty, Aritra; Sibley, Sean; Rohde, Daniel; and Marmor, Meir T., "Validating pose-derived entropy metrics for video-based surgical skill assessment: a pilot study in orthopaedic surgery" (2026). Office of Research DMU Research Symposium. 25.
https://digitalcommons.dmu.edu/researchsymposium/2025rs/2025abstracts/25
Validating pose-derived entropy metrics for video-based surgical skill assessment: a pilot study in orthopaedic surgery
Introduction: A previous study demonstrated that automated hand motion analysis can aid in evaluating surgical performance. We hypothesized that single-camera video capture of surgical tasks, processed with OpenPose, a 2D pose estimation algorithm, could identify upper extremity movement patterns that correlate with Global Rating Scale (GRS) scores.
Methods: In a cadaveric laboratory, medical students (n = 7), residents (n = 12), fellows (n = 2), and attending surgeons (n = 3) were filmed performing standardized surgical tasks: incision, suturing, one- and two-handed ties, reduction clamp application, and K-wire and lag screw placement. Videos were captured on an iPhone and analyzed with OpenPose. Joint coordinates were processed in Python to compute kinematic and entropy-based metrics. Expert evaluators, blinded by face occlusion, scored each task recording with a validated GRS.
Results: Across all tasks, multiple upper extremity metrics demonstrated moderate to strong correlations with GRS scores (r = +0.43 to −0.72, p < 0.05). Time to task completion showed the strongest correlation (r = −0.62 to −0.84, p < 0.001). Reduced directional joint entropy between paired joints consistently correlated negatively with GRS, observed in five of eight tasks (r = −0.42 to −0.72, p < 0.05).
Discussion: Features extracted from single-camera 2D video strongly correlated with expert-evaluated scores, showing that pose estimation algorithms have the potential to provide an unbiased framework for evaluating orthopaedic surgical skills. By utilizing an objective evaluation method, it aims to reduce evaluator burden, standardize assessments, provide real-time intraoperative feedback, and ultimately enhance surgical training to improve patient outcomes.
