The Stanford Microsurgery and Resident Training (SMaRT) Scale: validation of an on-line global rating scale for technical assessment.
Academic Article
Overview
abstract
INTRODUCTION: We previously reported results of our on-line microsurgery training program, showing that residents who had access to our website significantly improved their cognitive and technical skills. In this study, we report an objective means for expert evaluators to reliably rate trainees' technical skills under the microscope, with the use of our novel global rating scale. METHODS: "Microsurgery Essentials" (http://smartmicrosurgery.com) is our on-line training curriculum. Residents were randomly divided into 2 groups: 1 group reviewed this online resource and the other did not. Pre- and post-tests consisted of videotaped microsurgical sessions in which the trainee performed "microsurgery" on 3 different models: latex glove, penrose drain, and the dorsal vessel of a chicken foot. The SMaRT (Stanford Microsurgery and Resident Training) scale, consisting of 9 categories graded on a 5-point Likert scale, was used to assess the trainees. Results were analyzed with ANOVA and Student t test, with P less than 0.05 indicating statistical significance. RESULTS: Seventeen residents participated in the study. The SMaRT scale adequately differentiated the performance of more experienced senior residents (PGY-4 to PGY-6, total average score=3.43) from less experienced junior residents (PGY-1 to PGY-3, total average score=2.10, P<0.0001). Residents who viewed themselves as being confident received a higher score on the SMaRT scale (average score 3.5), compared to residents who were not as confident (average score 2.1) (P<0.001). There were no significant differences in scoring among all 3 evaluators (P>0.05). Additionally, junior residents who had access to our website showed a significant increase in their graded technical performance by 0.7 points when compared to residents who did not have access to the website who showed an improvement of only 0.2 points (P=0.01). CONCLUSIONS: Our SMaRT scale is valid and reliable in assessing the microsurgical skills of residents and other trainees. Current trainees are more likely to use self-directed on-line education because of its easy accessibility and interactive format. Our global rating scale can help ensure residents are achieving appropriate technical milestones.
