Assessing Medical Student Competency in Community-Engaged Service Learning: Development and Validity Evidence of a Multisource Feedback Tool.
Academic Article
Overview
abstract
PURPOSE: Community-engaged service learning (CSL) fosters partnerships to address health disparities and instills cultural humility and social responsibility in medical students. Despite its increasing integration into medical education, learner assessment remains a challenge; most assessments include reflection or participation but overlook behavioral and attitudinal skills. The authors aimed to cocreate a multisource feedback tool to assess observable competencies during the experiential phase of CSL. METHOD: From 2022 to 2024, the authors developed a multisource assessment tool with community partners and students in iterative phases while collecting validity evidence using Kane's framework. The tool was pilot-tested with 170 students and 39 site leaders at Albert Einstein College of Medicine, Bronx, New York; self, peer, and site leader assessments were collected for each student. After exploratory factor analysis, the authors implemented the revised tool with the next cohort of 168 students. The authors conducted confirmatory factor analysis, tested α and bivariate correlations among competency domains, and computed a mixed-effects regression model to affirm the tool was able to distinctly measure respondent-specific effects. Assessment data were compared with assessment data from other courses with similar learning objectives. RESULTS: The initial tool included 36 items in 5 competency domains: cultural humility, communication, teamwork, social responsibility, and professionalism. The second iteration of the tool included 29 items organized in the same 5 competency domains and 12 subdomains. The α coefficients for 3-item constructs exceeded .76, and bivariate correlations of 2-item constructs exceeded .47 ( P < .001). Comparative analysis indicated the tool's ratings for certain subdomains correlated with assessments in other courses. CONCLUSIONS: The authors developed and collected preliminary validity evidence of a novel multisource assessment tool to measure students' behavioral and attitudinal skills in CSL. The tool and process of development may inform assessment practices in other CSL programs.
