Validity Evidence for STATUS To Assess Resident Tolerance for and Competence in Communicating Medical Ambiguity.
Academic Article
Overview
abstract
OBJECTIVE: No assessment instrument with validity evidence exists to assess resident competence in communicating medical ambiguity. Here, validity evidence was collected for STATUS (Scalable Tolerating Ambiguity/Uncertainty Tool Utilizing Simulation). METHODS: Using avatar patients and two simulated cases, investigators created a guidebook and trained ten faculty in STATUS use. Pediatric residents completed two video-recorded simulated cases. Residents self-assessed tolerance for communicating medical ambiguity. Two faculty reviewed each video, assessing participants for communicating medical ambiguity. Validity evidence collected included: content, response process, internal structure, and relationship to other variables. Generalizability theory analysis was conducted to understand the assessment tools' reliability. RESULTS: Of 89 eligible residents, forty-three (48.3%) had sessions recorded. Eighty-six videos were analyzed. Faculty rater training increased inter-rater reliability by 0.34 units. The Φ-coefficient was 0.72 for the resident self-assessment tool and 0.26 for the faculty rater assessment tool. The decision study found that with 11 faculty raters and 11 scenarios, the Φ-coefficient would be 0.70. Resident self-assessment negatively associated faculty rater assessment with a Spearman correlation of -0.21 overall, indicating a possible weak correlation. CONCLUSION: Results provide sufficient reliability to measure resident self-assessment of tolerance for communicating medical ambiguity. More scenarios would likely result in higher reliability for faculty assessment.
