Implementation and Initial Construct Validity Evidence of a Tool, myTIPreport, for Interactive Workplace Feedback on ACGME Milestones.

Overview

OBJECTIVE: To describe implementation of myTIPreport for milestone feedback and to initiate construct validity testing of myTIPreport for milestones. DESIGN: myTIPreport was used to provide workplace feedback on Accreditation Council for Graduate Medical Education required milestone sets. Performance of senior learners (postgraduate year [PGY]-4s) was compared to that of junior learners (PGY-1s) to begin the process of construct validity testing for myTIPreport. SETTING: A convenience-based site selection of Obstetrics and Gynecology (OBGYN) residency programs. PARTICIPANTS: OBGYN residents and faculty. RESULTS: Amongst the 12 participating OBGYN residency programs, there were 444 unique learners and 343 unique faculty teachers. A total of 5293 milestone feedback encounters were recorded. Mean PGY-4 performance was rated higher than mean PGY-1 performance on all 25 of the compared milestone sets, with statistically significant differences seen for 19 (76%) of these 25 milestone sets and nonsignificant differences in the predicted direction observed for the other 6 milestone sets. CONCLUSIONS: myTIPreport detected differences between senior and junior learners for the majority of compared feedback encounters for OBGYN residents. Findings support the emerging construct validity of myTIPreport for milestone feedback.