The ABCs of PEMs: Using Artificial Intelligence to Enhance the Readability of Patient Educational Materials in Pediatric Orthopaedics.
Academic Article
Overview
abstract
BACKGROUND: While the American Medical Association and National Institute of Health recommend patient educational materials (PEMs) be written at a 6th-grade reading level, studies consistently show that PEMs in orthopaedics are written at the 10th-grade level or higher. This mismatch disproportionately affects patients with limited health literacy, who are at increased risk for poor clinical outcomes. This study investigates the potential of artificial intelligence (AI) platforms, including ChatGPT and OpenEvidence, to generate PEMs in pediatric orthopaedics that reach readability standards without sacrificing clinical accuracy. METHODS: Fifty-one of the most common pediatric orthopaedic conditions were selected using the American Academy of Orthopaedic Surgeons OrthoInfo PEM database. For each condition, PEMs were generated using two AI platforms: ChatGPT 4 and Open Evidence utilizing a standardized prompt requesting a sixth-grade level explanation that included relevant anatomy, symptoms, physical exam findings, and treatment options. Readability was assessed using eight validated readability metrics via the Python Textstat library. PEMs were scored for accuracy and completeness by four blinded, pediatric orthopaedic surgeons. Interrater reliability was assessed using intraclass correlation coefficients (ICCs), and statistical comparisons were performed using paired t-tests. RESULTS: ChatGPT-generated PEMs had the lowest average reading grade level (8.7) compared to OrthoInfo (10.8) and Open Evidence (10.1). OrthoInfo PEMs were rated highest for accuracy and completeness (total accuracy: 6.95; total completeness: 6.98), compared to Chat GPT (total accuracy: 6.15; total completeness: 5.90) and Open Evidence (total accuracy: 3.25; total completeness: 3.05), but ChatGPT approached OrthoInfo in several subdomains, including treatment descriptions, timeline, and follow-up recommendations. CONCLUSIONS: This study demonstrates the promise of AI platforms in generating readable, patient-friendly educational materials in pediatric orthopaedics. While OrthoInfo remains the gold standard in content accuracy and completeness, it falls short of national readability guidelines. AI tools like ChatGPT and OpenEvidence produced significantly more readable PEMs and, in some categories, approached the quality of expert-validated materials. These findings suggest a potential role for AI-assisted content creation in bridging the health literacy gap. However, concerns surrounding accuracy, hallucinations, and source transparency must be addressed before AI-generated PEMs can be safely integrated into clinical practice. KEY CONCEPTS: (1)Artificial intelligence (AI) refers to computer systems capable of performing tasks that typically require human intelligence, such as language processing; in this study, AI was used to revise and assess the readability of patient education materials.(2)Patient education materials (PEMs) are written or visual tools designed to inform patients and families about medical conditions, treatments, and procedures; they play a critical role in supporting shared decision-making in pediatric orthopaedics.(3)Readbility refers to how easily a written text can be understood by a target audience; improving the readability of PEMs ensures that patients and caregivers can comprehend essential health information.(4)Health literacy is the ability of individuals to obtain, process, and understand basic health information needed to make informed decisions; enhancing the readability of PEMs supports improved health literacy in pediatric populations.(5)Natural language processing (NLP) is a branch of AI that enables computers to understand and generate human language; in this study, NLP was used to revise PEMs and improve their readability and accessibility. LEVEL OF EVIDENCE: IV.
