Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1

Julie Ayre, Carissa Bonner, Danielle M Muscat, Adam G Dunn, Eliza Harrison, Jason Dalmazzo, Dana Mouwad, Parisa Aslani, Heather L Shepherd, Kirsten J McCaffery

Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a "text preparation" mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor's ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.