Ji Meng, Liu Yanmeng, Zhao Mengdan, Lyu Ziqing, Zhang Boren, Luo Xin, Li Yanlin, Zhong Yin
School of Languages and Cultures, University of Sydney, Sydney, Australia.
School of Foreign Languages, Jiangsu University of Science and Technology, Zhenjiang, China.
JMIR Med Inform. 2021 May 6;9(5):e28413. doi: 10.2196/28413.
Improving the understandability of health information can significantly increase the cost-effectiveness and efficiency of health education programs for vulnerable populations. There is a pressing need to develop clinically informed computerized tools to enable rapid, reliable assessment of the linguistic understandability of specialized health and medical education resources. This paper fills a critical gap in current patient-oriented health resource development, which requires reliable and accurate evaluation instruments to increase the efficiency and cost-effectiveness of health education resource evaluation.
We aimed to translate internationally endorsed clinical guidelines to machine learning algorithms to facilitate the evaluation of the understandability of health resources for international students at Australian universities.
Based on international patient health resource assessment guidelines, we developed machine learning algorithms to predict the linguistic understandability of health texts for Australian college students (aged 25-30 years) from non-English speaking backgrounds. We compared extreme gradient boosting, random forest, neural networks, and C5.0 decision tree for automated health information understandability evaluation. The 5 machine learning models achieved statistically better results compared to the baseline logistic regression model. We also evaluated the impact of each linguistic feature on the performance of each of the 5 models.
We found that information evidentness, relevance to educational purposes, and logical sequence were consistently more important than numeracy skills and medical knowledge when assessing the linguistic understandability of health education resources for international tertiary students with adequate English skills (International English Language Testing System mean score 6.5) and high health literacy (mean 16.5 in the Short Assessment of Health Literacy-English test). Our results challenge the traditional views that lack of medical knowledge and numerical skills constituted the barriers to the understanding of health educational materials.
Machine learning algorithms were developed to predict health information understandability for international college students aged 25-30 years. Thirteen natural language features and 5 evaluation dimensions were identified and compared in terms of their impact on the performance of the models. Health information understandability varies according to the demographic profiles of the target readers, and for international tertiary students, improving health information evidentness, relevance, and logic is critical.
提高健康信息的易懂性可显著提高针对弱势群体的健康教育项目的成本效益和效率。迫切需要开发基于临床知识的计算机化工具，以便快速、可靠地评估专业健康和医学教育资源的语言易懂性。本文填补了当前以患者为导向的健康资源开发中的一个关键空白，该领域需要可靠且准确的评估工具，以提高健康教育资源评估的效率和成本效益。
我们旨在将国际认可的临床指南转化为机器学习算法，以促进对澳大利亚大学国际学生的健康资源易懂性进行评估。
基于国际患者健康资源评估指南，我们开发了机器学习算法，以预测来自非英语背景的澳大利亚大学生（年龄在25至30岁之间）对健康文本的语言易懂性。我们比较了极端梯度提升、随机森林、神经网络和C5.0决策树在自动评估健康信息易懂性方面的表现。与基线逻辑回归模型相比，这5种机器学习模型在统计上取得了更好的结果。我们还评估了每个语言特征对这5种模型中每种模型性能的影响。
我们发现，在评估具有足够英语技能（国际英语语言测试系统平均分数为6.5）和高健康素养（健康素养英语简短评估测试平均分为16.5）的国际大学生的健康教育资源的语言易懂性时，信息清晰度、与教育目的的相关性以及逻辑顺序始终比算术技能和医学知识更为重要。我们的结果挑战了传统观点，即缺乏医学知识和数字技能构成了理解健康教育材料的障碍。
开发了机器学习算法来预测25至30岁国际大学生对健康信息的易懂性。确定了13个自然语言特征和5个评估维度，并比较了它们对模型性能的影响。健康信息的易懂性因目标读者的人口统计学特征而异，对于国际大学生而言，提高健康信息的清晰度、相关性和逻辑性至关重要。
