Xuto Piyanut, Prasitwattanaseree Piyaporn, Chaiboonruang Tareewan, Chaiwuth Sujitra, Khwanngern Podjanee, Nuntakwang Chadchadaporn, Nimarangkul Karnjana, Suwansin Wara, Khiaokham Lawitra, Bressington Daniel
Faculty of Nursing, Chiang Mai University, Chiang Mai 50200, Thailand.
Srisavarindhira Thai Red Cross Institute, Bangkok 10330, Thailand.
Nurs Rep. 2025 Feb 26;15(3):80. doi: 10.3390/nursrep15030080.
Artificial intelligence (AI) can potentially enhance cognitive assessment practices in maternal and child health nursing education. : To evaluate the reliability, accuracy and precision, and external validity of an AI-assisted answer assessment (4A) program for cognitive assessments in nursing education. : This study is a validation study. Initially, 170 nursing students from northern Thailand participated, with 52 randomly selected for detailed testing. Agreement testing between the 4A program and human experts was conducted using the intraclass correlation coefficient (ICC). Accuracy and precision testing compared 4A scores with human expert assessments via the McNemar test. External validation involved 138 participants to compare the 4A program's assessments against national examination outcomes using logistic regression. : Results indicated a high level of consistency between the 4A program and human experts (ICC = 0.886). With an accuracy of 0.808 and a precision of 0.913, compared to the human expert's accuracy of 0.923 and precision of 1.000. The McNemar test (χ = 0.4, = 0.527) showed no significant difference in evaluation performance between AI and human experts. Higher scores on the 4A program significantly predicted success in the national nursing examination (OR: 1.124, = 0.031). : The 4A program demonstrates potential in reliably assessing nursing students' cognitive abilities and predicting exam success. This study advocates for the continued integration of AI in educational assessments and the importance of refining AI systems to better align with traditional assessment methods.
人工智能（AI）有可能改善母婴健康护理教育中的认知评估实践。：评估用于护理教育认知评估的人工智能辅助答案评估（4A）程序的可靠性、准确性和精确性以及外部有效性。：本研究是一项验证性研究。最初，来自泰国北部的170名护理专业学生参与其中，随机选择52名进行详细测试。使用组内相关系数（ICC）对4A程序与人类专家之间进行一致性测试。准确性和精确性测试通过McNemar检验将4A分数与人类专家评估进行比较。外部验证涉及138名参与者，使用逻辑回归将4A程序的评估与国家考试结果进行比较。：结果表明4A程序与人类专家之间具有高度一致性（ICC = 0.886）。其准确率为0.808，精确率为0.913，而人类专家的准确率为0.923，精确率为1.000。McNemar检验（χ = 0.4， = 0.527）表明人工智能与人类专家在评估性能上没有显著差异。4A程序得分较高显著预测了在国家护理考试中取得成功（OR：1.124， = 0.031）。：4A程序在可靠评估护理专业学生的认知能力和预测考试成功方面显示出潜力。本研究主张在教育评估中持续整合人工智能以及完善人工智能系统以更好地与传统评估方法保持一致的重要性。
