Alemneh Esubalew, Kebebaw Tegegn, Nigatu Dabere, Azage Muluken, Misgan Eyaya, Abate Enyew
ICT4D Research Center, Bahir Dar Institute of Technology, Bahir Dar University, Bahir Dar, Ethiopia.
School of Public Health, College of Medicine and Health Sciences, Bahir Dar University, Bahir Dar, Ethiopia.
PLOS Digit Health. 2024 May 13;3(5):e0000494. doi: 10.1371/journal.pdig.0000494. eCollection 2024 May.
An innovative electronic Maternal and Child Health (eMCH) application was developed to support operational and clinical decision-making in maternal and child health services. End-user-based evaluation of eHealth application is a critical step to ascertain how successfully users can learn and use it, and improve the technology. Therefore, this study aimed to evaluate the eMCH tool usability, feasibility, and acceptability among healthcare providers (HCPs) in the Amhara region, Ethiopia. A cross-sectional study was conducted among HCPs working in six public healthcare facilities. The usability evaluation was done on 24 HCPs across three professional categories using the ISO 9241-11 usability guideline. One hundred nine HCPs were participated in the feasibility and acceptability study. Data were collected using a standard usability tool, think-aloud protocol, a self-administered approach, and Open Broadcaster Software Studio version 26.1.1 video recorder. Descriptive statistics were used to describe the data. A Kruskal-Wallis test was used to measure the association between mean scores and categories of HCPs. The recorded videos were used for the log file analysis method. None of the HCP categories were able to complete all the tasks without errors. The average number of errors and restarts were 7.5 and 2.8, respectively. The average number of restarts was directly proportional to the average number of errors. The participants successfully completed more than 70% of the tasks without requiring any assistance or guidance. Forty-seven comments or errors were identified from the think-aloud analysis and 22 comments from the usability metrics analysis. Overall, statistically significant performance differences were observed among the three HCP groups across the majority of the usability evaluation metrics. Fifty-seven percent of HCPs scored higher than the mean on the feasibility study. Slightly higher than half, 56 (51.4%), of the HCPs scored higher than the mean score on the acceptability study. The usability evaluation identified vital comments and usability flaws that were essential for the eMCH tool to be upgraded. The tool was feasible and acceptable as reported by end-users. Therefore, the errors and usability flaws of the tool should be fixed before deployment to other healthcare settings.
开发了一种创新的电子妇幼保健(eMCH)应用程序,以支持妇幼保健服务中的业务和临床决策。基于最终用户对电子健康应用程序进行评估,是确定用户学习和使用该程序的成功程度以及改进该技术的关键步骤。因此,本研究旨在评估埃塞俄比亚阿姆哈拉地区医疗保健提供者(HCPs)对eMCH工具的可用性、可行性和可接受性。对在六个公共医疗保健机构工作的HCPs进行了一项横断面研究。使用ISO 9241-11可用性指南,对三个专业类别的24名HCPs进行了可用性评估。109名HCPs参与了可行性和可接受性研究。使用标准可用性工具、出声思维协议、自填式方法和Open Broadcaster Software Studio 26.1.1版录像机收集数据。使用描述性统计来描述数据。使用Kruskal-Wallis检验来衡量HCPs平均得分与类别之间的关联。录制的视频用于日志文件分析方法。没有一个HCP类别能够无错误地完成所有任务。错误和重新启动的平均次数分别为7.5次和2.8次。重新启动的平均次数与错误的平均次数成正比。参与者在无需任何帮助或指导的情况下成功完成了超过70%的任务。从出声思维分析中识别出47条评论或错误,从可用性指标分析中识别出22条评论。总体而言,在大多数可用性评估指标上,三个HCP组之间观察到了具有统计学意义的性能差异。在可行性研究中,57%的HCPs得分高于平均分。在可接受性研究中,略高于一半(56名,51.4%)的HCPs得分高于平均分。可用性评估确定了对eMCH工具进行升级至关重要的评论和可用性缺陷。正如最终用户所报告的,该工具是可行且可接受的。因此,在将该工具部署到其他医疗保健环境之前,应修复其错误和可用性缺陷。