School of Languages and Cultures, University of Sydney, Sydney 2006, Australia.
Department of Computer Science, City University of Hong Kong, Hong Kong 518057, China.
Int J Environ Res Public Health. 2021 Sep 19;18(18):9873. doi: 10.3390/ijerph18189873.
Machine translation (MT) technologies have increasing applications in healthcare. Despite their convenience, cost-effectiveness, and constantly improved accuracy, research shows that the use of MT tools in medical or healthcare settings poses risks to vulnerable populations.
We aimed to develop machine learning classifiers (MNB and RVM) to forecast nuanced yet significant MT errors of clinical symptoms in Chinese neural MT outputs.
We screened human translations of MSD Manuals for information on self-diagnosis of infectious diseases and produced their matching neural MT outputs for subsequent pairwise quality assessment by trained bilingual health researchers. Different feature optimisation and normalisation techniques were used to identify the best feature set.
The RVM classifier using optimised, normalised (L normalisation) semantic features achieved the highest sensitivity, specificity, AUC, and accuracy. MNB achieved similar high performance using the same optimised semantic feature set. The best probability threshold of the best performing RVM classifier was found at 0.6, with a very high positive likelihood ratio (LR+) of 27.82 (95% CI: 3.99, 193.76), and a low negative likelihood ratio (LR-) of 0.19 (95% CI: 0.08, 046), suggesting the high diagnostic utility of our model to predict the probabilities of erroneous MT of disease symptoms to help reverse potential inaccurate self-diagnosis of diseases among vulnerable people without adequate medical knowledge or an ability to ascertain the reliability of MT outputs.
Our study demonstrated the viability, flexibility, and efficiency of introducing machine learning models to help promote risk-aware use of MT technologies to achieve optimal, safer digital health outcomes for vulnerable people.
机器翻译（MT）技术在医疗保健领域的应用越来越广泛。尽管它们具有方便、经济高效和准确性不断提高的特点，但研究表明，在医疗或医疗保健环境中使用 MT 工具会给弱势群体带来风险。
我们旨在开发机器学习分类器（MNB 和 RVM），以预测中文神经 MT 输出中临床症状的细微但重要的 MT 错误。
我们筛选了 MSD 手册中的人类翻译信息，以了解传染病的自我诊断，并为随后的双语健康研究人员进行配对质量评估生成了他们的匹配神经 MT 输出。使用不同的特征优化和归一化技术来确定最佳特征集。
使用优化、归一化（L 归一化）语义特征的 RVM 分类器实现了最高的敏感性、特异性、AUC 和准确性。MNB 使用相同的优化语义特征集也实现了类似的高性能。最佳 RVM 分类器的最佳概率阈值发现为 0.6，具有非常高的阳性似然比（LR+）为 27.82（95%CI：3.99，193.76），而阴性似然比（LR-）为 0.19（95%CI：0.08，046），表明我们的模型预测疾病症状的 MT 错误概率具有很高的诊断效用，可以帮助没有足够医学知识或无法确定 MT 输出可靠性的弱势群体避免潜在不准确的自我诊断疾病。
我们的研究证明了引入机器学习模型的可行性、灵活性和效率，以帮助促进对 MT 技术的风险意识使用，为弱势群体实现最佳、更安全的数字健康结果。
