Department of Computing, FSKIK, Universiti Pendidikan Sultan Idris, Tanjung Malim 35900, Malaysia.
Department of Computing, FSKIK, Universiti Pendidikan Sultan Idris, Tanjung Malim 35900, Malaysia.
J Infect Public Health. 2020 Oct;13(10):1381-1396. doi: 10.1016/j.jiph.2020.06.028. Epub 2020 Jul 1.
This study presents a systematic review of artificial intelligence (AI) techniques used in the detection and classification of coronavirus disease 2019 (COVID-19) medical images in terms of evaluation and benchmarking. Five reliable databases, namely, IEEE Xplore, Web of Science, PubMed, ScienceDirect and Scopus were used to obtain relevant studies of the given topic. Several filtering and scanning stages were performed according to the inclusion/exclusion criteria to screen the 36 studies obtained; however, only 11 studies met the criteria. Taxonomy was performed, and the 11 studies were classified on the basis of two categories, namely, review and research studies. Then, a deep analysis and critical review were performed to highlight the challenges and critical gaps outlined in the academic literature of the given subject. Results showed that no relevant study evaluated and benchmarked AI techniques utilised in classification tasks (i.e. binary, multi-class, multi-labelled and hierarchical classifications) of COVID-19 medical images. In case evaluation and benchmarking will be conducted, three future challenges will be encountered, namely, multiple evaluation criteria within each classification task, trade-off amongst criteria and importance of these criteria. According to the discussed future challenges, the process of evaluation and benchmarking AI techniques used in the classification of COVID-19 medical images considered multi-complex attribute problems. Thus, adopting multi-criteria decision analysis (MCDA) is an essential and effective approach to tackle the problem complexity. Moreover, this study proposes a detailed methodology for the evaluation and benchmarking of AI techniques used in all classification tasks of COVID-19 medical images as future directions; such methodology is presented on the basis of three sequential phases. Firstly, the identification procedure for the construction of four decision matrices, namely, binary, multi-class, multi-labelled and hierarchical, is presented on the basis of the intersection of evaluation criteria of each classification task and AI classification techniques. Secondly, the development of the MCDA approach for benchmarking AI classification techniques is provided on the basis of the integrated analytic hierarchy process and VlseKriterijumska Optimizacija I Kompromisno Resenje methods. Lastly, objective and subjective validation procedures are described to validate the proposed benchmarking solutions.
本研究从评估和基准测试的角度，对人工智能（AI）技术在 2019 年冠状病毒病（COVID-19）医学图像检测和分类中的应用进行了系统综述。使用了五个可靠的数据库，即 IEEE Xplore、Web of Science、PubMed、ScienceDirect 和 Scopus，以获取给定主题的相关研究。根据纳入/排除标准，进行了几次筛选和扫描阶段，以筛选获得的 36 项研究；然而，只有 11 项研究符合标准。对这些研究进行了分类，并根据综述和研究两个类别进行了分类。然后，对这些研究进行了深入分析和批判性评价，以突出给定主题学术文献中概述的挑战和关键差距。结果表明，没有相关研究评估和基准测试用于 COVID-19 医学图像分类任务（即二进制、多类、多标签和层次分类）的 AI 技术。如果进行评估和基准测试，将面临三个未来挑战，即每个分类任务中的多个评估标准、标准之间的权衡以及这些标准的重要性。根据讨论的未来挑战，评估和基准测试用于 COVID-19 医学图像分类的 AI 技术的过程被认为是多复杂属性问题。因此，采用多准则决策分析（MCDA）是解决问题复杂性的一种重要而有效的方法。此外，本研究提出了一种评估和基准测试用于 COVID-19 医学图像所有分类任务的 AI 技术的详细方法，作为未来的方向；该方法是基于三个连续阶段提出的。首先，基于每个分类任务的评价标准和 AI 分类技术的交集，提出了构建四个决策矩阵（即二进制、多类、多标签和层次）的识别过程。其次，提出了基于集成层次分析法和 VIseKriterijumska Optimizacija I Kompromisno Resenje 方法的 MCDA 方法来开发用于基准测试 AI 分类技术的方法。最后，描述了客观和主观验证程序，以验证所提出的基准解决方案。
