Head of the Sector of the Development of Systems for the Implementation of Intelligent Medical Technologies; Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health care Department, 24/1 Petrovka St., Moscow, 127051, Russia.
Head of the Department of Medical Informatics, Radiomics and Radiogenomics; Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health care Department, 24/1 Petrovka St., Moscow, 127051, Russia.
Sovrem Tekhnologii Med. 2023;15(2):19-25. doi: 10.17691/stm2023.15.2.02. Epub 2023 Mar 29.
is to evaluate the efficacy of approaches to sampling during periodic quality control of the artificial intelligence (AI) results in biomedical practice.
The approaches to sampling based on point statistical estimation, statistical hypothesis testing, employing ready-made statistical tables, as well as options of the approaches presented in GOST R ISO 2859-1-2007 "Statistical methods. Sampling procedures for inspection by attributes" have been analyzed. We have considered variants of sampling of different sizes for general populations from 1000 to 100,000 studies.The analysis of the approaches to sampling was carried out as part of an experiment on the use of innovative technologies in computer vision for the analysis of medical images and their further application in the healthcare system of Moscow (Russia).
Ready-made tables have specific statistical input data, which does not make them a universal option for biomedical research. Point statistical estimation helps to calculate a sample based on given statistical parameters with a certain confidence interval. This approach is promising in the case when only a type I error is important for the researcher, and a type II error is not a priority. Using the approach based on statistical hypothesis testing makes it possible to take account of type I and II errors based on the given statistical parameters. The application of GOST R ISO 2859-1-2007 for sampling allows using ready-made values depending on the given statistical parameters.When evaluating the efficacy of the studied approaches, it was found that for our purposes, the optimal number of studies during AI quality control for the analysis of medical images is 80 items. This meets the requirements of representativeness, balance of the risks to the consumer and the AI service provider, as well as optimization of labor costs of employees involved in the process of quality control of the AI results.
评估在生物医学实践中人工智能（AI）结果定期质量控制期间采样方法的效果。
分析了基于点统计估计、统计假设检验、使用现成统计表格的采样方法，以及 GOST R ISO 2859-1-2007“统计方法。属性检验抽样程序”中提出的方法的选项。我们考虑了从 1000 到 100000 项研究的一般人群中不同大小的采样变体。采样方法的分析是在使用计算机视觉创新技术分析医学图像及其在莫斯科（俄罗斯）医疗保健系统中的进一步应用的实验的一部分中进行的。
现成的表格具有特定的统计输入数据，这使得它们不是生物医学研究的通用选项。点统计估计有助于根据给定的统计参数和置信区间计算样本。当仅对研究人员来说，I 类错误很重要，而 II 类错误不是优先事项时，这种方法很有前途。使用基于统计假设检验的方法可以根据给定的统计参数考虑 I 类和 II 类错误。根据 GOST R ISO 2859-1-2007 进行采样可以根据给定的统计参数使用现成的值。在评估所研究方法的效果时，发现对于我们的目的，在医学图像分析的 AI 质量控制期间，最佳 AI 数量为 80 项。这符合代表性、消费者和 AI 服务提供商的风险平衡以及参与 AI 结果质量控制过程的员工的劳动力成本优化的要求。
