University Clinic for Radiology, Westfälische Wilhelms-University Muenster and University Hospital Münster, Albert-Schweitzer-Campus 1, 48149, Muenster, Germany.
Institute of Neuroradiology, Kantonsspital Aarau, Aarau, Switzerland.
Sci Rep. 2022 Aug 11;12(1):13648. doi: 10.1038/s41598-022-18028-8.
To investigate the applicability and performance of automated machine learning (AutoML) for potential applications in diagnostic neuroradiology. In the medical sector, there is a rapidly growing demand for machine learning methods, but only a limited number of corresponding experts. The comparatively simple handling of AutoML should enable even non-experts to develop adequate machine learning models with manageable effort. We aim to investigate the feasibility as well as the advantages and disadvantages of developing AutoML models compared to developing conventional machine learning models. We discuss the results in relation to a concrete example of a medical prediction application. In this retrospective IRB-approved study, a cohort of 107 patients who underwent gross total meningioma resection and a second cohort of 31 patients who underwent subtotal resection were included. Image segmentation of the contrast enhancing parts of the tumor was performed semi-automatically using the open-source software platform 3D Slicer. A total of 107 radiomic features were extracted by hand-delineated regions of interest from the pre-treatment MRI images of each patient. Within the AutoML approach, 20 different machine learning algorithms were trained and tested simultaneously. For comparison, a neural network and different conventional machine learning algorithms were trained and tested. With respect to the exemplary medical prediction application used in this study to evaluate the performance of Auto ML, namely the pre-treatment prediction of the achievable resection status of meningioma, AutoML achieved remarkable performance nearly equivalent to that of a feed-forward neural network with a single hidden layer. However, in the clinical case study considered here, logistic regression outperformed the AutoML algorithm. Using independent test data, we observed the following classification results (AutoML/neural network/logistic regression): mean area under the curve = 0.849/0.879/0.900, mean accuracy = 0.821/0.839/0.881, mean kappa = 0.465/0.491/0.644, mean sensitivity = 0.578/0.577/0.692 and mean specificity = 0.891/0.914/0.936. The results obtained with AutoML are therefore very promising. However, the AutoML models in our study did not yet show the corresponding performance of the best models obtained with conventional machine learning methods. While AutoML may facilitate and simplify the task of training and testing machine learning algorithms as applied in the field of neuroradiology and medical imaging, a considerable amount of expert knowledge may still be needed to develop models with the highest possible discriminatory power for diagnostic neuroradiology.
为了研究自动化机器学习（AutoML）在诊断神经放射学中的潜在应用的适用性和性能。在医疗领域，对机器学习方法的需求迅速增长，但只有数量有限的相应专家。AutoML 的相对简单的处理应该能够使非专家以可管理的工作量开发出足够的机器学习模型。我们旨在研究开发 AutoML 模型与开发传统机器学习模型相比的可行性以及优缺点。我们将讨论与医疗预测应用的具体示例有关的结果。在这项回顾性 IRB 批准的研究中，纳入了 107 例接受完全脑膜瘤切除术的患者队列和 31 例接受次全切除术的患者队列。使用开源软件平台 3D Slicer 半自动执行肿瘤增强部分的图像分割。通过对每个患者的预处理 MRI 图像的手动勾画 ROI，共提取了 107 个放射组学特征。在 AutoML 方法中，同时训练和测试了 20 种不同的机器学习算法。作为比较，训练和测试了神经网络和不同的传统机器学习算法。就用于评估 Auto ML 性能的本研究中使用的示例性医疗预测应用程序而言，即脑膜瘤可切除状态的预处理预测，AutoML 实现了几乎与具有单个隐藏层的前馈神经网络相当的出色性能。然而，在考虑到的临床案例研究中，逻辑回归优于 AutoML 算法。使用独立的测试数据，我们观察到以下分类结果（AutoML/神经网络/逻辑回归）：平均曲线下面积=0.849/0.879/0.900，平均准确性=0.821/0.839/0.881，平均kappa=0.465/0.491/0.644，平均敏感性=0.578/0.577/0.692 和平均特异性=0.891/0.914/0.936。因此，AutoML 的结果非常有希望。然而，我们研究中的 AutoML 模型尚未显示出传统机器学习方法获得的最佳模型的相应性能。虽然 AutoML 可以简化和简化神经放射学和医学成像领域应用的机器学习算法的训练和测试任务，但为了诊断神经放射学开发具有最高鉴别力的模型，可能仍然需要相当多的专家知识。
