Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Helmholtzweg 4, 07743, Jena, Germany; Leibniz Institute of Photonic Technology, Member of Leibniz Health Technologies, Member of the Leibniz. Centre for Photonics in Infection Research (LPI), Albert Einstein Straße 9, 07745, Jena, Germany.
Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Helmholtzweg 4, 07743, Jena, Germany; Leibniz Institute of Photonic Technology, Member of Leibniz Health Technologies, Member of the Leibniz. Centre for Photonics in Infection Research (LPI), Albert Einstein Straße 9, 07745, Jena, Germany.
Anal Chim Acta. 2024 Dec 15;1332:343346. doi: 10.1016/j.aca.2024.343346. Epub 2024 Oct 16.
BACKGROUND: Machine learning algorithms for bacterial strain identification using Raman spectroscopy have been widely used in microbiology. During the training phase, existing datasets are augmented and used to optimize model architecture and hyperparameters. After training, it is presumed that the models have reached their peak performance and are used for inference without being further enhanced. Our methodology combines Monte Carlo Dropout (MCD) with convolutional neural networks (CNNs) by utilizing dropout during the inference phase, which enables to measure the model uncertainty, a critical but often ignored aspect in deep learning models. RESULTS: We categorize unseen input data into two subsets based on the uncertainty of their prediction by employing MCD and defining the threshold using the Gaussian Mixture Model (GMM). The final prediction is obtained on the subset of testing data that exhibits lower model uncertainty, thereby enhancing the reliability of the results. To validate our method, we applied it to two Raman spectra datasets. As a result, we have observed an increase in accuracy of 9 % for Dataset 1 (from 83.10 % to 92.10 %) and 12.82 % for Dataset 2 (from 83.86 % to 96.68 %). These improvements were observed within specific subsets of the data: 826 out of 1206 spectra in Dataset 1 and 1700 out of 3000 spectra in Dataset 2. This demonstrates the effectiveness of our approach in improving prediction accuracy by focusing on data with lower uncertainty. SIGNIFICANCE: Different from routine prediction based on mere probabilities, we believe this uncertainty-guided prediction is more effective to ensure a high prediction rate rather than the prediction on the entire dataset. By guiding the decision-making of a model on higher-confidence subsets, our methodology can enhance the accuracy of classification in critical areas like disease diagnosis and safety monitoring. This targeted approach is to advance microbial identification and produces more trustworthy predictions.
背景:使用拉曼光谱对细菌株进行识别的机器学习算法已在微生物学中得到广泛应用。在训练阶段,使用现有数据集进行扩充,并优化模型结构和超参数。在训练之后,假定模型已经达到最佳性能,并且无需进一步增强即可用于推断。我们的方法通过在推断阶段使用随机失活(Monte Carlo Dropout,MCD)结合卷积神经网络(convolutional neural networks,CNNs),利用模型不确定性来衡量模型的可信度,这是深度学习模型中一个关键但经常被忽视的方面。
结果:我们通过使用 MCD 对未见输入数据进行分类,根据其预测的不确定性将其分为两个子集,并使用高斯混合模型(Gaussian Mixture Model,GMM)定义阈值。然后在具有较低模型不确定性的测试数据子集中进行最终预测,从而提高结果的可靠性。为了验证我们的方法,我们将其应用于两个拉曼光谱数据集。结果表明,我们的方法使第一个数据集的准确率提高了 9%(从 83.10%提高到 92.10%),第二个数据集的准确率提高了 12.82%(从 83.86%提高到 96.68%)。这些改进是在数据的特定子集中观察到的:在第一个数据集的 1206 个光谱中有 826 个,在第二个数据集的 3000 个光谱中有 1700 个。这表明我们的方法通过关注具有较低不确定性的数据来提高预测准确性是有效的。
意义:与基于单纯概率的常规预测不同,我们认为这种基于不确定性的预测方法更有效,能够确保高预测率,而不是对整个数据集进行预测。通过指导模型在更可信的子集中做出决策,我们的方法可以提高疾病诊断和安全监测等关键领域的分类准确性。这种有针对性的方法可以促进微生物鉴定,并产生更可信的预测。
Anal Chim Acta. 2024-12-15
Spectrochim Acta A Mol Biomol Spectrosc. 2025-1-15
Sensors (Basel). 2021-10-30
Comput Methods Programs Biomed. 2024-10
Comput Methods Programs Biomed. 2024-8
NPJ Syst Biol Appl. 2025-7-7