Graduate School of Cancer Science and Policy, National Cancer Center, Goyang-si, Gyeonggi-do, Republic of Korea.
Cancer Data Center, National Cancer Control Institute, National Cancer Center, Goyang-si, Gyeonggi-do, Republic of Korea.
J Med Internet Res. 2020 Dec 16;22(12):e18418. doi: 10.2196/18418.
Despite excellent prediction performance, noninterpretability has undermined the value of applying deep-learning algorithms in clinical practice. To overcome this limitation, attention mechanism has been introduced to clinical research as an explanatory modeling method. However, potential limitations of using this attractive method have not been clarified to clinical researchers. Furthermore, there has been a lack of introductory information explaining attention mechanisms to clinical researchers.
The aim of this study was to introduce the basic concepts and design approaches of attention mechanisms. In addition, we aimed to empirically assess the potential limitations of current attention mechanisms in terms of prediction and interpretability performance.
First, the basic concepts and several key considerations regarding attention mechanisms were identified. Second, four approaches to attention mechanisms were suggested according to a two-dimensional framework based on the degrees of freedom and uncertainty awareness. Third, the prediction performance, probability reliability, concentration of variable importance, consistency of attention results, and generalizability of attention results to conventional statistics were assessed in the diabetic classification modeling setting. Fourth, the potential limitations of attention mechanisms were considered.
Prediction performance was very high for all models. Probability reliability was high in models with uncertainty awareness. Variable importance was concentrated in several variables when uncertainty awareness was not considered. The consistency of attention results was high when uncertainty awareness was considered. The generalizability of attention results to conventional statistics was poor regardless of the modeling approach.
The attention mechanism is an attractive technique with potential to be very promising in the future. However, it may not yet be desirable to rely on this method to assess variable importance in clinical settings. Therefore, along with theoretical studies enhancing attention mechanisms, more empirical studies investigating potential limitations should be encouraged.
尽管深度学习算法在预测性能方面表现出色，但由于其不可解释性，限制了其在临床实践中的应用价值。为了克服这一局限性，注意力机制作为一种解释性建模方法被引入临床研究。然而，临床研究人员尚未明确认识到使用这种有吸引力的方法的潜在局限性。此外，缺乏向临床研究人员介绍注意力机制的入门信息。
本研究旨在介绍注意力机制的基本概念和设计方法。此外，我们旨在根据基于自由度和不确定性意识的二维框架，从预测和可解释性性能方面实证评估当前注意力机制的潜在局限性。
首先，确定了注意力机制的基本概念和几个关键注意事项。其次，根据基于自由度和不确定性意识的二维框架，提出了四种注意力机制方法。第三，在糖尿病分类建模设置中评估了预测性能、概率可靠性、变量重要性的集中程度、注意力结果的一致性以及注意力结果对常规统计的可推广性。第四，考虑了注意力机制的潜在局限性。
所有模型的预测性能都非常高。具有不确定性意识的模型的概率可靠性较高。当不考虑不确定性意识时，变量重要性集中在几个变量上。当考虑不确定性意识时，注意力结果的一致性较高。无论采用何种建模方法，注意力结果对常规统计的可推广性都较差。
注意力机制是一种很有前途的技术，具有很大的潜力。然而，在临床环境中，依靠这种方法来评估变量的重要性可能还不太理想。因此，除了进行理论研究以增强注意力机制外，还应鼓励开展更多的实证研究来调查其潜在局限性。
