Department of Computer Science and Technology, Xi'an Jiaotong University, No. 28 Xianning West Road, Xi'an, 710049, Shaanxi, China.
Department of Computer Science and Engineering, The Ohio State University, 2015 Neil Ave, Columbus, OH, 43210, USA.
BMC Med Inform Decis Mak. 2020 Dec 30;20(Suppl 11):307. doi: 10.1186/s12911-020-01331-7.
The availability of massive amount of data enables the possibility of clinical predictive tasks. Deep learning methods have achieved promising performance on the tasks. However, most existing methods suffer from three limitations: (1) There are lots of missing value for real value events, many methods impute the missing value and then train their models based on the imputed values, which may introduce imputation bias. The models' performance is highly dependent on the imputation accuracy. (2) Lots of existing studies just take Boolean value medical events (e.g. diagnosis code) as inputs, but ignore real value medical events (e.g., lab tests and vital signs), which are more important for acute disease (e.g., sepsis) and mortality prediction. (3) Existing interpretable models can illustrate which medical events are conducive to the output results, but are not able to give contributions of patterns among medical events.
In this study, we propose a novel interpretable Pattern Attention model with Value Embedding (PAVE) to predict the risks of certain diseases. PAVE takes the embedding of various medical events, their values and the corresponding occurring time as inputs, leverage self-attention mechanism to attend to meaningful patterns among medical events for risk prediction tasks. Because only the observed values are embedded into vectors, we don't need to impute the missing values and thus avoids the imputations bias. Moreover, the self-attention mechanism is helpful for the model interpretability, which means the proposed model can output which patterns cause high risks.
We conduct sepsis onset prediction and mortality prediction experiments on a publicly available dataset MIMIC-III and our proprietary EHR dataset. The experimental results show that PAVE outperforms existing models. Moreover, by analyzing the self-attention weights, our model outputs meaningful medical event patterns related to mortality.
PAVE learns effective medical event representation by incorporating the values and occurring time, which can improve the risk prediction performance. Moreover, the presented self-attention mechanism can not only capture patients' health state information, but also output the contributions of various medical event patterns, which pave the way for interpretable clinical risk predictions.
The code for this paper is available at: https://github.com/yinchangchang/PAVE .
大量数据的可用性使得临床预测任务成为可能。深度学习方法在这些任务上取得了有前景的性能。然而，大多数现有的方法都存在三个局限性：（1）对于真实值事件，存在大量的缺失值，许多方法会对缺失值进行插补，然后基于插补值训练模型，这可能会引入插补偏差。模型的性能高度依赖于插补的准确性。（2）许多现有的研究只将布尔值的医疗事件（例如诊断代码）作为输入，但忽略了真实值的医疗事件（例如实验室检查和生命体征），这些对急性疾病（例如败血症）和死亡率预测更为重要。（3）现有的可解释模型可以说明哪些医疗事件有助于输出结果，但无法说明医疗事件之间的模式的贡献。
在这项研究中，我们提出了一种新颖的可解释模式注意力与值嵌入（PAVE）模型，用于预测某些疾病的风险。PAVE 以各种医疗事件的嵌入、它们的值以及相应的发生时间作为输入，利用自注意力机制关注医疗事件之间有意义的模式，以进行风险预测任务。由于只有观察到的值被嵌入到向量中，我们不需要对缺失值进行插补，从而避免了插补偏差。此外，自注意力机制有助于模型的可解释性，这意味着所提出的模型可以输出导致高风险的模式。
我们在一个公开的 MIMIC-III 数据集和我们专有的 EHR 数据集上进行了败血症发作预测和死亡率预测实验。实验结果表明，PAVE 优于现有的模型。此外，通过分析自注意力权重，我们的模型输出了与死亡率相关的有意义的医疗事件模式。
PAVE 通过结合值和发生时间来学习有效的医疗事件表示，从而提高风险预测性能。此外，所提出的自注意力机制不仅可以捕捉患者的健康状态信息，还可以输出各种医疗事件模式的贡献，为可解释的临床风险预测铺平了道路。
本文的代码可在以下网址获取：https://github.com/yinchangchang/PAVE。
