Tajgardoon Mohammadamin, Cooper Gregory F, King Andrew J, Clermont Gilles, Hochheiser Harry, Hauskrecht Milos, Sittig Dean F, Visweswaran Shyam
Intelligent Systems Program, School of Computing and Information, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
JAMIA Open. 2020 Dec 31;3(4):602-610. doi: 10.1093/jamiaopen/ooaa058. eCollection 2020 Dec.
Patient information can be retrieved more efficiently in electronic medical record (EMR) systems by using machine learning models that predict which information a physician will seek in a clinical context. However, information-seeking behavior varies across EMR users. To explicitly account for this variability, we derived hierarchical models and compared their performance to nonhierarchical models in identifying relevant patient information in intensive care unit (ICU) cases.
Critical care physicians reviewed ICU patient cases and selected data items relevant for presenting at morning rounds. Using patient EMR data as predictors, we derived hierarchical logistic regression (HLR) and standard logistic regression (LR) models to predict their relevance.
In 73 pairs of HLR and LR models, the HLR models achieved an area under the receiver operating characteristic curve of 0.81, 95% confidence interval (CI) [0.80-0.82], which was statistically significantly higher than that of LR models (0.75, 95% CI [0.74-0.76]). Further, the HLR models achieved statistically significantly lower expected calibration error (0.07, 95% CI [0.06-0.08]) than LR models (0.16, 95% CI [0.14-0.17]).
The physician reviewers demonstrated variability in selecting relevant data. Our results show that HLR models perform significantly better than LR models with respect to both discrimination and calibration. This is likely due to explicitly modeling physician-related variability.
Hierarchical models can yield better performance when there is physician-related variability as in the case of identifying relevant information in the EMR.
通过使用机器学习模型预测医生在临床环境中会查找哪些信息，可在电子病历（EMR）系统中更高效地检索患者信息。然而，不同EMR用户的信息查找行为存在差异。为了明确考虑这种变异性，我们推导了分层模型，并将其在识别重症监护病房（ICU）病例中相关患者信息方面的性能与非分层模型进行了比较。
重症监护医生查看了ICU患者病例，并选择了与晨间查房相关的数据项。以患者EMR数据作为预测因子，我们推导了分层逻辑回归（HLR）和标准逻辑回归（LR）模型来预测其相关性。
在73对HLR和LR模型中，HLR模型的受试者工作特征曲线下面积为0.81，95%置信区间（CI）[0.80 - 0.82]，在统计学上显著高于LR模型（0.75，95% CI [0.74 - 0.76]）。此外，HLR模型的预期校准误差（0.07，95% CI [0.06 - 0.08]）在统计学上显著低于LR模型（0.16，95% CI [0.14 - 0.17]）。
医生评审人员在选择相关数据时表现出变异性。我们的结果表明，HLR模型在区分度和校准方面均显著优于LR模型。这可能是由于对与医生相关的变异性进行了明确建模。
当存在与医生相关的变异性时，如在EMR中识别相关信息的情况，分层模型可以产生更好的性能。
