Carli Delphine, Fahrni Guillaume, Bonnabry Pascal, Lovis Christian
Division of Pharmacy, University Hospitals of Geneva, Geneva, Switzerland.
School of Pharmaceutical Sciences, University of Geneva, University of Lausanne, Geneva, Switzerland.
JMIR Med Inform. 2018 Jan 24;6(1):e3. doi: 10.2196/medinform.7170.
Computerized decision support systems have raised a lot of hopes and expectations in the field of order entry. Although there are numerous studies reporting positive impacts, concerns are increasingly high about alert fatigue and effective impacts of these systems. One of the root causes of fatigue alert reported is the low clinical relevance of these alerts.
The objective of this systematic review was to assess the reported positive predictive value (PPV), as a proxy to clinical relevance, of decision support systems in computerized provider order entry (CPOE).
A systematic search of the scientific literature published between February 2009 and March 2015 on CPOE, clinical decision support systems, and the predictive value associated with alert fatigue was conducted using PubMed database. Inclusion criteria were as follows: English language, full text available (free or pay for access), assessed medication, direct or indirect level of predictive value, sensitivity, or specificity. When possible with the information provided, PPV was calculated or evaluated.
Additive queries on PubMed retrieved 928 candidate papers. Of these, 376 were eligible based on abstract. Finally, 26 studies qualified for a full-text review, and 17 provided enough information for the study objectives. An additional 4 papers were added from the references of the reviewed papers. The results demonstrate massive variations in PPVs ranging from 8% to 83% according to the object of the decision support, with most results between 20% and 40%. The best results were observed when patients' characteristics, such as comorbidity or laboratory test results, were taken into account. There was also an important variation in sensitivity, ranging from 38% to 91%.
There is increasing reporting of alerts override in CPOE decision support. Several causes are discussed in the literature, the most important one being the clinical relevance of alerts. In this paper, we tried to assess formally the clinical relevance of alerts, using a near-strong proxy, which is the PPV of alerts, or any way to express it such as the rate of true and false positive alerts. In doing this literature review, three inferences were drawn. First, very few papers report direct or enough indirect elements that support the use or the computation of PPV, which is a gold standard for all diagnostic tools in medicine and should be systematically reported for decision support. Second, the PPV varies a lot according to the typology of decision support, so that overall rates are not useful, but must be reported by the type of alert. Finally, in general, the PPVs are below or near 50%, which can be considered as very low.
计算机化决策支持系统在医嘱录入领域引发了诸多希望和期待。尽管有大量研究报告了其积极影响，但人们对这些系统的警报疲劳及实际效果的担忧日益增加。所报告的警报疲劳的根本原因之一是这些警报的临床相关性较低。
本系统评价的目的是评估决策支持系统在计算机化医生医嘱录入（CPOE）中所报告的阳性预测值（PPV），以此作为临床相关性的替代指标。
使用PubMed数据库对2009年2月至2015年3月间发表的关于CPOE、临床决策支持系统以及与警报疲劳相关的预测值的科学文献进行系统检索。纳入标准如下：英文文献、有全文（免费或付费获取）、评估了药物、直接或间接的预测值水平、敏感性或特异性。如有可能，根据所提供的信息计算或评估PPV。
在PubMed上的附加检索共找到928篇候选论文。其中，基于摘要有376篇符合要求。最后，26项研究符合全文审查标准，17项研究提供了足以满足研究目的的信息。从已审查论文的参考文献中又补充了4篇论文。结果表明，根据决策支持的对象不同，PPV差异巨大，从8%到83%不等，大多数结果在20%至40%之间。当考虑患者的特征，如合并症或实验室检查结果时，能观察到最佳结果。敏感性也存在重要差异，范围从38%到91%。
在CPOE决策支持中，越来越多地报告了对警报的忽略。文献中讨论了几个原因，其中最重要的是警报的临床相关性。在本文中，我们试图使用一个近乎强有力的替代指标——警报的PPV，或任何表达它的方式，如实假阳性警报率，来正式评估警报的临床相关性。在进行这项文献综述时，得出了三个推论。第一，很少有论文报告直接或足够的间接因素来支持PPV的使用或计算，而PPV是医学中所有诊断工具的金标准，对于决策支持应系统地报告。第二，PPV根据决策支持的类型差异很大，因此总体率并无用处，而必须按警报类型报告。最后，一般来说，PPV低于或接近50%，可认为非常低。
