Dexter Franklin, Ledolter Johannes, Hindman Bradley J
From the Division of Management Consulting, Department of Anesthesia, Department of Management Sciences, and Department of Anesthesia, University of Iowa, Iowa City, Iowa.
Anesth Analg. 2014 Sep;119(3):679-685. doi: 10.1213/ANE.0000000000000342.
We describe our experiences in using Bernoulli cumulative sum (CUSUM) control charts for monitoring clinician performance. The supervision provided by each anesthesiologist is evaluated daily by the Certified Registered Nurse Anesthetists (CRNAs) and/or anesthesia residents with whom they work. Each of 9 items is evaluated (1 = never, 2 = rarely, 3 = frequently, 4 = always). The score is the mean of the 9 responses. Choosing thresholds for low scores is straightforward, <2.0 for CRNAs and <3.0 for residents. Bernoulli CUSUM detection of low scores was within 50 ± 14 (median ± quartile deviation) days rather than 182 days without use of CUSUM. The true positive detection of anesthesiologists with incidences of low scores greater than the chosen "out-of-control" rate was 14 of 14. The false-positive detection rate was 0 of 29. This CUSUM performance exceeded that of Shewhart individual control charts, for which the smallest threshold sufficiently large to detect 14 of 14 true positives had false-positive detection of 16 of 29 anesthesiologists. The Bernoulli CUSUM assumes that scores are known right away, which is untrue. However, CUSUM performance was insensitive to this assumption. The Bernoulli CUSUM assumes statistical independence of scores, which also is untrue. For example, when an evaluation of an anesthesiologist 1 day by a CRNA had a low score, there was an increased chance that another CRNA working in a different operating room on the same day would also give that same anesthesiologist a low score (P < 0.0001). This correlation among scores does affect the Bernoulli CUSUM, such that detection is more likely. This is an advantage for our continual process improvement application since it flags individuals for further evaluation by managers while maintaining confidentiality of raters.
我们描述了使用伯努利累积和(CUSUM)控制图监测临床医生表现的经验。每位麻醉医生的监督工作由与其共事的注册护士麻醉师(CRNA)和/或麻醉住院医师每日进行评估。对9项内容分别进行评估(1 = 从不,2 = 很少,3 = 频繁,4 = 总是)。分数为9项回答的平均值。为低分选择阈值很简单,CRNA为<2.0,住院医师为<3.0。使用伯努利CUSUM检测低分的时间在50±14(中位数±四分位数偏差)天内,而不使用CUSUM则为182天。对低分发生率高于选定“失控”率的麻醉医生的真阳性检测率为14/14。假阳性检测率为0/29。这种CUSUM的表现超过了休哈特个体控制图,对于休哈特个体控制图,要检测出14/14的真阳性,最小阈值足够大时,29名麻醉医生中有16名出现假阳性检测。伯努利CUSUM假定分数能立即得知,但这并不符合实际情况。然而,CUSUM的表现对这一假设不敏感。伯努利CUSUM假定分数具有统计独立性,这同样不符合实际情况。例如,当一名CRNA在某一天对一名麻醉医生的评估得分为低分,那么同一天在不同手术室工作的另一名CRNA也给该麻醉医生打低分的可能性就会增加(P<0.0001)。分数之间的这种相关性确实会影响伯努利CUSUM,使得检测更有可能。这对我们的持续过程改进应用来说是一个优势,因为它能标记个体以便管理人员进一步评估,同时保持评分者的保密性。