Engeltjes Bernice, Rosman Ageeth, Bertens Loes C M, Wouters Eveline, Cronie Doug, Scheele Fedde
Athena Institute for Transdisciplinary Research, Faculty of Science, VU University, Amsterdam, the Netherlands.
Department of Healthcare Studies, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands.
Risk Manag Healthc Policy. 2021 Aug 7;14:3247-3254. doi: 10.2147/RMHP.S319564. eCollection 2021.
Safety and efficiency of emergency care can be optimized with a triage system which uses urgency to prioritize care. The Dutch Obstetric Telephone Triage System (DOTTS) was developed to provide a basis for assessing urgency of unplanned obstetric care requests by telephone. Reliability and validity are important components in evaluating such (obstetric) triage systems.
To determine the reliability of Dutch Obstetric Telephone Triage, by calculating the inter-rater and intra-rater reliability.
To evaluate the urgency levels of DOTTS by testing inter-rater and intra-rater reliability, 90 vignettes of possible requests were developed. Nineteen participants, from hospitals where DOTTS had been implemented, rated in two rounds a set of ten vignettes. The five urgency levels and five presenting symptoms had an equal spread and had to be entered in accordance with DOTTS per vignette. Urgency levels were dichotomized into high urgency and intermediate urgency. Inter-rater reliability was rated as degree of agreement between two different participants with the same vignette. Intra-rater reliability was rated as agreement by the same participants at different moments in time. The degree of inter-rater and intra-rater reliability was tested using weighted Cohen's Kappa and ICC.
The agreement of urgency level between participants in accordance with predefined urgency level per vignette was 90.5% (95% CI 87.5-93.6) [335 of 370]. Agreement of urgency level between participants was 88.5% (95% CI 84.9-93.0) [177 of 200] and 84.9% (95% CI 78.3-91.4) after re-rating [101 of 119]. Inter-rater reliability of DOTTS expressed as Cohen's Kappa was 0.77 and as ICC 0.87; intra-rater reliability of DOTTS expressed as Cohen's Kappa was 0.70 and as ICC 0.82.
Inter-rater and intra-rater reliability of DOTTS showed substantial correlation, and is comparable to other studies. Therefore, DOTTS is considered reliable.
通过使用紧急程度来确定护理优先级的分诊系统，可以优化急诊护理的安全性和效率。荷兰产科电话分诊系统（DOTTS）的开发是为评估通过电话提出的非计划产科护理请求的紧急程度提供依据。可靠性和有效性是评估此类（产科）分诊系统的重要组成部分。
通过计算评分者间信度和评分者内信度来确定荷兰产科电话分诊的可靠性。
为通过测试评分者间信度和评分者内信度来评估DOTTS的紧急程度水平，编制了90个可能请求的案例 vignettes。来自已实施DOTTS的医院的19名参与者对一组10个案例进行两轮评分。五个紧急程度水平和五个呈现症状分布均匀，每个案例必须按照DOTTS进行填写。紧急程度水平被二分法分为高紧急程度和中等紧急程度。评分者间信度被评定为两名不同参与者对同一案例的一致程度。评分者内信度被评定为同一参与者在不同时间点的一致性。使用加权科恩kappa系数和组内相关系数（ICC）来测试评分者间信度和评分者内信度的程度。
根据每个案例预先定义的紧急程度水平，参与者之间紧急程度水平的一致性为90.5%（95%可信区间87.5 - 93.6）[370个中的335个]。重新评分后，参与者之间紧急程度水平的一致性为88.5%（95%可信区间84.9 - 93.0）[200个中的177个]和84.9%（95%可信区间78.3 - 91.4）[119个中的101个]。以科恩kappa系数表示的DOTTS评分者间信度为0.77，以ICC表示为0.87；以科恩kappa系数表示的DOTTS评分者内信度为0.70，以ICC表示为0.82。
DOTTS的评分者间信度和评分者内信度显示出显著相关性，并且与其他研究相当。因此，DOTTS被认为是可靠的。
