Lebet Ruth, Hayakawa Jennifer, Chamblee Tracy B, Tala Joana A, Singh Nakul, Wypij David, Curley Martha A Q
Ruth Lebet, MSN, CCNS-P, is Lecturer, School of Nursing, University of Pennsylvania, Philadelphia. Jennifer Hayakawa, DNP, PCNS-BC, CNRN, CCRN, is Clinical Nurse Specialist, Pediatric Intensive Care Unit, CHOC Children's Hospital, Orange, California, and Clinical Faculty, Western University of Health Sciences, Pomona, California. Tracy B. Chamblee, PhD, APRN, PCNS-BC, is Clinical Nurse Specialist, Pediatric Intensive Care Unit, Children's Medical Center Dallas, Texas. Joana A. Tala, MD, is Research Coordinator, Pediatric Intensive Care Unit, Yale New Haven Hospital/Yale University, Connecticut. Nakul Singh, ScM, is Biostatistician, Department of Cardiology, Boston Children's Hospital, Massachusetts. David Wypij, PhD, is Senior Biostatistician, Department of Cardiology, Boston Children's Hospital; Associate Professor, Department of Pediatrics, Harvard Medical School; and Senior Lecturer, Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts. Martha A. Q. Curley, RN, PhD, FAAN, is Ellen and Robert Kapito Professor in Nursing Science, School of Nursing, University of Pennsylvania, Philadelphia, and Nurse Scientist, Boston Children's Hospital, Massachusetts.
Nurs Res. 2017 Jul/Aug;66(4):323-329. doi: 10.1097/NNR.0000000000000224.
RESTORE (Randomized Evaluation of Sedation Titration for Respiratory Failure) was a cluster randomized clinical trial evaluating a sedation strategy in children 2 weeks to <18 years of age with acute respiratory failure supported on mechanical ventilation. A total of 31 U.S. pediatric intensive care units (PICUs) participated in the trial. Staff nurse rater agreement on measures used to assess a critical component of treatment fidelity was essential throughout the 4-year data collection period.
The purpose of the study is to describe the method of establishing and maintaining interrater agreement (IRA) of two core clinical assessment instruments over the course of the clinical trial.
IRA cycles were carried out at all control and intervention sites and included a minimum of five measurements of the State Behavioral Scale (SBS) and Withdrawal Assessment Tool-Version 1 (WAT-1). Glasgow Coma Scale scores were also obtained. PICUs demonstrating <80% agreement repeated their IRA cycle. Fleiss's kappa coefficient was used to assess IRA.
Repeated IRA cycles were required for 8% of 226 SBS cycles and 2% of 222 WAT-1 cycles. Fleiss's kappa coefficients from more than 1,350 paired assessments were .86 for SBS and .92 for WAT-1, demonstrating strong agreement and similar to .91 for the Glasgow Coma Scale. There was no difference in Fleiss's kappa for any of the instruments based on unit size or timing of assessment (earlier or later in the study). For SBS scores, Fleiss's kappa was significantly different in larger and smaller PICUs (.82 vs. .92, p = .003); however, Fleiss's kappa for both groups indicated excellent agreement.
Monitoring measurement reliability is an essential step in ensuring treatment fidelity and, thus, the validity of study results. Standardization on the use of these core assessment instruments among participating sites was achieved and maintained throughout the trial.
RESTORE（呼吸衰竭镇静滴定随机评估）是一项整群随机临床试验，旨在评估2周龄至未满18岁接受机械通气支持的急性呼吸衰竭儿童的镇静策略。共有31个美国儿科重症监护病房（PICU）参与了该试验。在整个4年的数据收集期间，护士评级人员就用于评估治疗依从性关键组成部分的措施达成一致意见至关重要。
本研究的目的是描述在临床试验过程中建立和维持两种核心临床评估工具的评定者间一致性（IRA）的方法。
在所有对照和干预地点进行IRA周期，包括至少五次对状态行为量表（SBS）和戒断评估工具第1版（WAT-1）的测量。还获取了格拉斯哥昏迷量表评分。一致性低于80%的PICU重复其IRA周期。使用Fleiss卡方系数评估IRA。
226个SBS周期中的8%和222个WAT-1周期中的2%需要重复IRA周期。超过1350对评估的Fleiss卡方系数，SBS为0.86，WAT-1为0.92，显示出高度一致性，与格拉斯哥昏迷量表的0.91相似。基于单位规模或评估时间（研究早期或晚期），任何工具的Fleiss卡方系数均无差异。对于SBS评分，大型和小型PICU的Fleiss卡方系数有显著差异（0.82对0.92，p = 0.003）；然而，两组的Fleiss卡方系数均表明一致性极佳。
监测测量可靠性是确保治疗依从性以及研究结果有效性的重要步骤。在整个试验过程中，参与地点对这些核心评估工具的使用实现并维持了标准化。
