Peiró S, Meneu R, Roselló M L, Portella E, Carbonell-Sanchís R, Fernández C, Lázaro G, Llorens M A, Martínez-Mas E, Moreno E, Ruano M, Rincón A, Vila M
Institut Valencià d'Estudis en Salut Pública (IVESP), Hospital La Malva-rosa, Valencia.
Med Clin (Barc). 1996 Jun 22;107(4):124-9.
The study was carried out in order to assess the inter-observer reliability and validity in respect of clinical appraisal given by the Appropriateness Evaluation Protocol (AEP), in the context of the Spanish Public Hospital System.
In order to assess the reliability a total of 614 hospital stays chosen at random from 56 hospital admissions were independently analysed by three reviewers (two doctors and one nurse). In order to assess the validity, the findings obtained by the nurse were compared with the majority opinion given by the 7 hospital specialists in respect of each of hospital stays under evaluation. As part of the analytical procedure, indices for observed agreement, and specific agreement were calculated, as well as the Kappa statistic, all forming of various random samples of 614 hospital stays. In order to assess the predictive validity of the AEP, its sensitivity, specificity and predictive values were all measured against the majority clinical judgement.
The study exhibited a high degree of inter-observer reliability (specific agreement > 64%, kappa > 0.75) and a reasonable validity in comparison with the consensus of opinions formed by a least 4 or 5 of its 7 clinical reviewers (specific agreement > 61%, kappa > 0.64), these values decreasing notably when the consensus of 6 or 7 of the reviewers was required. The AEP revealed a high degree of sensitivity and a low degree of specificity in comparison with the majority clinical assessment, thus minimising the occurrence of false results when the stay was regarded as appropriate, and producing false negatives (appropriate hospital stays regarded as inappropriate) varying in degree from moderate to very high.
The results showing high reliability and moderate validity regarding clinical assessment shows the AEP to be a useful instrument in the sifting-out of inappropriate use of hospitalisation, although they do not allow a definitive judgement to be made concerning the efficiency of hospital services nor judgements regarding individual cases.
本研究旨在评估在西班牙公立医院系统背景下，适宜性评估方案（AEP）所做临床评估的观察者间信度和效度。
为评估信度，三位评审员（两名医生和一名护士）对从56例住院病例中随机选取的614次住院进行了独立分析。为评估效度，将护士得出的结果与7位医院专家针对每例评估住院病例给出的多数意见进行了比较。作为分析程序的一部分，计算了观察一致性指数、特定一致性指数以及Kappa统计量，所有这些均来自614次住院的各种随机样本。为评估AEP的预测效度，对照多数临床判断对其敏感性、特异性和预测值进行了测量。
该研究显示出高度的观察者间信度（特定一致性>64%，kappa>0.75），并且与7位临床评审员中至少4位或5位形成的意见共识相比具有合理的效度（特定一致性>61%，kappa>0.64），当要求6位或7位评审员达成共识时，这些值显著降低。与多数临床评估相比，AEP显示出高度的敏感性和低度的特异性，因此当住院被视为适当时，可将假结果的发生率降至最低，但会产生程度从中度到非常高不等的假阴性结果（将适当的住院视为不适当）。
结果表明AEP在筛选不适当住院使用方面是一种有用的工具，尽管它不能对医院服务效率做出确定性判断，也不能对个别病例做出判断，但在临床评估方面具有高信度和中度效度。
