NSW Primary Industries, 1243 Bruxner Highway, Wollongbar Primary Industries Institute, Wollongbar 2477, NSW, Australia.
Prev Vet Med. 2012 May 1;104(3-4):240-8. doi: 10.1016/j.prevetmed.2011.11.005. Epub 2011 Dec 7.
Results from laboratory assays for detection of animal disease are often assessed for repeatability (agreement within laboratory) and reproducibility (agreement between laboratories). This work aimed to understand the strengths and limitations of available methods for describing these quantities. Five major veterinary laboratories in Australia volunteered to participate in a designed evaluation based on repeat testing of twenty bovine sera. Sampling was stratified so that ten of the sera were negative to the virus neutralisation test (VNT) for antibody to bovine herpes virus 1 (BHV-1) and the remaining ten sera were VNT positive. Each serum was divided into 50 replicates and each laboratory assayed one replicate of each serum on a weekly basis using a commercial ELISA for BHV-1. Laboratories were blinded to the identity of sera. The data on sample to positive control ratio (S/P) for these 1000 individual assays were collated, sources of variance analysed using a random effects model, and reliability coefficients (ρ) obtained from the variance estimates as quantitative measures of within and between laboratory agreement. Coefficient of variation (CV) was calculated for combinations of sera and laboratory. CV was found to be higher for sera with the lowest mean S/P values (VNT -ve sera). For VNT -ve sera, agreement of S/P within laboratory was low to moderate (ρ: 0.01-0.27) and the agreement between all labs was low (ρ=0.02). Reliability coefficients for VNT +ve sera were very high for agreement within laboratories (ρ: 0.63-0.92) and moderate for agreement between laboratories (ρ=0.52). As well, simulation demonstrated that sero-prevalence has a dramatic affect on the reliability coefficient if sampling were to be irrespective of VNT status. We conclude that there are some limitations with the available approaches for assessing agreement within and between laboratories. Although reliability coefficients have some drawbacks they are an attractive way of reducing reliance on subjective assessment of agreement.
结果从实验室检测动物疾病经常评估可重复性 （实验室内部协议） 和再现性 （实验室之间的协议）。这项工作旨在了解现有方法的优缺点，用于描述这些数量。澳大利亚的五个主要兽医实验室自愿参加了一个基于重复测试二十牛血清的设计评估。采样分层，使十血清对病毒中和试验 （VNT） 抗体阴性牛疱疹病毒 1 （BHV-1） 和其余十个血清 VNT 阳性。每个血清被分为 50 个重复，每个实验室每周用商业 ELISA 检测一个血清的一个重复 BHV-1。实验室对血清的身份是盲的。对这些 1000 个单独的检测的样本与阳性对照比值 （S/P） 的数据进行了整理，使用随机效应模型分析方差来源，并从方差估计中获得可靠性系数 （ρ），作为实验室内部和实验室之间协议的定量测量。计算了血清和实验室的变异系数 （CV）。对于具有最低平均 S/P 值的血清 （VNT 阴性血清），发现 CV 更高。对于 VNT 阴性血清，实验室内部的 S/P 协议低至中等 （ρ：0.01-0.27），所有实验室之间的协议低 （ρ=0.02）。对于 VNT 阳性血清，实验室内部的 S/P 协议可靠性系数非常高 （ρ：0.63-0.92），实验室之间的协议中等 （ρ=0.52）。此外，模拟表明，如果采样不取决于 VNT 状态，血清流行率对可靠性系数有很大影响。我们得出结论，评估实验室内部和实验室之间协议的现有方法存在一些局限性。虽然可靠性系数有一些缺点，但它们是减少对协议的主观评估依赖的一种有吸引力的方法。
