Programa de Pós-Graduação em Agroecologia, Departamento de Agronomia, Universidade Estadual de Maringá, Maringá, Brasil.
Núcleo de Pesquisa em Biotecnologia Aplicada, Universidade Estadual de Maringá, Maringá, Brasil.
Plant Dis. 2020 Sep;104(9):2440-2448. doi: 10.1094/PDIS-08-19-1708-RE. Epub 2020 Jul 10.
Standard area diagrams (SADs) are plant disease severity assessment aids demonstrated to improve the accuracy and reliability of visual estimates of severity. Knowledge of the sources of variation, including those specific to a lab such as raters, specific procedures followed including instruction, image analysis software, image viewing time, etc., that affect the outcome of development and validation of SADs can help improve standard operating practice of these assessment aids. As reproducibility has not previously been explored in development of SADs, we aimed to explore the overarching question of whether the lab in which the measurement and validation of a SAD was performed affected the outcome of the process. Two different labs (Lab 1 and Lab 2) measured severity on the individual diagrams in a SAD and validated them independently for severity of gray mold (caused by ) on Gerbera daisy. Severity measurements of the 30 test images were performed independently at the two labs as well. A different group of 18 raters at each lab assessed the test images first without, and secondly with SADs under independent instruction at both Lab 1 and 2. Results showed that actual severity on the SADs as measured at each lab varied by up to 5.18%. Furthermore, measurement of the test image actual values varied from 0 to up to 24.29%, depending on image. Whereas at Lab 1 an equivalence test indicated no significant improvement in any measure of agreement with use of the SADs, at Lab 2, scale shift, generalized bias, and agreement were significantly improved with use of the SADs ( ≤ 0.05). An analysis of variance indicated differences existed between labs, use of the SADs aid, and the interaction, depending on the agreement statistic. Based on an equivalence test, the interrater reliability was significantly ( ≤ 0.05) improved at both Lab 1 and Lab 2 as a result of using SADs as an aid to severity estimation. Gain in measures of agreement and reliability tended to be greatest for the least able raters at both Lab 1 and Lab 2. Absolute error was reduced at both labs when raters used SADs. The results confirm that SADs are a useful tool, but the results demonstrated that aspects of the development and validation process in different labs may affect the outcome.
标准区域图 (SAD) 是一种植物疾病严重程度评估辅助工具，已被证明可以提高视觉估计严重程度的准确性和可靠性。了解影响 SAD 开发和验证结果的变异来源，包括特定于实验室的变异来源，如评估者、遵循的特定程序，包括指令、图像分析软件、图像查看时间等，可以帮助改进这些评估辅助工具的标准操作实践。由于以前没有探索过 SAD 开发中的可重复性，我们旨在探讨一个总体问题，即进行 SAD 测量和验证的实验室是否会影响该过程的结果。两个不同的实验室 (实验室 1 和实验室 2) 分别在 SAD 中的个别图表上测量严重程度，并分别对非洲菊上的灰霉病严重程度进行验证。在两个实验室中，还独立地对 30 个测试图像的严重程度进行了测量。每个实验室的另一组 18 名评估者在没有和有 SAD 辅助的情况下，首先在两个实验室的独立指令下对测试图像进行评估。结果表明，每个实验室测量的 SAD 实际严重程度差异高达 5.18%。此外，测试图像实际值的测量值因图像而异，从 0 到高达 24.29%。尽管在实验室 1 中，等价检验表明使用 SAD 没有显著提高任何一致性度量，但在实验室 2 中，使用 SAD 显著提高了比例尺偏移、广义偏差和一致性（≤0.05）。方差分析表明，一致性统计数据存在差异取决于实验室、SAD 辅助工具的使用以及交互作用。基于等价检验，由于 SAD 作为严重程度估计的辅助工具，在实验室 1 和实验室 2 中，评估者之间的可靠性都有显著提高（≤0.05）。在实验室 1 和实验室 2 中，最不熟练的评估者的一致性和可靠性衡量指标的提高幅度最大。当评估者使用 SAD 时，两个实验室的绝对误差都降低了。结果证实 SAD 是一种有用的工具，但结果表明，不同实验室的开发和验证过程的各个方面可能会影响结果。
