Lanovaz Marc J, Giannakakos Antonia R, Destras Océane
1École de Psychoéducation, Université de Montréal, C.P. 6128, succursale Centre-Ville, Montreal, QC H3C 3J7 Canada.
2Manhattanville College, Purchase, NY USA.
Perspect Behav Sci. 2020 Jan 21;43(1):21-38. doi: 10.1007/s40614-020-00244-0. eCollection 2020 Mar.
Visual analysis is the most commonly used method for interpreting data from single-case designs, but levels of interrater agreement remain a concern. Although structured aids to visual analysis such as the dual-criteria (DC) method may increase interrater agreement, the accuracy of the analyses may still benefit from improvements. Thus, the purpose of our study was to (a) examine correspondence between visual analysis and models derived from different machine learning algorithms, and (b) compare the accuracy, Type I error rate and power of each of our models with those produced by the DC method. We trained our models on a previously published dataset and then conducted analyses on both nonsimulated and simulated graphs. All our models derived from machine learning algorithms matched the interpretation of the visual analysts more frequently than the DC method. Furthermore, the machine learning algorithms outperformed the DC method on accuracy, Type I error rate, and power. Our results support the somewhat unorthodox proposition that behavior analysts may use machine learning algorithms to supplement their visual analysis of single-case data, but more research is needed to examine the potential benefits and drawbacks of such an approach.
视觉分析是解释单病例设计数据最常用的方法，但评分者间的一致性水平仍是一个问题。尽管诸如双标准（DC）法等用于视觉分析的结构化辅助工具可能会提高评分者间的一致性，但分析的准确性仍可能受益于改进。因此，我们研究的目的是：（a）检验视觉分析与源自不同机器学习算法的模型之间的一致性，以及（b）将我们每个模型的准确性、I型错误率和检验效能与DC法产生的结果进行比较。我们在一个先前发表的数据集上训练我们的模型，然后对非模拟和模拟图表进行分析。我们所有源自机器学习算法的模型比DC法更频繁地与视觉分析者的解释相匹配。此外，机器学习算法在准确性、I型错误率和检验效能方面优于DC法。我们的结果支持了这一有点非正统的观点，即行为分析者可以使用机器学习算法来补充他们对单病例数据的视觉分析，但需要更多研究来检验这种方法的潜在利弊。
