Centre for Human Factors and Sociotechnical Systems, Faculty of Arts, Business and Law, University of the Sunshine Coast, Sippy Downs, Australia.
Southern Queensland Rural Health (SQRH), Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, Australia.
Ergonomics. 2024 May;67(5):695-715. doi: 10.1080/00140139.2023.2240048. Epub 2023 Jul 31.
Accident analysis methods are used to model the multifactorial cause of adverse incidents. Methods such as AcciMap, STAMP-CAST and recently AcciNet, are systemic approaches that support the identification of safety interventions across sociotechnical system levels. Despite their growing popularity, little is known about how reliable systems-based methods are when used to describe, model and classify contributory factors and relationships. Here, we conducted an intra-rater and inter-rater reliability assessment of AcciMap, STAMP-CAST and AcciNet using the Signal Detection Theory (SDT) paradigm. A total of 180 hours' worth of analyses across 360 comparisons were performed by 30 expert analysts. Findings revealed that all three methods produced a weak to moderate positive correlation coefficient, however the inter-rater reliability of STAMP-CAST was significantly higher compared to AcciMap and AcciNet. No statistically significant or practically meaningful differences were found between methods in the overall intra-rater reliability analyses. Implications and future research directions are discussed.
事故分析方法用于对不良事件的多因素原因进行建模。AcciMap、STAMP-CAST 等方法，以及最近的 AcciNet，都是支持在社会技术系统各级识别安全干预措施的系统方法。尽管它们越来越受欢迎，但对于系统方法在描述、建模和分类促成因素和关系方面的可靠性知之甚少。在这里，我们使用信号检测理论（SDT）范式对 AcciMap、STAMP-CAST 和 AcciNet 进行了内部评分者和外部评分者可靠性评估。由 30 名专家分析师进行了总计 360 次比较，分析了 180 小时的数据。研究结果表明，所有三种方法都产生了弱到中度的正相关系数，但是 STAMP-CAST 的评分者间可靠性明显高于 AcciMap 和 AcciNet。在整体内部评分者可靠性分析中，方法之间没有发现统计学上显著或实际有意义的差异。讨论了影响和未来的研究方向。
