Lopetegui Marcelo A, Bai Shasha, Yen Po-Yin, Lai Albert, Embi Peter, Payne Philip R O
Department of Biomedical Informatics, The Ohio State University, Columbus, OH, USA.
Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH, USA.
AMIA Annu Symp Proc. 2013 Nov 16;2013:889-96. eCollection 2013.
Understanding clinical workflow is critical for researchers and healthcare decision makers. Current workflow studies tend to oversimplify and underrepresent the complexity of clinical workflow. Continuous observation time motion studies (TMS) could enhance clinical workflow studies by providing rich quantitative data required for in-depth workflow analyses. However, methodological inconsistencies have been reported in continuous observation TMS, potentially reducing the validity of TMS' data and limiting their contribution to the general state of knowledge. We believe that a cornerstone in standardizing TMS is to ensure the reliability of the human observers. In this manuscript we review the approaches for inter-observer reliability assessment (IORA) in a representative sample of TMS focusing on clinical workflow. We found that IORA is an uncommon practice, inconsistently reported, and often uses methods that provide partial and overestimated measures of agreement. Since a comprehensive approach to IORA is yet to be proposed and validated, we provide initial recommendations for IORA reporting in continuous observation TMS.
理解临床工作流程对研究人员和医疗保健决策者至关重要。当前的工作流程研究往往过于简化,未能充分体现临床工作流程的复杂性。持续观察时间动作研究(TMS)可以通过提供深入工作流程分析所需的丰富定量数据来加强临床工作流程研究。然而,有报告称持续观察TMS存在方法上的不一致性,这可能会降低TMS数据的有效性,并限制其对知识总体状况的贡献。我们认为,标准化TMS的一个基石是确保人类观察者的可靠性。在本手稿中,我们回顾了以临床工作流程为重点的TMS代表性样本中的观察者间可靠性评估(IORA)方法。我们发现,IORA是一种不常见的做法,报告不一致,并且经常使用提供部分和高估一致性度量的方法。由于尚未提出和验证一种全面的IORA方法,我们提供了在持续观察TMS中进行IORA报告的初步建议。