Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
Clinical Epidemiology Program, Ottawa Hospital Research Institute, 1053 Carling Avenue, Ottawa, Ontario, Canada.
BMC Med Res Methodol. 2024 Oct 29;24(1):256. doi: 10.1186/s12874-024-02366-4.
Dichotomisation of statistical significance, rather than interpretation of effect sizes supported by confidence intervals, is a long-standing problem.
We distributed an online survey to clinical trial statisticians across the UK, Australia and Canada asking about their experiences, perspectives and practices with respect to interpretation of statistical findings from randomised trials. We report a descriptive analysis of the closed-ended questions and a thematic analysis of the open-ended questions.
We obtained 101 responses across a broad range of career stages (24% professors; 51% senior lecturers; 22% junior statisticians) and areas of work (28% early phase trials; 44% drug trials; 38% health service trials). The majority (93%) believed that statistical findings should be interpreted by considering (minimal) clinical importance of treatment effects, but many (61%) said quantifying clinically important effect sizes was difficult, and fewer (54%) followed this approach in practice. Thematic analysis identified several barriers to forming a consensus on the statistical interpretation of the study findings, including: the dynamics within teams, lack of knowledge or difficulties in communicating that knowledge, as well as external pressures. External pressures included the pressure to publish definitive findings and statistical review which can sometimes be unhelpful but can at times be a saving grace. However, the concept of the minimally important difference was identified as a particularly poorly defined, even nebulous, construct which lies at the heart of much disagreement and confusion in the field.
The majority of participating statisticians believed that it is important to interpret statistical findings based on the clinically important effect size, but report this is difficult to operationalise. Reaching a consensus on the interpretation of a study is a social process involving disparate members of the research team along with editors and reviewers, as well as patients who likely have a role in the elicitation of minimally important differences.
将统计显著性二分法化，而不是解释置信区间支持的效应大小，是一个长期存在的问题。
我们向英国、澳大利亚和加拿大的临床试验统计学家分发了一份在线调查，询问他们在解释随机试验的统计结果方面的经验、观点和实践。我们报告了封闭式问题的描述性分析和开放式问题的主题分析。
我们在广泛的职业阶段（24%教授；51%高级讲师；22%初级统计学家）和工作领域（28%早期试验；44%药物试验；38%卫生服务试验）获得了 101 份回复。大多数（93%）人认为应该通过考虑（最小）治疗效果的临床重要性来解释统计发现，但许多人（61%）表示量化临床重要的效应大小很困难，而且较少人（54%）在实践中采用这种方法。主题分析确定了在研究结果的统计解释上达成共识的几个障碍，包括：团队内部的动态、缺乏知识或沟通知识的困难，以及外部压力。外部压力包括发表明确发现和统计审查的压力，这些压力有时可能没有帮助，但有时也可以挽救局面。然而，最小重要差异的概念被认为是一个特别定义不明确、甚至模糊的概念，它是该领域存在许多分歧和困惑的核心。
大多数参与的统计学家认为，根据临床重要的效应大小来解释统计发现很重要，但他们报告说这很难实施。对研究解释达成共识是一个涉及研究团队不同成员、编辑和审稿人以及可能在最小重要差异征集中发挥作用的患者的社会过程。
