Ogunsola Ayobami S, Marinier Michael C, Hlas Arman C, Elkins Jacob M
Department of Orthopedics and Rehabilitation, University of Iowa, 200 Hawkins Drive, Iowa City, IA, 52242, USA.
J Orthop Surg Res. 2025 Mar 1;20(1):216. doi: 10.1186/s13018-025-05538-0.
Utilization of large-volume clinical registries for observational research has gained popularity in orthopaedic literature. However, concerns exist regarding inadequate reporting of methodology in this type of research. Despite these concerns, the reproducibility of such studies has not been adequately assessed in existing literature. This study aims to assess the reproducibility of American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) arthroplasty studies on smoking as a risk factor for poor surgical outcomes by employing identical datasets and statistical methods.
A systematic PubMed search between 2013 and 2023 identified ACS-NSQIP studies involving hip or knee arthroplasty and smoking as a potential risk factor for poor surgical outcomes. Each study's methods were reproduced by a trained statistician based on the reported methodology. In cases where certain steps were not explicitly stated, the statistician made informed decisions to reproduce those steps. Adjusted odds ratios (aORs) and p-values (α = 0.05) were compared between the original and reanalyzed datasets.
The initial search yielded 43 studies, with 11 meeting inclusion criteria resulting in the reanalysis of 268 aORs. Upon reanalysis, 12.69% of the original studies' aORs changed in interpretation, while 13.43% experienced a change in statistical significance. The average magnitude change of each aOR across all studies was 17.22%, and the sample size (N) in reanalysis varied by up to 47.84%. Among the 11 commonly cited studies, approximately one in eight objective conclusions changed in interpretation or statistical significance.
Inconsistent reproducibility exists across many arthroplasty studies that utilize the ACS-NSQIP database. These findings highlight the importance of rigorous reporting of study methodology, data collection, and statistical analyses when utilizing large-volume databases in orthopaedic research. This burden of responsibility should be shared among authors, peer reviewers, and orthopaedic journals to confirm the accuracy and validity of published database research.
This study systematically reviewed and analyzed, in attempt to reproduce, published arthroplasty studies utilizing ACS-NSQIP database to assess smoking as a potential risk factor for poor surgical outcomes. All analyzed studies included Level III Evidence, therefore this current study compares reproduced Level III Evidence to the original Level III Evidence.
在骨科文献中，利用大容量临床登记数据库进行观察性研究越来越普遍。然而，这类研究的方法报告不充分令人担忧。尽管存在这些担忧，但现有文献中尚未对这类研究的可重复性进行充分评估。本研究旨在通过使用相同的数据集和统计方法，评估美国外科医师学会国家外科质量改进计划（ACS - NSQIP）关于吸烟作为手术不良结局危险因素的关节置换术研究的可重复性。
在2013年至2023年间对PubMed进行系统检索，以确定涉及髋或膝关节置换术且将吸烟作为手术不良结局潜在危险因素的ACS - NSQIP研究。一位训练有素的统计学家根据报告的方法重现每项研究的方法。在某些步骤未明确说明的情况下，统计学家做出明智的决定来重现这些步骤。比较原始数据集和重新分析数据集之间的调整比值比（aORs）和p值（α = 0.05）。
初步检索产生43项研究，其中11项符合纳入标准，从而对268个aORs进行了重新分析。重新分析后，原始研究中12.69%的aORs解释发生变化，而13.43%的aORs统计学显著性发生变化。所有研究中每个aOR的平均变化幅度为17.22%，重新分析中的样本量（N）变化高达47.84%。在11项经常被引用的研究中，约八分之一的客观结论在解释或统计学显著性方面发生了变化。
许多使用ACS - NSQIP数据库的关节置换术研究存在不一致的可重复性。这些发现凸显了在骨科研究中使用大容量数据库时，严格报告研究方法、数据收集和统计分析的重要性。作者、同行评审人员和骨科期刊应共同承担这一责任，以确认已发表的数据库研究的准确性和有效性。
本研究系统回顾并分析了已发表的利用ACS - NSQIP数据库评估吸烟作为手术不良结局潜在危险因素的关节置换术研究，试图进行重现。所有分析的研究均为Ⅲ级证据，因此本研究将重现的Ⅲ级证据与原始的Ⅲ级证据进行比较。
