Quality of reporting and fragility index for randomized controlled trials in the vesicoureteral reflux literature: where do we stand?

BACKGROUND/INTRODUCTION: Randomized controlled trials (RCTs) are considered the "gold standard" methodology for examining the effects of clinical interventions, yet only 1% of urology literature employs this design. The Consolidated Standards of Reporting Trials (CONSORT) statement contains a standardized checklist of 37 items to be included when reporting RCTs to ensure transparency and completeness of information [2]. Despite the robust design of RCTs, the number of events can greatly change the significance of the results, which can be represented by the fragility index (FI).

OBJECTIVE

The objective was to assess the quality of reporting of RCTs in the pediatric vesicoureteral reflux (VUR) literature using the 2010 CONSORT statement and, for studies with significant positive findings, to determine the FI as a measure of robustness of the results.

STUDY DESIGN

A comprehensive search was conducted through MEDLINE® and Embase® to identify RCTs in VUR literature from 2000 to 2016. Two reviewers independently selected articles, which were evaluated using the CONSORT checklist. An overall quality of reporting score (OQS) (%) was calculated by dividing the number of checklist items present in each study by the maximum possible score (34) and expressed as a percentage. Studies were classified as low (<40%), moderate (40-70%) and high quality (>70%) based on the modified assessing the methodological quality of systematic reviews (AMSTAR) checklist. Of the 2052 initial matches, 98% were excluded due to methodology or content, a further 28 studies were found not to meet inclusion criteria after full text review. The FI was calculated for the 7 studies that met inclusion criteria with significantly different results by manually adding events to the study groups until p>0.05.

RESULTS

Twenty-two studies met inclusion criteria. The mean OQS was 46+17% with 9 (41%) identified as low quality (score <40%), 11 (50%) as moderate (40-70%) and 2 (9%) as high quality (>70%). There was no significant difference in OQS between RCTs with a sample size > 100 (n=15) versus <100 patients (n=7) (45+17% vs. 47+17%, p=0.7). However, we noted a difference when we compared RCTs with biostatistician support (n=4) vs. those without (n=18) (59+20% vs. 43+15%, p<0.05). Seven studies reported significant positive results making calculation of FI possible. The mean FI was 5.8+5.1 indicating that most studies were fragile. There was no correlation between the OQS and FI.

DISCUSSION AND CONCLUSION

The mean OQS of VUR RCTs was suboptimal (46%), with most studies having low FI scores indicating the instability of the findings. The only variable that significantly impacted the OQS was biostatistician support. Implementation of the CONSORT checklist with a minimum of 50% inclusion as a prerequisite for submission of manuscripts may improve the quality and transparency of reporting. Calculation of the FI could provide readers with an objective measure of robustness for the published trials, allowing for appropriate interpretation of the results.