Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials.

BACKGROUND

Protocols are often unavailable to peer-reviewers and readers. To detect outcome reporting bias (ORB), readers usually have to resort to publicly available descriptions of study design such as public clinical trial registries. We compared primary outcomes in protocols, ClinicalTrials.gov and publications of oncology trials and evaluated the use of ClinicalTrials.gov as compared with protocols in detecting discrepancies between planned and published outcomes.

METHOD

We searched for phase III oncology trials registered in ClinicalTrials.gov and published in the Journal of Clinical Oncology and New England Journal of Medicine between January 2014 and June 2015. We extracted primary outcomes reported in the protocol, ClinicalTrials.gov and the publication. First, we assessed the quality of primary outcome descriptions by using a published framework. Second, we evaluated modifications of primary outcomes between each source. Finally, we evaluated the agreement, specificity and sensitivity of detecting modifications between planned and published outcomes by using protocols or ClinicalTrials.gov.

RESULTS

We included 65 trials, with 81 primary outcomes common among the 3 sources. The proportion of primary outcomes reporting all items from the framework was 73%, 22%, and 75% for protocols, ClinicalTrials.gov and publications, respectively. Eight (12%) trials presented a discrepancy between primary outcomes reported in the protocol and in the publication. Twelve (18.5%) trials presented a discrepancy between primary outcomes registered at ClinicalTrials.gov and in publications. We found a moderate agreement in detecting discrepant reporting of outcomes by using protocols or ClinicalTrials.gov [κ = 0.53, 95% confidence interval (0.25-0.81)]. Using ClinicalTrials.gov to detect discrepant reporting of outcomes showed high specificity (89.5%) but lacked sensitivity (75%) as compared with use of protocols.

CONCLUSION

In oncology trials, primary outcome descriptions in ClinicalTrials.gov are often of low quality and may not reflect what is in the protocol, thus limiting the detection of modifications between planned and published outcomes.