Limitation of site-stratified cox regression analysis in survival data: a cautionary tale of the PANAMO phase III randomized, controlled study in critically ill COVID-19 patients.

Current guidelines tend to focus on a p-value threshold of a pre-specified primary endpoint tested in randomized controlled clinical trials to determine a treatment effect for a specific drug. However, a p-value does not always provide evidence on the treatment effect of a drug, especially when stratification of the data does not account for unforeseen variables introduced into the analysis. We report and discuss a rare case in which investigational site stratification in the pre-specified analysis method of a primary endpoint results in a loss of statistical power in the evaluation of the treatment effect due to data attrition of almost 17% of outcome data in the phase III randomized, controlled PANAMO study in critically ill COVID-19 patients. Other analyses utilizing no or different stratification (e.g., stratifying by country, region, pooling low enrollment clinical sites) evaluates 100% of patient data resulting in p-values suggesting a positive treatment effect (p < 0.05). We demonstrate how this technical artifact occurs by adjustment for site stratification within the Cox regression analysis for survival outcomes and how alternative stratification corrects this discrepancy.