Counterfactual Groups to Assess Vaccine or Treatment Efficacy in HIV Prevention Trials in High-Risk Populations in Uganda.

BACKGROUND

Assessment of efficacy in HIV prevention trials remains a challenge in the era of widespread use of active controls. We investigated use of counterfactual groups to assess treatment efficacy.

METHODS

We used data from placebo arms of two previous HIV prevention efficacy trials (Pro2000 vaginal microbicide trial, 2005-2009: ISRCTN64716212 and dapivirine vaginal ring trial, 2013-2016: NCT01539226) and four observational cohorts (two in each of the periods; (a) during the conduct of a simulated HIV vaccine efficacy trial (SiVET), 2012-2017, and (b) prior to SiVET (2005-2011)) and compared HIV prevention efficacy trial targeted outcomes with SiVETs. SiVET participants were administered a licensed hepatitis B vaccine at 0, 1 and 6 months mimicking an HIV vaccine efficacy trial schedule. Participants were tested for HIV quarterly for one year. The probability of the SiVET assignment conditioned on the measured participants' baseline characteristics were estimated using propensity scores (PS) and matched between SiVET and placebo arm of trials. Similar calculations were repeated for observational cohorts in the pre- and during SiVET periods. We compared HIV incidence rate ratio (IRR) between SiVET and the trials or observational data before and after PS matching.

RESULTS

This analysis involved data from 3387 participants; observational cohorts before SiVET 1495 (44.2%), placebo arms of previous trials 367 (10.8%), observational cohorts during SiVET conduct 953 (28.1%) and SiVETs 572 (16.9%). Before propensity score matching (PSM), there were significant imbalances in participants' baseline characteristics between SiVET, and all the other studies and HIV incidence was lower in SiVET. After PSM, the participants' characteristics were comparable. The HIV incidence in SiVET was similar to that in the previous trial, IRR = 1.01 95% CI: 0.16-4.70), = 0.968, and observational data during SiVET, IRR = 0.74, 95% CI 0.34-1.54), = 0.195, but much lower compared to the observational data pre-SiVET, IRR = 0.48, 95% CI: 0.20-1.04), = 0.023.

CONCLUSIONS

PSM can be used to create counterfactual groups from other data sources. The best counterfactual group for assessing treatment effect is provided by data collected in the placebo arm of previous trials followed by that from observational data collected concurrently to the current trial (SiVET). Even with PSM, observational data collected prior to the current trial may overestimate treatment effect.