Structural assumptions in infectious disease models, such as the choice of network or compartmental model type or the inclusion of different types of heterogeneity across individuals, might affect model predictions as much as or more than the choice of input parameters. We explore the potential implications of structural assumptions on HIV model predictions and policy conclusions. We illustrate the value of inference robustness assessment through a case study of the effects of a hypothetical HIV vaccine in multiple population subgroups over eight related transmission models, which we sequentially modify to vary over two dimensions: parameter complexity (e.g., the inclusion of age and HCV comorbidity) and contact/simulation complexity (e.g., aggregated compartmental vs. individual/disaggregated compartmental vs. network models). We find that estimates of HIV incidence reductions from network models and individual compartmental models vary, but those differences are overwhelmed by the differences in HIV incidence between such models and the aggregated compartmental models (which aggregate groups of individuals into compartments). Complexities such as age structure appear to buffer the effects of aggregation and increase the threshold of net vaccine effectiveness at which aggregated models begin to overestimate reductions. The differences introduced by parameter complexity in estimated incidence reduction also translate into substantial differences in cost-effectiveness estimates. Parameter complexity does not appear to play a consistent role in differentiating the projections of network models.