Data harmonization is necessary for removing confounding effects in multi-site diffusion image analysis. One such harmonization method, LinearRISH, scales rotationally invariant spherical harmonic (RISH) features from one site ("target") to the second ("reference") to reduce confounding scanner effects. However, reference and target site designations are not arbitrary and resultant diffusion metrics (fractional anisotropy, mean diffusivity) are biased by this choice. In this work we propose MidRISH: rather than scaling reference RISH features to target RISH features, we project both sites to a mid-space. We validate MidRISH with the following experiments: harmonizing scanner differences from 37 matched patients free of cognitive impairment, and harmonizing acquisition and study differences on 117 matched patients free of cognitive impairment. We find that MidRISH reduces bias of reference selection while preserving harmonization efficacy of LinearRISH. Users should be cautious when performing LinearRISH harmonization. To select a reference site is to choose diffusion metric effect-size. Our proposed method eliminates the bias-inducing site selection step.