Intracellular accumulation of misfolded proteins causes serious human proteinopathies. The transmembrane emp24 domain 9 (TMED9) cargo receptor promotes a general mechanism of cytotoxicity by entrapping misfolded protein cargos in the early secretory pathway. However, the molecular basis for this TMED9-mediated cargo retention remains elusive. Here, we report cryo-electron microscopy structures of TMED9, which reveal its unexpected self-oligomerization into octamers, dodecamers, and, by extension, even higher-order oligomers. The TMED9 oligomerization is driven by an intrinsic symmetry mismatch between the trimeric coiled coil domain and the tetrameric transmembrane domain. Using frameshifted Mucin 1 as an example of aggregated disease-related protein cargo, we implicate a mode of direct interaction with the TMED9 luminal Golgi-dynamics domain. The structures suggest and we confirm that TMED9 oligomerization favors the recruitment of coat protein I (COPI), but not COPII coatomers, facilitating retrograde transport and explaining the observed cargo entrapment. Our work thus reveals a molecular basis for TMED9-mediated misfolded protein retention in the early secretory pathway.