Multidrug resistance (MDR) and targeted therapies present major challenges in tumor chemotherapy. Nanoparticles (NPs) hold promise for use in cancer theranostics due to their advantages in terms of tumor-targeted cytotoxicity and imaging. In this study, we developed N-((2-hydroxy-3-trimethylammonium) propyl) chitosan chloride (HTCC)/alginate-encapsulated Fe3O4 magnetic NPs (HTCC-MNPs) and applied them to MDR gastric cancer both in vivo and in vitro. HTCC-MNPs were fabricated from sodium alginate (ALG), Fe3O4 and HTCC using an ionic gelation method. The sizes and physical characteristics of the NPs were determined using dynamic light scattering, transmission electron microscopy (TEM) and zeta potential analysis. The HTCC-MNPs exhibited excellent water solubility and biocompatibility as well as significantly reduced cell viability in the drug-resistant cancer cell line SGC7901/ADR, but not in normal gastric cells (P < 0.05). An analysis of LC3 expression demonstrated the involvement of autophagy in HTCC-MNP cytotoxicity. Additionally, apoptosis was verified using a DNA content assay. HTCC-MNPs led to mitochondrial membrane potential loss, decreased ATP production and excessive reactive oxygen species (ROS) generation compared to a control group (P < 0.05). Magnetic resonance imaging showed enrichment of HTCC-MNPs in tumor-bearing mice. In vivo bioluminescence imaging and tumor volume measurements revealed that HTCC-MNPs markedly inhibited in vivo tumor growth (P < 0.05). In conclusion, HTCC-MNPs significantly inhibited MDR gastric tumor growth and reduced tumor volume via the induction of cellular autophagy and apoptosis, which was attributed to mitochondrial dysfunction and excessive ROS accumulation.