Selenium nanoparticles (SeNPs), as a potential cancer therapeutic agent, have attracted extensive attention due to their high anticancer activity and low toxicity. Polysaccharides could be the modifiers and stabilizers to improve the stability and dispersibility of SeNPs in aqueous solution. This study aimed to investigate the physicochemical characterization, stability, and anti-pancreatic cancer cell activities of SeNPs stabilized by a heteroxylan PVP3-1 extracted from the clusters of Prunella vulgaris Linn. Our results showed that PVP3-1 with Mw of 154 kDa was composed of →4)-β-D-Xylp(1→, →2, 4)-β-D-Xylp(1→, t-α-L-Araf(1→ and 4-MeO-α-D-GlcpA(1→. Red, zero-valent, and uniform spherical SeNPs with an average diameter of about 60 nm and high stability in aqueous solution were constructed successfully by polysaccharide PVP3-1. Anti-pancreatic cancer cell activity assays showed that PVP3-1-SeNPs could inhibit the proliferation and migration of pancreatic cancer cells in vitro. Furthermore, PVP3-1-SeNPs induced apoptosis and autophagy of pancreatic cancer cells through inhibiting mTOR signaling pathway. In conclusion, these results indicated that PVP3-1-SeNPs could be potential anti-tumor nanoparticles for treating pancreatic cancer.