Protein tyrosine phosphatase 1B (PTP1B) is a critical negative modulator of insulin signaling and has attracted considerable attention in treating type 2 diabetes mellitus (T2DM). Low-molecular-weight polymannuronic acid phosphate (LPMP) was found to be a selective PTP1B inhibitor with an IC of 1.02 ± 0.17 μM. Cellular glucose consumption was significantly elevated in insulin-resistant HepG2 cells after LPMP treatment. LPMP could alleviate oxidative stress and endoplasmic reticulum stress, which are associated with the development of insulin resistance. Western blot and polymerase chain reaction (PCR) analysis demonstrated that LPMP could enhance insulin sensitivity through the PTP1B/IRS/Akt transduction pathway. Furthermore, animal study confirmed that LPMP could decrease blood glucose, alleviate insulin resistance, and exert hepatoprotective effects in diabetic mice. Taken together, LPMP can effectively inhibit insulin resistance and has high potential as an anti-diabetic drug candidate to be further developed.