Systematic inflammatory response syndrome (SIRS) and the accompanying sepsis pose a huge threat to human health worldwide. Heparin is a part of the standard supportive care for the disease. However, the molecular mechanism is not fully understood yet, and the potential signaling pathways that play key roles have not yet been elucidated. In this paper, the main findings regarding the molecular mechanisms associated with the beneficial effects of heparin, including inhibiting HMGB-1-driven inflammation reactions, histone-induced toxicity, thrombo-inflammatory response control and the new emerging mechanisms are concluded. To set up the link between the preclinical research and the clinical effects, the outcomes of the clinical trials are summarized. Then, the structure and function relationship of heparin is discussed. By providing an updated analysis of the above results, the paper highlights the feasibility of heparin as a possible alternative for sepsis prophylaxis and therapy.