An outstanding characteristic of gliomas is their infiltration into brain parenchyma, a property that impairs complete surgical resection; consequently, these tumors might recur, resulting in a high mortality rate. Gliomas invade along preferential routes, such as those along white matter tracts and in the perineuronal and perivascular spaces. Brain extracellular components and their partners and modulators play a crucial role in glioma cell invasion. This review presents an extensive survey of the literature, showing how the brain extracellular matrix (ECM) is modulated during the glioma infiltration process. We explore aspects of ECM interaction with glioma cells, reviewing the main glycosaminoglycans, glycoproteins and proteoglycans. We discuss the roles of ECM-binding proteins, including CD44, RHAMM, integrins and axonal guidance molecules, and highlight the role of proteases and glycosidases in glioma infiltration; in binding and release chemokines, cytokines and growth factors; and in generating new bioactive ECM fragments. We also consider the roles of cytoskeletal signaling, angiogenesis, miRNAs and the glial-to-mesenchymal transition linked to glioma invasion. We closely discuss therapeutic approaches based on the modulation of the extracellular matrix, targeting the control of glioma infiltration, its relative failure in clinical trials, and potential means to overcome this difficulty.