Marine sediments are a large reservoir of recalcitrant organic matter and host microbes at subsurface depths exceeding 2.4 kilometers and temperatures up to 120°C, yet the mechanisms supplying bioavailable substrates remain unclear. Here, we investigated 7.8-million-year-old sediment from IODP Site C0012 off the Nankai Trough, Japan, through incubations at 20°, 35°, 55°, and 85°C to simulate burial temperatures. Using 3D fluorescence spectroscopy and ultrahigh-resolution mass spectrometry, we tracked changes in dissolved organic matter (DOM). At 35°C, humic-like DOM was released alongside metal ions, exhibiting low bioavailability. At 55°C, abiotic decomposition of humic compounds generated smaller, more bioavailable DOM, promoting fermentation. At 85°C, large nitrogen-containing humic compounds decomposed, producing labile H and acetate mainly through abiotic processes, bypassing fermentation. Our findings show how abiotic thermal processes activate the refractory organic matter pool, advancing our understanding of long-term carbon sequestration in marine sediments and its implications for global carbon cycling.