The ever-increasing demand for energy and environmental concerns have driven scientists to look for renewable and eco-friendly alternatives. Bio-based thermoelectric (TE) composite materials provide a promising solution to alleviate the global energy crisis due to their direct conversion of heat to electricity. Cellulose, the most abundant bio-polymer on earth with fascinating structure and desirable physicochemical properties, provides an excellent alternative matrix for TE materials. Here, recent studies on cellulose-based TE composites are comprehensively summarized. The fundamentals of TE materials, including TE effects, TE devices, and evaluation on conversion efficiency of TE materials are briefly introduced at the beginning. Then, the state-of-the-art methods for constructing cellulose-based TE composites in the forms of paper/film, aerogel, liquid, and hydrogel, are highlighted. TE performances of these composites are also compared. Following that, applications of cellulose-based TE composites in the fields of energy storage (e.g., supercapacitors) and sensing (e.g., self-powered sensors) are presented. Finally, opportunities and challenges that need investigation toward further development of cellulose-based TE composites are discussed.