Recently, the influence of the concept of environmental sustainability has increased, which includes environmentally friendly measures related to reducing the consumption of petrochemical fuels and converting post-production feedstocks into raw materials for the synthesis of polymeric materials, the addition of which would improve the performance of the final product. In this regard, the development of bio-based polyurethane foams can be carried out by, among other things, modifying polyurethane foams with vegetable or waste fillers. This paper investigates the possibility of using walnut shells (WS) and the mineral fillers vermiculite (V) and perlite (P) as a flame retardant to increase fire safety and thermal stability at higher temperatures. The effects of the fillers in amounts of 10 wt.% on selected properties of the polyurethane composites, such as rheological properties (dynamic viscosity and processing times), mechanical properties (compressive strength, flexural strength, and hardness), insulating properties (thermal conductivity), and flame retardant properties (e.g., ignition time, limiting oxygen index, and peak heat release) were investigated. It has been shown that polyurethane foams containing fillers have better performance properties compared to unmodified polyurethane foams.