The formation of lithium dendrites is suppressed using a LiAlGe(PO)-poly(ethylene oxide) (LAGP-PEO) composite solid electrolyte and a PEO (lithium bis(trifluoromethane)sulfonimide) [PEO (LiTFSI)]-modified lithium metal anode in all-solid-state lithium batteries. The effects on the anode performance based on the PEO content in the composite solid electrolyte and the molecular weight of PEO used to modify the Li anode are studied. The structure, surface morphology, and stability of the composite solid electrolyte are examined by X-ray diffraction spectroscopy, scanning electron microscopy, and electrochemical tests. Results show that the presence of a PEO-500000(LiTFSI) film on a Li anode results in good mechanical properties and satisfactory interface contact features. The film can also prevent Li from reacting with LAGP. Furthermore, the formation of lithium dendrites can be effectively inhibited as the composite solid electrolyte is combined with the PEO film on the Li anode. The ratio of PEO in the composite solid electrolyte can be reduced to a low level of 1 wt %. PEO remains stable even at a high potential of 5.12 V (vs Li/Li). The assembled Li-PEO (LiTFSI)/LAGP-PEO/LiMnFePO all-solid-state cell can deliver an initial discharge capacity of 160.8 mAh g and exhibit good cycling stability and rate performance at 50 °C.