Seaweed polysaccharides are versatile both in their functions in seaweed physiology and in their practical applications in society. However, their content and quality vary greatly. This review discusses the main factors that influence the yield and quality of polysaccharides, specifically carrageenans and agars (sulfated galactans) found in red algae species (Rhodophyta). In addition, its historical, current, and emerging applications are also discussed. Carrageenan has been influenced mainly by photosynthetically active radiation (PAR) and nitrogen, while its relationship with temperature has not yet been replicated by recent studies. Agar's seasonal trend has also been found to be more ambiguous than stated before, with light, temperature, nutrients, and pH being influencing factors. In this review, it is also shown that, depending on the compound type, seaweed polysaccharides are influenced by very different key factors, which can be crucial in seaweed aquaculture to promote a high yield and quality of polysaccharides. Additionally, factors like the extraction method and storage of polysaccharides also influence the yield and quality of these compounds. This review also highlights the drawbacks and inadequacy inherent from the conventional (or current) extraction technology approaches.