The aim of this cross-sectional study was to assess the level of fear related to the SARS-CoV-2 virus and the association of fear, and of sociodemographic and clinical characteristics, with health-related quality of life (HRQoL). A large sample of the Greek general population (N = 583) completed the validated versions of the Fear of COVID-19 scale (FCV-19s) and the 12-item Short Form (SF-12), and provided data on socio-demographic status, health history and COVID-19 protective behaviors. Variables were compared with Mann-Whitney and Kruskal-Wallis tests and associations with Spearman’s correlations. Gamma regression models investigated the influence of sociodemographic and COVID-related variables on HRQoL. The mean FCV-19s score for the sample was 18.3 ± 5.6, and physical and mental component summary scores were 50.2 ± 7.9 and 46.7 ± 10.1, respectively. More fear of COVID-19 was expressed by females (p < 0.001), individuals with comorbidities (p < 0.01), those with contacts with comorbidities (p < 0.001), and individuals not having caught COVID-19 (p < 0.05). Contrastingly, less fear was expressed by unvaccinated individuals and those with less frequent intake of information about the pandemic. Item level and overall FCV-19s scores were negatively associated with SF-12 summary scores, and fear of COVID-19 was the most important predictor of both physical and mental HRQoL. The findings from this and other similar studies could help to identify specific population groups in need of interventions to improve their physical and mental health, which had deteriorated due to the pandemic.