BACKGROUND: Use of virtual reality (VR)-based biofeedback (BF) represents an emerging nonpharmacological intervention for enhancing sleep quality in individuals exhibiting depressive symptoms, anxiety symptoms, or both. However, empirical evidence regarding its efficacy in addressing sleep disturbances remains limited and inconclusive. OBJECTIVE: This 3-arm randomized controlled trial aimed (1) to compare the efficacy of VR-based BF with conventional BF in improving sleep quality, as measured by the Pittsburgh Sleep Quality Index (PSQI), among individuals with depressive symptoms, anxiety symptoms, or both (DAS); (2) to examine the effects of VR-based BF in a demographically similar healthy control (HC) group; and (3) to evaluate between-group differences in sleep quality improvements at the 4-week follow-up. METHODS: Participants scoring ≥10 on the Patient Health Questionnaire-9 or ≥9 on the Panic Disorder Severity Scale were allocated to a group with DAS while others were assigned to a HC group. The DAS group was subsequently randomized into VR-based BF or conventional BF interventions with a therapist. All participants attended sessions at weeks 0, 2, and 4, completing assessments including the Montgomery-Asberg Depression Rating Scale, State-Trait Anxiety Inventory, and Visual Analog Scale in interviews. The PSQI was administered at baseline and postintervention to evaluate alterations in sleep quality over a 4-week period. RESULTS: A total of 118 participants were randomized into a VR-based BF group (DAS/VR, n=40) or a conventional BF group (DAS/BF, n=38), and a control group (HC/VR, n=40) received VR-based BF. Sleep disturbance scores of both DAS/VR and DAS/BF groups had significant improvements (mean reductions of -0.58, SD 0.75 and -0.66, SD 0.75, respectively) compared to those preintervention, showing no significant difference after adjusting for age and sex (P=.49). The DAS/VR group had a greater improvement in sleep disturbance (mean -0.08, SD 0.53; P=0.0014) than the HC/VR group. Global PSQI scores in both DAS/VR and DAS/BF groups improved compared to those preintervention, showing decreases by -2.50 (SD 2.89) and -3.39 (SD 2.80), respectively. The difference between the 2 groups was not statistically significant (P=.14). The Global PSQI score in the DAS/VR group showed significant improvement (-0.95, SD 2.09; P=.01) compared to that in the HC/VR group. CONCLUSIONS: This study provides evidence that both VR-based BF and conventional BF with a therapist are efficacious psychological interventions for enhancing sleep quality in individuals with depressive symptoms, anxiety symptoms, or both, with no significant differences observed between these 2 approaches. Both interventions showed significant improvements compared to baseline measurements. These findings suggest potential applications of these interventions in clinical settings to improve sleep quality and mental well-being.