This study explores the genetic and epidemiologic correlates of long-term photoplethysmography-derived pulse rate variability (PRV) measurements with anxiety disorders. Individuals with whole-genome sequencing, Fitbit, and electronic health record data (N = 920; 61,333 data points) were selected from the All of Us Research Program. Anxiety polygenic risk scores (PRS) were derived with PRS-CS after meta-analyzing anxiety genome-wide association studies from three major cohorts- UK Biobank, FinnGen, and the Million Veterans Program (N =364,550). PRV was estimated as the standard deviation of average five-minute pulse wave intervals over full 24-hour pulse rate measurements (SDANN). Antidepressant exposure was defined as an active antidepressant prescription at the time of the PRV measurement in the EHR. Anxiety PRS and antidepressant use were tested for association with daily SDANN. The potential causal effect of anxiety on PRV was assessed with one-sample Mendelian randomization (MR). Anxiety PRS was independently associated with reduced SDANN (beta = -0.08; p = 0.003). Of the eight antidepressant medications and four classes tested, venlafaxine (beta = -0.12, p = 0.002) and bupropion (beta = -0.071, p = 0.01), tricyclic antidepressants (beta = -0.177, p = 0.0008), selective serotonin reuptake inhibitors (beta = -0.069; p = 0.0008) and serotonin and norepinephrine reuptake inhibitors (beta = -0.16; p = 2×10) were associated with decreased SDANN. One-sample MR indicated an inverse effect of anxiety on SDANN (beta = -2.22, p = 0.03). Anxiety and antidepressants are independently associated with decreased PRV, and anxiety appears to exert a causal effect on reduced PRV. Those observational findings provide insights into the impact of anxiety on PRV.