Reduced heart rate variability (HRV) and fatigue are common after COVID-19 infection and both are potentially influenced by physical activity (PA). We compared resting HRV, PA from accelerometers and questionnaires, and self-reported fatigue in 41 COVID-19 survivors (~8 months postinfection, 38 ± 17 years) with 41 matched controls. Differences in HRV were observed on acceleration capacity (p = 0.041), deceleration capacity (p = 0.032), high-frequency peak frequency (p = 0.019), absolute low-frequency power (p = 0.042), relative very low-frequency power (p = 0.012), SD2 (from Poincare plot; p = 0.047), and DFA2 (slope of long-term detrended fluctuation analysis; p = 0.004). Fatigue was greater in COVID-19 survivors (p < 0.001) with no differences in PA. Moderate-vigorous physical activity (MVPA) (Standardized Beta = -0.427, p = 0.003) and steps per day (Standardized Beta = -0.402, p = 0.007) were associated with DFA2 in COVID-19 survivors after controlling for age, sex, and body fat percentage. Fatigue was correlated to less MVPA (Spearman's rho = 0.342, p = 0.031) and fewer steps per day (rho = 0.329, p = 0.038) in COVID-19 survivors, and was indirectly linked to HRV through these PA mediators (Estimate = -0.20; p = 0.040). We present a model showing the complex relations between HRV, PA, and fatigue that provides the foundation for strategies to improve outcomes and rehabilitation after COVID-19 infection.