INTRODUCTION: Limited research has examined how technology and digital literacy may affect patients' use of video visits. This study explored the relationship of demographic factors and patient-reported confidence in digital literacy skills to access to video visits among patients who never used them during the COVID-19 pandemic. METHODS: Using existing survey data, the current study examined data from respondents who did not engage in video appointments but instead attended face-to-face appointments between April and December 2020 for nonemergent health concerns. A multivariable logistic regression model was used to investigate whether demographic and social determinants of health factors, context of care (primary care or psychiatry/psychology), and digital literacy confidence were associated with video visit engagement. Collinearity was assessed using the variance inflation factor. RESULTS: This study found that living in rural areas and having a self-reported lack of confidence in logging video appointments using the Mayo Clinic patient portal were associated with persistent nonuse of video appointments in a cohort of patients who did not use video visits at this institution during the early part of the COVID-19 pandemic. DISCUSSION: The research findings reported herein reveal that individuals living in rural areas and those who lack confidence in logging into patient portals to access video visits tend to persistently avoid using video appointments. More investment is needed at the federal and corporate levels to improve digital connectivity. Digital navigators and community involvement can promote digital adoption. CONCLUSION: To encourage digital competency in rural communities, it is important to implement support strategies through community stakeholders and other resources.