OBJECTIVES: Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet primary care settings face time constraints that present barriers to routine screening. MyCog Mobile is a self-administered, smartphone-based cognitive screener that has the potential to help overcome these screening obstacles. We compared MyCog Mobile to 'gold-standard' measures to support its reliability and validity and examined performance differences between in-person versus remote assessment. DESIGN: Cross-sectional convenience sample study comparing MyCog Mobile measures to established cognitive assessments. SETTING: Five in-person research facility locations in Georgia, Texas, New Jersey, Florida and Arizona. PARTICIPANTS: 200 adults aged 65-87 years (M=72.56, SD=5.11) who spoke English. PRIMARY OUTCOME MEASURES: Convergent and divergent validity were examined via Spearman's rho correlations between MyCog Mobile measures (MyPictures, MyFaces, MySorting and MySequences) and established measures (verbal paired associates, digit span, letter-number sequencing, Color-Word Interference Test and Trail Making Test), as well as a confirmatory factor analysis using these same measures. Internal consistency was assessed via Spearman-Brown split-half correlations. Score distributions were compared between in-person and remote administration. RESULTS: Internal consistency for all MyCog Mobile measures was above acceptable thresholds, ranging from ρ=0.71 to 0.89. Strong correlations were found between MyCog Mobile subtests and their tablet counterparts (MyPictures-Picture Sequence Memory: ρ=0.52; MySorting-dimensional change card sorting: ρ=0.55). MyPictures and MyFaces demonstrated large correlations with convergent memory measures (verbal paired associates immediate ρ=0.51 and 0.56, respectively) and small correlations with discriminant measures. Confirmatory factor analysis supported a two-factor model with subtests loading onto episodic memory and executive functioning factors as expected (χ²(25)=39.136; Comparative Fit Index (CFI) = 0.965; Tucker-Lewis Index (TLI) = 0.950; Root Mean Square Error of Approximation (RMSEA) = 0.053; Standardized Root Mean Square Residual (SRMR) = 0.040). Remote participants scored significantly higher on the MyPictures task overall (t(249)=2.98, p=0.004), while in-person participants scored higher on the MyFaces First Letter subtest and MySequences. CONCLUSIONS: MyCog Mobile offers a reliable and valid assessment of executive functioning and episodic memory in older adults. Performance may differ by setting, but further research is needed. These findings support future clinical validation studies to determine the diagnostic accuracy of MyCog Mobile to detect cognitive impairment.