Purpose Cochlear implant (CI) performance varies considerably across individuals and across domains of auditory function, but clinical testing is typically restricted to speech intelligibility. The goals of this study were (a) to develop a basic auditory skills evaluation battery of tests for comprehensive assessment of ecologically relevant aspects of auditory perception and (b) to compare CI listeners' performance on the battery when tested in the laboratory by an audiologist or independently at home. Method The battery included 17 tests to evaluate (a) basic spectrotemporal processing, (b) processing of music and environmental sounds, and (c) speech perception in both quiet and background noise. The battery was administered online to three groups of adult listeners: two groups of postlingual CI listeners and a group of older normal-hearing (ONH) listeners of similar age. The ONH group and one CI group were tested in a laboratory by an audiologist, whereas the other CI group self-tested independently at home following online instructions. Results Results indicated a wide range in the performance of CI but not ONH listeners. Significant differences were not found between the two CI groups on any test, whereas on all but two tests, CI listeners' performance was lower than that of the ONH participants. Principal component analysis revealed that four components accounted for 82% of the variance in measured results, with component loading indicating that the test battery successfully captures differences across dimensions of auditory perception. Conclusions These results provide initial support for the use of the basic auditory skills evaluation battery for comprehensive online assessment of auditory skills in adult CI listeners.