BACKGROUND: There is an extensive library of language tests, each with excellent psychometric properties; however, many of the tests available take considerable administration time, possibly bearing psychological strain on patients. The Short and Tailored Evaluation of Language Ability (STELA) is a simplified, tablet-based language ability assessment system developed to address this issue, with a reduced number of items and automated testing process. OBJECTIVE: The aim of this paper is to assess the administration time, internal consistency, and validity of the STELA. METHODS: The STELA consists of a tablet app, a microphone, and an input keypad for clinician's use. The system is designed to assess language ability with 52 questions grouped into 2 comprehension modalities (auditory comprehension and reading comprehension) and 3 expression modalities (naming and sentence formation, repetition, and reading aloud). Performance in each modality was scored as the correct answer rate (0-100), and overall performance expressed as the sum of modality scores (out of 500 points). RESULTS: The time taken to complete the STELA was significantly less than the time for the WAB (mean 16.2, SD 9.4 vs mean 149.3, SD 64.1 minutes; P<.001). The STELA's total score was strongly correlated with the WAB Aphasia Quotient (r=0.93, P<.001), supporting the former's concurrent validity concerning the WAB, which is a gold-standard aphasia assessment. Strong correlations were also observed at the subscale level; STELA auditory comprehension versus WAB auditory comprehension (r=0.75, P<.001), STELA repetition versus WAB repetition (r=0.96, P<.001), STELA naming and sentence formation versus WAB naming and word finding (r=0.81, P<.001), and the sum of STELA reading comprehension or reading aloud versus WAB reading (r=0.82, P<.001). Cronbach α obtained for each modality was .862 for auditory comprehension, .872 for reading comprehension, .902 for naming and sentence formation, .787 for repetition, and .892 for reading aloud. Global Cronbach α was .961. The average of the values of item-total correlation to each subscale was 0.61 (SD 0.17). CONCLUSIONS: Our study confirmed significant time reduction in the assessment of language ability and provided evidence for good internal consistency and validity of the STELA tablet-based aphasia assessment system.