Single-cell RNA sequencing technologies have enabled unprecedented insights into gene expression and are poised to transform clinical diagnostics. At present, most computational approaches for interpreting single-cell data operate at the level of individual cells, predicting labels or properties based on isolated transcriptomic profiles. This approach overlooks a key class of signals: the composition of cells within a sample or defined population. Such signals are often critical for inferring tissue identity, disease state, or other sample-level phenotypes. To address this limitation, we introduce TissueFormer, a Transformer-based neural network that analyzes groups of single-cell RNA profiles to infer population-level labels while retaining single-cell resolution. Applied to predict the cortical area of groups of cells sampled from spatial transcriptomic data from mouse brains, TissueFormer outperformed both single-cell foundation models and machine learning methods applied to pseudobulk and cell type composition. This higher performance enables the automated construction of high-resolution brain region maps in individual animals directly from spatial transcriptomic data. More broadly, TissueFormer provides a framework for predicting any population-level phenotypes which are influenced by cellular diversity and tissue-level organization.