Cell populations in the tumor microenvironment (TME), including their abundance, composition, and spatial location, are critical determinants of patient response to therapy. Recent advances in spatial transcriptomics (ST) have enabled the comprehensive characterization of gene expression in the TME. However, popular ST platforms, such as Visium, only measure expression in low-resolution spots and have large tissue areas that are not covered by any spots, which limits their usefulness in studying the detailed structure of TME. Here, we present TESLA, a machine learning framework for tissue annotation with pixel-level resolution in ST. TESLA integrates histological information with gene expression to annotate heterogeneous immune and tumor cells directly on the histology image. TESLA further detects unique TME features such as tertiary lymphoid structures, which represents a promising avenue for understanding the spatial architecture of the TME. Although we mainly illustrated the applications in cancer, TESLA can also be applied to other diseases.