Sarcoidosis is a complex immune-mediated disease characterized by clusters of immune cells called granulomas. Despite major steps in understanding the cause of this disease, many questions remain. In this Review, we perform a mechanistic interrogation of the immune activities that contribute to granuloma formation in sarcoidosis and compare these processes with its closest mimic, tuberculosis, highlighting shared and divergent immune activities. We examine how Mycobacterium tuberculosis is sensed by the immune system; how the granuloma is initiated, formed, and perpetuated in tuberculosis compared with sarcoidosis; and the role of major innate and adaptive immune cells in shaping these processes. Finally, we draw these findings together around several recent high-resolution studies of the granuloma in situ that utilized the latest advances in single-cell technology combined with spatial methods to analyze plausible disease mechanisms. We conclude with an overall view of granuloma formation in sarcoidosis.