Immune protection and lasting memory are accomplished through the generation of phenotypically and functionally distinct CD8 T cell subsets. Understanding how these effector and memory T cells are formed is the first step in eventually manipulating the immune system for therapeutic benefit. In this review, we will summarize the current understanding of CD8 T cell differentiation upon acute infection, with a focus on the transcriptional and epigenetic regulation of cell fate decision and memory formation. Moreover, we will highlight the importance of high throughput sequencing approaches and single cell technologies in providing insight into genome-wide investigations and the heterogeneity of individual CD8 T cells.