In acute myeloid leukemia (AML), a small cell population that contains stem cell features such as lack of differentiation, self-renewal potential, and drug resistance, can be identified. These so-called leukemic stem cells (LSCs) are thought to be responsible for relapse initiation after initial treatment leading to successful eradication of the bulk AML cell population. Since many studies have aimed to characterize and eliminate LSCs to prevent relapse and increase survival rates of patients, LSCs are one of the best characterized cancer stem cells. The specific elimination of LSCs, while sparing the healthy normal hematopoietic stem cells (HSCs), is one of the major challenges in the treatment of leukemia. This review focuses on several surface markers and intracellular transcription factors that can distinguish AML LSCs from HSCs and, therefore, specifically eliminate these stem cell-like leukemic cells. Moreover, previous and ongoing clinical trials of acute leukemia patients treated with therapies targeting these markers are discussed. In contrast to knowledge on LSCs in AML, insight into LSCs in acute lymphoid leukemia (ALL) is limited. This review therefore also addresses the latest insight into LSCs in ALL.