Polyoxometalates (POMs) are an emerging class of inorganic metal oxides, which over the last decades demonstrated promising biological activities by the virtue of their great diversity in structures and properties. They possess high potential for the inhibition of various tumor types; however, their unspecific interactions with biomolecules and toxicity impede their clinical usage. The current focus of the field of biologically active POMs lies on organically functionalized and POM-based nanocomposite structures as these hybrids show enhanced anticancer activity and significantly reduced toxicity towards normal cells in comparison to unmodified POMs. Although the antitumor activity of POMs is well documented, their mechanisms of action are still not well understood. In this Review, an overview is given of the cytotoxic effects of POMs with a special focus on POM-based hybrid and nanocomposite structures. Furthermore, we aim to provide proposed mode of actions and to identify molecular targets. POMs are expected to develop into the next generation of anticancer drugs that selectively target cancer cells while sparing healthy cells.