Emotions organize human and animal behaviour by automatically adjusting their actions at multiple physiological and behavioural scales. Recently, pattern recognition techniques have emerged as an important tool for quantifying the neural, physiological, and phenomenological organization of emotions in humans. Here we review recent advances in our understanding of the human emotion system from the viewpoint of pattern recognition studies, focussing on neuroimaging experiments. These studies suggest, in general, clear and consistent categorical structure of emotions across multiple levels of analysis spanning expressive behaviour, subjective experiences, physiological activity, and neural activation patterns. In particular, the neurophysiological data support the view of multiple discrete emotion systems that are organized in a distributed fashion across the brain, with no clear one-to-one mapping between emotions and brain regions. However, these techniques are inherently limited by the choice of a priori emotion categories used in the studies, and cannot provide direct causal evidence for brain activity-emotion relationships.