Machine Learning With Neuroimaging: Evaluating Its Applications in Psychiatry.

Psychiatric disorders are complex, involving heterogeneous symptomatology and neurobiology that rarely involves the disruption of single, isolated brain structures. In an attempt to better describe and understand the complexities of psychiatric disorders, investigators have increasingly applied multivariate pattern classification approaches to neuroimaging data and in particular supervised machine learning methods. However, supervised machine learning approaches also come with unique challenges and trade-offs, requiring additional study design and interpretation considerations. The goal of this review is to provide a set of best practices for evaluating machine learning applications to psychiatric disorders. We discuss how to evaluate two common efforts: 1) making predictions that have the potential to aid in diagnosis, prognosis, and treatment and 2) interrogating the complex neurophysiological mechanisms underlying psychopathology. We focus here on machine learning as applied to functional connectivity with magnetic resonance imaging, as an example to ground discussion. We argue that for machine learning classification to have translational utility for individual-level predictions, investigators must ensure that the classification is clinically informative, independent of confounding variables, and appropriately assessed for both performance and generalizability. We contend that shedding light on the complex mechanisms underlying psychiatric disorders will require consideration of the unique utility, interpretability, and reliability of the neuroimaging features (e.g., regions, networks, connections) identified from machine learning approaches. Finally, we discuss how the rise of large, multisite, publicly available datasets may contribute to the utility of machine learning approaches in psychiatry.