Psychiatric illnesses are heterogeneous in nature. No illness manifests in the same way across individuals, and no two patients with a shared diagnosis exhibit identical symptom profiles. Over the last several decades, group-level analyses of in vivo neuroimaging data have led to fundamental advances in our understanding of the neurobiology of psychiatric illnesses. More recently, access to computational resources and large, publicly available datasets alongside the rise of predictive modeling and precision medicine approaches have facilitated the study of psychiatric illnesses at an individual level. Data-driven machine learning analyses can be applied to identify disease-relevant biological subtypes, predict individual symptom profiles, and recommend personalized therapeutic interventions. However, when developing these predictive models, methodological choices must be carefully considered to ensure accurate, robust, and interpretable results. Choices pertaining to algorithms, neuroimaging modalities and states, data transformation, phenotypes, parcellations, sample sizes, and populations we are specifically studying can influence model performance. Here, we review applications of neuroimaging-based machine learning models to study psychiatric illnesses and discuss the effects of different methodological choices on model performance. An understanding of these effects is crucial for the proper implementation of predictive models in psychiatry and will facilitate more accurate diagnoses, prognoses, and therapeutics.