There is an increasing understanding in recent years that mental health and psychiatric illnesses can be interpreted as complex dynamical systems. This understanding is largely derived from the complexity of dynamics that is observed in time series that are closely related to mental health. This complexity is quantified using a range of metrics from information theory and nonlinear time series analysis. Interpreting these metrics correctly and discerning how they vary as the nature of the dynamics changes is important to correctly identify the effect of mental illness. In this perspective article I attempt to do this, by first describing complexity of time series and the metrics that are used to quantify this complexity. I then analyze the behavior of these metrics as dynamics transition between periodicity, chaos and noise. Finally, I explore these changes in the context of depression by studying existing literature, and interpret what this implies for the nature of its underlying dynamics. There are divergent trends across studies and across domains such as actigraphy, EEG, and ECG. These findings emphasize the need for a nuanced interpretation of complexity metrics and their role in advancing our understanding of the nonlinear dynamics underlying mental health conditions like depression.