The role of correlations between neuronal responses is crucial to understanding the neural code. A framework used to study this role comprises a breakdown of the mutual information between stimuli and responses into terms that aim to account for different coding modalities and the distinction between different notions of independence. Here we complete the list of types of independence and distinguish activity independence (related to total correlations), conditional independence (related to noise correlations), signal independence (related to signal correlations), coding independence (related to information transmission), and information independence (related to redundancy). For each type, we identify the probabilistic criterion that defines it, indicate the information-theoretic measure used as statistic to test for it, and provide a graphical criterion to recognize the causal configurations of stimuli and responses that lead to its existence. Using this causal analysis, we first provide sufficiency conditions relating these types. Second, we differentiate the use of the measures as statistics to test for the existence of independence from their use for quantification. We indicate that signal and noise correlation cannot be quantified separately. Third, we explicitly define alternative system configurations used to construct the measures, in which noise correlations or noise and signal correlations are eliminated. Accordingly, we examine which measures are meaningful only as a comparison across configurations and which ones provide a characterization of the actually observed responses without resorting to other configurations. Fourth, we compare the commonly used nonparametric approach to eliminate noise correlations with a functional (model-based) approach, showing that the former approach does not remove those effects of noise correlations captured by the tuning properties of the individual neurons, and implies nonlocal causal structure manipulations. These results improve the interpretation of the measures on the framework and help in understanding how to apply it to analyze the role of correlations.