For accurate EEG/MEG source analysis it is necessary to model the head volume conductor as realistic as possible. This includes the distinction of the different conductive compartments in the human head. In this study, we investigated the influence of modeling/not modeling the conductive compartments skull spongiosa, skull compacta, cerebrospinal fluid (CSF), gray matter, and white matter and of the inclusion of white matter anisotropy on the EEG/MEG forward solution. Therefore, we created a highly realistic 6-compartment head model with white matter anisotropy and used a state-of-the-art finite element approach. Starting from a 3-compartment scenario (skin, skull, and brain), we subsequently refined our head model by distinguishing one further of the above-mentioned compartments. For each of the generated five head models, we measured the effect on the signal topography and signal magnitude both in relation to a highly resolved reference model and to the model generated in the previous refinement step. We evaluated the results of these simulations using a variety of visualization methods, allowing us to gain a general overview of effect strength, of the most important source parameters triggering these effects, and of the most affected brain regions. Thereby, starting from the 3-compartment approach, we identified the most important additional refinement steps in head volume conductor modeling. We were able to show that the inclusion of the highly conductive CSF compartment, whose conductivity value is well known, has the strongest influence on both signal topography and magnitude in both modalities. We found the effect of gray/white matter distinction to be nearly as big as that of the CSF inclusion, and for both of these steps we identified a clear pattern in the spatial distribution of effects. In comparison to these two steps, the introduction of white matter anisotropy led to a clearly weaker, but still strong, effect. Finally, the distinction between skull spongiosa and compacta caused the weakest effects in both modalities when using an optimized conductivity value for the homogenized compartment. We conclude that it is highly recommendable to include the CSF and distinguish between gray and white matter in head volume conductor modeling. Especially for the MEG, the modeling of skull spongiosa and compacta might be neglected due to the weak effects; the simplification of not modeling white matter anisotropy is admissible considering the complexity and current limitations of the underlying modeling approach.