Recent studies have demonstrated that the mathematical model used for analyzing and interpreting fMRI data in gray matter (GM) is inappropriate for detecting or describing blood-oxygenation-level-dependent (BOLD) signals in white matter (WM). In particular the hemodynamic response function (HRF) which serves as the regressor in general linear models is different in WM compared to GM. We recently reported measurements of the frequency contents of resting-state signal time courses in WM that showed distinct power spectra which depended on local structural-vascular-functional associations. In addition, multiple studies of GM have revealed how functional connectivity between regions, as measured by the correlation between BOLD time series, varies dynamically over time. We therefore investigated whether and how BOLD signals from WM in a resting state varied over time. We measured voxel-wise spectrograms, which reflect the time-varying spectral patterns of WM time courses. The results suggest that the spectral patterns are non-stationary but could be categorized into five modes that recurred over time. These modes showed distinct spatial distributions of their occurrences and durations, and the distributions were highly consistent across individuals. In addition, one of the modes exhibited a strong coupling of its occurrence between GM and WM across individuals, and two communities of WM voxels were identified according to the hierarchical structures of transitions among modes. Moreover, these modes are coupled to the shape of instantaneous HRFs. Our findings extend previous studies and reveal the non-stationary nature of spectral patterns of BOLD signals over time, providing a spatial-temporal-frequency characterization of resting-state signals in WM.