On the optimality of nonunitary filter banks in subband coders.

This paper investigates the energy compaction capabilities of nonunitary filter banks in subband coding. It is shown that nonunitary filter banks have larger coding gain than unitary filter banks because of the possibility of performing half-whitening in each channel. For long filter unit pulse responses, optimization of subband coding gain for stationary input signals results in a filter bank decomposition, where each channel works as an optimal open-loop DPCM system. We derive a formula giving the optimal filter response for each channel as a function of the input power spectral density (PSD). For shorter filter bank responses, good gain is obtained by suboptimal half-whitening responses, but the impact on the theoretical coding gain is still highly significant. Image coding examples demonstrate that better performance is achieved using nonunitary filter banks when the input images are in correspondence with the signal model.