Low latency secondary transforms for intra/inter prediction residual.

In this paper, we present a transform scheme where a secondary transform is applied after the conventional DCT for intra as well as inter prediction residues. Our approach is applicable to any block-based video codec that employs transforms along the horizontal and vertical direction separably. The secondary transform is applied to the lower K ( K=4 or 8) frequency coefficients of the output of conventional DCT at block with dimensions 8 and larger. The proposed transform scheme has low complexity as it is applied only to the top-left portion of the DCT output, especially in the context of large blocks such as 32 × 32 where an alternate non-DCT 32 × 32 transform would have a prohibitive implementation hardware cost. The proposed technique is single-pass, and the choice of whether to use the secondary transform is solely based on the prediction direction for intra residue, and on transform unit location in the prediction unit for the inter residue. The scheme requires no additional signaling information or R-D search. Our simulation results show that the proposed transform scheme provides significant BD-rate improvement over the conventional DCT-based coding scheme. Finally, we also show how to implement the proposed secondary transforms with low latency in hardware.