Low-Complexity Transform and Quantization in H.264/AVC - Malvar, Henrique S.; Hallapuro, Antti; Karczewicz, Marta; Kerofsky, Louis (IEEE Transactions on Circuits and Systems, 2003)
This paper presents an overview of the transform and quantization designs in H.264. Unlike the popular 8x8 discrete cosine transform used in previous standards, the 4x4 transforms in H.264 can be computed exactly in integer arithmetic, thus avoiding inverse transform mismatch problems. The new transforms can also be computed without multiplications, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity, especially for low-end processors. By using short tables, the new quantization formulas use multiplications but avoid divisions.