NTHU H.264/AVC Video Encoder & Decoder

1. NTHU H.264/AVC Video Encoder & Decoder Youn-Long Lin Department of Computer Science National Tsing Hua University Hsin-Chu, TAIWAN 300 ylin@cs.nthu.edu.tw 2007/02/10 IC-DFN, Las Vegas

2. ITU-T Standards H.261 H.263 H.263+ Joint ITU-T/ISO Standards H.262/ MPEG-2 H.264 ISO Standards MPEG-1 MPEG-4 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 Evolution of Video Coding Standards DVD/DTV VCD MP3 DV/IPCam YLLIN NTHU-CS

3. Get More (Quality) for Less (Bit-rate) H.264 MPEG2 YLLIN NTHU-CS

4. Effectiveness of basic techniques Ref: G. Sullivan & T. Wiegand, “Video Compression—From Concepts to the H.264/AVC Standard”, Proceedings of the IEEE, Vol 93, No.1, Jan 2005 • No MC; • Adding Skip mode to form a CR coder. • Allow only zero-valued MVs. • Allow integer-pel MC. • Allow half-pel MC • Allowing 4-MV; • Allowing quarter-pel MC. YLLIN NTHU-CS

5. Features of Video Coding Standards YLLIN NTHU-CS

6. Competing Standards • H.264 Advanced Video Coding (H.264/AVC) • Also Called MPEG-4 Part 10 • WMV/VC-1 (MicroSoft) • Chinese AVS (Audio Video Coding Standard) YLLIN NTHU-CS

7. H.264/AVC Profiles Extended profile SP, SI slice B slice Interlace Main profile Data partition Weighted prediction CABAC FREext (High) profile Slice group I slice 8x8 transform ASO P slice Quantization matrix Baseline profile Redundant Slice CAVLC Color Sampling 8/10/12 bit sampling YLLIN NTHU-CS

8. Global UniChip Multimedia SOC Platform CPU Accelerator (FPGA) USB(PHY) Daughter Board ROM/ Flash Memory SRAM SDRAM FPGA VIC USB 2.0 Static memory SDRAM Controller(4-CH) High-Speed Bus JPEG Codec DMA SRAM PWM WDT TIMER APB Bridge Capture Display Controller Peripheral Bus DAI SSI SD SM UART GPIO 12C Audio Codec I2S Flash memory with SSI Flash Card Button LED Video-In CCIR601 TV/LCD YLLIN NTHU-CS

9. H.264/AVC Decoder System Diagram UART TV Timer SD Card ARM926EJS Slave Master Slave Slave AHB1 Slave Slave Slave SDRAM SDC SDC H264 Slave Slave Master SDRAM LM AHB2 YLLIN NTHU-CS

10. SD Card SDRAM Input/Ref./Display Frame CPU Storage Device Display AHB MV SRAM Ref idx SRAM Para SRAM Parser reconstruct SRAM MC unfilter SRAM Pred SRAM MBinfo SRAM DF CABAD Pic Rec Intra pred CAVLD Coeff SRAM IQ/IDCT Residual SRAM DECODER H.264/AVC Decoder Architecture YLLIN NTHU-CS

11. AMBA interface AHB A LM slave wrapper control register H.264 Decoder MFU VLC & TV OUT DF & MC SDC arbiter 1 arbiter 2 master wrapper 1 master wrapper 2 YLLIN NTHU-CS AHB B

12. NTHU Design Flow Software spec. in C & Acceleration specify User Spec. SW lib. C models, drivers Platform spec. API System configuration System.h Embedded Software Compilation System description Acceleration Acceleration Software image System generation HW lib. HDL IPs HW IP Synthesizer Co-Sim HW/SW co-simulation Parameterized ISS Accelerator.v System.v System Integrate Area & Timing & Power evaluation Evaluation Integration Platform model No Performance constraint Yes Pin assignment & Hardware compilation FPGA prototyping Hardware image FPGA Verify YLLIN NTHU-CS

13. Demo Video YLLIN NTHU-CS

14. Encoder Demo – Video Capturing YLLIN NTHU-CS

15. NTHU-PKU Collaboration Porting NTHU H.264/AVC Encoder and Decoder to PKU SOC Platform

16. Participants YLLIN NTHU-CS

17. YLLIN NTHU-CS

18. Job Function YLLIN NTHU-CS

19. Schedule YLLIN NTHU-CS

20. Schedule YLLIN NTHU-CS

21. System Overview YLLIN NTHU-CS

22. FPGA Prototyping YLLIN NTHU-CS

23. Results YLLIN NTHU-CS

24. Future Systems YLLIN NTHU-CS

25. High-Performance QFHD YLLIN NTHU-CS

26. SuperHDTV (3840x2160 QFHD) 3840x2160 - SuperHDTV YLLIN NTHU-CS

27. Summary • H.264/AVC Encoder & Decoder IP • Two Platforms: UniChip and PKU • Very High Performance (Cycles per MB) • Future Directions • Super High Resolution (4X HDTV, 16X HDTV) • Low Power Design • Enhanced Platform and Design Flow • Chip Implementation YLLIN NTHU-CS