Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005

1. A hybrid error concealment scheme for MPEG-2 video transmission based on best neighborhood matching algorithm Li-Wei Kang and Jin-Jang Leou Journal of Visual Communication & Image Representation 2005

2. Outline • Introduction • Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames • Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames • Simulation results

3. Introduction (1/2) • Transmission errors and error propagation Original pictures Encoded stream Network Encoded stream Decoded pictures

4. Introduction (2/2) • Error control • Channel coding • Error resilient coding • Detection and correction • Detection and concealment Encoded stream Encoder control Encoded stream + = Encoded stream Encoded stream Decoder control previous current future current

5. Proposed hybrid error concealment scheme for MPEG-2 • I frame • Spatial search for similar blocks • Selection between H.26L test model and FBNM • P frame • Temporal search for similar blocks • Determination of block behavior • Small motions, middle motions, and large motions • Search for motion vectors

6. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (1/7) • H.26L test model [25] • Best neighborhood matching (BNM) [15] a2 a1 a3 a4 Best match L N+m N+m H

7. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (2/7) • Proposed fast BNM • Step 1: Initial candidate block by MAD • Step 2: Hexagon-based fast search by MAD

8. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (3/7) • Proposed hybrid error concealment scheme • Fitness function • H.26L or fast BNM? • Check the smoothness of the boundaries between the concealed block and its neighbors • Check the similarity between the concealed block and its neighbors • Components • AIDB • AMD • AVD

9. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (4/7) • Average intersample difference across the block boundaries between a concealed block and its four neighbors (AIDB) • Smoothness of the boundaries

10. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (5/7) • Average mean difference between a concealed block and its eight neighbors (AMD) mean mean mean mean mean mean mean mean mean

11. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (6/7) • Average variance difference between a concealed block and its eight neighbors (AVD)

12. Proposed hybrid error concealment scheme for MPEG-2 intra-coded I frames (7/7) • Fitness function • Choose of concealed block • Purpose of TC: BH.26L is usually smaller than BFBNM, but it’s visual quality is not always better than BFBNM

13. Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames (1/2) • MV magnitude • Determination of types of the corrupted blocks • Block matching Tl Th Small motion blocks Candidates: MV0 MVave Middle motion blocks Candidates: MVi for i = 1 to 8 MVave MVmedium Large motion blocks Candidates: Fast search algorithm

14. Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames (2/2) • Fast search algorithm for large motion blocks 2D+1 2D+1 Search range for the corrupted block

15. Simulation results (1/8) • MPEG-2 • GOP size = 12 • IBBP structure • 1.5 Mbps • 30 fps • One packet contains one slice • CIF • Coastguard • Football • Salesman • Table tennis

16. Simulation results (2/8) • parameters • I frame • N = 16; m = 4; L = H = 120 • P or B frame • Tl = 8; Th = 12; TC = 200; D = 2 L N+m N+m H Tl Th

17. Simulation results (3/8) • Discussions • H and L for I frame • Smaller: good candidates may be excluded • Larger: bad candidates may be included • Tl and Th for P frame • Smaller: better concealment results but high computation cost • Larger: worst concealment results but low computation cost • D for P frame • Smaller: worst concealment results but low computation cost • Larger: better concealment results but high computation cost

18. Simulation results (4/8) • 1. replace all corrupted blocks by zeros • 2. I frame: H.26L; P/B frame: overlapped region matching [21] • 3. I frame: split matching; P/B frame: forward-backward block matching [22] • 4. I frame: H.26L; P/B frame: neighbor matching [23] • 5. Proposed MV interpolation

19. Simulation results (5/8) • Football • Salesman • Processing time (10% PLR) 4 2 3 1

20. Simulation results (6/8) • Visual quality (15% PLR)

21. Simulation results (7/8) • Visual quality (15% PLR)

22. Simulation results (8/8) • Football • PLR = 15%

23. References [15] Z. Wang, Y. Yu, D. Zhang, Best neighborhood matching: an information loss restoration technique for block-based image coding systems, IEEE Trans. Image Process. 7 (7) (1998) 1056–1061. [21] M.C. Hong, L. Kondi, H. Scwab, A.K. Katsaggelos, Video error concealment techniques, Signal Process.: Image Commun. 14 (1999) 437–492. [22] S. Tsekeridou, I. Pitas, MPEG-2 error concealment based on block-matching principles, IEEE Trans. Circuits Syst. Video Technol. 10 (4) (2000) 646–658. [23] S. Valente, C. Dufour, F. Groliere, D. Snook, An efficient error concealment implementation for MPEG-4 video streams, IEEE Trans. Consumer Electron. 47 (3) (2001) 568–578. [25] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, The error concealment feature in the H.26L test model, in: Proceedings IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.