1 / 44

Error Control and Concealment for Video Communication

Error Control and Concealment for Video Communication. CMPT820 Summer 2008 Michael Jia. Reference. “Error Control and Concealment for Video Communication : A Review”. YAO WANG, Member, IEEE QIN-FAN ZHU, Member, IEEE. Outline. Introduction Error Detection Error Concealment at Coder

platt
Download Presentation

Error Control and Concealment for Video Communication

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Error Control and Concealment for Video Communication CMPT820 Summer 2008 Michael Jia

  2. Reference “Error Control and Concealment for Video Communication : A Review” YAO WANG, Member, IEEEQIN-FAN ZHU, Member, IEEE

  3. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  4. Introduction • 2 Types of Transmission Errors • Random Bit Errors • Bit inversion, bit insertion, bit deletion • Erasure Errors • Packet loss, burst errors, system failures • More destructive • Common usage of VLC makes them no differences in video streaming

  5. Introduction • Lossless recovery • FEC (Forward Error Correction) • ECC (Error Control Coding) • ARQ (Automatic Retransmission Request) • Not necessary for video transmission, human eyes can tolerate a certain degree of distortion • Focus on signal-reconstruction and error-concealment techniques

  6. Introduction • Typical structure of a video communication system

  7. Introduction • 3 groups of error-concealment techniques • Forward error concealment • Encoder plays the primary role • Error concealment by post-processing • Decoder fulfills the task • Interactive error concealment • Main concerns • Effectiveness • Required delay • Bit-rate overhead • Processing complexity

  8. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  9. Error Detection • At transport layer • Adding header information (sequence number) • H.223 • FEC (Forward Error Correction) • H.223, H.261 • More reliable • Need more bandwidth

  10. Error Detection • At decoder • Based on characteristics of natural video signals • Pixel value differences of neighboring lines (compare to a threshold) • Differences between boundary pixels in a block and its four neighbor blocks • Obvious false value of quantization step size or DCT coefficients • Insert synchronization code word at the end of line of blocks • No additional bits or very few • Rely on smoothness of signal

  11. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  12. Error Concealment at Coder (1) • Layered Coding with Transport Prioritization • Most popular and effective (MPEG-2)

  13. Error Concealment at Coder (1) • Layered Coding • Base layer – most important layer, with acceptable quality • Transport prioritization • Deliver base layer with higher degree of error protection • High priority channel • Re-transmission and/or FEC • No explicit bit-rate overhead • Complicate structure and coding overhead • H.264 AVC/SVC • Redundant pictures • Data Partitioning

  14. Error Concealment at Coder (2) • Multiple-Description Coding (MDC) • Several parallel channels • Independent error events • Small probability of “all channels down” • Multiple “Descriptions” • Several coded bit streams • Transmitted over separate channels • Any one will work

  15. Error Concealment at Coder (2) • Multiple-Description Coding (MDC)

  16. Error Concealment at Coder (3) • Joint Source and Channel Coding • Source-channel interaction at a lower level • Given channel error characteristics, design quantizer and entropy coder for to minimize the effect of errors • For general sources, noisy channel  coarse quantizer is better • For image signals, noisy channel  • fewer bits to high-frequency coefficients • more bits to low-frequency coefficients

  17. Error Concealment at Coder (4) • Robust Waveform Coding • Intentionally keep some redundancy in source-coding • Layered coding and MDC both belong to this category • Adding auxiliary information in waveform coder • MPEG-2: sending motion vectors for microblocks in I-frames • Restricting prediction domain • H.263/H.264: prediction is confined within each slice

  18. Error Concealment at Coder (5) • Robust Entropy Coding • Add redundancy in entropy-coding • To help detect bit errors and prevent error propagation • Self-Synchronizing entropy coding • Add a synchronization code word • H.261, H.263, MPEG-4 • Error-Resilient entropy coding (EREC) • MPEG-4 uses RVLC (reversible VLC)

  19. Error Concealment at Coder (7) • Transport-Level Control • Add redundancies at transport level • Prioritized transport for layered coding • Robust packetization • Spatial block interleaving • Dual transmission of important information • H.264 AVC/SVC • NAL unit syntax structure • Parameter Sets

  20. Error Concealment at Coder (Summary)

  21. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  22. Error Concealment at Decoder • Preview • Perform error concealment at the decoder • Can be used in conjunction with the auxiliary information provided by the source coder • Low frequency components dominate images of natural scenes • Color values of adjacent pixels vary smoothly except sharp edges • Human eyes tolerate more distortion to high-frequency components

  23. Error Concealment at Decoder (1) • Motion-Compensated Temporal Prediction • Replace damaged MB with the motion compensated block • Very effective when all the motion information in the base layer • Widely used (MPEG-2) • What if motion information or coding mode is lost? • Will discuss in next slide

  24. Error Concealment at Decoder (2) • Recovery of Motion Vectors and Coding Modes • What if we lost motion vectors or coding modes? • Interpolated from spatially and temporally adjacent blocks • Estimation of coding modes • Simple: treat as ‘intracoded’ • More sophisticate: MPEG-2 (See tables)

  25. Error Concealment at Decoder (2) • Estimation of motion vectors • Set to zeros (works well for low motion video) • Use MV of the corresponding block in the previous frame • Use the average of MVs from spatially adjacent blocks • Use the median of MVs from spatially adjacent blocks • Select one of the above methods depending on least boundary matching error

  26. Error Concealment at Decoder (3) • Maximally Smooth Recovery • A constrained energy minimization approach • Minimize a measure of spatial and temporal variation between adjacent pixels in this block and its spatially and temporally neighboring blocks • Measure differences in 3 domains • Spatial – adjacent blocks • Temporal – prediction block in previous frame • Frequency – received coefficients for this block

  27. Error Concealment at Decoder (3) • Maximally Smooth Recovery • 2 sample spatial smoothness measures

  28. Error Concealment at Decoder (4) • Spatial- and Frequency-Domain Interpolation • A coefficient in a damaged block is likely to be close to the corresponding coefficients in spatially adjacent blocks • Interpolate from four neighbor blocks • Maybe not accurate (8-pixel is too far) • Interpolate from four 1-pixel-wide boundaries • 2 pixels in 2 nearest boundaries • 4 pixels in all 4 boundaries • See picture in next slide

  29. Error Concealment at Decoder (4) • Spatial- and Frequency-Domain Interpolation • Interpolate from four 1-pixel-wide boundaries

  30. Error Concealment at Decoder (Summary)

  31. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  32. Interactive Error Concealment • Preview • If a backward channel is available, can achieve better performance by cooperation • Based on the feedback • At source coder – coding parameters can be adapted • At transport level – adjust the portion of bandwidth used for error control • Decoding delay issue

  33. Interactive Error Concealment (1) • Selective Encoding for Error Concealment • Simple – code next frame in intramode • Error stopped in about one round-trip time • Will cause bit-rate increase • Send identity info back, perform error concealment at the same time • Continue encode without using the affected area • Perform same error concealment procedure (need a prediction frame buffer) • See picture in next slide

  34. Interactive Error Concealment (1) • Selective Encoding for Error Concealment

  35. Interactive Error Concealment (1) • Selective Encoding for Error Concealment

  36. Interactive Error Concealment (1) • H.263 uses more prediction frame buffers (reference picture selection mode)

  37. Interactive Error Concealment (2) • Adaptive Transport for Error Concealment • Retransmission is unacceptable for real-time video applications? • Not always • For one-way video broadcast, we may tolerate a few seconds delay • For multipoint video conferencing, use MCU (multipoint control unit) • If retransmission is controlled properly, end-to-end quality can be improved • Both H.323 and H.324 defined such mechanisms

  38. Interactive Error Concealment (3) • Retransmission Without Waiting • Wait for the retransmission data • Not good, may freeze the display • Cause transit delay and accumulation delay • Without waiting • Request the retransmission • Conceal the error • Track the affected pixels • Correct them upon the arrival of the retransmission data

  39. Interactive Error Concealment (4) • Prioritized, Multi-copy Retransmission • Effective in very lossy channels • Video streaming via PSTN • Are you kidding me? • Send multiple copies of a lost packet • Use layered coding • # of retransmission trials and # of copies are proportional to the importance of the layer

  40. Interactive Error Concealment (Summary)

  41. Outline • Introduction • Error Detection • Error Concealment at Coder • Error Concealment at Decoder • Interactive Error Concealment • Conclusion

  42. Conclusion • Real-time video communication doesn’t require lossless delivery; signal-reconstruction and error-concealment techniques are more effective. • Add redundancy when encoding or delivering • Estimate missing information when decoding • Inform sender what is lost • Burstiness has a significant impact on the choice of algorithms

  43. Questions?

  44. Thank You

More Related