School of computing science Simon Fraser University

1. School of computing scienceSimon Fraser University CMPT 820 : Error Mitigation Schaar and Chou,Multimedia over IP and Wireless Networks: Compression, Networking, and Systems Presented by: AzinDastpak

2. Outline • Motivation • Operational Encoder Control • Adaptive Intra Updates • Interactive Error Control • Mode 1:Acknowledged Reference Area Only • Mode 2:Synchronized Reference Frames • Mode 3:Regular Prediction with Limited Error Propagation • Mode 4:Unrestricted Reference Areas with Expected Distortion Update • Conclusion • Questions

3. Motivation • Error propagation is the major problem when transmitting MCP-coded video over lossy channels. • Encoder shall change its encoding behavior if it’s aware that decoder has experienced loss, which may sacrifice some compression efficiency.

4. Motivation a)no error robustness b)adaptive intra updates c)interactive error control

5. Operational Encoder Control • The encoder is responsible to select one of many encoding parameters • Motion vectors • MB modes • Quantization parameters • Reference frames • Spatial and temporal resolution • considering constraints • bit rate • encoding and transmission delay • complexity • Buffer size

6. Operational Encoder Control

7. Operational Encoder Control • Bad decisions in parameter selection may lead to poor results in: • Coding efficiency • Error resilience • Both..!

8. Operational Encoder Control – Compression Efficiency • The distortion of MB b coded with option m: • The coding mode selected for MB b:

9. Adaptive Intra Updates • In the presence of errors, introducing more intra-coded MBs is important. • Sequence characteristics and bit rate influence the appropriate percentage of intra updates. • Modify the selection of the coding modes to take into account the influence of lossy channels. • Replace the encoding distortion by decoder distortion.

10. Adaptive Intra Updates • The channel behavior is not deterministic and channel realization observed by the decoder is unknown • The receiver can send reports on the experienced loss and delays statistics for the encoder to compute expected distortion:

11. Interactive Error Control • In previous scenario only the statistics of the channel process C’ was known to the encoder. • In this scenario the -frame delayed version of the loss process experienced at the receiver is known at the encoder. • Sending Acks for correctly received data units • Sending Nacks for missing slices • Both • The goal is to reduce, limit or even completely avoid error propagation by integrating the decoder state information in the encoding process

12. Interactive Error Control • At encoder side, each generated data unit Pi is assigned a decoder state. • This information will be integrated in a modified rate-distortion optimized operational encoder control

13. Interactive Error Control • We have four different operation modes, differing only in set of coding option available at encoder, O’ and the applied distortion metric • Mode 1: Acknowledged Reference Area Only • Mode 2: Synchronized Reference Frames • Mode 3: Regular Prediction with Limited Error Propagation • Mode 4: Unrestricted Reference Areas with Expected Distortion Update.

14. Mode 1: Acknowledged Reference Area Only • Only the decoded representations of data units that have positively acknowledged at the encoder are allowed to be referenced in the encoding process. • Encoding options are restricted to acknowledged areas only, • If no reference area is available or no satisfying match is found in the accessible reference area, intra coding is applied.

15. Mode 1: Acknowledged Reference Area Only • Error may be visible in the presentation of a single frame but error propagation and reference frame mismatch are completely avoided.

16. Mode 1: Acknowledged Reference Area Only Average PSNR versus bit rate for feedback mode 1

17. Mode 2: Synchronized Reference Frames • In this mode not only positively acknowledged data units but also a concealed version data units with decoder state Nack are allowed to be referenced. • For the reference frames to be synchronized the encoder must apply exactly the same error concealment as the decoder.

18. Mode 2: Synchronized Reference Frames

19. Mode 2: Synchronized Reference Frames • The advantage of of mode 2 is for : • Low bit rates • Delays < Nref -1

20. Mode 3: Regular Prediction with Limited Error Propagation • Mode 1 and 2 are mainly suitable in cases of higher loss rates, when the loss rates are low the performance is degraded by longer prediction chains due to feedback delay. • In this case we only alter the prediction in the encoder in case of the reception of a NAK. In this case the reference area and option set will be changed to acknowledged and concealed image parts.

21. Mode 3: Regular Prediction with Limited Error Propagation

22. Mode 3: Regular Prediction with Limited Error Propagation • For delay=0 mode 2 and 3 are the same. • For increasing delays, mode 3 performs significantly worse than mode 2.

23. Mode 4: Unrestricted Reference Areas with Expected Distortion Updates • In this case the set of encoding options is not altered, that is O’=O. • Only for the data units with outstanding acknowledgement at the encoder, the randomness of the observed channel state is considered. For all other data units the observed channel state is no longer random.

24. Mode 4: Unrestricted Reference Areas with Expected Distortion Updates • Compared to mode 1 and 2 this method is beneficial if the feedback is significantly delayed. • Compared to mode 3, it reduces the unsatisfying performance in case of error propagation • For this mode turns to mode selection without any feedback at all • For this mode is identical to mode 2 and 3.

25. Conclusion • Error propagation is the most important issue in error-prone video transmission • If no feedback is available, increased percentage of intra MBs, selected by channel-adapted optimization schemes, performs best • If feedback is available, interactive error control can be applied • For short delays or low error rates: Modify the prediction only in case of Nack message reception. • In all other cases: Reference only those areas for which the encoder is sure that the decoder has exactly the same are

26. Questions?