1 / 31

Internet-based interactive HDTV

Internet-based interactive HDTV. Bin Yu, Klara Nahrstedt Department of Computer Science University of Illinois at Urbana-Champaign ACM/Springer Multimedia Systems Journal, 9(5) March, 2004. Interactive HDTV – Traditional Approach. TV cable network + set-top box + TV set

esommers
Download Presentation

Internet-based interactive HDTV

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet-based interactive HDTV Bin Yu, Klara Nahrstedt Department of Computer Science University of Illinois at Urbana-Champaign ACM/Springer Multimedia Systems Journal, 9(5) March, 2004 DING Feng HT040297J

  2. Interactive HDTV – Traditional Approach • TV cable network + set-top box + TV set • High quality video delivery • Not compatible, proprietary format • Not scalable, all processing is done in the TV studio • Interactivity, flexibility and customization are not easy DING Feng HT040297J

  3. Interactive HDTV – Traditional Approach DING Feng HT040297J

  4. Interactive HDTV – Traditional Approach • PC + Internet • Easy interactivity, flexibility and customization • No high quality HDTV display • Current software solutions are inefficient DING Feng HT040297J

  5. HDControl – Authors’ Approach • Internet + PC + TV set • Combine good features of previous solutions • PC function as open set-top box • Standard video format: MPEG2 • Distributed architecture DING Feng HT040297J

  6. HDControl – Architecture DING Feng HT040297J

  7. Typical Application • Picture-in Picture (PiP) • Logo/ticker insertion • Email browsing • … DING Feng HT040297J

  8. Visual Information Embedding (VIE) • Naïve solution – not feasible • Decode the video stream into raw pixels • Overlay embedded content • Re-encode to MPEG2 stream DING Feng HT040297J

  9. MPEG2 Encoder Diagram – Recall DING Feng HT040297J

  10. Typical GOP Structure DING Feng HT040297J

  11. Visual Information Embedding • Inverse Variable Length Coding (VLC) converts bit-stream to Motion Compensation (MC) domain • MC contains motion vectors and prediction errors (residual) in quantized DCT format • Do MC to get reconstructed DCT (RD) domain data • Replace part of the image with embedded information • Re-encode data to MPEG2 format DING Feng HT040297J

  12. VIE • Wrong reference problem • MB2 use MB1 as a reference for prediction DING Feng HT040297J

  13. VIE • Fix reference • RD domain data of MB2, MB1 and MB0 are required • Worst case: IBBPBBPBBPBBPBB • Maximum search distance: 16 macroblocks • Potentially all the MBs in I and P frames need to be decoded for future use DING Feng HT040297J

  14. Two Types of Macroblocks • d-MBs • MB0 and MB1, their data should be decoded to RD domain for future reference • c-MBs • MB2, their reference blocks are wrong and their MC data have been changed DING Feng HT040297J

  15. Efficient Motion Compensation • Only those macroblocks surrounding the foreground window are affected by VIE (Chang et al. ) • Future motion prediction pattern is unknown • Completely reconstruct the reference frame DING Feng HT040297J

  16. Efficient Motion Compensation • Introduce delay • Buffering a GOP • All c-MBs can be detected by testing whether their reference MBs are in the foreground area • The reference MBs for both c-MBs and d-MBs are d-MBs DING Feng HT040297J

  17. Efficient Motion Compensation • Back tracking • c-MB -> d-MB -> … ->d-MB • Perform motion compensation for c-MBs and d-MBs to get RD domain data • Perform motion estimation for c-MBs to get their new motion vectors and prediction errors DING Feng HT040297J

  18. Optimizations • Bi-direction prediction -> uni-direction prediction • B frames has two reference frames • One reference frame is in the foreground area • Delete one motion vector DING Feng HT040297J

  19. Optimizations • Mark out sensitive area • Foreground window may only occupy a small area • Define c-sensitive area which may contains c-MBs; define d-sensitive are which may contains d-MBs. • For slices in insensitive area, copy them directly from input to output without decoding DING Feng HT040297J

  20. Optimizations • Shorten the delay • Select a shorter GOP size • Start back tracking process earlier • The sensitive area for the last P frame is mainly within the foreground window • IBBPBBPBBPBBPBB DING Feng HT040297J

  21. Resynchronization • MPEG2 • Decoding timestamp (DTS) and presentation timestamp (PTS) for each frame are determined by sender clock • Synchronization point (PCR) in multiplexed stream • Distance between PCR is constant DING Feng HT040297J

  22. De-multiplexing and Multiplexing DING Feng HT040297J

  23. Resynchronization • VIE problem • The frame size will be changed by VIE process • The constant spacing of PCR will be violated DING Feng HT040297J

  24. Two solutions • Simple padding • Pad Null packet in the empty space to keep the PCR positions • Can only handle bit-rate reduction • Waste bandwidth • Time-invariant bit-rate scaling • To scale the bit-rate to another constant value with scaling factor SF DING Feng HT040297J

  25. Determine SF • Measure the maximal ratio (R) in number of packets before and after VIE offline • Monitor the actual ratio (AR) online • New SF DING Feng HT040297J

  26. Evaluation • Experimental environment • HDTV testbed • Live high-definition digital TV stream from the satellite or storage device is fed into the server • Encode video into MPEG2 format and then multicast over LAN • Player PC decode the stream and send it to TV set • Video editing servants receive the stream and perform editing operation • Multicast result stream DING Feng HT040297J

  27. PiP FG: football_sd.mpg(480*256, 30fps) BG: trees1.mpg(1920*1088, 30fps) DING Feng HT040297J

  28. Evaluation • Distribution of d-MBs DING Feng HT040297J

  29. Comparison with the previous approach The number of MBs to be convert from MC domain to RD domain DING Feng HT040297J

  30. Resynchronization results DING Feng HT040297J

  31. Thank you! DING Feng HT040297J

More Related