Presents IMTC/Wainhouse Research European Forum 2003

1. Presents IMTC/Wainhouse Research European Forum 2003

2. Putting H.264 into Good Use:High-Resolution Video Conferencing Presented by Stephan Wenger TeleSuite Corp.

3. Outline • Where are we now? • What is feasible now? • A Few Tricks • A Real-World System • Cross-Codec Rate Control • MCU-less Continuous Presence • Summary and Outlook IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

4. Where Are We Now? Assumption: High-End, Professional Video-Conferencing • H.263+ or H.263++ (w/ Annex U for background restoration) • 384 kbit/s or 768 kbit/s • At these bit rates, and with cooperative users, few coding artifacts • TDM/ISDN (H.320) or IP (H.323) • CIF (352x288) or CIF-Interlace 2x(352x288) • Capture and rendering typically interlace • 25/30 fps, 50/60 fields per second • Round-trip delay around 400ms + • Round-trip delay for multipoint (w/ MCU) more than one second IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

5. And the Result is… IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

6. What is Feasible Now? Assumptions • Reasonable BOM, small number of DSPs • Full frame rate (25Hz / 30Hz) and progressive scan pictures What is Possible • H.264 video encoding (in a product design) • 640x480 at full frame rate on one high-end DSP • Limit: RAM bandwidth • Bit rate around 384 kbit/s yields acceptable picture quality • Subjective quality significantly better than H.263++ w/ CIF at same bit rate • Objective quality (PSNR) slightly lower • but keep in mind that the pixels are much smaller IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

7. A Few Tricks (Point-to-Point) Identified Problem: Low Quality, and High Delay • Progressive scan hardware throughout (no A/D and D/A losses) • Better quality and good delay characteristics • Interlace throughout also has good delay characteristics • 250ms round-trip delay over IP can be demonstrated • A fixed environment that discourages uncooperative use of the technology • E.g. fixed camera position, fixed background, studio lighting, seating • Improves quality by itself…… and saves cycles of the DSPs which can be spent elsewhere • Background restoration through ERPS is very efficient • Variable, content-dependent capture rate • Not just simple frame skipping • Requires picture segmentation beyond the usual MBs/Slices schemes IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

8. A Few Tricks (Multipoint) Identified Problems: Low Quality, and Very High Delay • Both a result of evil transcoding • Transcoding currently required for all “video mixing” applications • To avoid transcoding • Identical media coding schemes • Create all representations needed in the sending endpoint…… and simulcast them • Ideally suited: IP Multicast…… however, also possible with bridging infrastructure need some innovative changes in the control and mux protocols • Without transcoding, vastly improved user experience • No transcoding artifacts whatsoever • No additional end-to-end delay for multipoint when using IP-Multicast • Very little additional delay when IP Multicast is not available • 10 ms or so for packet/stream duplication IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

9. A Real-World System:2560 Camera/Codec Series • Video resolution 2560x480 • Screen size roughly 5x1 m2 , viewing distance minimal 1.5m • Pixel size 2x2 mm2 • IMAX-style experience, you stop watching a TV screen • Single CCD camera with custom, wide-angle lens • Progressive scan, programmable frame rate, CameraLink digital interface, low noise, operation point adjusted to studio light • Multiple DSPs, distributed codec • Codec processes four sub-images of 640x480 • Cross-codec rate control shifts bit rate to where it is needed • Each encoder can generate 640x480…… or 640x240, 320x240, 320x120, 160x120, 160x60 simultaneously • Carrier-grade, compact-PCI hardware architecture • Not cheap, but reliable • Connectivity 1.5 Mbit./s minimum, 3MBit/s typical IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

10. MVA Hardware IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

11. Fixed Environment IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

12. Cross-Codec Rate Control (1/4) Subjects are seated in discussion Data load impact per screen 25% 25% 25% 25% IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

13. Cross-Codec Rate Control (2/4) One participant stands Data load impact per screen 22% 34% 22% 22% IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

14. Cross-Codec Rate Control (3/4) Participant leaves the room Data load impact per screen 40% 20% 20% 20% IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

15. Cross-Codec Rate Control (4/4) Subjects continue their discussion Data load impact per screen 25% 25% 25% 25% IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

16. MCU-less Continuous Presence Effective meeting management with multi-point conferencing Point-to-point meeting IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

17. MCU-less Continuous Presence Effective meeting management with multi-point conferencing Multi-point meeting (3 locations) IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

18. MCU-less Continuous Presence Effective meeting management with multi-point conferencing Multi-point meeting (4 locations, only two camera capture fields used) IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

19. MCU-less Continuous Presence Effective meeting management with multi-point conferencing Multi-point meeting (6 locations) IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

20. And the Result is… TeleSuiteNetwork 50 – 120 hrs/month vs. Traditional “talking heads” utilization 0-5 hrs/month IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

21. Summary: Why is H.264 Essential for TeleSuite? • High compression makes bigger picture formats possible • MPEG-2 quality at 1/3 of the bit rate • ERPS allows for efficient, standard-compliant background restore • No visible “trailing artifacts” • This is useful primarily in a controlled environment • ERPS and FMO allow for good error resilience • Helpful for IP networks and especially IP Multicast • Arbitrary picture sizes commonly implemented • In contrast to H.263+ or MPEG-2 • H.264 is by far less complex than generally assumed • One document, few conformance points help reduce implementation complexity • Very DSP-friendly • Modern technology, marketing argument IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

22. Outlook • Even higher resolution • 1024 x 768 per field, 4096 x 768 per screen • 21 DSPs today, 13 DSPs of the 2004 generation • Target bit rate between 500kbit/s and 2 Mbit/s per field is realistic • At this resolution, the pixel structure is invisible • 1.3x1.3 mm2 pixels at 1.5m viewing distance • Provides unmatched, “beyond TV” experience • Matched audio performance • Surround-sound audio with appropriate echo canceling • Audio-zoning (especially difficult w/ more complex continuous presence scenarios) • Higher frame rates? • Does it make sense to go beyond 30 fps progressive? IMTC/Wainhouse Research European Forum – May 2003 – Geneva, Switzerland

23. Thank You Stephan WengerTeleSuite Corp. stewe@telesuite.com