An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet

1. 6th IT Excellence Awards (Post Secondary) 2004/06/09 An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith Video over InternEt and Wireless Technologies Lab., CUHK

2. Video over InternEt and Wireless Technologies Laboratory • VIEWLab is a research and application development laboratory • Computer Science and Engineering, CUHK • Prof. Michael R. Lyu • Focus on • Video Information Processing Technologies • Mobile and Wireless Application Development

3. Presentation Outline • Motivation • Functionality & Design • Demonstration • Innovation • Prospect for future commercialization • Q&A

4. Motivation • Needs of mobile video content is increasing • Video information processing technologies developed • A prototype system is created to automate the video authoring process • Enabling faster and cheaper mobile video entertainment

5. Functionality & Design • An automatic video information processing and conversion system for MMS and Mobile Internet • Targets • Fully automation • Mechanical processing with pre-defined knowledge • Various video content type • Short processing time • Configurable for different needs

6. Advertisement Removal Program Identification Scene Analysis Overview of System TV Broadcast MMS Component Creation MMS (Pictures + Sound) / Mobile Video Video Streaming / Progressive Download Mobile Internet Video Clip Conversion

7. Video Information Processing • These are the technologies to extract information from video using visual and audio channels • By extracting/identifying the information, “effective” condensation/summary techniques can be applied through information fusion or recognition steps • We define a type of information a “modality” • Transcript/Speech • On-screen characters • Face • Visual features

8. Every Bit Counts • MMS size is limited to within hundred KB • In order to delivery “useful” information to viewer • Condensation/summary techniques are required to locate/prioritize the most important bits • Bandwidth cost is relatively higher than other channels • the “value” of the content transmitted is curial to successfully MMS business • Utilize the “value” of the bandwidth use, only transmitting the important bits

9. Main Features of Video2MMS • Advertisement removal • TV program identification • Video condensation/summary • MMS creation and adaptation for different mobile handsets

10. Advertisement Removal • Based on the logo detection techniques developed, TV program and non TV program segmentscan be separated automatically TV Broadcast Non Program Program Non Program

11. TV Program Identification • Based on the visual features and audio characteristics, the desire TV program can be identified

12. Video Summary Video Condensation / Summary • Condense the long video into short slide show presentation • Segment the video into different scene • Two pass minimum difference color histogram • Choose the most representative shots • Video editing style • Longest scene

13. MMS Creation and Adaptation • Different handsets have different MMS size and screen size • Nokia 7650, 3650 - 100KB, 176 x 136 • SonyEricsson T68i, T300 – 30KB, 101 x 80 • Panasonic GD88 – 45KB, 176 x 136 • Convert the pictures to correct screen size • Utilize the MMS size for most information (number of pictures and duration) • Configurable for new handsets

14. MMS Matrix MMS Size = audio size + (picture size * n) Audio size = length of audio * 0.8k, (4.75 bit-rate) Picture size = f(screen size, number of colors)

15. Mobile Internet Video Clips Conversion • Control third party modules to cut and convert video into short video clips • Different handsets have different MMS size and screen size • 3GPP format • RealVideo format • Microsoft Windows Media Format • Clips size configurable for different usage • 100KB for MMS delivery • >100KB for video streaming

16. Demonstration Video (1) ATV News Headline AMR + Animated Gif Nokia 3650 100KB 20 seconds 174 x 136 128 colors Animated Gif (6 pic.) AMR Audio (click to play)

17. Demonstration Video (2) ATV News Headline 3GP for Mobile Internet Nokia 3650 195KB 20 seconds 174 x 136 256 colors (click to play)

18. Demonstration Video (3) ATV Weather Report AMR + Animated Gif Nokia 3650 100KB 60 seconds 174 x 136 128 colors Animated Gif (5 pic.) AMR Audio (click to play)

19. Demonstration Video (4) ATV Hourly News AMR + Animated Gif Nokia 3650 100KB 40 seconds 174 x 136 128 colors Animated Gif (6 pic.) AMR Audio (click to play)

20. Demonstration Video (5) Horse Racing 3GP in MMS Nokia 3650 95KB 20 seconds 174 x 136 (click to play)

21. Demonstration Video (6) Horse Racing RealVideo in MMS Nokia 3650 95KB 20 seconds 120 x 96 (click to play)

22. Demonstration Video (7) Soccer Goal Shot RealVideo in MMS Nokia 3650 92KB 5 seconds 120 x 96 (click to play)

23. System Demonstration Main Panel Modules Editor Task Profile Task Scheduler

24. Innovation • We believed • this is the first application • applied Video Information Processing technologies • on automating the content preparation processes • for mobile handset in Hong Kong

25. Prospect for future commercialization • Base on this prototype, our laboratory technology commercialization team developed a production system called Video2MMS™ • Video2MMS™ was licensed to HKCSL in spring 2003 to prepare daily news and weather video clips and MMS • Press release • http://www.hkcsl.com/show.jsp?pid=4&cnid1=407&type1=1&rhtID1=3&lid=2

26. Tested Handsets • MMS • Nokia 7560, 3650 • SonyEricssion T68i, T300 • Panasonic GD88 • VideoClips (Download) • MS PocketPC, XDA

27. Recommended System Requirements • Intel P4 1.0G Hz • 512MB RAM • Microsoft Windows 2000 or above • DirectX 8.1 • TV Capture Card (MPEG-1) • Optional third party software • Real Encoder • Nokia 3GP Encoder • Sony Ericsson AMR Convertor • Microsoft Windows Media Encoder

28. Q&A