VIDOS: Web-based Video Editing & Customization Service

1. The development of VIDOS:a Web-based video editing, customization and repurposing service

2. Personal computing at present • Desktop-centred • Requires multiple individual copies of all applications, and multiple backup facilities • Hardware and software become rapidly outdated and are expensive to replace • Problems with application and version incompatibility between colleagues • Limited use of the network for co-operative computing over long distances • Network bandwidth inadequate for real-time viewing of videos or high quality video conferencing • Use of mass-produced standardized media entities

3. The new paradigm for personal computing • Network-centred • Single copy of most recent version of application on powerful central server • Simple and cost-effective • Enhances inter-personal co-operation CENTRAL NETWORKED SERVERS • Requires advanced infrastructure with high bandwidth network • Permits effective use of central databases and archives • Real-time video viewing possible • Permits customization of personal versions of media entities

4. Two examples of network-centred computing • The BioImage Database, an on-line database for multi-dimensional images and videos of biological specimens • VIDOS, a Web-based video editing, customization and repurposing service www.bioimage.org www.vidos.ac.uk

5. Outline of my presentation • The VIDOS prototype • Current VIDOS developments • A VIDOS demonstration • The future of VIDOS: • Improving the user interface and spatial editing – the J-Video player • Integration with IVANA for semantic content analysis • Continuing the VIDOS service as part of VideoWorks for the Grid • Integration with video databases • The SONGBIRD project for semantics and ontology generation

6. The VIDOS prototype

7. Schematic diagram of the prototype VIDOS system

8. VIDOSa Web-based video customization system • Edit a digital video spatially and temporally over the Web • Adapts the video content, size, format and compression quality to its intended purpose, or to match the capabilities of one’s PC • The video may be in a VIDOS-enabled database or anywhere else on the Web, including the user’s own Web server • VIDOS gives the user basic video editing capabilities without the need to purchase expensive video editing software • By reducing the size of digital video files, VIDOS can improve efficiency by speeding network transfers, reducing bandwidth bottlenecks, and minimizing disc storage

9. Conference presentation Terena Networking Conference Lisbon, 22-25 May 2000 Session 6A, ‘Coming soon to a monitor near you’ Chair: Egon Verharen VIDOS - a system for editing and customizing videos over the Web David M. Shotton and Thomas Boudier Department of Zoology University of Oxford, UK

10. Journal publication Boudier and Shotton (2000) Computer Networks 34: 931-944

11. The current development of VIDOS

12. The limitations of the VIDOS prototype • Underlying lack of flexibility in the software implementation • Limited video format and codec choice, determined by the outdated underlying Silicon Graphics digital media library employed for video format trans-coding • User interface showed only first and last frames, making temporal editing difficult on videos with scene changes • Inability to splice together different video clips

13. The VIDOS second phase - objectives • To rewrite VIDOS from scratch, to provide a more efficient and extendable software implementation and overcome the limitations of the prototype • To enhance VIDOS functionality by expanding the choice of video output formats, and by updating the choice of codecs to include those using the most modern compression algorithms • To enhance the user interface for temporal editing • To permit the splicing together of separate video clips to create a single new output video file • To launch a VIDOS service for academia via the VIDOS web site

14. The VIDOS Home Page

15. The VIDOS second phase –architecture The VIDOS hardware configuration: • A VIDOS Web server – VIDOS slave processor farm topology The VIDOS Web server: • To undertake user authentication • To serve Web pages and Java applets, and receive parameters • To commission jobs for the slave processors to implement • To control the video filestore and send customized videos to users An expandable farm of slave processors: • To undertake individual video conversion tasks

16. Current VIDOS architecture

17. VIDOS functionality

18. The GALLOP test video Original video details: QuickTime Format M-JPEG compressed Compression quality 0.6 One stride of 15 frames repeated six times.giving 90 frames at 30 fps File size: 3,731 Kb Preview video details: Highly compressed AVI preview Zoom factor 50% Compressed file size: 420 Kb Credits Photographer: Eaduard Muybridge, 1878 Racehorse: Annie, a Californian thoroughbred Jockey: unknown

19. The VIDOS input video selection interface

20. The VIDOS output format selection interface

21. The VIDOS spatio-temporal editing interface

22. The VIDOS spatio-temporal editing interface

23. The VIDOS conversion request interface

24. The VIDOS conversion request interface

25. The VIDOS conversion request interface

26. The final VIDOS- customized video Original video: QuickTime format, M-JPEG compressed at quality 0.6, 90 frames (6 strides) at 30 fps File size 3,731 Kbytes Customized video: Selected area: Jockey and horse’s head and shoulders only, 50% zoom Selected time: first 45 frames only (three strides) at 10 fps Selected output format: AVI format, Indeo compressed at quality 0.3 File size 91 Kbytes Compressed file approximately 41 times smaller than original

27. Let’s try a demo ! Start VIDOS

28. The benefits of VIDOS • For individuals and academic institutions, VIDOS provides: • enhanced download times and reduced video storage requirements, • free video editing, customization and format conversion facilities, • reduced hardware and software costs, and • enhanced personal and corporate efficiency • For distributed organizations (e.g. a university with regional campuses): • a system for distributed client-based video editing from a centralized server • In education and training: • the ability to select and repurpose video clips from a central resource for inclusion in live lectures or computer-based training programs • For academic video databases and video content providers: • added value by enabling users to edit and customize selected videos • particularly powerful if coupled with a Query-by-Content system

29. Registration of VIDOS Users • We are now seeking VIDOS Users who, as well as using VIDOS for their own productive work, can evaluate VIDOS and advise us on functionality enhancements • If you would like to register as a VIDOS User, please log on to www.vidos.ac.uk, and complete the Registration Form that you will find under the Registration button. • You will then be sent a password that will give you unrestricted access to VIDOS from any computer

30. Functional developments of VIDOS Further developments of VIDOS functionality will require a break from our previous philosophy of enabling VIDOS to run on any modern browser platform, and will instead require that users first install from the Sun Web site the latest versions of Java and Java Media Framework. This will enable us: • To further enhance the functionality of VIDOS for temporal editing of videos, by incorporating • J-Video, our new Java video player that uses Java Media Framework and permits the splicing together of separate video clips • IVANA(Interactive Video ANalysis Application) for semantic content analysis • To integrate VIDOS with our other video tools

31. CTL-mediated target cell death

32. The J-Video java video player

33. The J-Video java video player

34. The J-Video java video player

35. IVANA for semantic content analysis

36. Semantic content analysis classes • Video A temporal sequence of scenes • Segment A distinct sequence of video frames • Frame A single video frame • Character Animate characters (e.g. cells, racing drivers) • Object Inanimate objects (e.g. pipettes, racing cars) • BackgroundRegion of image not occupied by objects, etc. • Event An instantaneous or extended happening involving one or more characters or objects • Sound An item on the video soundtrack • Area of interest A geometric area enclosing items of interest • Annotation A textual label relating to a character or object • Insert A separate image, animation or movie insert

37. Query by Content using VANQUIS • We now plan to combine • J-Video for client video manipulation, • IVANA(Interactive Video ANalysis Application) for interactive video content analysis, and our alternative • VAMPORT (Video Analysis and Metadata Production by Object Recognition and Tracking) system for automated analysis of simple videos, with • VideoStore, our semantic content metadata database, and a • Query by Content interface, to create • VANQUIS (Video Analysis and QUery Interface System) • This will enables users to make such queries as: • “Find me a sequence showing Lord Jenkins” • “Find me the times of all target cell deaths that occur in the upper left corner of the screen” • “Which CTL are serial killers? Show me them at work!”

38. The future of VIDOS as part of VideoWorks for the Grid, an Oxord University e-Science Centre testbed project

39. The VIDEOWORKS partnership • VideoWorks for the Grid is a new umbrella project for these video e-services, launched in the context of the current UK Grid initiatives • It is the first testbed project of the Oxford e-Science Centre • Our VideoWorks industrial partners are • IBM • Virage • Telestream • Square Box Systems, and • The International DOI Foundation

40. Functionality of VideoWorks • Central to the video e-services within VideoWorks will be the on-line video editing and customization services of VIDOS • The same Java software structure and server-slave protocols as presently developed for VIDOS will be expanded for VideoWorks • We will use VANQUIS for interactive video semantic content analysis • VANQUIS semantic metadata will be stored a VideoStore database modelled upon the BioImage Database • We will use • Telestream’s FlipFactory for efficient video transcoding • Virage’s VideoLogger for automated structural video analysis • IBM’s Video Charger for streaming preview videos for editing and analysis

41. TheVideo-WorksfortheGrid system

42. Links from VideoWorks to video databases • Links will be made to major academic video databases such as BioImage and the British Film and Video Council’s MAAS Media Online • User of these databases can upload videos of interest to VideoWorks either for semantic content analysis using VANQUIS or for editing and customization using VIDOS, before downloading • After semantic content analysis, the semantic content metadata stored within VideoStore will be available for subsequent query by content using the VANQUIS query interface. Videos located by QbC may also be customized using VIDOS before downloading

43. From image to knowledge:the state of the art of image bioinformatics Shotton, D. M. (Nov 2000) Microscopy and Analysis 80: 23-25. In that paper I said that harvesting knowledge from digital videos would require five components: • a searchable image database - The BioImage Database • an automatic video content analysis system for simple videos - VAMPORT • an interactive video analysis system - IVANA • a query by content system – VANQUIS • a web-based video customization system - VIDOS Combined together, these form a unique and powerful combination to facilitate the conversion of raw video data into knowledge. VideoWorks is the realization of that vision.

44. Thomas BoudierMy research assistant1997-1999Author of the VIDOS prototype John PybusVIDOS Research and Development ProgrammerHas written VIDOS 3.0 Danny TorbicaVIDOS Project Development Manager Mac expert and Webmaster Jamie ShottonComputer Science Student and VIDOS Java Programmer Has written our Java video player and semantic content analysis module