WLAP: Improving acquisition

Workshop on digital video archiving 22 June 2001, CERN WLAP: Improving acquisition Hector Sanchez San Martin Universitat Jaume I Ing. Informatica CERN ETT-DH

Background introduction WLAP@CERN: Status report Possible improvements for the acquisition Unsolved problems & future work Contents Hector Sanchez 22 June 2001 @ CERN

Lots of courses, seminars and tutorials are given at CERN WLAP: Web Lecture Archive Project Web Lectures (WL) Background introduction Need for keeping an archive Internet allows making them world wide available Started at CERN in 1999 Aim: Implement an e-archival system for slide based lectures on the Internet Made of video, slides and timing information Nowadays: Set of HTML files produced by SyncOmat Future: Standard for describing WL Lecture Object Hector Sanchez 22 June 2001 @ CERN

For having a persistent archive of WL is necessary to keep the data in a format which allows to obtain any desired output We will focus on the acquisition of data Which are the current needs? WLAP@CERN A time persistent archive Don’t keep WLs but “raw” data (video, slides, timing) The way of storing this data has to be consistent We will be able to generate any kind of output: syncOmat, lecture objects, only video,... Need of a real system of WL production Current archive is not scalable (syncOmat WL are static) Acquiring data is the first step towards an archive Hector Sanchez 22 June 2001 @ CERN

Web Main Page Bulletin WLAPer Speaker Organiser Contact the speaker* Contact the speaker Publish Info Lecture Give transparencies Obtain lecture info Digitalise video Record lecture Produce WL Scan slides Publish WL Edit video Timing WLAP@CERN: Initial scenario (I) ... Hector Sanchez 22 June 2001 @ CERN

The process is carried out in a quite manual way Some tasks are performed more than once Dependence on one person: WLAPer Actors making technical tasks in which they are not specialised WLAP@CERN: Initial scenario (II) Some tools have been developed in the production part Acquisition is too heavy Contacting the speaker, publishing info about the lecture,… Contacting the speaker, recording, making web pages… Loss of quality in the final product Hector Sanchez 22 June 2001 @ CERN

Scanning service Web Main Page Bulletin WLAPer Speaker Organiser Video service Record lecture & timing Contact the speaker* Contact the speaker Publish Info Lecture Give transparencies Obtain lecture info Digitalise video Produce WL Publish WL Scan slides Edit video WLAP@CERN: Current scenario (I) ... Hector Sanchez 22 June 2001 @ CERN

STILL: Manual process, repeated tasks New techniques tested and currently in production More people involved means more channels of communication Process is more distributed Specialised professionals dealing with technical tasks WLAP@CERN: Current scenario (II) Video & scanning Better quality “raw” data STILL dependence on WLAPer to archive “raw” data The way for transmitting information between the actors is manual This could make the process more inaccurate and slower Hector Sanchez 22 June 2001 @ CERN

Video service WLAPer Speaker Organizer Scanning service Bulletin Web Main Page Introduce Lecture Info Submit transparencies Contact the speaker Publish Info Lecture Obtain lecture info Record lecture Digitalis video Produce WL Scan slides Publish WL Edit video Timing WLAP@CERN: Future scenario (I) ? ... Hector Sanchez 22 June 2001 @ CERN

The archival system is automatic and transparent to actors The system is autonomous Some work is needed to achieve this situation WLAP@CERN: Future scenario (II) Actors only do their work informing the system about the lecture The system will archive the files as necessary Ex: WLAPer will only notice that all files concerning a lecture are avaliable Communication between actors is carried on inside the system No need for human co-ordination Re-organising the process Develop SW for supporting it Hector Sanchez 22 June 2001 @ CERN

Notice Lecture data Transp. Mail Organiser Speaker Video service Scanning service Creates a bibliographic record E-documents CDS Agenda Maker CDS Submit Upload script Upload script Lecture data Video [Timing] Lecture Id & Scanned trasp. Webcast FS Library DB Doc FS Link Manager WLAP@CERN: How? *CDS Search *Bulletin *Organiser web page *Webcast pages ... Hector Sanchez 22 June 2001 @ CERN

Once the video of the lecture is encoded it needs to be edited The timing file needs to be resynchronised This two adjusts need human intervention The main idea is taking some data during the recording of the lecture that allows to do the editing automatically Editing the video: The basis Adjust it to the real duration of the lecture Normally the timing tool is started before the lecture starts Hector Sanchez 22 June 2001 @ CERN

Starts live encoding Starts recording tape Starts timing Starts lecture Ends lecture Ends recording tape Ends live encoding First slide change Tse 10:50:00 Tsl 11:00:00 Tel 12:00:00 00:00:00 00:10:00 01:10:00 Between 00:10:00 and 01:10:00 Tsl-Tse Tel-Tse Editing the video: Example time How has the video to be cut? Hector Sanchez 22 June 2001 @ CERN

Lecture Data & events Lecture recorder File with global times Taketime Editvideo Webcast FS Edited video Editing the video: How? Script that will run in batch Hector Sanchez 22 June 2001 @ CERN

This can be used also to synchronise the timing file Technology used: Python+Tk (tkinter) Some tests have been made but more testing is needed Very easy to use: No need for a computer specialist Editing the video: Final notes We could reduce human intervention to do some clicks Can be run in different platforms: Win, UNIX, MAC Hector Sanchez 22 June 2001 @ CERN

The archival system is based on the Talk Number Id The process has to be transparent for people involved This is the sw support for achieve an automated archive of raw data Upload scripts: The basis Numbers are always difficult to handle Transfer of files to servers Lecture Id vs. Lecture data It’s a kind of interface for archive “raw” data The user selects the lecture his material is related to The script will obtain the talk number and archive the files Hector Sanchez 22 June 2001 @ CERN

Upload script for encoding&uploading video User Organiser mail MySQL Db pymod ftplib pymod smtplib pymod encoded video Encoded video GUI CORE notification Webcast FS Agenda DB Config File Lectures data & id RealProducer MySQL Client Lecture data Upload scripts: Architecture Hector Sanchez 22 June 2001 @ CERN

Technology used: Python+Tk (tkinter) + MySQL This will be extended to other parts of the process First version of the script will be tested next week Almost automatic publication of videos Upload scripts: Final notes Submission of scanned transparencies Submission of timing files The user only selects the lecture to record The video of the lecture will be available a few minutes after it is finished Ready for Summer Student lectures For now ASCII interface Hector Sanchez 22 June 2001 @ CERN

Timing acquisition Agreement between people involved Master copy of the digital video: MPEG??? Better tool integration Unsolved problems & future development When not using MS PWPT another person is needed Very high quality video file which can be converted to any format RealMedia is not the better master format A lot of different tools could be integrated (timing, encoding, editing,…) Hector Sanchez 22 June 2001 @ CERN

WLAP: Improving acquisition