Mixed Information types

1. Mixed Information types Multimedia Internet Communication Lecture 6

2. Multimedia ? • When various media types combined into single coherent object • Examples - text, database, stills, animation, graphic, video, audio • Communication links use channels which can be shared by different media types • Recent advances in computer/communications has significantly expanded possibilities for multimedia use • Multimedia not unique to computers - e.g. children’s story tapes/books

3. Multimedia examples: • Sound + Video • advs: familiar concept, effective • disadvs: familiarisation = contempt , …TV/cinema, passive etc. • digitisation: various transfer + synchronisation methods • Sound + Image • e.g. slide show + audio commentary/music • advs: portable, flexible, simple • disadvs: needs preplanning, lacks movement • digitisation: straightforwardtransfer + synchronisation not critical

4. Multimedia - more examples • Hardcopy Image + Text • e.g. picture album, brochure • advs: familiar, simple, portable • disadvs: lacks movement, content inflexible, copying (?) • digitisation: design important • Sound + Text • e.g. language learning pack • advs: easy to use, familiar • disadvs: selected replays difficult, lacks visuals • digitisation: more effective links+flexibility, easily added visuals

5. Selection of media • The examples given indicate the diversity available • Long history of traditional uses of mixed media types • Some mixtures and styles of mixing are more recent • Digitisation tends to present more options, but opens up more complex design issues (HCI) • Availability of information in a particular form often means inclusion but…… • Selection and design should reflect delivery purpose and user preferences/needs

6. Linkage between Media Types • Traditional links between media types used in an application tend to be retained in digitised formats • Use of single device (computer) often simplifies linking and synchronisation • Two forms of links • different types present for ‘automatic’ viewing • e.g. text with an inserted picture (in-line link) • explicit optional link from text to a musical soundtrack • external hyperlink

7. Synchronisation • Not particularly problematic - except for some cases of audio/video transmission • Even ‘off-the-shelf’ broadcast packages have difficulties with attaining ‘lip-synch’ quality • Audio can by ‘synched’ to the beginning of frames but constraints relating to channel capacity and hardware/ software affect performance • Audio and image/text synchronisation is much easier • image change(s) at specific point(s) in audio track

8. Multimedia Issues • Vary depending on application • Core issues: • access • how achieved • how regulated • cost restrictions • equipment constraints • awareness ? • Bandwidth conflicts ? • Timeliness • Confidentiality • Socialising

9. Another core issue - content • Content control: • regulatory control (law and professional code of conduct) • code of practice (self-imposed/conditions of access) • IRC/Chat Forums can reach over international boundaries • Local rules (e.g. Video conferencing/Closed Group forum, discussion board) • Parental control packages

10. Interface requirements • Computer/Internet based Multimedia is ‘open’ to many platforms and environments via standards • Standards can be open, proprietary, or from other sources • Some inconsistency • e.g Media player, RealPlayer, QuickTime • Min requirements can specify • Screen size • Colour depth – no. of bits • Sound sample processing - sampling rate – sample size • Video resolution, frame rate • Video capture/audio standard etc., • Internet connection/network

11. Usage implications (computer based) • Technical capability to achieve planned service needs to be considered • E.g video links need to have appropriate bandwidth and machine/peripheral availability • Cater for average and ‘peak’ activities • Specialist technical support • Testing and training • Costing

12. Standards • Many available • Streaming • Audio • Video • International standards • JPEG, MPEG, SMIL

13. SMIL • Based on XML • Mark-up elements for • Media types • Audio, Video, Text, Image • Synchronisation and sequencing • Parallel and sequential playback • Layout • Specified screen area use for individual media types

14. Example SMIL file (slideshow) <smil> <head> <layout type="text/smil-basic-layout"> <root-layout width="320" height="240" background‑color="black"/> <region id="title_region" left="0" top="0" width="320" height="240" /> <region id="pix_region" left="0" top="0" width="320" height="240" /> </layout/> </head> <body> <par> <seq> <text src="example1.rt" region="title_region"/> <img src="example1.rp" region="pix_region" fill="freeze"/> </seq> <audio src="example1.rm"/> </par> </body> </smil>

15. Example file details • File created by Real Slideshow using • .rt file - various text elements • .rp - a sequence of pictures • .rm - encoded audio file • Other tools create different styles • Basic elements stay the same

16. Summary • Mixed media is not new • Educational use for decades • Computers make it more useable • Easier to construct and maintain • Streaming allows most mixed media to be delivered easily over the Internet • Broadband allows greater use of video

17. Multimedia on the web • Many sites available to demonstrate streamed multimedia • See assessment • Some academic literature available via IEEE digital library at • http://www.computer.org/multimedia/