Comp311/Comp511

1. Comp311/Comp511 Lecture01 - Introduction

2. Unit Plan • Text Book: Fundamentals of Multimedia by Ze-Nian Li and Mark S. Drew • Theme - ”Platform independent multimedia” • What we will not do: – How to become computer graphic artists! – How to drive commercial multimedia suits!

3. Contents • Foundational aspects: Our study about this is minimal but there is five lectures covering • Human Vision and Colour models. • Image data representations. • Data compression. • Transform coding. • Video compression.

4. Contents • Rest of the lectures are on programming aspects including • Java Media APIs (Java2D, JAI, JMF, little bit of Java3D) • SMIL – Synchronized Multimedia Integration language • A XML based language. • VRML/X3D – Virtual Reality Markup Language • X3D is the current XML based VRML specification • Basic Java Game programming.

5. Assessment • Assessment: 60% Assignments and 40% Exam • Must obtain at least 50% from each component to pass the unit. • There are 3 assignments covering programming and multimedia development. • The first assignment will be released today. • Lecturer: Ashoka Jayawardena • Room 171 • Extension: 2267 • Tutor: TBA

6. Multimedia • Multimedia involves multiple modalities of text, audio, images, drawings, animation, and video.

7. Applications • Video teleconferencing. • Distributed lectures for higher education. • Tele-medicine. • Co-operative work environments. • Searching in (very) large video and image databases for target visual objects. • etc.

8. Hypermedia and Multimedia • A hypertext system: meant to be read nonlinearly, by following links that point to other parts of the document, or to other documents. • Hypermedia: not constrained to be text-based, can include other media, e.g., graphics, images, and especially the continuous media – sound and video: The World Wide Web (WWW) – the best example of a hypermedia application. • Multimedia means that computer information can be represented through audio, graphics, images, video, and animation in addition to traditional media.

9. Multimedia Software Tools • Music Sequencing and Notation • Digital Audio • Graphics and Image Editing • Video Editing • Animation • Multimedia Authoring

10. Music Sequencing and Notation • Cakewalk: now called Pro Audio. • The term sequencer comes from older devices that stored sequences of notes (events, in MIDI). • MIDI – Musical Instrument Digital Interface. • MIDI is a protocol adopted by electronic music industry that enables computers, synthesizers, keyboards, and other musical devices to communicate with each other. • It is a scripting language that codes events that stand for the production of certain sounds. • It is also possible to insert WAV files and Windows MCI commands (for animation and video) into music tracks (MCI is a ubiquitous component of the Windows API.) • Cubase: another sequencing/editing program, with capabilities similar to those of Cakewalk. It includes some digital audio editing tools. • Macromedia Soundedit: mature program for creating audio for multimedia projects and the web that integrates well with other Macromedia products such as Flash and Director.

11. Digital Audio • Digital Audio tools deal with accessing and editing the actual sampled sounds that make up audio: • Cool Edit: a very powerful and popular digital audio toolkit; emulates a professional audio studio —multi-track productions and sound file editing including digital signal processing effects. • Sound Forge: a sophisticated PC-based program for editing audio WAV files. • Pro Tools: a high-end integrated audio production and editing environment — MIDI creation and manipulation; powerful audio mixing, recording, and editing software.

12. Graphics and Image Editing • Adobe Illustrator: a powerful publishing tool from Adobe. Uses vector graphics; graphics can be exported to Web. • Adobe Photoshop: the standard in a graphics, image processing and manipulation tool. • Allows layers of images, graphics, and text that can be separately manipulated for maximum flexibility. • Filter factory permits creation of sophisticated lighting-effects filters. • Macromedia Fireworks: software for making graphics specifically for the web. • Macromedia Freehand: a text and web graphics editing tool that supports many bitmap formats such as GIF, PNG, and JPEG.

13. Video Editing • Adobe Premiere: an intuitive, simple video editing tool for nonlinear editing, i.e., putting video clips into any order: • Video and audio are arranged in ”tracks”. • Provides a large number of video and audio tracks, super-impositions and virtual clips. • A large library of built-in transitions, filters and motions for clips ) effective multimedia productions with little effort. • Adobe After Effects: a powerful video editing tool that enables users to add and change existing movies. Can add many effects: lighting, shadows, motion blurring; layers. • Final Cut Pro: a video editing tool by Apple; Macintosh only.

14. Multimedia Authoring • Multimedia authoring is the creation of multimedia productions, sometimes called ”movies” or ”presentations”. We are mostly interested in interactive applications.

15. Multimedia Authoring Metaphors • Scripting Language Metaphor: use a special language to enable interactivity (buttons, mouse, etc.), and to allow conditionals, jumps, loops, functions/macros etc. • Slide Show Metaphor: A linear presentation by default, although tools exist to perform jumps in slide shows. • Hierarchical Metaphor: User-controllable elements are organised into a tree structure – often used in menu-driven applications. • Iconic/Flow-control Metaphor: Graphical icons are available in a toolbox, and authoring proceeds by creating a flow chart with icons attached. • Frames Metaphor: Like Iconic/Flow-control Metaphor; however links between icons are more conceptual, rather than representing the actual flow of the program. • Card/Scripting Metaphor: Uses a simple index-card structure – easy route to producing applications that use hypertext or hypermedia; used in schools.

16. Multimedia Authoring Metaphors • Cast/Score/Scripting Metaphor: • Time is shown horizontally; like a spreadsheet: rows, or tracks, represent instantiations of characters in a multi- media production. • Multimedia elements are drawn from a cast of characters, and scripts are basically event procedures or procedures that are triggered by timer events. • Director, by Macromedia, is the chief example of this metaphor. Director uses the Lingo scripting language, an object-oriented event-driven language. • Hypermedia Linkage – The Hypermedia Linkage paradigm is similar to the Frame paradigm in that it shows conceptual links between elements; however, it lacks the Frame paradigm’s visual linkage metaphor. • Tagging – The Tagging paradigm uses tags in text files (for instance, SGML/HTML, SMIL (Synchronised Media Integration Language), VRML) to link pages, provide interactivity and integrate multimedia elements.

17. Multimedia Authoring Tools • Macromedia Flash: allows users to create interactive movies by using the score metaphor, i.e., a timeline arranged in parallel event sequences. • Macromedia Director: uses a movie metaphor to create interactive presentations—very powerful and includes a built-in scripting language, Lingo, that allows creation of complex interactive movies. • Authorware: a mature, well-supported authoring product based on the Iconic/Flow-control metaphor. • Quest: similar to Authorware in many ways, uses a type of flowcharting metaphor. However, the flowchart nodes can encapsulate information in a more abstract way (called frames) than simply subroutine levels.

18. Programming vs. Authoring • Authoring involves the assembly and bringing together of Multimedia with possibly high level graphical interface design and some high level scripting. • Programming involves low level assembly and construction and control of Multimedia and involves real languages like C++ and Java. • Later in this course we will study: • Java media programming (Java2D, JAI, JMF (Java Media Framework), Basic Java game programming, Java3D. • SMIL - Synchronized Multimedia Integration Language. • VRML/X3D - Virtual Reality Modelling Language.