Multimedia Systems Design

1. Multimedia Systems Design Introduction

2. Introduction • “A Picture is worth a thousand words” • There have been ongoing attempts to improve productivity of knowledge workers. • Reducing paper flow (reports, memos) has been one important area where electronic mail and groupware technologies are beginning to have impact. • As the demands of business increased, the 1980s were distinguished by overnight mail and fax as the means of communicating important information that did not require face to face meeting.

3. Introduction • Electronic Mail • Electronic Meetings • Video Conferencing • ISDN • HDTV

4. Introduction • The technologies at the core of the computing revolution have reached a point where one can envision a computing system composed of a number of elements together by various communication methods, all striving to serve user.

5. Introduction • The machine user interaction will be at conversational level rather that through the typed cryptic commands of yesterday or the mouse movements still in use. • As screens become larger and feature increasingly higher resolutions, the demand to see higher-resolution images is increasing. • New technologies especially developed for GUI allows multiple applications to operate simultaneously using multiple windows, thereby placing increasing demands on screen estates.

6. Introduction • The main challenge for the designers of multimedia systems will be to pack large volumes of information in compact packets. • Local area network provides bandwidth ranging from 10mbits/sec to several gigabits per second. • WAN depends on land, sea or satellite based communication channels that carry a large number of conversations. • With such bandwidths compression and decompression of data is important. • The size of compressed data is important because it determines the bandwidth necessary to meet acceptable performance requirements.

7. What constitutes Multimedia? • Multimedia meant a combination of text with images. • Document image management was an outgrowth of facsimile technology. • Facsimile provided a means of scanning and converting a document into coded information that described each pixel as white or black. • When the number of pixel per inch was low, the information was easily manageable. • When the pixel densities increased as better fax machines were developed, the information became very large.

8. What Constitutes Multimedia? • Wide variety of multimedia applications are in use or under development like: 1. Video Conferencing 2. Medical applications, such as analysis of surgical procedures and high resolution. 3. Real estates on-line video clips with property descriptors 4. Multimedia help and training material 5. Security systems for employee identification

9. Multimedia Elements • Multimedia applications require dynamic handling of data consisting of a mix of text, voice, audio components, video components, and image animation. • Integrated multimedia application allows user to cut sections of all or any of these components and paste them in a new document or in another application. • There are several components of Multimedia:

10. Facsimile • Fax. A process by which fixed graphic material including pictures, text, or images is scanned and the information converted into electrical signals which are transmitted via telephone to produce a paper copy of the graphics on the receiving fax machine. • Facsimile transmissions were the first practical means of transmitting document images over a telephone line. • It uses group3 compression standards. • Typical pixel densities used for facsimile are in the 100 to 200 dpi range. • The higher resolution are used to enhance the clarity of documents.

11. Document Images • Document images are used for storing business documents that must be retained for long periods of time or may need to be accessed by a large number of people. • Providing multimedia access to such documents removes the need for making several copies of the original for storage or distribution. • For storage of such documents minimum dpi required is 300 dpi. • An uncompressed A-size(8 -½ inch x 11 -½ inch image) is over 1 Mbyte. • Even with group 3 compression this size reduces to approximately 75kbytes.

12. Document Images • For images that are gray scaled or color, the sizes are much larger to accommodate the pixel color information. • Scanning of document images at that a high resolution requires very efficient compression and decompression technologies.

13. Photographic Image • Photographic images are used for wide range of applications such as employee records for instant identification at a security desk, real estate systems with photographs of house in the database containing the descriptions of house. • For bank signature cards, patient medical history, fingerprint cards. • To create photographic image with laser printer resolution of 600 dpi is considered essential .

14. Geographic information system maps • Maps created in GIS are widely used for natural resource and wild life management as well as urban planning. • These systems store the graphical information of the map along with a database containing information relating high lighted map elements. • Two kinds of technologies are used for storage and display of geographic maps. • Raster technology to display natural resources. • Another application combines raster image and vector overlay showing railroads or highways and other human made structure.

15. Voice Commands • Voice commands allow the user to direct computer operation by spoken commands. • Voice commands allow hands free usage of computer applications by allowing command via short voice commands rather than a keyboard or pointing device. • Recognition of commands requires specialized techniques and powerful processing capabilities to compensate for differences in pitch, accents, and voice modulation techniques of users.

16. Voice Synthesis • Transforming computer output to voice or audio output. • It is easier to achieve in compare to voice recognition. • The cadence(the consistency with which the spoken words are stung together) of the output should be good for message to be clear. • Digital signal processors designed for such an application have the processing power to perform the computations and maintain correct cadence.

17. Audio Messages • Audio messages are substitute of text messages. • Computers having microphone can record audio message and can embed it into electronic mail. • Audio messages also require large volumes of storage. • Compression techniques are required to manage the storage efficiently. • With image speed of decompression and display is important, speed of decompression and playback of audio message with proper cadence is important for audio message.

18. Video Messages • Similar to audio messages, video messages can be embedded into electronic mail messages. • The storage and playback requirements are even more complex with video messages because of storage of each video shots.

19. Full motion stored and live video • Full motion video act as very useful idea for online training manuals. • Full motion video can be used for video conferencing, live video presentations. • Full motion video requires high bandwidth for communication, massive storage requirements and high compression techniques. • High-definition television (or HDTV) is a digital television broadcasting system with higher resolution than traditional television systems • Again digital HDTV(High definition TV) places another major demand on the design- that of special effects such as zoom, freeze frame, image merging and image reconstruction.

20. Holographic Images • Holographic images extend the concept of virtual reality by allowing the user to get inside a part, such as, an engine and view its operation from the inside.

21. Multimedia Applications • Document Imaging • Image Processing and Image Recognition • Full Motion Digital Video Applications • Electronic Messaging

22. Document Imaging • Organizations such as insurance agencies, law offices, state governments, and the federal government, including the department of defense, manage large volumes of documents. • Document imaging makes it possible to store, retrieve, and manipulate very large volumes of drawings, documents, and other graphical representations. • Ex: Sending large volumes of engineering data about complex systems in electronic form rather than on paper.

23. Document Imaging • Almost all document image systems use below work flow: 1. Scanning images 2. Performing Quality checks 3. Performing data entry based on the contents of the images. 4. Indexing them 5. Storing them on optical media

24. Image Processing and Image Recognition • Image processing involves: Image Recognition Image Enhancement Image Synthesis Image Reconstruction

25. Image Processing and Image Recognition • The original image is not altered in document image workflow. • Image processing system actually alter the contents of the image itself. • Ex: Recognition of images: in factory floor quality assurance systems, • Image Enhancement: Satellite reconnaissance systems • Image Synthesis: Law enforcement suspect identification • Image Reconstruction: In Plastic surgery design systems

26. Image Enhancement • Image enhancement improves the quality of images for human viewing. • Removing blurring and Noise, increasing contrast . • For example an image might be taken of any cell of body which might be of low contrast range could enhance the image. • Increasing sensitivity and contrast makes picture darker by making borderline pixels black or increasing the gray scale level of pixels.

27. Image Animation • Computer created or scanned images can be displayed sequentially at controlled display speeds to provide image animation. • Image animation is a technology developed by Walt Disney and brought into every home in the form of cartoons.

28. Optical Character recognition • OCR technology is used for data entry by scanning typed or printed words in a form. • OCR technology used as a means for data entry may be used for as simple a task as reading the number of an invoice, or for capturing entire paragraphs of text.

29. Handwriting Recognition • Handwriting Recognition has been subject of intensive research now a days. • Pen based systems are designed to allow user to write commands on an electronic tablet. • Key Design Constraint: Ability to recognize writer independent continuous cursive handwriting accurately in real time. • Two factors are important: Strokes or shapes being inputted and velocity of input. • Parse recognizer will identify topology of strokes.

30. Handwriting Recognition • The stroke is compared with the prototype character set until a match is found or all predefined prototypes are checked.

31. Full Motion Digital Video Applications • Full Motion video is the most complex and the most demanding component of multimedia applications. • Full Motion Video Applications: 1. Training and Manuals: On-Line Reference CD-Rom Training 2. Business Applications; E-Mail Video Conferencing Presentations Demos

32. Full Motion Digital Video Applications • Games and Entertainment Interactive TV CD-ROM Interactive Games

33. Full Motion Digital Video Requirements • It should be possible to attach full motion video clips to other documents such as memos, presentations. • Users should be able to take sections of a video clip and combine the sections with sections from other video clips to form their own new video clip. • Features such as rewind, fast-forward, play, and search should be available. • It should be possible to view the same clip on a variety of display terminal types with varying resolution capabilities without storing multiple copies in different formats. • It should be possible for users to move and resize the window displaying the video clip.

34. Full Motion Digital Video Requirements • The users should be able to adjust the contrast and brightness of the video clip and also adjust the volume of the associated sound.

35. Evolving Technologies for multimedia systems • Hypermedia documents Hypertext Hyper speech • HDTV and UDTV • 3-D Technologies and Holography • Fuzzy Logic • Digital Signal Processing

36. Hypermedia documents • Technical and business documents in electronic form. • It contains text, embedded or linked multimedia objects such as image, audio or full motion video. • Hypermedia has its roots in hypertext.

37. Hypertext • Hypertext allows authors to • Link information together, • Create information paths through a large volume of related text in documents • Explicit connections or links allow readers to move from one location to another in a document or to other documents. • Hypermedia is an extension of hypertext in that these electronic documents, in addition to containing text will also include any kind of information that can be stored electronically.

38. Hypertext • An important issue arise with Hypermedia document is storage. • It does not make sense to embed full copies of each component of a hypermedia document within document file. • Rather, since the embedded components may also be included in other documents. • Hypermedia document only store references to the documents. • The user sees a single document, but the locations of the various components that are in document are transparent.

39. HDTV and UDTV • The electronics industry is attempting to raise the resolution levels of commercial television broadcasting. • Among the better-known television broadcasting standards are NTSC, PAL, SECAM, NHK, and others. • These standards range in resolution from 525 lines for NTSC to 819 lines for the French standard. • A hot debate has been in progress for bringing the world together on a single high-definition television (HDTV) broadcasting standard. • A 1125-line digital HDTV has been developed and is being commercialized. • NHK of Japan is trying to leapfrog the digital technology to develop ultra definition television (digital UDTV) featuring approximately 3000 lines. Figure 1-5 shows the progression in the resolution of television pictures..

40. HDTV and UDTV • There are some key technologies necessary to make the jump to a 3000-line UDTV standard. • It requires the development of ultra- resolution displays at a commercially viable price, high-speed video-processing ICs, and ultra broadband communications bandwidths for WAN services such as ISDN.

41. Fuzzy Logic • It includes graphics rendering, data compression for images, voice and video, voice recognition and synthesis, as well as signal processing for video, high resolution facsimile. • Graphics rendering involves the painting of three dimensional objects onto a two-dimensional display • Any application that is computationally intensive can benefit from the mathematical principles behind fuzzy logic.

42. Types of images

43. Visible Images • Visible Images include Drawings such as Blueprints, Engineering Drawings, Space maps for Offices, n Town Layouts, Paintings, Photographs, Documents, Still Frames • Non-visible Images: Non-visible Images are not stored as images but it is displayed as images. The examples include Pressure Gauges Temperature Gauges Other metering displays

44. Abstract Images • Abstract Images are not really images. They exist as real world objects or representations. • They are computer generated images based on some arithmetic calculations. The examples are • Kaleidoscope – Shows different patterns due to relative positions of glass beads when it is rotated • The mathematical used for generating Abstract Images: • Discrete Functions – It results in still images that remain constant. • Continuous Functions – It is used to show animated images and operations such as image fading or dissolving into another image

45. Multimedia data interface Standards • Video Processing Standards: Intel’s DVI Apple’s Quick Time Microsoft’s AVI

46. Intel’s DVI • DVI is actually both the name of the Digital Video Interactive hardware system sold by Intel and the file format associated with that system. DVI technology is essentially a PC-based interactive audio/video system used for multimedia applications

47. Apple’s Quick time • QuickTime (sometimes called QTM) is the native method of storing audio and motion video information on the Apple Macintosh platform. • It is used to record and play back multimedia information and store the data on magnetic or optical media. • QuickTime, however, is not only a data-storage format. It is also a collection of tools (the Movie Toolbox) that allows QuickTime movies to be modified (edit, cut, copy, paste, and so on), just as a word processor is capable of modifying an ordinary text file.

48. Apple’s Quick time • Quick time includes: System Software File formats Compression/Decompression algorithms Human interface standards

49. Microsoft’s AVI • Microsoft’s Audio video interleave standard, similar to apple’s quick time offers low cost, low resolution video processing. • It allows users to set parameters such as window size, quality, compression algorithm