Multimedia- and Web-based Information Systems

1. Multimedia- and Web-based Information Systems Lecture 5

2. Multimedia: Color- and Video-technology

3. Video-Technology • Television- and Video-Technology form the basis of the medium motion picture • Generation • Recording from the real world • Synthesis on the basis of a description • Analogous and digital technology

4. Representation of the video signal • Representation of the video signal contains • Visual representation • Transmission • Digitalization

5. Visual Representation • Presentation of the video signal trough a CRT (Cathode Ray Tube) • In television and computer screens • Representation of a scene as realistic as possible • Delivery of the space and time content of a scene

6. Fundamentals of visual representation • Resolution • Width W • Height H • E.g. W=833, H=625 • Width/height-relation • 4:3 or 16:9 • Perception of depth • In the natural preception trough the use of both eyes (different view angles onto one scene) • Focus-depth of the camera, appearance of the material of an object

7. Fundamentals of visual representation • Luminance / Chrominance • Motion picture resolution / continuity • Discreet sequence of single pictures can be perceived as a continually sequence • Boundary of motion picture resolution • 15 pictures/sec (video used 30 pictures/sec) • No boundary with acoustic signals

8. Fundamentals of visual representation • Flicker • With small refresh rate • Eg. 50 or 60 Hz • Full and half pictures (interlacing)

9. RGB Color Coding • RGB (Red Green Blue) • Additive color blend • Normalization of values (R+G+B=1)

10. YUV Color Coding • For the human eye, brightness is more important than color information • Brightnessinformation (Luminance) • 1 channel of luminance (Y) • Color Information (Chrominance) • 2 channels of chrominance (U and V)

11. Component Coding YUV • Y = 0.30 R + 0.59 G + 0.11 B • U = 0.493 (B-Y) • V = 0.877 (R-Y) • Errors in Y are more severe • Y to be encoded with high bandwidth • YUV Coding often specified with a raito of the channels (4:2:2)

12. Component Coding YUV • YIQ (similar to YUV) • Derived from NTSC • Y = 0.30 R + 0.59 G + 0.11 B • I = 0.60 R + 0.28 G + 0.32 B • Q = 0.21 R + 0.52 G + 0.31 B

13. Shared Signal • Individual components (RGB, YUV, YIQ) need to be combined to one signal • Methods of modulation to avoid interference

14. Video formats • Resolution of a picture (frame) • Quantisation • Framerate • Video controller • Dedicated video memory

15. Video formats • CGA (Color Graphics Adapter) • 320x200, 4 colors, 16.000 bytes • EGA (Enhanced Graphic Adapter) • 640x350, 16 colors, 112.000 bytes • VGA (Video Graphic Array) • 640x480, 256 colors, 307.200 bytes • XVGA (eXtended Video Graphic Array) • 1024x768, 256 colors, 768.423 bytes • XGA (eXtended Graphic Array) • 1024x768, 16M colors, 2304 kbytes • Many more

16. Conventional Systems • NTSC (National Television Systems Commitee) • From the USA, oldest standard, widely used, 30 Hz, 525 lines • SECAM (Sequential Coleur avec Memoire) • France, Eastern Europe, 25 Hz, 625 lines • PAL (Phase Alternating Line) • Western Europe, 25 Hz, 625 lines

17. High-Definition Television (HDTV) • Resolution • 1440x1152 / 1920x1152 • Frame rate • 50 or 60 Hz • No longer interlaced

18. Digitalisation of video signals • Conversion into a digital representation • Nyquist-Theorem (bandwidth = half the sampling rate) • Of the components • Quantisation • 2 Alternatives • Shared Coding • Component Coding

19. Shared Coding • Scanning of the whole of the analogue video signal (e.g. composite video) • Dependant on the standard • Bandwidth the same for all components • Disadvantage: low in contrast

20. Component Coding • Separate digitalisation of the components (e.g. YUV) • Ratio 4:2:2 • 864 scan values for luminance • 432 scan values for chrominancy

21. Digital Television • Digital Television Broadcasting (DTVB) • Digital Video Broadcasting (DVB) • DVB-T (terrestric broadcast) • System description • Implementation of HDTV • Employs MPEG-2 • Coding of Audio and Video

22. Advantages of DVB • Increase in the number of TV-channels • Adaptable picture and sound quality • Encryption possible for Pay-TV • New Services: Data broadcast, Multimedia broadcast, Video-on-Demand • Convergence of PC and TV

23. Multimedia: Data Compression

24. Data Compression • Audio and Video require lots of storage space • Increasing Demand • Text – Single Pictures – Audio – Motion Picture • Data rates influence • Transmission • Processing • Efficient Compression • Theory • Standards

25. Storage Space / Bandwidth • Considerable storage capacity for uncompressed pictures, audio and video data • For uncompressed Video, even a DVD is not sufficient • Uncompressed Audio-/Videodata requires very high bandwidth

26. Required Storage Space • Text • 80 x 60 * 2 bytes = 9600 bytes = 9,4 KByte • Figures • 500 primitives * 5 Bytes for properties = 2500 bytes • Voice • 8 kHz, 8 bit quantisation = 8 kByte / s • Audio • 2 x 44100*16 bit / 8 bit * 1 byte = 172 Kbyte / s • Video • 640 x 480 * 3 x 25 frames = 22,500 Kbyte /s

27. Important Methods • JPEG (JPEG 2000) • For single pictures • H.261 and H.263 • Video sequences of small resolution • MPEG 1,2 and 4 • Motion Picture and Audio (MPEG Layer 3)

28. Demands on Methods • Good quality • Small complexity • Effective implementation • Time boundaries with decompression (and compression) • MPEG-1: high effort with compression

29. Demands in Dialogue mode • End-to-End latency • Part of the (De-)Compression < 150 ms • 50 ms -> natural dialogue • Additionally all latencies of the network, communication protocols and of the in- and output devices

30. Demands in Query mode • Fast Forward / Rewind with simoultaneuos display of the data • Random access to single frames • < 0.5 s • Decompression of single pictures without interpretation of all the frames before them

31. Demands in Dialogue and Query mode • Format independent of screen size and refresh rate • Audio and video in different qualities (to adapt to the respective circumstances) • Synchronisation of Audio and Video • Implementation in software

32. Classification of compression methods • Entropy coding • Lossless methods • Source coding • Often lossy • Hybrid coding • Combined application of both of the methods above for a specific scenario

33. Entropy coding • Independent of media specific properties • Data to compress is a sequence of digital data values • Losslessness • Data before and after the compression/decompression are identical

34. Source coding • Usage of the semantics of the information • Compression ratio depends on the specific medium • Data before and after the compressen/decompression are very similar to each other but no longer identical

35. Hybrid coding • Combination of entroy and souce coding, used e.g. In • JPEG • MPEG • H.263

36. Decompression • Inverse function of the compression • Decompression possible in real time? • Symmetric methods • Similar effort for coding and decoding • Assymetric method • Decoding possible with smaller effort

37. Run length encoding • Sequence of identical bytes • Number of repeating bytes • Mark M (e.g. „!“) • Stuffing if M is in the data space • Example 1: 0, „!“, 256 • Example 2: „!“, „!“ (Stuffing) • In what cases does it help? Maximum saving?

38. Suppression of null values • Special case of run length encoding • Selection of a single character that is repeated often (e.g. „0“) • Mark M, after that number of repetitions • In what cases does it help? Maximum saving?

39. Vector quantisation • Splitting of the data stream into blocks of n bytes • Table with patterns for blocks • Index into the table to the entry most similar to the block • Multi-dimensional table -> vector • Approximation of the original data stream • Example

40. Pattern Substitution • Patterns of frequent occurence replaced by one byte • Mark M, then index into a table • Well suited for text • e.g. keywords in programming languages

41. Diatomic Encoding • Putting together of two bytes of data at a time • Determination of the byte-pairs occuring most frequently • e.g. in the English language • „E“, „T“, „TH“, „RE“, „IN“, ... (8 in total) • Special bytes not occuring in the text used to represent 2 letters • Reduction in data of ca. 10%

42. Static encoding • Frequency of occurence of a character • Different coding length for characters • Basis of the Morse code • Important: unambigous decompression

43. Huffmann coding • Regards the probability of occurence • Minimum number of bits for given probability of occurence • Characters occuring most often get the shortest code words • Binary tree (Nodes contain probabilities, edges bit 0 or 1)

44. Huffmann coding • P(A)=0.16, P(B)=0.51, P(C)=0.09, P(D)=0.13 and P(E)=0.11

45. Huffmann Coding P(ADCEB)=1.0 • w(A)=001, w(B)=1, w(C)=011, w(D)=000, w(E)=010 1 0 P(ADCE) P(B)=0.51 0 1 P(CE)=0.20 P(AD)=0.29 0 1 1 0 P(C)=0.09 P(E)=0.11 P(D)=0.13 P(A)=0.16

46. Transformation coding • Data transformed into a better suited mathematical space • Inverse Transformation needs to be possible • Discrete Cosine-Transformation (DCT) • Fast-Fourier-Transformation (FFT) • See example in the JPEG lecture

47. Prediction or relative encoding • Forming the difference to the previous value • Data do not differ much • Combination of methods • e.g. homogenous areas in pictures • DPCM, DM and ADPCM

48. Further Methods • Color tables • with pictures (video) • Muting • Threshold for sound volume