Lecture 8: DIGITAL VIDEO technology

1. Lecture 8:DIGITAL VIDEO technology Email: evi@fsktm.upm.edu.my EVI INDRIASARI MANSOR Tel ext: 1741

2. Outline • Using video • How video works? • Broadcast video standards • Analog video • Digital video • Video recording and tape formats • Video editing technology • Optimizing video files for CD-ROM

3. Learning Outcomes • Summarize the implications of using digital video in multimedia • Illustrate video recording and its relation to multimedia production • Prepare digital video and images for conversion to television

4. Using video • Video is an excellent for delivering multimedia • Video places the highest performance demand on computer and its memory and storage • Digital video has replaced analog as the method of choice for making and delivering video for multimedia

5. Using video (cont) • Digital video device producesexcellent finished productsat a fraction of the cost of analog • Digital video eliminates theimage-degradinganalog-to-digital conversion • Manydigital video sourcesexist, but getting the rights can be difficult, time-consuming, and expensive

6. How video works • Light reflected from an object through the camera’s lens is converted into electronic signal by charge-coupled device (CCD) • This electronic signal containsthree channels of color information and synchronization pulses (sync) • Several video standards exist that deal with the amount of separation between the components of the signal

7. How video works (cont)

8. Broadcast video standards • National Television Standards Committee (NTSC): • These standards define a method for encoding information into electronic signal that creates a television picture • It has screen resolution of 525 horizontal scan lines and a scan rate of 29.97 frames per second

9. Broadcast video standards (cont) • Phase Alternate Line (PAL) and Sequential Color and Memory (SECAM): • PALhas a screen resolution of 625 horizontal lines and a scan rate of 25 frames per second • SECAMhas a screen resolution of 625 horizontal lines and is a 50 Hz system • SECAM differs from NTSC and PAL color systems in its basic technology and broadcast method

10. Your TV • Each frame is a single picture • When played rapidly one after another, it gives the illusion of motion  Everybody knows that, right? • Take NTSC as an example: • The overall signal has 525 lines (only 480 are actually visible, the extra 45 gives the TV time to reset the electronic beam to the top of the screen) • A single frame is ‘painted’ on the screen in an INTERLACED fashion (line-by-line) • Firstly: Odd-numbered lines (i.e. 1,3,5, …, 479) – takes ~1/60 seconds • Then: Even-numbered lines (i.e. 2,4,6, …, 480) – takes ~1/60 seconds • You might notice after this seeing 480i (or 525i). This means 480-interlaced!  [Later you’ll see 480p? Apaitu?] • BTW, all the above is called Standard Definition TV – Digital TV format which is 480i

11. Your TV ... As it gets bigger  • When screens get bigger, scan lines during interlacing become visible. This produces ‘jaggedies’… No body likes ‘jaggedies’… (19” vs 60” example) • Solution: PROGRESSIVE SCANNING!!! • ALL the lines from 1-480 are ‘painted’ on the screen at just one single pass!!! • Feasible with today’s technology since coding and transfer speeds have dramatically improved • This called for a new name…  Henceforth – Enhanced Definition TV was born [more of a marketing gimmick though… but hey, who really cares?] • 480p, 525p

12. EDTV’s advantage • Most TVs cannot handle 480p  • And what if we’re still getting interlaced signals? Won’t a progressive scan system be a waste, then? • EDTV can take interlaced signals, and pass them through a DEINTERLACER/LINE-DOUBLER • 1st: All the odd and even numbered scan lines are combined. This also improves smoother (‘jaggedies’ are smoothed out)  525i is converted to 480p here  • BTW, all of this is also done @ 1/60 second • 2nd: The next 1/60 is used to feed the frame a second time! • Increases brightness and stability of the frame • Eliminates or reduces jaggedies, giving a cleaner picture

13. HDTV (High Definition TV) • What HDTV does? • 1. Increases number of scan lines • 2. Widens aspect ration from 4:3 to 16:9  (baru la mcmwayangsikit, ye dok?) • Well.. That’s basically it… Comes with the same variation • 1080i – interlaced. However, since there are so many lines, the jaggedies aren’t that visible (illusion of better quality!) • 1080p – progressive scanning… but, do you really need it? … You yourself will be the decider (bionic eyes vs. normal human eyes) • Marketing hype however, will tell you otherwise  • Full HD – max resolution possible (1920X1080) • HD-ready – Meaning ure non-HD TV can be connected to an HD external box if you want to received HD signals • HD – Takes the full HD signal and resizes it down a bit

14. HDTV (High Definition TV) • Alternatively, the 720-lines progressive scan or 720p is also available • However, some interlacing artifacts might still be visible at 1080i • Therefore, video with rapid motion would not be suitable at 1080i, and would be nicer to view @ 720p • For drama swasta(soap operas), 1080i would be nice 

15. SDTV, EDTV and HDTV side-by-side

16. The different TV resolutions 

17. Analog video • Video information that is stored using television video signals, film, videotape or othernon-computermedia • Each frame is represented by a fluctuating voltage signal known as an analogue wave form or composite video

18. Analog video (cont) • Composite analogue video has all the video components: • brightness, colourandsynchronization • Then combined into one signal for delivery • Example : traditional television • Problems: colour blending, low clarity, high generation lost, difficult to edit

19. Analog video (cont) • Video color: • Television sets use composite input. Hence colors are less pure and less accurate than computers using RGBcomponent • NTSCtelevision uses a limited color palette and restricted luminance (brightness) levels and black levels. • Some colors generated by a computer that display fine on a RGB monitor may be illegal for display on a NTSC TV. • While producing a multimedia project, consider whether it will be played on a RGB monitor or a conventional television set.

20. Analog video (cont) • Interlacing effects: • In television, the electron beam makes two passes on the screen while drawing a single video frame • It first lays down all the odd-numbered lines, and then all the even-numbered lines, hence they are interlaced • While capturing images from a video signal, they can be filtered through a de-interlacing filter provided by image-editing applications

21. Analog video (cont) • Text and titles for television and taking care of analog tapes: • Titles for video productions can be created with an analog character generator. • Computers can create titles digitally using video and image-editing software. • New tapes should always be fast-forwarded to the end and then rewound, to ensure even tape tension.

22. Digital video • Video clip stored on any mass-storage device can be played back on a computer’s monitor without special hardware • Setting up a production environment for making digital video, requires somehardware specifications • Some specifications include computer with FireWire connection and cables, fast processor, plenty of RAM, fast and big hard disk

23. Digital video (cont) • Digital video architecture • Digital video architecture consists of aformat for encoding and playing back video files by a computer • Architecture includes a player that can recognize and play files created for that format

24. Digital video (cont) • Digital video compression • Digital video compression schemes or codecsis the algorithm used to compress (code) a video for delivery • The codec then decodes the compressed video in real-time for fast playback • Streaming audio and video starts playback as soon as enough data has transferred to the user’s computer to sustain this playback

25. Digital video (cont) • Digital video compression (cont) • MPEGis a real-time video compression algorithm • MPEG-4includes numerous multimedia capabilities and is a preferred standard • MPEG-7(or Multimedia Content Description Interface) integrates information about motion video elements with their use [NOT A COMPRESSION STANDARD]

26. Digital video (cont) • Digital video compression (cont) • The video compression/decompressionprograms are used so that video can fit on a single CD and the speed of transferring video from a CD to the computer can be increased • Let us say that a sequence of 25fps video is about 25MB • CD-ROM transfer rate is calculated as follows: • 1X= 150KB per second • 10X=1.5 MB per second • 100X= 15 MB per second • To overcome large video size, CODECS were developed

27. Compresses when saved CODEC File format used such as: Avi, Mpeg, Mov Decompresses when needed for display Digital video (cont) • Digital video compression (cont)

28. FULL VIDEO Taken from the CD into memory buffer DISPLAY ON SCREEN MEMORY BUFFER Digital video (cont) • Digital video compression (cont) • Digital video compression schemes orcodecsis the algorithm used to compress (code) a video for delivery • Thecodec then decodes the compressed video in real-time for fast playback • Streaming audio and video starts playback as soon as enough data has transferred to the user’s computer to sustain this playback

29. Digital video (cont) • Digital video compression (cont) • Two types of COMPRESSION: • Lossless compression • preserves the exact image throughout the compression and decompression process • e.g.,: text images is to identify repeating words and assign them a code

30. Digital video (cont) • Digital video compression (cont) • Lossy compression • Eliminates some of the data in the image and therefore provides greater compression ratios than lossless compression • Applied to video because some drop in the quality is not noticeable in moving images

31. Digital video (cont) • Digital video compression (cont) • Two types of CODEC (lossy): • Spatial compression • Digital compression of video data that compresses the size of the video file by compressing the image data of each frame • Compression is done by removing redundancy from data in the same frame

32. Digital video (cont) • Digital video compression (cont) • Temporal compression • Digital compression of video data that uses similarities of sequential frames over time to determine and store only the image data that differs from frame to frame • Compression is done by removing similarity between successive video frames

33. Digital video (cont) • Digital video compression (cont) • Flavors of file formats brand to choose: • Microsoft’s AVI format • QuickTime • MPEG • Div-X • Wmv (Windows Media Video)

34. Digital video (cont) • Digital video compression (cont) • Standards have been established for compression programs, including • JPEG(Joint Photographic Experts Group) and • MPEG(Motion Picture Experts Group)

35. Digital video (cont) • Digital video compression (cont) • JPEG (spatial) • Often areas of an image (especially backgrounds) contain similar information • JPEG compression identifies these area and stores them as blocks of pixels instead of pixel by pixel reducing the amount of information needed to store the image • These program reduce the file size of graphic images by eliminating redundant information

36. Digital video (cont)

37. Digital video (cont) • Digital video compression (cont) • MPEG (temporal) • The changes in the image fromframe to frame • Key frames are identified every few frames the changes that occur from key frame • Provide greater compression ratios than JPEG • Initially, it requires extra hardware for multimedia

38. Digital video (cont) • Digital video compression (cont) • MPEG (temporal) – (cont) • MPEG – real-time video compression algorithm • MPEG-4 includes numerous multimedia capabilities and is a preferred standard • MPEG-7 (or Multimedia Content Description Interface) integrates information about motion video elements with their use

39. MPEG compression • On how MPEG compression works, please view these videos • http://www.youtube.com/watch?v=kyztYavfFMs • http://www.youtube.com/watch?v=P7abyWT4dss • Skip the technical (maths) parts in the second video, instead focus on the concept of how things are done 

40. Video recording and tape formats • File size and formats • There is an important consideration: • File size in digitized video which included • Frame rate • Image size • Color depth

41. Video recording and tape formats (cont) • File size and formats (cont) • Frame Rate • animation is an illusion caused by the rapid display of still images. • television and movies play at 30 fps but acceptable playback can be achieved with 15 fps • [Show VirtualDub Example]

42. Video recording and tape formats (cont) • File size and formats (cont) • Image Size • A standard full screen resolution is640x480pixels but to safe storing space a video with 320x240 for a computer display is still acceptable • New high-definition televisions (HDTV) are capable of resolutions up to 1920×1080 , p60, • 1920 pixels per scan line by 1080 scan lines, progressive, at 60 frames per second

43. Digital video (cont)

44. Video recording and tape formats (cont) • File size and formats (cont) • Color Depth • The quality of video is dependent on the color quality (related to the number of colors) for each bitmap in the frame sequence

45. Video recording and tape formats (cont) • File size and formats (cont) • The color depth below 256 colors is poorer-quality image • The frame rate to below 15 fps causes a noticeable and distracting jerkiness that unacceptable • Changing theimage sizeand compressingthe file therefore become primary ways of reducing file size 24 bit 16 bit 8 bit (256 colors)

46. Video Compression • Digitizing full-motion video in the computer requires transfer of ENORMOUS amounts of data in a short period of time  • Consider this: • Frame rate = 25 fps (FR) • Color depth of each frame = 65,536 colors (thus a bit depth of 16-bits) (BD) • Frame size = 320 X 240 pixel dimension (D) • FORMULA FOR VIDEO SIZE (per second) = FR X BD X D • For this particular video… the UNCOMPRESSED video would be: • 25 X 16 X (320 X 240) • 30,720,000 bits / second • 3.67 Mbytes / second • This is just for 1-SECOND!!! And we’re not even talking about AUDIO yet  • A 1-hour video would thus be  3.67 X 60 X 60 = 13,212 Mbytes • Which is equals to = 12.9 Gigabytes  … Now… try fitting that onto a CD 

47. Video Compression • A simpler example with less big numbers • Consider this: • Frame rate = 2 fps … meaning, in every second of the video, TWO pictures are being displayed continuously (FR) • Color depth of each frame = 4 colors (thus a bit depth of 2-bits) … meaning that each frame of the video has a maximum of 4 colors (BD) • Frame size = 320 X 240 pixel dimension … each frame’s size (D) • FORMULA FOR VIDEO SIZE (per second) = FR X BD X D • For this particular video… the UNCOMPRESSED video would be: • 2 X 4 X (320 X 240) • 614,400 bits / second • 0.073 Mbytes / second  614,400 / 8 / 1024 / 1024 • As you can see… this rate is very2 acceptable due to the small data rate  • BUT! Your video is going to be very2 UN-colorful :P If there exists such a word… • Not to mention it being very2 flicker-y…

48. Video about compression • The basics of Video Compression • MPEG video • Codecs and Containers (formats) – 1 • Codecs and Containers (formats) – 2

49. Video editing technology • Linear • It plays end to end in one direction, usually pertains to videotape editing specifically the editing of linear tape segments into one final master tape

50. Video editing technology (cont) • Non-Linear • Refers to the editing of disk-based digital video. • The software provides an on screen map of what the final video sequences should look like incorporating the edits, splices, special effects, transitions and sound tracks.