Digital Media

1. Digital Media Lecture 9: Video, TV & Film Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan

2. In the next several lectures… Film & TV & Video & Animation Issues that arise from conversion Analog vs Digital

3. Test Pattern

4. TV Broadcast…Digital replaces Analog • Why Digital Broadcast? • reduced spectrum use • greater capacity • multiple programs on one freq • better quality picture • HDTV • can use compression • allows multiple HD signals on one freq. • allows user interaction

5. TV Broadcast…Difference with poor receptionAnalog vs Digital • Analog… • as signal gets weaker • image gets less distinct • “ghosts (white shadows) appear” • gracefully degrades

6. TV Broadcast…Difference with poor receptionAnalog vs Digital • Digital… • with digital, you either have signal • or you don’t have signal so… • lose signal • everything goes black • audio stops • ungraceful degrading

7. Two ways to make Moving Pictures:Video & Animation • In this class: • Video • shot with a camera • captures images from the world • then play them back • Animation • create frames individually • using inkscape and blender • play them back

8. Two ways to make Moving Pictures:Video & Animation • In this class: • Video • shot with a camera • captures images from the world • then play them back • Animation • create frames individually • using inkscape and blender • play them back

9. Video (and Film) • Works because of persistence of vision • human perception causes still images played in rapid succession to fuse into motion • http://en.wikipedia.org/wiki/Frame_rate • Fusion frequency • ~ 40 frames per second • depends on the brightness of the image relative to the viewing environment • Less than that • first flickering • then individual images appear losing the illusion of motion

10. Film: How it works • Plays at 24 frames per second • Show the image • Block the light to make it dark • Move to the new image • Allow the light through to show the new image • Without “blacking out” the change from one image to the next the image would be blurred

11. Film Trivia 1 • Films are longer than one reel long • How does the person who runs the cameras know when to change? • There are two projectors, one running, one waiting • A black hole in the film appears ~5 sec before the switch is made • Another black hole in the film appears and the projector operator switches

12. Film Trivia 2 • Watch the credits… • Foley artist? • 24 frame manager? • Telecine?

13. Video & TV • Two versions • Interlaced • Rising from a TV legacy • Progressive scan • Rising from a computer legacy

14. Interlaced • Captured (and displayed) as “fields” • First the odd numbered lines are captured (or displayed) • Then the even numbered lines are captured (or displayed) • …

15. Interlaced • … • This reduced the bandwidth needed to transmit images that moved for early TV • The glowing phosphor of the CRT stayed glowing for a while after the electron beam was turned off • Allowing the other field to be drawn and complete the TV image

16. Interlaced fieldsRaster scan

17. Interlaced scan

18. Interlaced problem: • Rapid motion resulted in the “comb effect”

19. Progressive scan

20. Progressive scan • Each line on the screen is painted one after the other from top to bottom • Electronics are faster now so interlacing is not required • If captured progressively, then the playback is straight forward • If captured as interlaced fields, playing them back progressively is problematic • disadvantage of progressive scan is that it requires higher bandwidth than interlaced video that has the same frame size

21. Interlace problem: the left-column images are progressive scan the center-column images are interlaced the right-column images use line doublers bottom images are anti-aliased http://en.wikipedia.org/wiki/File:Indian_Head_interlace.gif

22. Video… it’s bigHow do you deal with it? • Playback degradation (graceful degradation) • Compression

23. Video… it’s bigHandling with Playback • Transport or playback not fast enough to keep up with the story? • something’s got to give • there’s too much data to either transport or display • Some players just freeze the image and halt the audio • this kills the ability to tell the story • Some players (like quicktime) make attempts to “degrade gracefully”

24. Video: Graceful degradation • Graceful degrading allows the story to continue • Some players drop frames • first showing as a “slide show” while continuing to play the audio • then holding the last image while continuing to play the audio stream • this effectively loses the illusion of motion but continues the “story” as an audio stream • …

25. Video: Graceful degradation • … • Some play lower resolution images while remaining synched to the audio stream • this continues the illusion of motion (at a lower resolution) and continues the “story” with the audio stream

26. Video: The “progress bar”Quicktime example • Click to play a quicktime video • Quicktime window opens • It is in “play” mode (the pause icon is showing • Doesn’t begin to play, instead a gray colored bar starts filling the progress bar • At some point it begins to play • Why? It’s predicting how long the download will take

27. The progress bar

28. Video is big so: reduce its size using compression • On the capture side • Digitization & compression can be carried out by hardware to be fast • Can be done in the camera (hardware) • Can be done in the computer (software)

29. hardware vs software compression • Hardware compression... user has no control over it... it is hardwired • It can be in the camera • Software compression... is computationally expensive... it’s a slow process • Provides for the most flexibility since it can be changed • Can use different software coder-decoders (codec), picking and choosing what fits your needs better

30. Compression in the camera:hardware compression • Our cameras? • Mini DV format • Compress each captured image into a jpeg image • This is called intra-frame compression • Present a digital stream of bits to the computer over a firewire connection • With compression you get artifacts

31. with software compression… • Analog is presented to the computer through a video capture card • Compression is done (usually) in the video capture card • Allows for a really small camera because the work (the compression and the analog to digital conversion) is done elsewhere

32. More aboutAnalog vs Digital • An analog signal to the computer is susceptible to noise corruption • Digital signal is not • What’s the big deal? • Consider compressing a video of a wall painted a solid color • Analog noise will cause small fluctuations from pixel to pixel • RLE can’t compress it because each pixel is a bit different

33. Comparing cameras iSight to MiniDV • iSight (or a webcam) is built into the Macs in this room • Presents an analog signal to the computer • Subject to analog noise • The cameras we can check out from the library are Mini DV format and record on tape • Presents a digital signal to the computer

34. Our video cameras compress using jpeg analog signal !!!NOISE!!! computer webCam video capture card compression the scene digital signal iMovie miniDV compression

35. We’ve seen… • Converting TV to Video is problematic • Interlacing • comb effect Next: • Converting Film to Video is problematic • Matching 24 frames to 30 frames • Telecine problem

36. Film to Video • Problematic (interleaved) • video is 30 frames per second • film is 24 frames per second • How do you make 30 frames from 24? • One way: The 3-2 pull down… • AKA Telecine

37. Film to interlaced video:

38. Refer to Supplemental text: Moving Images: Film   Moving Images: TV   Moving Images: Video

39. Next • Lecture 10: Video & compression techniques