460 likes | 641 Views
Brushing Up on Upconversion Bruce Jacobs Twin Cities Public Television. At Stake. Millions of hours of SD content New consumer HD displays At best, 480i becomes 720p At worst 360i becomes 1080p!. UC. 360i. 1080p. 720. 1920. Brush Up. Upconverter Theory - Digitize - Deinterlace
E N D
Brushing Up on UpconversionBruce JacobsTwin Cities Public Television
At Stake Millions of hours of SD content New consumer HD displays At best, 480i becomes 720p At worst 360i becomes 1080p! UC 360i 1080p 720 1920
Brush Up Upconverter Theory - Digitize - Deinterlace - Scale Upconverter Practice - Software - Hardware
References Larry Thorpe: "The HDTV Camcorder and the March to Marketplace Reality" SMPTE Journal, March 1998 Jed Deame: "Motion Compensated De-Interlacing: The Key to the Digital Video Transition" Presented at the SMPTE 141st Technical Conference, November 19-22, 1999 Charles Poynton: "Digital Video and HDTV: Algorithms and Interfaces" Morgan Kaufmann, 2003
Tip #1 To successfully reconstruct a digitized signal, we must sample at more than twice the highest input frequency – the "Nyquist Rate".
Four Sample Examples At Nyquist Rate f(s) = 2 * f(max) Below Nyquist Rate f(s) = 1.3 * f(max) Double Nyquist Rate f(s) = 4 * f(max) Just Above Nyquist Rate f(s) = 2.3 * f(max) (Rec 601 = 13.5/5.75=2.3)
Tip #2 In a camera, direct pixel mapping isn't good enough, despite the "PC" experience. - See Tip #1
Below Nyquist Rate Alias = 1/3 f !
Pan Right ¼λ Alias Moved ¾λ Right!
Tip #3 Aliasing is bad enough in audio where it is "stationary". In video, aliasing also moves, the wrong distance, often in the wrong direction, resulting in obvious and annoying artifacts.
Pre-Filter ½ f(s) f(s) Aliasing Lost Resolution
Tip #4 Avoid aliasing by using a steep LP filter before sampling. - Steep filters are impossible with optics - Camera designers say some aliasing is better than degraded resolution Avoid resolution loss by sampling at far more than double highest desired frequency. - Requires extra pixels and higher bit rate
Oversampling Pre-Filter Post-Filter & Sub-Sample ½ f(s1) ½ f(s2) f(s1)
Tip #5 Use Oversampling 1. Increase sample frequency 2. Increase pre-filter pass-band 3. Add steep digital LP post-filter 4. Subsample to original rate - Legacy technique in SD CCD cameras - Improves resolution - Reduces aliasing - No increase in bit rate - Does increase camera noise
Pan Right ¼λ Alias Moved ≈2λ Right!
Tip #6 Even with adequate sampling, mapping the output directly to perfect digital display does not produce perfect results!
Reconstruction Filter Pre-Filter ½ f(s) f(s) Aliasing Lost Resolution
Tip #7 To reconstruct the signal, a low pass filter is needed, to remove the sample frequencyimage. - Too steep results in ringing - An analog filter for analog output - Can be digital, if a higher frequency is used!
Reconstruction LPF Filter Magic
Tip #8 Using "reconstruction filters" in both dimensions will upconvert the image! - Doing this well (using many "taps") requires perhaps thousands of multiplications for every pixel, in every color channel, in every frame, all in real time!
Deinterlace The interlace problem: • - Simply using fields might leave only 180 V lines! 60 fields per second from source odd 1.1 2.1 3.1 4.1 5.1 even 1.2 2.2 3.2 4.2 5.2 60 Frames per second needed for upconversion ? ? ? ? ? ? ? ? ? ?
3/2 Pulldown 24 Fps 1 2 3 4 60 fps "3" "2" "3" "2" odd 1.1 1.1 2.1 3.1 4.1 even 1.2 2.2 3.2 3.2 4.2
Tip #9 If the 60i content came from 24p, weave matching fields back into frames, and ignore the duplicative fields.
Pulldown Removal 24 Fps 1 2 3 4 60 fps odd 1.1 1.1 2.1 3.1 4.1 even 1.2 2.2 3.2 3.2 4.2 60 Fps 1 1 1 2 2 3 3 3 4 4
Tip #10 If the two fields are well correlated when there is no motion, weave them back into a frame and repeat it. =
Tip #11 If the two fields are completely uncorrelated, bob between fields with half the vertical resolution for each. ≠
Tip #12 Adapt by using weave in regions of correlation, and bob between fields where not correlated. = ≠
Tip #13 Search for motion vectors, and compensate where possible.
Demonstration Software - Final Cut Pro - Apple Compressor Embedded Hardware - AJA Kona - Panasonic AJ-HD1400 DVCProHD VTR - Sony HDW-M2000 HDCam VTR Stand-Alone Hardware - Leitch X75 - Miranda XVP-811i - Snell and Wilcox Quasar PH.C - Teranex Xantus