1 / 56

华阳联讯

华阳联讯. Image Sensor Pipeline. CCD & CMOS Module. CCD. ISP. VD. CCD module. Video Decoder. CMOS. ISP. CMOS SOC. CCD. CMOS. CMOS Lens(very important,but no mention) Bad pixel Interpolation Gamma Backlighting Color Correction 3A vignetting Edge Extraction and Sharpness

ama
Download Presentation

华阳联讯

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 华阳联讯

  2. Image Sensor Pipeline • CCD & CMOS Module

  3. CCD ISP VD CCD module Video Decoder CMOS ISP CMOS SOC

  4. CCD CMOS CMOS Lens(very important,but no mention) Bad pixel Interpolation Gamma Backlighting Color Correction 3A vignetting Edge Extraction and Sharpness Detection and Compensation Anti Flicker Temporal Noise Reduction distortions Deblur(no mention) CCD Interlace Deinterlace Denosie 色彩增强 Color Space Color transform Resize 色彩调整 测试设备与人员 国家图像质量标准 新技术点

  5. Outline • SensorPerformance • Color Space • Color Transform • Interlace & deinterlace • Denoise&Deblur • Scale & Crop • New scheme

  6. 典型的安防市场用CCD模组 典型的安防市场用CMOS模组

  7. SensorPerformance Summary • SensorPixelSize • Read Noise • AD Converter • Signal-to-Noise Ratio • Full Well Capacity • Dynamic Range • Unity Gain • Low Light Sensitivity Factor • Noise Sources • Pixel Density

  8. Sensor Size 背景是35mm标准胶卷

  9. Sensor Pixel Size Full Well Capacity

  10. Color Array CFABayer The New Foveon Sensors

  11. Super CMOS Sensor Normally sensor pixels are square. Fujifilm's Super CCD sensors have octagonal pixels, as shown in this diagram. Therefore, the distance "d2" between the centers of two octagonal pixels is smaller than the distance "d1" between two conventional square pixels, resulting in larger (better) pixels.

  12. However, the information has to be converted to a digital image with square pixels. From the diagram you can see that, for a 4 x 4 area of 16 square pixels, only 8 octagonal pixel measurements were used: 2 red pixels, 2 blue pixels, and 4 green pixels (1 full, 4 half, and 4 quarter green pixels). In other words, 6 megapixel Super CCD images are based on the measurement by only 3 million effective pixels, similar to the above interpolated example, but with the advantage of larger pixels. In practice the resulting image quality is equivalent to about 4 megapixel. The drawback is that you have to deal with double the file size (leading to more storage and slower processing), while enjoying a quality improvement equivalent to only 33% more pixels. 简单的说，就是同样感光面积的的Sensor，能提供约多44%的有效像素，但是它需要 多处理1倍大小的图像。得不偿失？？？？？？？？？

  13. Sensitivity (ISO) ISO 800 - Red Channel ISO 100 ISO 800 ISO 100 - Red Channel Increase sensitivity-amplified -Noise increase

  14. Sensor Linearity

  15. Sensor Linearity 这就是需要做gamma校正的原因，当然实际的校正曲线往往是S型的曲线 这是因为为显示设备压缩动态范围的需要。

  16. AD Converter

  17. Table 1a Dynamic Range Well Capacity (electrons) / Read Noise (electrons), 理想Dynamic Range = Full Sensor，在完全黑暗中，读出应该为0. 根据sensor的噪声模型，最小信号为1时，噪声为1. 假设动态范围为1000，那么需要用2^12bits的AD表示这颗sensor。 另外：sensor的放大器，和AD会带来噪声，所以在不同的ISO值会有 不同的噪声叠加值。 所以，不同的ISO情况下，有不同的动态范围。 所以read noise, full well capacity, and dynamic range.三者关系复杂 Poisson Statistics

  18. High Dynamic Range • 影响动态范围的参数： • 噪声大小（越小越好） • 数据位宽（越宽越好） • Pixel物理面积大小（越大越好） 说明：Pixel物理尺寸越大，HDR性能越好 AD位宽越高，性能越好 说明：在高ISO时， HDR性能急速下降， 因为ISO增加，AD增益加大，噪声增加。

  19. The More Pixels, the Worse the Image! "As an independent test lab for digital cameras, we have observed a decrease in the image quality of digital cameras during the last 3 years,“ saysImage Engineering.www.image-engineering.de Newer cameras have an increasingly worse image quality and the reason for that is obvious:"The more pixels a camera has, the better it is” was true in the beginning of digital photography when compact cameras had a VGA resolution (640 x 480) When the pixel count exceeded 6 Megapixels in 2004, this was no longer the case but consumers still buy cameras with the highest pixel count.

  20. Color Space luma/chroma(TV) Hue/Saturation Mixtrue • YUV • YIQ • YCbCr • RGB • Lab • CMYK • HSV • HSB

  21. YUV A B BT656,PAL/SECAM ATV TV Color Space D C BT601SDTV,BT1120 BT709HDTV DTV To Be Created YIQ NTSC ATV YCbCr

  22. YUV Y' value of 0 YIQ YUV Y' value of 0.5 Y' value of 1 Note :From http://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion

  23. YCbCr Y' value of 0 Y' value of 0.5 Y' value of 1 Note :From http://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion

  24. RGB/CMY(K) RGB,表示范围大，适用主动发光物体色彩空间，如显示器。 CMKY，表示范围小，使用与被动显示物体，如打印相片

  25. RGB/HSB(HSV) RGB,非常适合产生R,G,B分量色彩。 RGB,非常适合调整饱和度，亮度，色度 。

  26. ColorTransform Color Transform ：基本上任何一款视频处理芯片都需要。 YUV-RGB，注意处理精度问题 HSB-RGB，算法复杂度，色彩损失

  27. YUYV YUV的色彩排列很重要，每颗芯片可能有自己定义的数据输入格式，这些格式在FOURCC中定义。YUV转换涉及到DownSampling和Upsampling技术 YVYU Y41P YV12 Y655 YUVTransform GREY http://www.codecs.onerivermedia.com

  28. RGBA RGB的数据格式定义：主要是某些数码相机中，输出的RGB格式排列有不同。 RAW BI_RLE8 BI_RGB RGB RGBTransform RLE http://www.codecs.onerivermedia.com

  29. Q&A about Color Space & Transform

  30. Interlace & Deinterlace • What is interlace?Is interlace a bug? Why interlace ? • How to Deinterlace? • Telecine

  31. AD. :Interlace! Since 1930s!

  32. This process of dividing frames into half-resolution fields at double the frame rate is known as interlacing. Resize Deinterlace From：http://en.wikipedia.org/wiki/Deinterlacing

  33. Why interlace （1）Clever processing for TV system Save half bandwidth.(Data compress) （2） Most modern displays, such as LCD, DLP and plasma displays, are not able to work in interlaced mode, because they are fixed-resoution displays and only support progressive scanning. (Bad luck to ATV) （3） In order to display interlaced signal on such displays, the two interlaced fields must be converted to one progressive frame with a process known as de-interlacing.

  34. How to deinterlace • 3D deinterlace • Memory cousume 复杂，效果最好 Motion Compensation Motion detection • Blend(to blur) • Area Base • To blur the mice teeth • Discard • Lose nothing Deinterlace Weave(do nothing) Field combination deinterlacing BOB Display every fields by 50FPS Field extension deinterlacing Note:None of them is perfect!!!!!!!!!!!(The main argument is that no matter how complex the deinterlacing algorithm may be, the artifacts in the interlaced signal cannot be completely eliminated because some information is lost between frames.)

  35. When is deinterlace bad? （1）when video source is not interlaced （2）Film to TV(just de telecine)

  36. telecine FilmTV (telecine) Interlaced video can carry progressive scan signal, and deinterlacing process should consider this as well. (More unfortunately) 24Fps25Fps on PAL TV system 24Fps59.94Fps on NTSC SECAM system

  37. 2:2 2:3

  38. Does deinterlacing work? • Interlace will be disappeared in future! • The European Broadcasting Union has argued against the use of interlaced video in production and broadcasting

  39. Q&A about Deinterlace

  40. Scale & Crop How to Scale?,(等比，非等比) How to Crop? Rotation? (no need)

  41. Resize Scale Crop

  42. Scale 近邻取样缩放到0.6倍 近邻取样缩放到1.6倍 原图 二次线性插值缩放到0.6倍 二次线性插值缩放到1.6倍 三次卷积插值缩放到0.6倍 三次卷积插值缩放到1.6倍

  43. Scale 近邻取样缩放到8倍 二次线性插值缩放到8倍 原图 三次卷积插值缩放到8倍 二次线性插值(近似公式)

  44. Crop

  45. Denoise & Deblur

  46. 背景噪声，白噪声 • TBD

  47. Video Enhancement

  48. Luminancepeaking ，补偿信道过程中高频损失，主要在YUV域中处理Y分量 • 数字亮度瞬态增强技术(DLTl）(sharpness) • 黑电平扩展，只处理Y • 白电平扩展，只处理Y • 对比度，亮度，处理Y • 色度，饱和度调整,之处理UV • 肤色校正

  49. 参考PW328

More Related