1 / 35

Evaluation of Noise Characteristics in Image Pipeline

Evaluation of Noise Characteristics in Image Pipeline. Annabel Huo, Hyung Suk Kim, Sung Hee Park 2009. 3. 19. Image Pipelines. Image Pipelines for Demosaic and Denoise. Overview of the Image Pipeline. Image Pipeline Overview. Overview of the Image Pipeline. Image Acquisition Module

melodie-clark
Download Presentation

Evaluation of Noise Characteristics in Image Pipeline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of Noise Characteristics in Image Pipeline Annabel Huo, Hyung Suk Kim, Sung Hee Park 2009. 3. 19.

  2. Image Pipelines • Image Pipelines for Demosaic and Denoise

  3. Overview of the Image Pipeline • Image Pipeline Overview

  4. Overview of the Image Pipeline • Image Acquisition Module • Use ISET to simulate image capturing process • Acquire both RGB images and color-mosaiced RAW format images • Change input noise levels

  5. Overview of the Image Pipeline • Demosaic Module • Bilinear interpolation method • Adaptive homogeneity algorithm • POCS (Projection onto Convex Sets) • Adaptive frequency domain method • Denoise Module • Linear filtering • BM3D (Block Matching and 3D Filtering) • BLS-GSM (Bayes Least Squares-Gaussian Scale Mixtures) • Bilateral filter • Joint Demosaic-Denoise Module

  6. Overview of the Image Pipeline • Image-Noise Evaluation Module • MSE (Mean Square Error) • s-CIELAB : Perceptual difference • Difference image • s-CIELAB image • Histogram : Error distribution • Variance-Intensity plot : Additive and multiplicative property, spatial correlation • 3D scatter in RGB space : Color channel correlation • Autocorrelation : Spatial correlation, periodic structure • FFT : Frequency domain characteristic • Data Analysis Module • Tableau : Data visualization tool

  7. Analysis of Demosaic Algorithms • Demosaic Artifacts • Find what kinds of artifacts we get from demosaicing • Color artifact, zipper effect

  8. Analysis of Demosaic Algorithms • Noise Propagation in Demosaic Algorithms • Assumptions for input noise • Mainly by photon noise (multiplicative) • Gaussian distribution • White noise (No spatial correlation) • Independent to color channel

  9. Analysis of Demosaic Algorithms • Overall Comparison for Input Noise and Demosaicing • Output noise is the combination of demosaic artifacts and input noise propagation

  10. Analysis of Denoise Algorithms • Denoising for RGB Images • Denoising for RAW Images

  11. Demosaic-Denoise Pipeline • Demosaic algorithms will be affected by input noise. • Demosaic algorithms will change the noise characteristics. • Denoise algorithms will show different result if they get different kinds of noise.

  12. Demosaic-Denoise Pipeline

  13. Demosaic-Denoise Pipeline • Noise after Demosaicing • High level noise causes weird noise pattern after demosaicing.

  14. Demosaic-Denoise Pipeline • Noise after Demosaicing • Noise is stretched to ellipsoidal point clouds. • Color channel correlation • Color noise -> Luminance noise

  15. Demosaic-Denoise Pipeline • Denoising for Color Noise and Luminance Noise

  16. Demosaic-Denoise Pipeline • Denoising for Color Noise and Luminance Noise

  17. Demosaic-Denoise Pipeline

  18. Denoise-Demosaic Pipeline • Applying denoising methods to a RAW images may not be applicable for some cases. • Reduced noise level will be better for demosaic process.

  19. Joint Demosaic and Denoise

  20. Problems with Dm->Dn • Demosaic algorithm would aim to preserve sharpe edges • With noise presence at the edge, sharpen high frequency noise • Interpolation in Dm adds structure to noise makes denoise more complicated

  21. Problems with Dn->Dm • Denoise single-color raw image • Raw image’s information is very limited • Insufficient noise removal Also.. It is difficult to optimize the two stages simultaneously

  22. Joint Demosaic and Denoise • Exploit the similarities between Demosaic and Denoise.e.g: they are both estimations • Optimize both stages simultaneously • Remove additive and multiplicative noise • Easy to tune, two input parameters k0 and k1

  23. Joint Demosaic and Denoise technique 1.To estimate ideal G(0,0) 2. Assume high freque-ncy RGB are similar 3. Low frequency RGB are dissimilar

  24. Joint Demosaic and Denoise Technique • Hence the ideal G(0,0) can be estimated by: is the filter coefficient,it satisfies the constrains sum(ared)=sum(ablue)=0. y(i,j) is the noisy pixel value from the sensor.

  25. The Result Fig1:notice that blurring is significant especially at area with high spatial frequency

  26. The Result The error image:

  27. The Result • As the noise level increases, the blurring is more significant on the picture. • Most information is lost in the high spatial frequency region

  28. Frequency dependent error Fig1:testing image Fig2:error image

  29. Frequency dependent error

  30. A final point to note • The running time of the algorithm is significantly longer than other methods.

  31. Comparison Between Each Pipeline

  32. Linear Comparison Demosaic First Denoise First

  33. Linear Comparison Demosaic First Denoise First

  34. Best Cases BLS-GSM -> AH Joint AH -> Bilateral

  35. Best Cases Joint AH -> Bilateral BLS-GSM -> AH

More Related