1 / 21

Image Demosaicing: a Systematic Survey

Image Demosaicing: a Systematic Survey. Xin Li 1 , Bahadir Gunturk 2 and Lei Zhang 3 1 Lane Department of CSEE, West Virginia University 2 Dept. of ECE, Louisiana State University 3 Dept. of Computing, The Hong Kong Polytechnic University. Growth of Image Demosaicing Community.

ronald
Download Presentation

Image Demosaicing: a Systematic Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Demosaicing: a Systematic Survey Xin Li1, Bahadir Gunturk2 and Lei Zhang3 1Lane Department of CSEE, West Virginia University 2Dept. of ECE, Louisiana State University 3Dept. of Computing, The Hong Kong Polytechnic University

  2. Growth of Image Demosaicing Community Distribution of researchers across the world Published papers since Year 1999

  3. Acknowledgement • Yap-peng Tan at Nanyang Technological University (NTU), Singapore • David Alleysson at University Pierre Mendes France (UPMF) • Daniele Menon at University of Padova, Italy • King-Hong Chung at the Hong Kong Polytechnic University (HKPU) • Dmitriy Paliy at Tampere University of Technology, Finland • Chung-Yen Su at National Taiwan Normal University • Darian Muresan previously with Cornell University • Keigo Hirakawa at Harvard University

  4. Outline of This Talk • Color science background • Scientific basis of color-filter-array (CFA) • Image demosaicing problem formulations • Deterministic in the frequency domain • Statistical in the spatial domain • Categorization of existing methods • Sequential vs. parallel reconstruction • Performance evaluation • Comparison results • Two image data sets: Kodak CD and IMAX HD • Concluding remarks and open questions • What have we learned? What lies ahead?

  5. Trichromatic Color Vision (S) (M) (L)

  6. Biological Demosaicing Problem Human vision system (HVS) solves this biological demosaicing problem so well that trichromacy does not affect spatial acuity1 1Alleysson, D.; Susstrunk, S.; Herault, J., "Linear demosaicing inspired by the human visual system," IEEE Transactions onImage Processing,, vol.14, no.4, pp. 439-449, April 2005

  7. Computational Demosaicing Problem S=(R,G,B) 3-CCD camera zS=(zR,zG,zB) single-CCD camera

  8. Computational Demosaicing Problem (Con’d) ^ Objective: minimize the distortion between S and S for the class of images of interests.

  9. Statistical vs. Deterministic Formulation Bayesian perspective Spectral perspective1 Original S zG zR/zB 1David Alleysson et al., “Frequency selection demosaicking: a review and a look ahead”

  10. Categorization of Existing Demosaicing Methods H Hinter Hintra sequential parallel Iterative methods Luminance Vector median filter Chrominance Neural network (NN) or VQ-based (Post-processing)

  11. Selected Example: Sequential Demosaicing (blue channel is processed in a similar fashion to red channel) luminance chrominance Edge-sensitive/directional interpolation, local polynomial approximation … …

  12. Experimental Set-up • Eleven latest algorithms have been used in our comparison • Lu & Tan (LT): TIP Oct. 2003 • Alternating projection (AP): TIP Sep. 2002 • Adaptive homogeneity-directed (AHD): TIP Mar. 2005 • Successive approximation (SA) with edge-weighted improvement: TIP Feb. 2005+TCE May 2006 • Lukac’s CCA method with post-processing: TCE 2004+ICIP2004 • Frequency-domain demosaicing (FD): TIP April 2005 • Directional filtering and a posteriori decision (DFPD): TIP Jan. 2007 • Variance of color difference (VCD): TIP Oct. 2006 • Directional Linear MMSE estimation (DLMMSE): TIP Dec. 2005 • Local polynomial approximation (LPA): IMA 2007 • Adaptive filtering (AF): TIP Oct. 2007

  13. Performance Evaluation Protocols KODAK test images IMAX test images • Objective measures: PSNR values + SCIELab metrics • Subjective evaluation: very limited (mainly to visually inspect the severity of various artifacts)

  14. PSNR Performance Comparison on KODAK set

  15. S-CIELab Measure Comparison on KODAK Set

  16. Subjective Performance Comparison Examples

  17. PSNR Performance Comparison on IMAX set

  18. Subjective Performance Comparison Examples

  19. Discussions and Perspectives • What have we learned? • Color-related problems are hard and our understanding of color demosaicing problem remains ad-hoc • Many demosaicing techniques might appear different but essentially follow a similar motivation • Kodak image set is a poor benchmark despite its popularity1 • What are important questions ahead? • Establishment of an alternative benchmark data set for demosaicing research • Design of new-generation CFA2 and video demosaicing techniques • Exploration of its relationship to other tasks such as compression, denoising and forensics • … … 1Xiaolin Wu et al., “Improved color demosaicking in weak spectral correlation” 2Keigo Hirakawa and Patrick J. Wolfe “Second-generation CFA and demosaicking designs”

  20. Conclusion Demosaicing is never an isolated problem; Instead of paying attention to PSNR values, it is often more fruitful to rethink this problem under the context of electronic imaging and ask the right question first.

  21. Ad-hoc Fusion of Different Demoisaicing Methods

More Related