1 / 27

Demosaicking for Multispectral Filter Array (MSFA)

Demosaicking for Multispectral Filter Array (MSFA). Lidan Miao AICIP Research August 24, 2004. Scene. Camera Len. MSFA. Sensor. Multispectral Image. Reconstruction. Mosaic Image. Background. Challenge in acquisition of multispectral images.

knoton
Download Presentation

Demosaicking for Multispectral Filter Array (MSFA)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Demosaicking for Multispectral Filter Array (MSFA) Lidan Miao AICIP Research August 24, 2004

  2. Scene Camera Len MSFA Sensor Multispectral Image Reconstruction Mosaic Image Background • Challenge in acquisition of multispectral images. The CyberEye 2100is developed in SCP Laboratory at Syracuse University. A synchronized filter wheel and optical system provides 12 independent spectral bands from 400-1000 nm . http://www.cs.ucf.edu/~jlee/www-scp/html/real-time_multispectral_camera.html • Multispectral filter array technique. • System Structure http://www.dpreview.com/learn/?/Glossary/ Camera_System/Color_Filter_Array_01.htm

  3. Terminology • MSFA samples • Mosaic Image • Demosaicking & Interpolation • Reconstructed image & Demosaicked image

  4. CFA Demosaicking Review • Bilinear Interpolation • Constant Hue-based Interpolation • Hue is defined as the ratio or difference of red and green, blue and green (i.e. R-G,B-G or R/G, B/G) • The HVS is more sensitive to hue artifacts than luminance errors. • Gradient Based Interpolation • Interpolate along the edges instead of across them. • Weighted sum Interpolation

  5. MSFA Review • Design requirement • Probability of appearance(POA) • Spectral consistency • Uniform distribution • A generic MSFA generation algorithm • Based on the binary tree decomposition. • Input: the number of spectral band and the probability of each band.

  6. Decomposition 1 1 2 2 1 2 1 1 2 2 1 1 2 2 1 1 2 2 Subsampling 3 4 5 6 5 5 3 3 4 4 6 6 7 8 5 5 3 3 4 4 6 6 Decomposition 7 8 7 8 3 7 3 8 6 4 6 4 3 8 3 7 6 4 6 4 MSFA Review Resulted MSFA

  7. Spectral Correlation • Why?? • Details are well preserved in spectral bands with more MSFA samples. • Different spectral bands possess similar edge information. • Validation metric: image similarity.

  8. Spectral correlation:example Similarity between different spectral bands

  9. Progressive Demosaicking Binary tree MSFA • Band selection • Pixel selection • Interpolation Pixel distribution of each band

  10. Band Selection • Determine the order of selecting different spectral bands for interpolation. • Band selection is associated with the tree level of each leaf, i.e. the probability of appearance. • The spectral band with the highest POA is interpolated first.

  11. Pixel Selection • Binary tree traversal from leaf nodes • Example: reconstruct spectral band “C”

  12. p3 p1 e p2 p4 Interpolation • Gradient Based Interpolation • Pros. vs. Cons. • Adaptive edge-sensing Interpolation: weighted sum of neighboring pixels. p1 p3 e p4 p2

  13. Edge-sensing Interpolation • Edge-likelihood in each direction of neighboring pixels. • Sobel edge detector and the second order derivative. • Weight estimator • Estimation of target pixel value

  14. Case studies POA=1/2 POA=1/8 POA=1/4

  15. Experiments-Data Description • Real Multispectral Data • 92AV3C9(9 bands) • FLC1(12 bands) • TIPJUL(7 bands) • Synthetic Multispectral Data • Eight objects

  16. Validation • Original multispectral images are sampled using MSFA to generate mosaic images. • Reconstruct the mosaic images using • proposed method with and without binary tree • bilinear interpolation with and without binary tree. • Compare between the original multispectral images and the reconstructed images. • Evaluation metric: subject visual comparison, objective RMSE, classification accuracy.

  17. Visual Comparison Proposed with binary tree Bilinear with binary tree Bilinear without binary tree Proposed without binary tree Original

  18. Visual Comparison Original Bilinear with and without tree Proposed with and without tree

  19. Experiments-RMSE • Root Mean Square Error where M and N are the size of image, and is the original image, represents the reconstructed image. The smaller the RMSE, the better the reconstructed image.

  20. Experiments-RMSE Comparison Real multispectral data set Synthetic multispectral data set

  21. Experiments-Pixel Classification • Real multispectral data • Each scene contains different classes. Choose the region we know the ground truth, from which we extract training and testing samples. • Knn classifier. • Synthetic multispectral data • Extract 20 training samples from each targets, all the targets pixels are testing samples. • Knn classifier

  22. Pixel classification accuracy Real multispectral data set Synthetic multispectral data set

  23. Experiments-Color Map Pixel classification of each targets, different colors represent different classification results Reconstructed Original Reconstructed Original Original Reconstructed

  24. Experiments-Object classification • Compare the spatial and spectral features. • Spatial: hu-moments (7D) • Spectral: average of target pixels (7D) • Study the effect of viewing distance. A schematic representation for extraction of training and testing set

  25. Object classification accuracy Original Bilinear Proposed

  26. Conclusion • The proposed progressive method outperforms non-progressive and bilinear interpolation. • The classification accuracy using reconstructed images is comparable with the one generated using original multispectral images. • The MSFA technique is a feasible solution for multispectral cameras.

  27. Thanks ?

More Related