1 / 36

Imager Design using Object-Space Prior Knowledge M. A. Neifeld University of Arizona

Imager Design using Object-Space Prior Knowledge M. A. Neifeld University of Arizona. OUTLINE 1. The Last Slot 2. Introduction 3. PSF Engineering 4. Feature-Specific Imaging. Neifeld IMA 2005. Neifeld IMA 2005. Introduction: objects are not iid pixels.

kaspar
Download Presentation

Imager Design using Object-Space Prior Knowledge M. A. Neifeld University of Arizona

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Imager Design using Object-Space Prior Knowledge M. A. Neifeld University of Arizona OUTLINE 1. The Last Slot 2. Introduction 3. PSF Engineering 4. Feature-Specific Imaging Neifeld IMA 2005

  2. Neifeld IMA 2005 Introduction: objects are not iid pixels. - Conventional cameras are designed to image iid pixels  impulse-like point-spread-functions (identity transformation)  generic metrics such as resolution, field of view, SNR, etc. - Real objects are not iid pixels so don’t estimate pixels - This keeps the compression guys employed! - (106 pixels)(3 colors/pixel)(8 bits/color) = 2.4x107 bits - (1011 people)(4x109 years)(109 images/year) = 4x1029 images  <100 bits - The set of “interesting” objects is small - Many ways to characterize “interesting” objects: power spectra, principal components, Markov fields, wavelet projections, templates, task-specific models, finite alphabets, etc. Information depends upon task: • Option 1 - this is a random image  I = 107 bits • Option 2 – this is a “battlefield” image  I = ? bits … how to quantify PDF! • Option 3 – this image either contains a tank or not  I = 1bit • … task-specific source model

  3. Neifeld IMA 2005 largest return rmse = 7.3mm Object Object Measurement IBP – 28% Viterbi rmse = 0.6mm Wiener rmse = 5.8mm IBPP – 24% 2D4 - 2% Introduction: post-processing exploits priors. - Linear Restoration: de-noising and de-blurring exploit noise statistics, object power spectra, principal components, wavelets, … - Nonlinear Restoration: super-resolution uses finite support, positivity, finite alphabet, power spectra, wavelets, principal components, isolated points, … - Recognition: features, templates, image libraries, syntax, invariance, … - Finite Alphabet Post-Processing Examples LADARMulti-Frame Super-Resolution Optical blur = 1.5 and pixel-blur = 2. Reconstruction from 2 images, σ = 1% Axial extent of target = Temporal pulse width = 30mm. Target feature size = Scan step size = 4.6mm

  4. Neifeld IMA 2005 rM r2 r1 rM r1 r2 r2 rM r1 rM … … … Introduction: plausibility of a single pixel imager. • Measure only what you want to know Imager • Fluorescent markers • Distant “bright” objects: aircraft, missile, stars Source volume r1 r2 y x M : Number of point sources z Strong Object Model: Conventional image • Equal-intensity monochromatic point sources • Scene is completely specified by sources positions: • Imager Goals: • Estimate point source position(s): { } • Conventional image may be formed as a post-processing step

  5. Neifeld IMA 2005 1 2 Source Volume phase mask 1 3 2 Detector d1 ,h1 Lens 3 Measurement log-pdf Measurement log-pdf Measurement log-pdf cubic phase random phase Introduction: information-based design. • Optimize imager based on information metric. • Maximize measurement entropy. • Select detector sizes and positions based on measurement pdf. 40cm 1cm3 NEP=2nW source power = 0.5mW

  6. Neifeld IMA 2005 Introduction: single pixel imager results. Single Source in Volume Multiple Sources in Volume • Object-space prior knowledge should inform the optical design • Let’s utilize this viewpoint in a more useful problem domain

  7. Neifeld IMA 2005 PSF ENGINEERING

  8. Neifeld IMA 2005 spatial ambiguity Frame K Frame 2 Frame 1 shift camera ….. PSF Engineering: Under-Sampled Imagers • Imagers for which pixel size > optical spot size. . • Large pixels result in under-sampling/aliasing. • Sub-pixel shifted measurements to resolve ambiguity. • Optical degrees of freedom not exploited. • We consider engineering optical point spread function.

  9. Neifeld IMA 2005 Imaging Model Object: f Imaging operator: H Measurements: g ….. Phase-mask Sub-pixel shifts N = 512x512 ….. • Sensor details: • Pixel = 7.5 mm • Under-sampling = 15x • Full well capacity = 49ke- • Spectral bandwidth = 10nm • Center wavelength = 550nm • Optics details: • Resolution = 0.2mrad/1mm • Field of view = 0.1 rad • Thickness = 5mm • Aperture = 2.75mm • F/# = 1/1.8 M = 34x34 • Single frame signal to noise ratio: SNR = 10log[sqrt(Ne)] = 23.3dB • SNR can be improved via multi-frame averaging ~ sqrt(K) • Total photon-count is kept constant over multiple-frames.

  10. Neifeld IMA 2005 • Linear imaging model: g = Hf + n (note: n is AWGN) • Block-wise shift-invariant imaging operator H is M x N • Problem: M << N (e.g., M=N/15) • Linear minimum mean square error (LMMSE) reconstruction:f = Wg • LMMSE operator: W = RfHt(HRfHt+Rn)-1 • No Priors = flat PSD • Priors = power law PSD or triangle PSD ^ Example training objects Linear Reconstruction Power Law PSD(f) = 1/f PSD model

  11. Neifeld IMA 2005 + + Performance Measures RMSE=8.6% • Root Mean Squared Error: n Composite Channel Hc g LMMSE Reconstruction Object • Angular resolution: n Reconstruction to Diffraction-limited sinc2 Composite Channel Hc g Point Object f = d(r) ^ f =0.4mrad

  12. Neifeld IMA 2005 Conventional/TOMBO Imager Results TOMBO Imager Conventional Imager Shift-sensor sub-pixel shift Sub-pixel shifted measurements Resolution for TOMBO RMSE for TOMBO

  13. Neifeld IMA 2005 Alternate PSF impulse-like PSF • Consider use of extended point spread function(PSF) • Design issue #1: retain full optical bandwidth • Design issue #2: tradeoff SNR for condition number extended PSF • Pseudo-Random Phase masks for extended PSF Realization of a spatial Gaussian random process.  - mask roughness  - mask correlation length Pseudo-Random Phase mask Enhanced Lens (PRPEL) Example PSF(=0.5,=10 ) Modulation Transfer Function

  14. Neifeld IMA 2005 Resolution for PRPEL and TOMBO Resolution Results • All designs use optimal roughness. • Note more rapid convergence of PRPEL compared to TOMBO. • Higher resolution achieved by PRPEL at reduced number of frames. • PRPEL achieves 0.3mrad resolution at K=5 compared to K=12 for TOMBO.

  15. Neifeld IMA 2005 RMSE Results TOMBO PRPEL RMSE for PRPEL and TOMBO K=1 K=1 K=2 K=2 K=3 K=3 • PRPEL makes effective use of prior knowledge at K=1 • Note more rapid convergence of PRPEL. • PRPEL consistently out-performs TOMBO.

  16. Neifeld IMA 2005 PRPEL Summary 4% RMSE requirement RMSE achieved at M=N/4 0.3mrad Resolution requirement Resolution achieved at M=N/4 • PRPEL imager achieves 60% improvement in resolution. • PRPEL imager obtains 22% improvement in RMSE.

  17. Neifeld IMA 2005 PSF Engineering via SPEL • Sine-Phase mask Enhanced Lens(SPEL) : Phase offset Spatial-frequency Amplitude   Phase-mask • Pick N=3: yields 12 free parameters for optimization. • Optimization criteria: • RMSE computed over object class using LMMSE operator. • PSF is optimized for each value of K.

  18. Neifeld IMA 2005 Optimized PSF Observations K=1 • Note smaller support of SPEL PSF compared to PRPEL PSF. • SPEL PSF also contains sub-pixel structure. • SPEL PSF has more efficient photon-distribution. Observations K=2 • PSF support reduces with increasing K. • SPEL PSF is array of delta pulses.

  19. Neifeld IMA 2005 Optimized PSF: System Implications K=16 Observations • SPEL PSF converges to delta pulses as K increases. • In limit K16 we observe that SPEL PSF to converge to TOMBO-like PSF.

  20. Neifeld IMA 2005 Results PRPEL SPEL RMSE : Power law PSD K=1 K=1 RMSE for SPEL, PRPEL, and TOMBO K=2 K=2 K=3 K=3 • SPEL provides best use of prior knowledge for K=1 • SPEL outperforms TOMBO by 47% in terms of RMSE(K=8). • SPEL improves RMSE by 35% compared to PRPEL (K=8).

  21. Neifeld IMA 2005 Results Angular resolution Resolution for SPEL,PRPEL and TOMBO • Note PSF optimization was performed over RMSE. • SPEL out-performs TOMBO. • SPEL performance compared to PRPEL improves with increasing K. • PSF engineering can exploit weak object prior knowledge to improve performance • Stronger object prior knowledge can enable non-traditional image measurement

  22. Neifeld IMA 2005 FEATURE-SPECIFIC IMAGING

  23. Neifeld IMA 2005 Passive Feature-Specific Imaging: Motivation Conventional imaging system PCA, ICA, Fisher, Wavelet, etc. Feature extraction Features Task Restoration, recognition, compression, etc. noisy image noise Feature-specific optics Features Task Feature-specific imaging system (FSI) noise • Feature-Specific Imaging (FSI) is a way of directly measuring linear features (linear combinations of object pixels). • Attractive solution for tasks that require linear projections of object space • Let’s consider a case for which task = pretty picture

  24. Neifeld IMA 2005 FSI for Reconstruction • PCA features provide optimal measurements in the absence of noise Noise-free reconstruction: PCA solution : General solution : photon count constraint Result using PCA features:

  25. Neifeld IMA 2005 RMSE = 124 RMSE = 12.9 RMSE = 12 RMSE = 11.8 Optimal Features in Noise • PCA features are not optimal in presence of noise Noise-free problem statement: • Object block size = 4x4 • Noise = AWGN • We use stochastic tunneling to optimize/search Note: PCA error is no longer monotonic in the number of features  trade-off between truncation error and photon count constraint

  26. Neifeld IMA 2005 Optimal Features in Noise • Error increases as number of feature increases for PCA solution • Reconstructed is improved significantly by using optimal solution • Optical implementation requires non-negative projections

  27. Neifeld IMA 2005 Passive FSI Result Summary • Optimal FSI is always superior to conventional imaging • Non-negative solution is a good experimental system candidate

  28. Neifeld IMA 2005 Passive FSI for Face Recognition • Face recognition from grayscale image feature measurements • Class of 10 faces, 600 images per face • Training = 3000 faces and testing = 3000 faces • Features: wavelet, PCA, Fisher, … • Recognition algorithms: - k – nearest neighbor based on Euclidean distance metric - 2-layer neural networks batch trained using back-propagation with momentum FSI Sample images from face database [Each image is 128x96] Conventional First Wavelet feature of the above images [Each feature is 8x6]

  29. Neifeld IMA 2005 Passive FSI Optical Implementations

  30. Neifeld IMA 2005 Object Illumination pattern Projector Conventional cameras Active Feature-Specific Imaging: Motivation • What is active illumination ? • Project known structure onto scene • Additional degrees of freedom • improve imager performance • Past work on active illumination focused on: • Obtain depth-information for 3D objects • Enhanced resolution for 2D objects • Our goals: • Improve object- and/or task-specific performance • Simplify light collection hardware

  31. Neifeld IMA 2005 FSAI System Flow Diagram • Illumination patterns are eigenvectors (refer as PCA - FSAI) PM P2 Sequence of illumination patterns 16 × 16 replication of eigenvector P1 16 × 16 detector Light Collection Object G 64 × 64 Photodetector noise (AWGN) H (optics operator) (Estimate of feature weight) • Advantages • Small number of detectors • High measurement SNR • Task is to produce object estimate using these values Vector of Measurements ece

  32. Neifeld IMA 2005 FSAI Post-Processing Linear post-processing Ŵ Measurement vector Ĝ = Ŵ R ≠ (suboptimal in noise) • Post-processing operator Ŵ is obtained by minimizing J • The MMSE operator is given by: N 2 = number of pixels, M =number of patterns • Metric to evaluate reconstructions : ece

  33. Neifeld IMA 2005 Illumination Using Optimal Patterns • PCA vectors are not optimal in presence of noise • Minimize the residual MMSE (JMMSE) with respect to both Pi’s and Ti’s • Optimal features depend on M, SNR SNR = 26 dB PCA PCA M = 4 M = 8 optimal optimal ece

  34. Neifeld IMA 2005 FSAI Results SNR = 26 dB (LOW NOISE) Original object PCA-FSAI (uniform T) PCA-FSAI (optimal T) Optimal FSAI M = 4 M = 8 • Minimum from PCA-FSAI • RMSE = 0.0633 • Minimum from optimal FSAI • RMSE = 0.0465 ece

  35. Neifeld IMA 2005 FSAI Results Summary ece

  36. Neifeld IMA 2005 Conclusions • Objects are not iid pixels • Pixel-fidelity should not be the goal of an imager • Need new non-traditional design metrics • Design should reflect prior knowledge of objects • Object-specific imagers (e.g., SPEL) • Joint design of optics and post-processing • Design should reflect prior knowledge of application • Task-specific imagers (e.g., FSI)

More Related