1 / 16

Halftoning-Inspired Methods for Foveation in Variable Acuity Superpixel Imager Cameras

Halftoning-Inspired Methods for Foveation in Variable Acuity Superpixel Imager Cameras. Thayne R. Coffman 1,2 Prof. Brian L. Evans 1 (presenting) Prof. Alan C. Bovik 1. 2 21 st Century Technologies, Inc. Austin, Texas.

deanna
Download Presentation

Halftoning-Inspired Methods for Foveation in Variable Acuity Superpixel Imager Cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Halftoning-Inspired Methods for Foveation in Variable Acuity Superpixel Imager Cameras Thayne R. Coffman1,2 Prof. Brian L. Evans1 (presenting) Prof. Alan C. Bovik1 221st Century Technologies, Inc. Austin, Texas 1 Center for Perceptual Systems Department of Electrical and Computer Engineering The University of Texas at Austin http://www.cps.utexas.edu November 2, 2005, IEEE Asilomar Conference on Signals, Systems, and Computers

  2. Motivation: Foveated Imagery • Foveated imagery has variable spatial resolution • Human visual system • Provides simultaneous • Wide field of view • High resolution on regions of interest • Low bandwidth • 19% bandwidth means 19% of “superpixels” • No compression in talk Full resolution (100% bandwidth) Variable resolution(19% bandwidth)

  3. Motivation: VASI™ Cameras • Variable Acuity Superpixel Imager (VASI) cameras • Generate foveated images by sharing charges on focal plane array • Achieve 1000-4000 frames/sec (e.g. to measure engine RPMs) • Pixel sharing reconfigured to achieve a particular frame rate • Use of 1x1, 2x2, and 4x4 pixel sharing [McCarley et al., 2002] VASI is a trademark of Nova Sensors, Inc. Images from [McCarley et al., 2002]

  4. The Catch • Desired spatial acuity (resolution) is usually specified as a continuous amplitude function on the range (0,1] • Translate desired resolution function to VASI™ binary share/no-share control signal at very high frame rates Foveation like the human eye (left pixelation) Two fovea (right pixelation)

  5. Halftoning for VASI Control Signals • Select a small number of test images • Manually specify desired resolution (using Gaussians) • Evaluate halftoning methods to control signal translation • Figures of merit to predict object recognition performance • Peak SNR (PSNR) • Weighted SNR (WSNR) • Universal Quality Index (UQI) • Percentage of Bandwidth (PBW) Shared up Shared left X Control signal for charge sharing at a pixel X

  6. Halftoning Methods Explored • Classical screening • 9-level clustered dot • 9-level dispersed dot • Block error diffusion • Floyd-Steinberg error diffusion • Blue noise dithering • White noise • Specialized (non-general) methods • vasiHalftone • vasiHalftone2 Dispersed dot screening F-S error diffusion White noise vasiHalftone

  7. Specialized Methods • Generate semi-regularly spaced squares • Square size varies with inverse of desired bandwidth • Side is 2K in vasiHalftone & unconstrained in vasiHalftone2 Full-resolution image Continuous desired resolution signal Binarized control (sharing) signal Foveated image

  8. Nontrivial Translation of Control Signal • Halftoning algorithms aim to achieve a specific ratio of white or black pixels, e.g. • For constant I(r,c)=0.1, 10% of pixels will be white (“don’t share”) • For constant I(r,c)=0.8, 80% of pixels will be white (“don’t share”) • But bandwidth and resolution are functions of geometry also Example 1 Example 2 Control signal Resulting image Control signal Resulting image 46% of pixels don’t share charge: 15% bandwidth 50% of pixels don’t share charge: 1% bandwidth

  9. Nontrivial Translation of Control Signal • Relationship between percent of “don’t share” pixels and bandwidth is different for every halftoning method • Eliminate nonlinearity by applying an inverse function • Implemented with lookuptables storing x = f-1(y) • Given target bandwidth andhalftoning method, findaverage value (x-axis) to usein continuous control signal • Stairstep patterns inrelationship limitcontrol over bandwidth • Floyd-Steinberg gives piecewise linear map and best bandwidth control

  10. Nontrivial Translation of Control Signal • Results are greatly improved • Better bandwidth control • Better foveation results • Floyd-Steinberg (F-S) results below Desired bandwidth =11.9% from ideal control signal Uncompensated control signal Achieved bandwidth = 2.6% Compensated control signal Achieved bandwidth = 12.5%

  11. Results: F-S Error Diffusion • Good performance and good bandwidth control • Good SNR in foveae means accurate object recognition • Good SNR in periphery means good object detection • Good bandwidth control means precise VASI frame rate control Original Sharing Signal Resulting Image PSNR = 17.5 dB (33.3 dB in ROI) WSNR = 16.4 dB (33.8 dB in ROI) Desired BW = 11.6% Actual BW = 12.1% Inflation = 4%

  12. Results: vasiHalftone and vasiHalftone2 • For a given desired resolution signal, methods consistently • Had better PSNR & WSNR than other methods • Overshot desired bandwidth by ~30-100% • Essentially “cheating” by using extra bandwidth Original Sharing Signal Resulting Image PSNR = 13.3 dB WSNR = 16.9 dB Desired BW = 9.6% Actual BW = 18.8% Inflation = 97%

  13. Results: Other Halftoning Methods Original Block error diffusion Blue noise Original Clustered dot Dispersed dot White noise

  14. Conclusions • Floyd & Steinberg error diffusion gives the best results while still being able to control bandwidth precisely • vasiHalftone and vasiHalftone2 • Consistently the best PSNR, WSNR • Poor bandwidth control – overshot specifications by 30-100% • Bandwidth inflation means it’s not a fair comparison (they’re cheating) • Stochastic methods (white & blue noise) perform poorly • Outperformed by deterministic approaches • Susceptible to “catastrophic gray-out” • Classical screening performs marginally and has poor bandwidth control

  15. Recent Work • vasiHalftone3 and vasiHalftone4 • Extensions to eliminate simplifying assumption that VASI™ shareUp and shareLeft signals are equal • This eliminates single-pixel artifacts in non-foveal regions • Eliminated lookup table (LUT) in F-S approach by determining closed-form inverse relationship • Significant speedup • Greatly shrank LUT in vasiHalftone & vasiHalftone2 approaches • Leveraged “stairstep” form of inverse relationship • 10x speedup in vasiHalftone, 4x speedup in vasiHalftone2 • 21st Century Technologies and Nova Sensors are actively collaborating on further work • Sponsored by U.S. Air Force Research Laboratory

  16. Background References • B.E. Bayer, “An optimum method for two level rendition of continuous-tone pictures,” Proc. IEEE Int. Conf. on Communications, Conf. Rec., pp. (26-11)-(26-15), 1973. • R. Floyd and L. Steinberg, “An adaptive algorithm for spatial grayscale,” Proc. SID’76, pp. 75-77, 1976. • P. McCarley, M. Massie, and J.P. Curzan, “Large format variable spatial acuity superpixel imaging: visible and infrared systems applications,” Proc. SPIE, Infrared Technology and Applications XXX, vol. 5406, pp. 361-369, Aug 2002. • V. Monga, N. Damera-Venkata, and B.L. Evans, Halftoning Toolbox for Matlab. Version 1.1 released November 7, 2002. Available online at http://www.ece.utexas.edu/~bevans/projects/halftoning/. • R.A. Ulichney, “Dithering with blue noise,” Proc. IEEE, vol. 76, pp. 56-79, Jan 1988. • Z. Wang, A.C. Bovik, and L. Lu, “Wavelet-based foveated image quality measurement for region of interest image coding,” Proc. IEEE Int. Conf. Image Proc., vol. 2, pp. 89-92, Oct 2001.

More Related