1 / 39

Software image stabilization comes of age

Software image stabilization comes of age. Michal Šorel. Removing camera motion blur. Alternative to OIS (optical image stabilization) systems Camera motion, not subject motion. Talk outline. How to describe the blur? (velocity field, space-variant PSF ) Common setups Single blurred image

ophira
Download Presentation

Software image stabilization comes of age

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software image stabilization comes of age Michal Šorel

  2. Removing camera motion blur • Alternative to OIS (optical image stabilization) systems • Camera motion, not subject motion

  3. Talk outline • How to describe the blur? (velocity field, space-variant PSF ) • Common setups • Singleblurred image • Multiple blurred images • Multiple underexposed/noisy images • One blurred and one underexposed image • Perspectives

  4. Stabilizer of 3D camera rotation • Rigid body – 6 degrees of freedom • Natural coordinate system • Camera rotation is mostly dominant • Blur is independent of scene depth (that is why optical image stabilizers can work)

  5. Camera rotates downwards ↓

  6. Velocity field

  7. Rotation about optical axis

  8. General 3D rotation

  9. Space-variant PSF • PSF h ... depends on position (x,y) • Generalized convolution • Convolution case – h is called convolution kernel or convolution mask

  10. (x2,y2) h(s,t; x2,y2) (x1,y1) h(s,t; x1,y1) h(s,t; x3,y3) (x3,y3) PSF for camera shake

  11. Talk outline • How to describe the blur? (velocity field, space-variant PSF ) • Common setups • Singleblurred image • Multiple blurred images • Multiple underexposed/noisy images • One blurred and one underexposed image • Perspectives

  12. Single image deblurring • Rob Fergus building on the work of James Miskin • Bayesian approach • Approximation – conditional distributions of PSF and image are considered independent • Priors on image gradients and blur kernels as a mixture of Gaussians and exponential functions

  13. Bayesian approach

  14. Image prior Intensity histogram Gradient histogram

  15. Image prior Gradient histogram

  16. Image priors Tikhonov regularization TV regularization

  17. PSF prior

  18. Functional to minimize

  19. Rob Fergus (Example I)

  20. Rob Fergus (Example II)

  21. Talk outline • How to describe the blur? (velocity field, space-variant PSF ) • Common setups • Singleblurred image • Multiple blurred images • Multiple underexposed/noisy images • One blurred and one underexposed image • Perspectives

  22. Multiple blurred images • Multichannel blind deconvolution • Convolution model of blurring • Solved by minimization of

  23. Multiple blurred images

  24. Talk outline • How to describe the blur? (velocity field, space-variant PSF ) • Common setups • Singleblurred image • Multiple blurred images • Multiple underexposed/noisy images • One blurred and one underexposed image • Perspectives

  25. Multiple noisy images N imagestime t’=t/N noise variance σ2/N • Noise variance of the sum of N images is the same as of the original image • The difficult part is registration • Main problem slow read-out 1 imagetime t =1snoise variance σ2

  26. Talk outline • How to describe the blur? (velocity field, space-variant PSF ) • Common setups • Singleblurred image • Multiple blurred images • Multiple underexposed/noisy images • One blurred and one underexposed image • Perspectives

  27. Blurred + underexposed image • noisy ~ underexposed (exposure time changes contrast) • patented in 2006 • since 2006 - several papers assuming convolution model • our space-variant version sent to BMVC 2008

  28. Deblurring algorithm Blurredimage Noisyimage

  29. Image registration • Small change of camera position – small stereo base • Static parts of the scene can be modelled by projective tranform found by RANSAC • Lens distortion can be neglected • Less important parts of scene can move

  30. Blurred + underexposed results

  31. Blur kernel adjustment • Regions lacking texture • Regions of pixel saturation

  32. Restoration • Minimization of functional • PSF h interpolated from estimated convolution kernels

  33. Shopping center (details)

  34. Bookcase example

  35. Bookcase (details)

  36. Summary/Perspectives • Denoising – artifactsorreadoutproblems • Single image approach – takes time, imprecise PSF, unable to distinguish intentional depth of focus, limited to convolution model • Multiple blurred images – computationally expensive, fewerartifacts • Blurred + underexposed image – fastest,algorithmforspace-variant deblurringexists

  37. Discussion, questions... Michal ŠorelInstitute of Information Theory and Automation of the ASCRsorel@utia.cas.cz

  38. Project ? • Segmentation of moving objects from noisy-blurry image pair • Restoration from one noisy and two blurred images • Space-variant single image deblurring • Combination with demosaicing to get full resolution image • Implementation in fix-point arithmetic? ...

  39. Cause of blur • Long exposure time -> apparent image motion more than about half pixel • Why do we need long time? • enough photons to avoid quantization and shot noise • Small aperture to achieve high depth of focus • Small aperture because of tele lens construction limitations

More Related