Motion Blur Estimation at Corners

1. Motion Blur Estimation at Corners Giacomo Boracchi and Vincenzo Caglioti giacomo.boracchi@polimi.it

2. Motion Blurred Image

3. Motion Blurred Image

4. Motion Blurred Image

5. Motion Blurred Image

6. Motion Blurred Image

7. Motion Blurred Image

8. Motion Blurred Image

9. Motion Blurred Image

10. Motion Blurred Image

11. Motion Blurred Image

12. Preliminary Remarks • Dealing with blurred images it is complicated (lack of information) • Blur is often assumed uniform, but this is restrictive • We propose to analyze blur on some image regions • We focus on regions containing a corners • We consider only motion blur • Blur is approximated as parametric – direction and length - • Estimation of corner linear displacement • Any other blurring phenomena are neglected (e.g out of focus blur).

13. Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Concluding Remarks

14. Blurred Image Model • A blurred noisy image given the original image • and the blur operator • we assume that blur locally is constant point spread function and

15. Blur Assumptions • Point spread function has 1D support, • Constant value (Uniform Speed) • Parametric approach, estimate and we call the corner displacement , the vector having direction and length • These assumptions hold only locally…

16. Presentation Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Concluding Remarks

17. Why Corners? • No Aperture Problem, when blurred. • Easy to Detect (Harris, Hessian) • Easy to Model • Meaningful for scene understanding

18. 2 2 1 1 The Corner Model • The Corner has to be binary in the considered region D • Not every displacement can be managed. D Exclude “self intersecting” corners

19. Presentation Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Concluding Remarks

20. Vector Relation • Consider an Image Region D containing a blurred corner • in the noise free case, • the aperture problem holds • However both at corners blurred edges may be used to solve this ambiguity

21. 2 T 3 ( ) r I x w ¡ ¡ n n 6 7 : : : ° ° T T 6 7 T ( ) ( ) [ ] r I A ¢ ~ ~ ( ) [ ] A ¢ x ~ i w x v v w w w = = ¡ 0 0 ° ¡ ° 6 7 x x n n x v a r g m n v w w w ; : : : ; ; : : : ; = 0 ¡ x v n n 6 7 ; : : : ; ; : : : ; ° ° 2 4 5 : : : T ( ) r I x w n n i ¡ < < w n n i ; Least Square Solution • A “good” image region should containpixels from both blurred edges • Several pixels have to be considered, for example weights

22. ° ° T ( ) [ ] A ¢ ~ i ¡ ° ° x v a r g m n v w w w = 0 ¡ x v n n ; : : : ; ; : : : ; ° ° 2 Drawbacks • Solution is not robust in presence of outliers and noise • Whenever image assumptions are not met (e.g. textured or shaded corners, smoothed contours, other image artifacts) solution is seriously corrupted. • Compute the solution on every pixel : method is slow • Requires a filtering procedure as every estimate depends on • Then, better look for a vector that satisfy the basic equation for a significant number of pixels, disregarding how far from the solution is for few pixels

23. Presentation Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Concluding Remarks

24. ( ) r I x ( ) N ¢ x = j j ( ) j j 2 r I x Robust Solution • Considering only two gradient vectors it would be enough if appropriately chosen

25. The Hough Transform • For each input data determine theset of possible solutions. • The solutions are represented in the parameter space • A vote (1) is assigned to all parameters that are compatible with a given data being , the parameters, the coordinates of end point (in pixels) • Evaluate all inputs and sum the votes • The most voted pair in the parameter space are taken as solution,as they represent the parameters satisfying most of data

26. 2 ¡ ¢ T h ³ ´ i ( ( ( ) ) ) ( [ ] ( ) ) k ` ` ` N R N ¡ x ¾ x u u u u u u ( ) = u ( ) ` ¼ µ 1 1 2 2 1 2 2 ¡ ¡ x x ´ ; ; ; u u e x p = 1 2 2 j j k ; 1 + u ¾ r 1 ´ The votes in parameter space • Consider also parameters close to the solutions • Assign them a fraction of vote (<1) • Assign a full vote to exact solutions • Being a tuning parameter and noise standard deviation • For every data , votesare assigned by this vote function opportunely rotated and translated

27. Robust Solution- Votes sum • Sum of votes

28. Presentation Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Concluding Remarks

29. ¡ ¢ ( ( ( ) ) ) ( ( ) ) ( ) ( ) ( ) » » I N K N 0 0 + + x x x x x x ´ y ¾ ¾ ´ x x » = » = » 1 2 ´ ; ; ; ; Experiment on Synthetic Images • Synthetic images constructed according to • where represents electronic noise represents differences between the binary corner and the synthetic image

30. Experiments on Synthetic images • Point Spread Function of 10° degrees and 20 pixels extents

31. Experiments on Synthetic images • Point Spread Function of 70° degrees and 30 pixels extents

32. Experiment on a Test Image • 5 regions containing a corner have been selected on house image

33. Results on a Test Image • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels Error in pixels : 2.07

34. Results on a Test Image • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels Error in pixels : 2.75

35. Results on a Test Image • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels Error in pixels : 3.19

36. Results on a Test Image • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels Error in pixels : 1.87

37. Results on a Test Image • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels • house image has been artificially blurred by motion blur psf having • direction 30 degrees • length 25 pixels Error in pixels : 2.04

38. Experiments on camera images • Triplets of images have been taken according to the following scheme • still image (A) • Blurred image moving the camera on a rack (B) • still image (C) • Motion has been estimated in selected image regions in B and compared with the ground truth obtained by matching the feature in the corresponding regions in A and in C.

39. A - still image at initial camera position

40. B – the Blurred Image

41. C – still image at final camera position

42. Five selected Image regions

43. Five selected Image regions

44. Five selected Image regions

45. Five selected Image regions

46. Five selected Image regions

47. Results from camera images

49. Presentation Outline • Image Model • Corner Model • Problem Solution • Robust Solution • Experiments • Ongoing Works

50. Conclusions • Fourier based methods usually fail at corners and on small regions • Method to estimate motion blur parameters at corners from a single blurred image • We handle space varying blur as every image region is considered separately