An Interactive Background Blurring Mechanism and Its Applications

1. An Interactive Background Blurring Mechanism and Its Applications NTU CSIE Yan Chih-Yu Advisor: Wu Ja-Ling, Ph.D.

3. Outline • Introduction • Related Work • Method • Object Segmentation • Depth Map Generation • Image Defocus • Experimental Result • Applications • Conclusion

4. Introduction • Depth of field • Circle of confusion Photo plane Real world

5. Introduction • Depth of field • Circle of confusion Photo plane Out of focus Real light Circle of Confusion

6. Introduction • Depth of field Focus plane Photo plane Out of focus – blur range Real light Depth of field Circle of Confusion - Readability range

7. Introduction • Shallow focus Photo plane Real world

8. Introduction • Shallow focus • Highlight the subject by softening background diffusion Shallow focus by DSLR NIKON D90(2008) Deep focus by DC NIKON E4300 (2003)

9. Depth Map Generation Object Segmentation Applications Image Defocus Perspective Box Lazy Snapping Camera Settings Pop-up Card Alpha Matting Defocus Blur Face Detection Privacy Preserving Photo Browser Method

10. Method User Stroke User Stroke Resize to 20% Resize to 20% • Segment • Lazy snapping • Mean shift • Alpha matting • Face detection Mean shift Mean shift Do hard graph cut by lazy snapping Do hard graph cut by lazy snapping Segmentation tri-Map Segmentation alpha Map Lazy Snapping. ACM Trans. On Graphics 2004. Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI 2002. A Bayesian Approach to Digital Matting. CVPR 2001.

11. Method Perspective Box Perspective Box Pop-up card Pop-up card • Depth map • Perspective Box • Vanish point • Rear wall Normal vector Normal vector Depth Map Depth Map Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH 1997

12. Method Perspective Box Pop-up card Pop-up card • Depth map • Pop-up card Normal vector Normal vector Depth Map Depth Map

13. Method Perspective Box Pop-up card • Depth map • Normal vector Normal vector Normal vector Depth Map Depth Map

14. (a) (b) ub vb ub vb b b d d u v u v Method Depth Map Segment Map Camera setting • Image defocus • Blur circle diameter Aperture size Focal length of the lens Distance of focus Blur circle diameter

15. Method • Defocus blur • bokeh

16. Method • Defocus blur

17. Method • Near-by object case Focus on the flower Focus on the flower

18. Method Blur circle diameter Objects in front of the focus Objects behind the focus • Near-by object case Blur shape as alpha map Text synthesis inpainting Defocus Blur Defocus Blur Interpolation by alpha map Shallow focus image

19. Experimental Result 1 • Defocus blur method proposed in the system comparing with other blur filter results Deep focus Shallow focus

20. Experimental Result 1 • Defocus blur method proposed in the system comparing with other blur filter results Gaussian blur Defocus blur Defocus blur + Bokeh

21. Experimental Result 1

22. Experimental Result 1 Original photograph • Defocus blur method proposed in the system comparing with other blur filter results Gaussian blur Defocus blur Defocus blur + Bokeh

23. Experimental Result 2 • with / without depth variation in the background Real photograph taken by DSLR

24. Experimental Result 2 • with / without depth variation in the background Depth variation Original photograph Result after post-processing Depth fixed

25. Experimental Result 3 • Near-by object case Focus on the flower Focus on the flower

26. Experimental Result 3 • Near-by object case Interpolation result Without inpainting

27. Experimental Result 4 • Comparing with Photoshop Original image Photoshop (1 Hour) Our system (3~5min)

28. Applications - Privacy Preserving

29. Applications -Partial Viewing

30. Applications -Partial Viewing

31. Applications - Image Transition at Photo Browser

32. Applications - Image Transition at Photo Browser

33. Applications - Image Transition at Photo Browser Movie with defocus blur effect

34. Applications - Image Transition at Photo Browser Concatenate two unrelated images

35. Conclusion • We proposed an interactive refocusing tool for background blurring • Simple user hint • Defocus blur kernel • Concatenate two related picture • Future work • concatenate two or more unrelated pictures • Color based image retrieval technique

36. Q&A

37. Experimental Result 1 • Defocus blur method proposed in the system comparing with other blur filter results Deep focus Shallow focus Difference

38. Experimental Result 1 • Defocus blur method proposed in the system comparing with other blur filter results Defocus blur PSNR:22.79 Gaussian blur PSNR:20.34 Shallow focus Difference

39. END THANK YOU