Improving Image Matting using Comprehensive Sampling Sets

1. Improving Image Matting using Comprehensive Sampling Sets CVPR2013 Oral

2. Outline • Introduction • Approach • Experiments • Conclusions

3. Introduction • Accurate extraction of a foreground object from an image is known as alpha or digital matting.

4. Introduction • Applications

5. Introduction Compositing Equation Observed color of pixel z Background color of pixel z Foreground color of pixel z Alpha value of pixel z

6. Introduction • Range of α : [ 0, 1] • α =1 , foreground. • α =0 , background.

7. Introduction Unknown Region • ill-posed problem • Typically, matting approaches rely on constraints • Assumption on image statistics • User constraints like Trimap Known Background Known Foreground

8. Introduction • Current alpha matting approaches can be categorized into • alpha propagation based method • color sampling based method

9. Introduction • Alpha propagation based method • Assume that neighboring pixels are correlated under some image statistics and use their affinities to propagate alpha values of known regions toward unknown ones.

10. Introduction • Color sampling based method • collect a set of known foreground and background samples to estimate alpha values of unknown pixels. • The quality of the extracted matte is highly dependent on the selected samples. • missing true samples problem

11. Introduction

12. Approach • Gathering comprehensive sample set • Choosing candidate samples • Handling overlapping color distributions • Selection of best(F, B)pair • Pre and Post-processing

13. Approach • Gathering comprehensive sample set • For each region, a two-level hierarchical clustering is applied. • first level, the samples are clustered with respect to color • second level , respect to spatial index of pixels.

14. Approach • Gathering comprehensive sample set

15. Approach • Choosing candidate samples • Each pixel in the unknown region collects a set of candidate samples that are in the form of a foreground-background pair

16. Approach • Handling overlapping color distributions

17. Approach • Selection of best(F, B)pair K : chromatic distortion S : spatial statistics of the image C : color statistics

18. Approach

19. Approach

20. Approach Cohen's d

21. Approach Trimap Expanded Trimap • Pre-processing • An unknown pixel z is considered as foreground if, for a pixel q ∈ F,

22. Approach • Post-processing • Eq. (2) is further refined to obtain a smooth matte by considering correlation between neighboring pixels. • Cost function [5] consisting of the data term a and a confidence valuef together with a smoothness term consisting of the matting Laplacian [10] [10] A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(1):228–242, 2007 [5] E. Gastal and M. Oliveira. Shared sampling for real time alpha matting. InProc. Eurographics , 2010, volume 29, pages 575–584, 2010.

23. Experiments • www.alphamatting.com

24. Experiments • www.alphamatting.com

25. Experiments

26. Experiments

27. Experiments

28. Conclusions • A new sampling based image matting method • New sampling strategy to build a comprehensive set of known samples. • This set includes highly correlated boundary samples as well as samples inside the F and B regions to capture all color variations and solve the problem of missing true samples.