420 likes | 1.04k Views
Image Matting with the Matting Laplacian. Chen-Yu Tseng 曾禎宇 Advisor: Sheng- Jyh Wang. Image Matting with the Matting Laplacian. Matting Laplacian
E N D
Image Matting with the Matting Laplacian Chen-Yu Tseng 曾禎宇 Advisor: Sheng-Jyh Wang
Image Matting with the Matting Laplacian • Matting Laplacian • A. Levin, D. Lischinski, Y. Weiss. A Closed Form Solution to Natural Image Matting. IEEE T. PAMI, vol. 30, no. 2, pp. 228-242, Feb. 2008. • Spectral Matting • A. Levin, A. Rav-Acha, D. Lischinski. Spectral Matting. IEEE T. PAMI, vol. 30, no. 10, pp. 1699-1712, Oct. 2008. • Matting for Multiple Image Layers • D. Singaraju, R. Vidal. Estimation of Alpha Mattes for Multiple Image Layers. IEEE T. PAMI, vol. 33, no. 7, pp. 1295-1309, July 2011. Center for Imaging Science, Department of Biomedical Engineering, The Johns Hopkins University
Image Matting • Extracting a foreground object from an image along with an opacity estimate for each pixel covered by the object Spectral Matting Result Input Image Conventional Segmentation Result
Image Compositing Equation = x α1 L1 Input Image x + α2 L2 x + α3 L3 Alpha Mattes Image Layers
Methodology • Supervised Matting • Unsupervised Matting • Spectral Matting Input Image Trimap (user’s constraint) Alpha Matte Input Image Matting Components
x x = + Local Models for Alpha Mattes , , and are unknown ill-posed problem Assume a and b are constant in a small window
Color Line Assumption Omer and M. Werman. Color Lines: Image Specific Color Representation. CVPR, 2004. Input Color Distributions
Local Models for Alpha Mattes for Multiple Layers Local Models Two color lines A color point and a color point Two color points and a single color line Four color points B G R
Local Models Two color lines A color plane and a color point Two color points and a single color line Four color points Color point Unknown color point Color plane
Local Models Two color lines A color plane and a color point Two color points and a single color line Four color points = Color point =++ Color plane
Local Models for Alpha Mattes for Multiple Layers Local Models Two color lines A color point and a color point Two color points and a single color line Four color points B G R
Overview of Spectral Matting Input Data Matting Laplacian Construction Input Image Local Adjacency Spectral Graph Analysis Data Component Generation Laplacian Matrix Output Components Components
Spectral Clustering Scatter plot of a 2D data set K-means Clustering Spectral Clustering U. von Luxburg. A tutorial on spectral clustering. Technical report, Max Planck Institute for Biological Cybernetics, Germany, 2006.
Graph Construction Similarity Graph Vertex Set Weighted Adjacency Matrix Similarity Graph • Similarity Graph • ε-neighborhood Graph • k-nearest neighbor Graphs • Fully connected graph Connected Groups
Graph Laplacian L: Laplacian matrix W: adjacency matrix For every vector D: degree matrix
L: Laplacian matrix W: adjacency matrix Example 2 1 3 4 Cost Function 5 Similarity Graph Good Assignment Poor Assignment * 2 1 2 1 3 3 4 4 5 5
LaplacianEigenvectors s.t.=1 : Eigenvector : Eigenvalue L is symmetric and positive semi-definite. The smallest eigenvalue of L is 0, the corresponding eigenvector is the constant one vector 1. L has n non-negative, real-valued eigenvalues 0= λ1 ≦ λ 2 ≦ . . . ≦ λ n. Input Image Smallest eigenvectors
K-means Projection into eigs space From Eigenvectors to Matting Components Smallest eigenvectors
Overview of Spectral Matting Input Data Graph Construction Input Image Local Adjacency Spectral Graph Analysis Data Component Generation Laplacian Matrix Output Components Components
Matting Laplacian x + = 1-α B x α F
Matting Laplacian Color Distribution
Matting Laplacian Matting affinity function Typical affinity function
Brief Summary K-means Clustering & Linear Transformation Input Image Laplacian Matrix Smallest Eigenvectors Matting Components
Supervised Matting Cost function with user-specified constraint: Foreground Background Unknown Input Trimap
Estimation Alpha Matte for Multi-Layers Karusch-Kuhn-Tucker (KKT) condition
Construction Assumption The vector of 1s lies in the null space of L, the solution automatically satisfies the constraint
Constrained Alpha Matte Estimation Image matting for n≥2 image layers with positivity + summation constraints
Karusch-Kuhn-Tucker (KKT) conditions For 0 < < 1 (i,i)=0 and(i,i)=0 Conventional Approaches Directly Clipping Equivalent to Introducing Lagrange Multipliers Refinement is neglected in conventional approaches
Algorithm 1. (b) Algorithm 2. (c) Spectral Matting. (d) SM-enhance.
Algorithm 1. (b) Algorithm 2. (c) Spectral Matting. (d) SM-enhance.
Summary • Image Matting with the Matting Laplacian • Construction of the Matting Laplacian • Image Compositing Model • Local-Color Affine Model • Supervised Closed-form Matting • Two-layer • Multiple-layer • Spectral Matting • Extended Applications
Depth Estimation Compositing Image Likelihood Prior Input Image Estimated Depth MAP Prior Refined Result Confidence Map
Input Image Transmission Prior Output Image Refined Transmission
Graph Laplacian and Non-linear Filters Global Optima Local Optima