1 / 28

Spectral Matting Anat Levin 1,2 Alex Rav-Acha 1 Dani Lischinski 1

Spectral Matting Anat Levin 1,2 Alex Rav-Acha 1 Dani Lischinski 1. 1 School of CS&Eng The Hebrew University. 2 CSAIL MIT. Hard segmentation and matting. compositing. Hard segmentation. Source image. matte. compositing.

haracha
Download Presentation

Spectral Matting Anat Levin 1,2 Alex Rav-Acha 1 Dani Lischinski 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spectral Matting Anat Levin1,2 Alex Rav-Acha1 Dani Lischinski1 1School of CS&Eng The Hebrew University 2CSAIL MIT

  2. Hard segmentation and matting compositing Hard segmentation Source image matte compositing

  3. Previous approaches to segmentation and matting Input Hard output Matte output

  4. Unsupervised Previous approaches to segmentation and matting Input Hard output Matte output Spectral segmentation: Shi and Malik 97 Yu and Shi 03 Weiss 99 Ng et al 01 Zelnik and Perona 05 Tolliver and Miller 06

  5. Supervised Unsupervised Previous approaches to segmentation and matting Input Hard output Matte output July and Boykov01 Rother et al 04 Li et al 04

  6. Supervised Unsupervised Previous approaches to segmentation and matting Input Hard output Matte output Trimap interface: Bayesian Matting (Chuang et al 01) Poisson Matting (Sun et al 04) Random Walk (Grady et al 05) Scribbles interface: Wang&Cohen 05 Levin et al 06 Easy matting (Guan et al 06)

  7. Supervised Unsupervised Previous approaches to segmentation and matting Input Hard output Matte output ?

  8. Unsupervised matting Input Automatically computed hard segments (Yu and Shi 03) Automatically computedmatting components

  9. Using components Building foreground object by simple components addition + + =

  10. x x = + Generalized compositing equation 2 layers compositing

  11. x x = + K layers compositing x x = + x x + + Matting components Generalized compositing equation 2 layers compositing

  12. x x = + x x + + Generalized compositing equation K layers compositing Matting components: “Sparse” layers- 0/1 for most image pixels

  13. Goals: • Automatically extract matting components from an image • Derive analogy between hard spectral segmentation and matting, and use similar tools. • Use matting components to automate matte extraction process and suggest new modes of user interaction

  14. Spectral segmentation Spectral segmentation: Analyzing smallest eigenvectors of a graph Laplacian L E.g.: Shi and Malik 97 Yu and Shi 03 Weiss 99 Ng et al 01 Maila and shi 01 Zelnik and Perona 05 Tolliver and Miller 06

  15. Null Spectral segmentation Fully separated classes: class indicator vectors belong to Laplacian nullspace General case: class indicators approximated as linear combinations of smallest eigenvectors Binary indicating vectors Laplacian matrix

  16. Zero eigenvectors Binary indicating vectors Laplacian matrix Smallest eigenvectors Linear transformation Spectral segmentation Fully separated classes: class indicator vectors belong to Laplacian nullspace General case: class indicators approximated as linear combinations of smallest eigenvectors Smallest eigenvectors- class indicators only up to linear transformation

  17. The matting Laplacian (Levin, Lischinski and Weiss CVPR06) • semidefinite sparse matrix • local function of the image:

  18. The matting Laplacian and user constrains Levin et al CVPR06- Input:Image+ user scribbles

  19. The matting Laplacian and user constrains Levin et al CVPR06- Input:Image+ user scribbles • Our goal: • Matting components from matting Laplacian- without user input • Build on hard spectral segmentation ideas

  20. Matting components and the matting Laplacian • Claim: • For an image consisting of “well separated” layers, the matting components belong to the matting Laplacian nullspace • In the general case, matting components are reasonably approximated as linear combinations of smallest eigenvectors Null Matting components Matting Laplacian

  21. linear transformation From eigenvectors to matting components

  22. Traditional Laplacian Matting Laplacian Smallest eigenvectors Linear transformation Binary class indicators Continuous matting components Hard segmentation- matting analogy

  23. K-means Projection into eigs space From eigenvectors to matting components 1) Initialization: projection of hard segments Smallest eigenvectors 2) Non linear optimization for sparse components

  24. Components with the scribble interface Components (our approach) Levin et al cvpr06 Wang&Cohen 05 Poisson Random Walk

  25. Components with the scribble interface Components (our approach) Levin et al cvpr06 Wang&Cohen 05 Poisson Random Walk

  26. Direct component picking interface Building foreground object by simple components addition + + =

  27. Limitations Need to set number of components: Too few - may not contain desired matte Too many - complicates computation and user interaction Cluttered images require a large number of components Input Ground truth matte 70 eigs approximation

  28. + + = Conclusions • Derived analogy between hard spectral segmentation to image matting • Automatically extract matting components from eigenvectors • Automate matte extraction process and suggest new modes of user interaction Ground truth data and code available online:vision.huji.ac.il/SpectralMatting

More Related