1 / 51

Primal Sketch Integrating Structure and Texture

Primal Sketch Integrating Structure and Texture. Ying Nian Wu UCLA Department of Statistics Keck Meeting April 28, 2006. Guo, Zhu, Wu (ICCV, 2003; GMBV, 2004; CVIU, 2006). A Generative Model for Natural Images. texture regions. input image. sketch graph. +. =. synthesized image.

levana
Download Presentation

Primal Sketch Integrating Structure and Texture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Primal Sketch Integrating Structure and Texture Ying Nian Wu UCLA Department of Statistics Keck Meeting April 28, 2006 Guo, Zhu, Wu (ICCV, 2003; GMBV, 2004; CVIU, 2006)

  2. A Generative Model for Natural Images texture regions input image sketch graph + = synthesized image sketchable image synthesized textures

  3. Outline Sparse coding Markov random field Primal sketch model Sketch pursuit algorithm

  4. Sparse Coding Olshausen and Field (1996)

  5. 500 bases 800 bases Matching Pursuit Mallat and Zhang (1993) matching pursuit input image

  6. Symbolic representation of 300 bases Reconstructed image Primak sketch

  7. Markov Random Fields Markov Property: MRF model = Gibbs distribution Besag (1974) Geman and Geman (1984) Cross and Jain (1983) One example of neighborhood

  8. MRF model & Image ensemble MRF model (Zhu, Wu & Mumford, 1997) Image ensemble (Wu, Zhu & Liu, 2000)

  9. Feature statistics: histograms of filter responses (Heeger and Bergen, 1995) Filtering – convolution original image I filter responses J of the “dy” filter a set of filters F

  10. Histogram of Filter Responses

  11. Average histogram error

  12. 800 bases A sample of image ensemble with 5*13=65 parameters 50*70 patch

  13. observed image sampled image from image ensemble Primak sketch

  14. Sparse Coding vs. MRF Sparse Coding models target low complexity patterns. const: related to the dictionary MRF models target high complexity patterns. p*: fitted MRF q: any distribution

  15. Primal Sketch Model Image pixels = Sketchable & Non-sketchable Sketchable: sparse coding using image primitives Non-sketchable: feature statistics/Markov random fields • Integration: • Non-sketchable interpolates sketchable • Non-sketchable recycles failed sketch detections

  16. Sketches Elder and Zucker, 1998

  17. Sketch Graph Sketch graph Vertices: 1,2,3 – corners 4,5,6,7 – end points 8,9,10 – junctions, etc

  18. Image Primitives b) Photometric a) Geometric

  19. Sketch Graph Model Geometric Photometric sketch image

  20. Sketchs = Gabor clusters Alignment across spatial and frequency domains

  21. Integrating structure and texture Sketch Graph Sketches Alignment Gabor filters Non-alignment Textures Pool marginal histograms

  22. Model fitting First: Sketch pursuit aided by Gabors Second: Non-sketchable texturing Sketchability test

  23. Sketch pursuit objective Approximated model

  24. Sketch Pursuit Phase I input image edge/ridge strength Edge/ridge map Proposals: a set of sketches as candidates. Select the sketches in the order of likelihood gain.

  25. Sketch Pursuit Phase IIRefinement Refinement Initialization Evolve the sketch graph by graph operators.

  26. Graph Operators

  27. A B Graph Editing A Phase I B Phase II

  28. Phase II Algorithm • Randomly choose a local sub-graph (S0) • Try all 10 pair of graph operators 1~ 5 steps, to generate a set of new graph candidates (S4,S2,S3) • Compare all new graph candidates • Select the one with the largest posterior gain (e.g. S4), accept the new graph. If no positive gain, no update. • Repeat 1~4 until no update S0 G1 G3 S1 S2 S3 G4 S4

  29. texture regions synthesized textures K-mean clustering Histograms in 7x7 window 7 filters x 7 bins

  30. Primal Sketch Model Result input image sketch graph sketchable image reconstructed image

  31. input image sketch graph reconstructed image sketchable image

  32. input image sketch graph reconstructed image

  33. input image reconstructed image sketch graph

  34. Lossy Image Coding codes for the vertices: 152*2*9 = 2,736 bits codes for the strokes: 275*2*4.7 = 2, 585 bits sketch graph codes for the profiles: 275*(2.4*8+1.4*2) = 6,050 bits Total codes for structures (18,185 pixels) 11,371 bits = 1,421 bytes sketchable image

  35. codes for the region boundaries: 3659*3 = 10,977 bits texture regions codes for the texture histograms: 7*5*13*4.5 =2,048 bits Total codes for textures (41,815 pixels) 13,025 bits = 1,628 bytes Total codes for whole image (72,000 pixels), 3,049 bytes synthesized textures

  36. Scaling

  37. Scaling

  38. Scaling Wu, Zhu, Bahrami, Li (2006)

More Related