1 / 76

Projective Analysis for 3D Shape Segmentation

SIGGRAPH Asia 2013 Yunhai Wang Shenzhen VisuCA Key Lab/SIAT Minglun Gong Tianhua Wang Daniel Cohen-Or Hao Zhang Baoquan Chen. Projective Analysis for 3D Shape Segmentation. Speaker : Sze. Outline. Introduction Related Work Overview

byron-vaughan
Download Presentation

Projective Analysis for 3D Shape Segmentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SIGGRAPH Asia 2013 YunhaiWang Shenzhen VisuCA Key Lab/SIAT MinglunGong TianhuaWang Daniel Cohen-Or HaoZhang Baoquan Chen Projective Analysis for 3D Shape Segmentation Speaker : Sze

  2. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion and Future Work

  3. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion and Future Work

  4. Introduction • We introduce projective analysis for semantic segmentation and labeling of 3D shapes.

  5. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion and Future Work

  6. Related Work • Shape Segmentation and labeling • Data-driven analysis • Projective shape analysis • Image and shape hybrid processing • Image retrieval • Image label transfer

  7. Related Work • Shape Segmentation and labeling • [Shamir 2008] • Difficult to develop precise mathematical models for what a meaningful shape part is

  8. Related Work • Data-driven analysis • [Kalogerakis et al. 2010; van Kaick et al. 2011] • [Golovinskiy and Funkhouser 2009; Xu et al. 2010; Sidiet al. 2011] • [Wang et al. 2012].

  9. Related Work • Projective shape analysis • [Muraseand Nayar1995], [Lindstrom and Turk 2000] • Our work applies projective analysis to a new application: semantic segmentation of 3D shapes

  10. Related Work • Image and shape hybrid processing • [Li et al. 2011], [Chang et al. 2009]

  11. Related Work • Image retrieval • [Xiao et al. 2010], [Baddeley 1992] • We do not only retrieve a 2D shape but also infer a semantic labeling of its interior

  12. Related Work • Image label transfer • [Shotton et al. 2006],[Liu et al. 2011a],[Zhang et al. 2010]

  13. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion, limitations and Future Work

  14. Overview • We assume that both the input and the objects captured in the labeled images are in their upright orientations

  15. Overview • Dataset • Large collection of images gathered from the Web • Organized into several semantic classes • Semantic parts of the object are manually segmented and labeled • Foreground object • Using Grabcut[Rother et al. 2004] • Normalizeall labeled images to the same size through uniform scaling

  16. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion, limitations and Future Work

  17. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  18. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  19. Retrieve labeled images using projections • [Chen et al. 2003] • Turn off the lighting • Apply the orthogonal projection

  20. Retrieve labeled images using projections • [Belongie et al. 2002;Ling and Jacobs 2007]

  21. Retrieve labeled images using projections • For each projection T, we first compute its slab representation • Use it to evaluate its dissimilarities to each labeled image S

  22. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  23. Matching between 1D shapes • Symmetric Hausdorff distance

  24. Matching between 1D shapes • Symmetric Hausdorff distance 1 0

  25. Matching between 1D shapes • Bi-class symmetric Hausdorffdistance • ,: all the white pixels in the line

  26. Matching between 1D shapes • n : number of horizontal (vertical) slabs • : row (column) number of the representative scanline • T[] : returns the scanline • : height (width)

  27. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  28. Matching between 2D shapes

  29. Matching between 2D shapes • Warp-aligned dissimilarity • T : unlabeled • S : labeled • S[] : scanlinesin S • w():axial-aligned mapping functionthatscales slabs in T to match those in S

  30. Matching between 2D shapes • Slab matching cost matrix M

  31. Matching between 2D shapes • Slab matching cost matrix M

  32. Matching between 2D shapes • Dynamic Time Warping (DTW) • [Berndt and Clifford 1994]

  33. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  34. Transfer label information to projections

  35. Transfer label information to projections

  36. Transfer label information to projections

  37. Transfer label information to projections • Confidence map

  38. Transfer label information to projections • Confidence map • A per-image term : / • A per-scanline term : • maps scanlines from T to S

  39. Transfer label information to projections • Confidence map • A per-pixel term : • : column number of the pixel in Sthat is used to label (i, j) in T • : Gaussian support (set to 150 in our experiments)

  40. Transfer label information to projections • Confidence map • A per-pixel term : T S

  41. Back-projection and Label Transfer • Retrieve labeled images using projections • Matching between 1D shapes • Matching between 2D shapes • Transfer label information to projections • Back-project labels and graph cuts optimization

  42. Back-project labels and graph cuts optimization

  43. Back-project labels and graph cuts optimization • : primitive in the 3D shape • : label • : The set containing all pixels • () : Dirac’s delta function • : label in S image

  44. Back-project labels and graph cuts optimization • Multi-label alpha expansion graph-cut algorithm [Boykov et al. 2001]

  45. Back-project labels and graph cuts optimization • V : Given by the primitives of the shape • E : {,} • : If the primitives represented by nodes and vare connected • : k nearest neighbors in Kd-tree

  46. Back-project labels and graph cuts optimization • : the labels assigned to nodes and v • : Weights

  47. Back-project labels and graph cuts optimization • : Length of the edge

  48. Back-project labels and graph cuts optimization • : the positive dihedral angle • : Euclidian distance between nodes and

  49. Outline • Introduction • Related Work • Overview • Back-projection and Label Transfer • Results • Discussion and Future Work

  50. Results • Experiment • Images resolution : 512 x512 • 30 seconds of matching a projection with 500 labeled images • Input meshes(20K triangles) • 1 minute of label transfer • 1 minute of back-projection

More Related