Detecting Curved Symmetric Parts using a Deformable Disc Model

1. Detecting Curved Symmetric Parts using a Deformable Disc Model Tom Sie Ho Lee, University of Toronto Sanja Fidler, TTI Chicago Sven Dickinson, University of Toronto Overview Motivation Robustness to taper Symmetric part detection as sequence finding • Recovering an object’s generic part structure is a key step in bottom-up object categorization • Symmetry has formed the basis of many 2D and 3D generic part representations, e.g. skeletons, shock graphs, generalized cylinders, geons • Our goal is to detect 2D symmetric parts in a cluttered image • Good symmetry follows a curvilinear axis • We find high-affinity sequences of disc hypotheses • Optimal sequences are computed using dynamic programming • Our sequence formulation avoids branching clusters • Tapered parts vary in scale along the axis • We allow parts to be composed along the axis from disc hypotheses of different scales Symmetric parts of objects detected by our method Robustness to curvature Related work • Symmetric parts are often curved • We capture curvature explicitly and recover the part in one piece • Classical skeletons [Blum ‘67; Brady ‘84] are inapplicable to cluttered scenes • Filter-based approaches require reliable templates • Contour-based approaches require quadratic grouping complexity • Region-based grouping [Levinshtein et al. ‘09] offers a good alternative • We demonstrate an improvement on [Levinshtein et al.] by capturing more shape variability and applying an optimal algorithm [ours] [Levinshtein et al.] [ours] [Levinshtein et al.] [Levinshtein et al.] [ours] 1. Representing symmetric parts 3. Finding sequences Results Deformable discs Weizmann Horse Database (WHD) Berkeley Segmentation Database (BSDS) Disc hypothesis graph • 81 images of horses • Manually annotated symmetric parts as groundtruth regions [Levinshtein et al.] • Count a hit when IoU-overlap > 40% between groundtruth and detected regions • The medial axis transform [Blum] decomposes a shape into the locus of maximal inscribed discs • We define a symmetric part as a sequence of deformable discs • Discs deform to the shape’s boundary while remaining compact • Organize deformable disc hypotheses into graph • Place edges between adjacent or overlapping discs • Find disc sequences in the graph with high affinity • Source of images of diverse objects on cluttered backgrounds • We manually annotated symmetric parts on 36 selected images Object part Maximal discs Superpixel approximation Multi-scale composition Superpixel approximation • Use compact superpixels as deformable disc hypotheses • Superpixels from different scales compose a single part disc hypothesis graph Cost of a disc sequence BSDS WHD • Score a sequence P = (d0, …, dn) in terms of local affinities σ(di-1,di) and σ(di-1,di,di+1) • Affinities favor local grouping of adjacent discs • Ternary affinities favor curvilinear axis (smoothness) • Convert affinities into binary costs {si-1,i = 1 - σ(di-1,di)} and ternary costs {ti-1,i,i+1 = 1 - σ(di-1,di,di+1)} 2. Deformable Disc Affinity Deformable ellipse Overview of affinity • Define affinity between adjacent disc hypotheses • High affinity reflects non-accidental symmetry • Adjacent discs occupy a region r on which to extract features • Train affinity on region symmetry features Qualitative results Ellipse fitted to a region hypothesis • Cost is normalized by number of discs in sequence • Growth term A favors longer sequences Deformation-invariant space Our ellipse is parameterized by bending and tapering, axis scaling, and rigid transformations • Evaluate symmetry in a warped space invariant to bending and tapering deformations • Determine warp by fitting a deformable ellipse to region • Extract spatial histogram of boundary edgels • Extract interior color and texture features Finding the optimal sequence boundary edgels • cost(P) can be globally minimized using dynamic programming • We use the algorithm of [Felzenszwalb & McAllester ‘06] to compute the global minimum P* Find parameters w that locally minimize non-linear least squares: • Perform best-first search using priority queue of candidate sequences • Dequeue candidate sequences and consider possible extensions • Repeat minimization to find multiple symmetric parts Affinity training • Learn to map a region r to its affinity σ(r) • Generate positive and negative training regions from annotated dataset • Extract features on each training region • Fit logistic regressor σ(r) to training examples Conclusions Candidate sequence extensions under consideration • Symmetric part detector trained on horse images generalizes to diverse objects • Symmetry is a powerful and ubiquitous shape regularity warp W spatial histogram