480 likes | 561 Views
Symmetry-based Segmentation and Recognition. Thomas B. Sebastian. Brown University. Summary. Three main contributions of my dissertation: Effective segmentation method for carpal bones in CT images Implements skeletal coupling between seeds
E N D
Symmetry-based Segmentation and Recognition Thomas B. Sebastian Brown University
Summary • Three main contributions of my dissertation: • Effective segmentation method for carpal bones in CT images • Implements skeletal coupling between seeds • Solves convergence problem of deformable models • Generic curve matching algorithm with several applications • Robust shape recognition algorithm for matching shock graphs • Gives excellent recognition rates for indexing large shape databases
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its applications • Shape recognition using shock graphs • Indexing into large shape databases
Medical application • Study 3D kinematics of carpal bones • Identify wrist injuries where radiographs are normal • Quantify shape of bone to characterize disease progression, e.g., in Kienbock disease • Compute curvature maps
Challenges • Weak and diffused edges • Gaps in bone boundary
Challenges (cont.) • Texture in spongy bone • Narrow inter-bone gaps
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its application • Shape recognition using shock graphs • Indexing into large shape databases
Deformable models Initialize multiple seeds, which grow under image-dependent forces • Drawbacks • Fail to converge near weak edges • Do not capture narrow inter-bone gaps
Seeded region growing [Biscoff PAMI] • Initializes seeds and grows them by annexing an adjacent pixel • Only the "closest" pixel is added at each iteration • Implements global competition among all seeds • Drawbacks • Leaks into small gaps as there are no geometric constraints
Region Competition [Zhu/Yuille, PAMI] • Initializes seeds and grows them using a combination of statistical and smoothing forces • Implements local competition between seeds when they contact each other • Allows for exchange of pixels between regions and recovery from errors
Region Competition (cont.) • Drawbacks • Merges some adjacent bones • Fails to capture low-contrast bones
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its application • Shape recognition using shock graphs • Indexing into large shape databases
Skeletal coupled deformable model (SCDM) • SCDM combines advantages of previous techniques • Subpixel nature of deformable models • Global competition of seeded region growing • Local competition of region competition
Growth of regions • In isolation, growth of regions depends on a local statistical force
l d Local competition by skeletal coupling • The inter-region skeleton is used to couple growing regions • The regions compete for pixels in the middle • When seeds come in contact, l=1, same as region competition
Long-range skeletal competition • The inter-region skeleton is viewed as predicted boundary and the growth of regions is modulated by its suitability
Total skeletally-coupled forces • The total statistical force is a combination of local and long-range forces
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its application • Shape recognition using shock graphs • Indexing into large shape databases
Convergence of SCDM • SCDM solves the convergence problem of traditional deformable models
SCDM segmentation results Results are clinically meaningful
3D model of carpal bones 3D model is created by stacking up 2D contours 3D model is useful in studying carpal kinematics, creating computational atlases, etc.
Carpal bone segmentation: Comparison SRG/RegComp SCDM • Rating by hand surgeons (RIH) • SCDM 83% • SRG 14% • RegComp 3%
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its applications • Shape recognition using shock graphs • Indexing into large shape databases
Curve matching [Sebastian et al, PAMI] • Goal is to find optimal alignment (pairing of points) and distance (deformation cost) • Cost is measured by length and curvaturedifferences of infinitesimal segments and summing them
Curve matching [Sebastian et al, PAMI] • Alignment is a pairing of points of C, C’ • Treat curves C and C’ as the x and y-axes • Alignment is then a curve in 2D space, called alignment curve • Dynamic programming to find optimal alignment curve
Prototype formation • Average curve can be computed by averaging corresponding sub-segments
Shape morphing • Weighted averages can be used to generate morph sequence
Handwritten character recognition • 327 characters (34 categories) • 98.5 % recognition rate
Gesture recognition Intuitive matches
Gesture recognition (cont.) Outperforms other methods Precision Recall
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its applications • Shape recognition using shock graphs • Indexing into large shape databases
Real Example Shock graph representation of shapes • Shocks (or medial axis or skeleton) are locus of centers of maximal circles that are bitangent to shape boundary Shape boundary Shocks
Distance between shapes • Shape space is collection of all shapes • Shape is a point • Shape deformation sequence is a path • Cost of optimal deformation sequence is distance from A to B • There are infinitely many deformation paths • Shape space has to be discretized
Changes in shock graph topology • Shock graph topology is unaltered for most shape deformations • At transition shapes small changes lead to abrupt changes in the shock graph topology
Partitioning of shape space • Shape cell: Collection of shapes with same shock graph topology • Transition shapes form the boundary between shape cells Shape cell 2 Shape cell 1
Discretization of deformation paths • Shape deformation bundle:Collection of deformation paths passing through the same set of shape cells • Represented by set of transition shapes it passes through Dynamic programming is used to find optimal edit sequence
Matching results* [Sebastian et al, ICCV 01] Edit-distance algorithm gives intuitive results * Same colors indicate matching edges; gray-colored edges are pruned
Robustness to transformations Boundary noise In optimal edit sequence “noisy” branches are pruned Articulation Edit-distance is robust in presence of part-based changes
Robustness to transformations (cont.) Viewpoint variation • Deform edit handles smooth changes • Splice and contract edits handle abrupt changes
Robustness to transformations (cont.) Partial occlusion Edit-distance is robust to partial occlusion
Indexing into shape databases* Edit-distance algorithm allows 100% shape recognition between different shape categories * Results duplicated in two databases: 99 shapes and 216 shapes
Indexing Results It also allows nearly 100% recognition between shapes in the same shape category
Contents • Segmentation using skeletally coupled deformable model • Introduction • Previous approaches • Skeletal coupling • Results • Curve matching and its applications • Shape recognition using shock graphs • Indexing into large shape databases
Indexing into large databases [ECCV 02] • Indexing results are excellent using a large database (1032 shapes, 200 exemplars) • Correct category is selected in top 1, 2 and 5 matches with 78%, 91%, and 99% success rate respectively
Number of exemplars • Using fewer exemplars suggests a hierarchical representation • 2-3 primary exemplars rule out 75% of categories • Additional 2-3 auxiliary exemplars rule out another 50% of categories
Acknowledgments • Collaborators/Advisors • Prof. Benjamin Kimia, Brown University • Prof. Philip Klein, Brown University • Dr. Joseph Crisco, RI Hospital
Selected References • SCDM segmentation • Segmentation of carpal bones from CT images using SCDM [MedIA02] • Segmentation of carpal bones using SCDM [MICCAI98] • Curve matching • On aligning curves [PAMI02] • Alignment-based recognition of shape outlines [IWVF01] • Constructing curve atlases [MMBIA00] • Shock-graph matching • Recognition of shapes by editing their shock graphs [ICCV01] • Shape matching using edit distance: An implementation [SODA01] • Indexing into databases • Shock-based indexing into large databases [ECCV02] • Metric-based shape retrieval in large databases [ICPR02] • Curves vs. skeletons for object recognition [ICIP01]