Technology Project: Shape-Based Retrieval of 3D Craniofacial Data

1. Technology Project: Shape-Based Retrieval of 3D Craniofacial Data PI: Linda Shapiro, Ph.D. Key Personnel: James Brinkley, M.D., Ph.D. Michael Cunningham, M.D., Ph.D. Collaborators: Carrie Heike, M.D. and Tim Cox, Ph.D. Postdoc: Katarzyna Wilamowska, Ph.D. Postdoc: Indriyati Atmosukarto, Ph.D. RA: Shulin Yang, MS RA: Jia Wu, MS RA: Sara Rolfe, MS Undergrad RA: Michael Lam

2. Progress on Specific Aims Aim 1: Software Tools for Quantification of Craniofacial Anatomy • New method for learning to compute the plane of symmetry for human faces (paper accepted for the ACM Conference on Bioinformatics, Biology, and Biomedicine) • Landmark-free framework for the detection and description of shape differences in chicken embryos (paper submitted to the IEEE Conference on Engineering in Medicine and Biology) Aim 2: Similarity Measures • Classification and interest-region localization on craniosynostosis skulls (paper accepted for the ACM Conference on Bioinformatics, Biology, and Biomedicine)

3. Aim 3: Organization and Retrieval • Subject database being set up at Seattle Children’s Hospital • De-identified subject database being set up at University of Washington including useful attributes for retrieval (age, gender, race, reason for scan, diagnosis) and pointers to image data files Aim 4: Retrieval System • Modules for 2D Azimuth-Elevation Histogram, Local Features, 2D Longitude-Latitude Signature Map, Pose Normalization, and Automatic Cranial Image Generation delivered to the HUB. • Reference manual has been delivered to the HUB. • Graphical user interface that can use these modules is in progress. • Retrieval system will be built (probably in year 4) to use both the database from Aim 3 and the completed feature extraction and similarity modules.

4. Learning to Compute the Plane of Symmetry for Human Faces • We have started to work with 3D mesh data from subjects • who have clefts. • Faces are no longer expected to be nearly symmetric. • Standard pose normalization is not guaranteed to work. • Instead, we have developed a method for computing the • plane of symmetry using regions about landmarks that are • learned from training data.

5. Methodology • 1. Use training data head meshes on which experts have marked landmarks • Train component detection classifiers to recognize regions (components) • surrounding these landmarks using the curvature of the mesh points • Using the known plane of symmetry, train component goodness classifiers • to determine which detected components are good for computing the plane • of symmetry: • components that lie on the plane of symmetry • component pairs that lie an equal distance from the plane of symmetry

6. On independent test data • Apply component detection classifiers to find components • Apply component goodness classifiers to select those to be • used for determining the plane of symmetry • Apply the RANSAC algorithm to fit the plane of symmetry to the • center points of single components and points halfway between • centers of pairs of components, while throwing out outliers. Single Components Pairs Good Computed Components Symmetry Plane

7. Landmark-Free Framework for the Detection and Description of Shape Differences in Embryos • Identify surface in problematic optical projection tomography (OPT) images. • Describe changes in shape during embryo development without the use of landmarks. • Differentiate normal shape changes from those due to cleft lip/palate defect. GOALS Problematic Tomographic Data Reconstructed 3D contour

8. Overview of Methodology Deformable Registration Low-level feature extraction Mid-level feature extraction Image 1 Image 2 Group similar features • Flow vectors represent change in shape. • The features extracted from the flow vectors are • vector magnitude • difference from surface normal • difference from reference vector • local similarity measure • local entropy ROIs

9. Feature Cluster Examples Brain Brain Eye Eye Midface Midface Embryo1 Embryo 2 Flow Vectors flow vector flow vector distance flow vector distance magnitude clusters from reference clusters from normal clusters (red = high)(groups of similar angles) (red = similar to normal)

10. Classification and Interest Region Localization for Craniosynostosis Skulls • In prior work, we developed the Cranial Image (CI) • representation of skull shape, a matrix of point distances. • Under FaceBase support, we developed an automatic • procedure for computing the CI, providing a general tool • for analysis of craniofacial shape. • Our tool allows users to select: • how many planes on which to detect points • where these planes should be located • how many points per plane • With 10 planes and 100 points per plane, the CI is • too big for many classification/description tasks.

11. Coronal Metopic Sagittal • We used several forms of machine learning to both • classify and quantify head shape and to • reduce the number of point pairs required. • logistic regression • L1-regularized logistic regression • fused lasso • clustering lasso • These machine-learning methods • identify the most useful point pairs for classification • provide a probability value for each classification • that can be used for quantification.

12. Misclassification rates are shown for each method. • Our new clustering lasso method is best overall. • The method also allows us to determine the most useful • point pairs for classification of each class vs. the other • two.

13. Pairs of Points Useful for Classification

14. The Graphical User Interface in Progress