Image/Volume Registration: A validation using the Finite Element Method

1. Image/Volume Registration: A validation using the Finite Element Method In collaboration with A. Abdel-Hakim and A. Elbaz R. Fahmi, A. Abdel Hakim, A. Elbaz and A. A. Farag, “New Deformable Registration Technique Using Scale Space and Curve Evolution Theory and A Finite Element Based Validation Framework”, EMBS’06. A. El-Baz, R. Fahmi, S. Yuksel, A. A. Farag, W. Miller, M. A. El-Ghar, and T. Eldiasty, “A New CAD System for the Evaluation of Kidney Diseases Using DCE-MRI”, MICCAI’06.

2. Image/Volume Registration • The process of spatially aligning two or more images so point-by-point correspondences can be established between them • Points corresponding to the same anatomical point are mapped to each other. • Two main families: • Feature-based: Extract and match salient features: edges, corners, line intersections,… • Area-based: directly match intensity maps: sensitive to noise, hard to solve anatomical correspondences pb, … • Other techniques: active contour-based approaches (Vemuri’03)… • New approach combining feature and level sets is proposed.

3. Finer Scales Step1: Feature extraction and global alignment • First, we build invariant feature descriptors which will be matched to find the correspondent pairs of control points • This stage involves three main steps: • Interest Points Detection: Scale Space Theory is used to detect the most stable features w.r.t. scale changes (Abdel Hakim and farag’07) The locations of extrema in the DoG pyramid correspond to the most stable features with respect to scale changes (Mikolajczyk’02, Lowe’04) compared to gradient, Hessian, Harris corner detector, …

4. Feature descriptor building (cont.) 2.To achieve invariance to scales, descriptors are built using the histograms of gradient orientations on neighborhood of interest points(Mikolajczyk & Shmidth’05, Abdel Hakim & Farag’07). 3.Feature matching using the Euclidean distance. • The matched features are then used to estimate the global alignment transformation: 5-parameter matrix in 2D cases and 9-parameter transformation matrix in 3D cases.

5. Local Alignment • After global alignment, iso-surfaces are evolved in one image to match those of the other image in four steps • Generate the distance map inside of the imaged organ (object of interest). • Use this distance map to generate iso-surfaces • Number to be set by user (trade-off between accuracy and speed). • Find correspondences between iso-surfaces using NCC and extracted features. • Evolve iso-surfaces in image A to match those in image B.

6. Our Evolution Approach • Let’s first define the followings :Iso-surface on source image :Iso-surface on target image :Euclidean Distance between two corresponding iso-surface points :Euclidean Distance between and

7. Interface Evolution Scenario Volume B Volume A

8. Speed Function • We define the evolution function V such that Otherwise • We then chose • We update the iso-contours as

9. 3D Example Global Alignment Before Alignment Local Alignment

10. 1 2 3 4 5 6 Mesh Generation (TetSplit) Mechanical Parameters Assignment B.C.’s and Loads Definitions Validation using Finite Element Method • 3D Validation on Brain MRI’s: Brain Tissue Segmentation (GM, WM, CSF) Three Deformations are Simulated: • 2x Gravity Induced Deformations (L.E. & Ogden H.E.) • Ventricles Contraction F.E. Solving (Abaqus)

11. 1 2 3 4 5 Let’s look at each component A MRF-based approach was used to segment the images into different tissue classes (GM, WM, CSF, …). We used “TetSplit” (SBIA/UPENN) to produce linear tetrahedral meshes conforming to quality measure required by Abaqus. Each voxel is assigned Mech. Par. of its underlying tissue class. Ventricles are modeled with hyperfoam material to anticipate contact between their walls (Miller & Chinzei’02). Points where falx meets skull are pinned and remaining points on outer surface are free to slide only in the plane tangent to brain surface (Miga’99 and Wasserman’99,07). • 3 deformations are simulated: • Two gravity-induced deformations: L.E. and H.E. models are resp. used. • Ventricle contraction using H.E. model.

12. Examples from case#3 Non deformed Mesh Deformed Mesh Z-plane Cut of Overlay

13. Generation of Deformed Images • Deformed images are generated by means of F.E. interpolations. • For each simulation case, compute the dense displacement within each finite element el : simulated nodal displacements shape functions • are piece-wise linear polynomial functions. • A program is written that takes the original gray level image, the mesh definition files, deformed nodes, and outputs a deformed gray level image.

14. Example Initial Positions Rigid Alignment Elastic Registration

15. Quantitative assessment Quantitative assessment of the registration accuracy. Error statistics for the three simulated cases. Displacements correspond to the simulated ones.

16. 100 randomly selected F.E. node positions and their corresponding recovered positions using our technique for Case#3 Red: Abaqus Green: Ours

17. Medical Application: Study of autism and Dyslexia • Autism is neuro-developmental disorder • Impairments in social interaction, communication. • Unusual behaviors and interests. • According to CDC, 1/150 American kids are autistic (4:1 ratio of boys to girls). • Challenges: • No definitive medical test for diagnosis. • No reliable cause is identified. • No cure. BUT: Therapies for specific symptoms.

18. Studies of Autism • Neuropathological and Neuroimaging studies • Most studies revealed macroencyphaly in autism. • Increased volume in cerebellar white matter. • Reduced size of the corpus callosum. • Inconsistencies between findings. • Increased WM volume is attributed to the large number of minicolumns in autistic brain (Casanova et al.’02,04,06). • Ongoing research: A structural MRI-based correlate to autism which relates to neural connectivity.

19. Current Achievements • Developed neuroimaging framework to classify autism • Gyrification window. • Used distance map inside of the WM as a discriminatory feature • Registration of the CC’s within a class to a chosen reference and building average deformation fields to classify a given subject. • Tested on postmortem and in-vivo brain MRI’s. • Several publications: ISBI’06, CARS’07, ISBI’07, J. Spec. Educ. Rehab.

20. Ongoing work in dyslexia research • Main Idea: • Given a sample set of brain MRI’s from each group, select a reference volume and register the remaining volumes to it. • For each group, create an average anatomical atlas in the space of the reference volume. Control Dyslexic • Average the deformations field obtained during the atlas creation processes and compute the cumulative distribution of their magnitude. • Given a test subject, register its data set with each atlas and compare the CDF of the generated disp. field with the appropriate average CDF representing each group.

21. Average Space Reference Space Groups Register Dyslexic Register Control Illustration and Results

22. Conclusions • Shape Registration (ICIP’07, ECCV’08) • A new dissimilarity criterion is proposed for global shape registration which can deal efficiently with scale variations. • Comparison with existing models and its application to statistical shape modeling and shape-based segmentation was highlighted. • A new energy formulation for elastic shape registration is introduced. • Potential of the proposed framework to solve the 3D face recognition problem in presence of facial expression showed promising results. 2. Image/Volume Registration (EMBS’06, MICCAI’06, ICPR’06) • A new image/volume registration framework is proposed. • Scale space theory is employed to extract robust feature descriptors and use them for global alignment. • Curve evolution theory is employed to handle local deformations • A novel validation framework based on finite element method is introduced.

23. Other achievements • FEM-related work • Soft tissue deformation (MICCAI’05, IEEE-TBE’08, 1 book chap.) • Autism and dyslexia work • (ISBI’07, 2book chapters, J. Spec. Educ. & Rehab.’06, CARS’08) • Segmentation Work • (SPIE’07, ICIP’08) • And more …

24. Future directions (Cont.) • More to be done to speed up the shape registration in 3D. • Relying on intensity map only is not very accurate in solving the anatomical correspondences problem. Other techniques are to be tested for the matching step prior to the iso-contour deformations. • Extend the fast implementation of the shape-based segmentation algorithm to the case of statistically learned model using for ex. PCA. • Autism: use the proposed shape registration framework to capture the morphological abnormalities of the CCand identify which specific callosal segment (s) contribute the most in the size deficit of the CC.

25. Thank You Questions?

26. Segmentation With Shape Prior and Pose Invariance Segmentation of the Corpus Callosum Input Image with noise & partial occlusion

27. Chan and Vese Segmentation Models • 2-Phase CV Model: One LSF • Evolving Curve • Two constants • n-Phase CV Model: mLSFs and • For n=4: 2 level set functions • Evolving curves: • Constant Vector:

28. C-V Segmentation Energies • 2-Phase Model • 4-Phase Model • Minimize using gradient descent to solve the corresponding Euler-Lagrange equations in a narrow band. • Energy functional has to be differentiable. • Computationally expensive (Non-linear Parabolic PDE’s) • Slow because of CFL condition. • Sensitive to initial condition.

29. C-V Segmentation with selective shape priors • For each LSF,a shape prior is implicitly represented by its SD, and a shape energy is added to the segmentation functional. • A labeling function, , is added so that the segmentation of other objects is not affected (Cremers et al.’03). Ex.

30. Proposed Algorithm • Idea: only the sign of LSFs is needed for segmentation not their values(B. Song and T. Chan, SIAM’03 and Gibou & Fedkiew, CVPR’02). • Direct manipulations of energy functional do not require any differentiability condition. Rachid Fahmi and Aly A. Farag, “ A Fast Algorithm for Multi-Phase Level Set Segmentation With Selective Shape Priors”, submitted to ICIP’08.

31. 2-Phase Case: • Initialize: partition image domain into and • Sweep: move current point, , from its region to the other and compute E, if E decreases then update • Repeat second step until E remains unchanged If is moved from A to B and vice-versa

32. 4-Phase Case: • Initialize: partition image domain into • Sweep: move current point, , from its region to the other 3. Assign to the region corresponding to largest decrease. • Repeat second step until E remains unchanged. Ex. If is moved from B to the other 3 regions

33. Results on Synthetic Images • 2-Phase Case & One Prior Images of size 150x150. Convergence in 1sweep < 0.1sec Segmentation With Shape Prior but no labeling Segmentation With Shape Prior and labeling Segmentation W/O Shape Prior Initial LSF and Labeling Function

34. Comparison with Standard CV algorithm 200iters=6.36sec 4sweeps=0.125sec Our segmentation Standard CV model Noisy Image Energy vs. iteration

35. One more comparison 6sweeps=0.21sec 140iters=25.85sec Our segmentation Standard CV model Noisy Image Energy vs. iteration

36. Results onSynthetic Images • 4-Phase Case & Two Prior 1sweeps=0.125sec 4sweeps=0.25sec Segmentation With Shape Priors but no labeling fncts Segmentation With Shape Prior and labeling fncts Segmentation w/o Shape Priors Labeling functions

37. Results onReal Images Pure CV 0.984sec Initial LSFs Partial Occlusion Using Shape Priors 0.234sec

38. Robustness to Noise & Partial Occlusion

39. The parameters of are SIMULTANEOUSLY updated during the course of evolution of the segmenting LSFs. • Application to the 2-phase and 4-phase CV models, using the actual values of LSFs not only their sign. Pose Invariance Formulation • The pose and orientation of familiar object (s) are no longer supposed known. • Assume the existence of an affine transformation between each known object and its corresponding prior. • Use our new SSD criterion to recover ‘s

40. Pose Invariance: Evolution Equations of LSF’s 2-Phase Case: 4-Phase Case:

41. Pose Invariance Formulation: 2-Phase Case W/O pose invariance With pose invariance

42. Pose Invariance Formulation: 4-Phase Case

45. Gradient flow w.r.t. • Initial conditions and Euler-Lagrange Equations • Gradient flow w.r.t.u Smoothing Operator