1 / 31

Intensity-based deformable registration of 2D fluoroscopic X-ray images to a 3D CT model

Intensity-based deformable registration of 2D fluoroscopic X-ray images to a 3D CT model. Aviv Hurvitz Advisor: Prof. Leo Joskowicz. Navigation in orthopedic surgery . Applications Determine position of surgical tools relative to anatomy Position surgical robots

albert
Download Presentation

Intensity-based deformable registration of 2D fluoroscopic X-ray images to a 3D CT model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Intensity-based deformable registration of 2D fluoroscopic X-ray images to a 3D CT model Aviv Hurvitz Advisor: Prof. Leo Joskowicz

  2. Navigation in orthopedic surgery Applications • Determine position of surgical tools relative to anatomy • Position surgical robots • Match pre-operative model to anatomy [hss.edu] [medtronic.com]

  3. Navigation in orthopedic surgery Related registration methods • Registration to fiducials • Implanted fiducials • On skin • Contact based registration (Point-cloud to surface registration) • Registration of fluoroscopic X-ray to CT

  4. Registration of fluoroscopic X-ray to CT [LaRose 2001; Tomazevic 2003; Knaan 2003; Livyatan 2003; and others] Preoperatively: • Acquire CT of anatomy of interest Intraoperatively: • Fix a tracking marker to the bone • Acquire 2-5 fluoroscopic X-ray images from various camera poses • For each image, record: • T(trackercamera) - camera position at acquisition time • T(trackerbone) - bone marker position at acquisition time • Obtain an initial transformation estimate T(boneCT) • Estimate transformation T(boneCT) with the algorithm we describe next Tracker updates T(trackerbone) continuously, enabling real-time navigation.

  5. Registration of fluoroscopic X-ray to CT Input: • For 2-5 X-ray images: • T(trackercamera) at acquisition time • T(trackerbone) at acquisition time • Initial T(boneCT) estimate Repeat until convergence: • For each camera position: • Define a virtual camera positioned relative to CT • Create DRR • Rate similarity of DRRs to X-ray images by a similarity metric • Determine the next T(boneCT) by an optimization algorithm

  6. DRR creation [Knaan et al., 03]

  7. Is it possible to do the registration without a CT? Advantages: • Save time before operation • Save costs • Decrease radiation exposure Disadvantages: • A patient-specific CT provides more information

  8. Is it possible to create a patient-specific model without a CT? Approach • Use prior information – everyone’s bones are similar • Deduce exact shapes of bones from the fluoroscopic images In absence of CT, need to search for both bone pose (6 d.o.f.) and the patient-specific bone shape

  9. Image analysis-by-synthesis Define a model of the bone which is parameterized by pose and by shape. Repeat until convergence: • Simulate X-ray imaging process to create DRRs • Rate similarity of DRRs to fluoroscopic images by similarity metric • Modify pose and shape parameters by optimization algorithm When the DRRs match the fluoroscopic images • We found the pose and the shape of the bone • We have registration: a transformation from every point in the model to its corresponding point in the patient anatomy. • We can map data from the model to the patient anatomy. E.g., map the surface of the bone.

  10. Active Appearance Model (AAM) [Cootes et al., 98; Matthews and Baker, 04] • Model shape variations and appearance variations compactly (with few parameters) • Can be trained from a dataset of samples • A generative model – can generate new images which are similar to the training images • Image analysis-by-synthesis [Edwards, Taylor and Cootes, 98]

  11. Shape • Manually label landmark points on training images • Define a triangular mesh between the landmarks [Matthews and Baker, 04] • Each shape is represented by a vector of landmark coordinates

  12. Procrustes analysis - align meshes to cancel global variations in translation, rotation and scale • PCA on shape vectors

  13. Appearance • Warp each training image to the base mesh so. This creates a set of shape-normalized images. • Represent pixels inside the base mesh so as vectors • PCA on shape-normalized image vectors:

  14. The resulting model [Matthews and Baker, 04]

  15. Instantiating the model • Input: • Shape parameters • Appearance parameters • Computation: • Instance’s shape-normalized appearance • Instance’s shape mesh

  16. Instantiating the model [Matthews and Baker, 04]

  17. Fitting AAMs to images • Input: • Image I(x) • AAM • Goal: find shape p and appearance λ that yield an AAM image similar to I(x). • Minimize SSD over λ and p: Sum differences over all pixels x in base mesh

  18. The optimization is nonlinear in p and linear in λ. • The parameters are found by iterative gradient descent. • Standard gradient descent is slow because it requires calculating the partial derivatives at every iteration. • There are tricks to estimate gradient quickly [Cootes, 98] , or use a pre-computed gradient.[Matthews and Baker, 04]. • After improvements, algorithm is fast enough for real-time face tracking

  19. Using AAMs for 2D-3D registration • The training dataset is CTs (3D images). ROIs of femoral head approximately 100x100x100 pixels. • The shapes are defined by tetrahedral meshes (approx. 3,000 nodes and 20,000 tetras). • It is too difficult to find landmarks manually.

  20. Finding landmarks automatically • Select one CT arbitrarily as a template. • Define a tetrahedral mesh on the template. • Perform 3D-3D deformable registration of each CT to the template. Defines a transformation Ti:TemplateCTi • Apply transformation to template mesh transfer mesh to the corresponding landmarks in each CT.

  21. Finding landmarks automatically • 3D-3D registration performed with Elastix software (an ITK wrapper) [Klein and Staring, 07] • Registration procedure • Preparation: match histogram to template • Rigid registration: • Initialize by aligning centerpoints of ROIs • Metric: Normalized Cross Correlation • Optimizer: Standard gradient descent • Deformable registration • Transform: 3D B-spline. (Grid spacing 8x8x8 voxels) • Metric: Sum of squared differences (SSD) • Optimizer: Conjugate gradient • Use image pyramids (3 levels) • Unsupervised • Each registration runs approx. 10 mins. • Compute all registrations in parallel on bmos cluster (mosix)

  22. Building AAM of femur CTs • Input: • 14 CTs and 14 corresponding meshes • Output: • Mean shape mesh and 6 significant basis vectors • Mean appearance image and 10 significant basis vectors

  23. Mean volume image - A0

  24. Shape model S0 s0 + 2 std s1 s0 + 2 std s2

  25. Using the AAM in 2D-3D registration Repeat until convergence: • Generate 2D DRRs out of AAM using current pose, shape, and appearance parameter estimates. • Rate similarity of DRRs to fluoroscopic images by similarity metric • Modify AAM shape parameters and the pose parameters by optimization algorithm. (We ignore appearance variations)

  26. Generating DRRs from AAM – Algorithm 1 • Generate a 3D image of the AAM instance for each pixel in 3D image: • find tetrahedron containing pixel center (use KD-tree) • find corresponding point in base mesh (an affine transform) • read value from appearance image • Proceed with standard ray casting of 3D image [Knaan et al., 03]

  27. Instance mesh Base mesh Shape-normalized volume image Generating DRRs from AAM – Algorithm 2 • Cast rays directly through tetrahedral mesh of AAM instance • Sample points at uniform intervals on ray • Find corresponding points in base mesh • Read values from appearance image • Requires identifying all ray-tetrahedron intersection points. We implemented algorithm of [Marmitt and Slusallek, 06]

  28. Parameter search issues • Accuracy and Robustness • Search from various initial locations • Genetic algorithm • Downhill simplex optimizer (ITK amoeba) • Speed • Multi-resolution • Limit DRR generation to ROIs

  29. Current status • The naïve search converges to unsatisfactory result (local minimum) • Need to improve both accuracy and speed • Search from multiple initialization points • ROIs • Think about a smarter optimization algorithm Future: • Experiment on real fluoroscopic images

  30. Intensity-based registration of 2D X-ray images to a 3D deformable model Thank you for listening

  31. Principal Component Analysis (PCA)

More Related