Deformable 3D Volume Registration Using Efficient MRFs Model with Decomposed Nodes

1. Deformable 3D Volume Registration Using Efficient MRFs Model with Decomposed Nodes Sep. 3, 2008 *LEE Kyong Joon KWON Dongjin YUN Il Dong LEE Sang Uk British Machine Vision Conference Leeds, 2008

2. Registration Example (2D) • Find pixel-wise displacements from source to target image source image target image difference

3. 2D Example: General Approach • Super-impose a mesh to reduce dimensionality • Rest of pixel displacement is calculated using interpolation source image target image deformed image

4. 2D Example: General Approach • Assuming local deformation, a control point searches for a matching point in a certain search range • Continuous method tries to use gradient of the energy function [Rueckert et al., IEEE MI 1999] source image target image deformed image • Energy to be minimized = CP matching cost + Mesh deformation cost

5. Registration as a Discrete Labeling Problem • Discretize a search space! [Glocker et al., IPMI 2007], [Tang et al., MICCAI 2007] • Independence of matching cost function from optimization methods • Benefit from state-of-the-art MRF framework source image target image deformed image

6. Basic Framework • MAP-MRF Formulation.. s t

7. Problem in the Model • Problem: |L| = |D|2 So far, we’ve seen 2D registration example.. |D|

8. Problem in the Model • Problem: Huge label spacefor 3D registration • Why does it make problem? • Discrete optimization methods can suffer from it • e.g.) Graph cuts complexity: O(|V||L|) • TRW-S complexity: O(|V||L|2) • Data cost calculation |L| = |D|3 |D|

9. Problem in the Model • Problem: Huge label spacefor 3D registration • Why does it make problem? • Discrete optimization methods can suffer from it • e.g.) Graph cuts complexity: O(|V||L|) • TRW-S complexity: O(|V||L|2) • Data cost calculation |L| = |D|3 |D| cf.) approach of [Glocker et al., IPMI 2007]: sparse search along 6 axes

10. Our Approach • Problem: Huge label space for 3D registration • Solution: Decompose a node into three node • cf.) 2D decomposed model [Shekhovtsov et al., CVPR 2007] |L| = |D| |L| = |D|3 |L| = |D| |D| |L| = |D|

11. Node Decomposition sx s sz sy • Proposed model introduces a higher(3rd) order clique!

12. Higher Order Clique sx • Since the clique potential must be a general type, BP based optimization is appropriate • Performance of TRW-S is proven to be great [Szeliski et al., ECCV 2006], [Kolmogorov, PAMI 2006] sz sy

13. Higher Order Clique sx • Convert to pairwise interaction to adapt TRW-S sz sy a

14. Conversion to Pairwise Interaction [Weiss et al., IEEE IT 2001], [Wainwright et al, IEEE IT 2005], [Kschischang et al., IEEE IT 2006]

15. Conversion to Pairwise Interaction [Weiss et al., IEEE IT 2001], [Wainwright et al, IEEE IT 2005], [Kschischang et al., IEEE IT 2006]

16. Experimental Result • Synthetically deformed data source target initial product proposed

17. Experimental Result • 10 sets of 256 × 256 × Sz where Sz ranges from 70 to 170 • Mesh spacing: 16 pixels • Search space covers: 15 × 15 × 15 •  # of labels in product model: 3,375 / in proposed model: 15 initial product proposed

18. Clinical Data: Liver CT Images • After injecting a contrast medium, two CT scans are obtained at different time points (e.g. "phase“) • Radiologists' task is identifying a lesion that shows a dynamic enhancement pattern different from that of the background liver, and then classifying this enhancement pattern arterial phase portal phase

19. Experimental Result • Clinical data source (arterial) target (portal) initial proposed

20. Conclusion • Proposed method can enhance the efficiency of the optimization algorithm by reducing number of labels. • Proposed method hardly degenerate accuracy or robustnessof registration result in both synthetically deformed data and clinical data. • Energy function can be minimized using modified (efficiently implemented) TRW-Sby converting higher order cliques to pairwise interaction. • Further research is needed to reduce the time for data cost calculation.

21. Thank you! • Question • Comment • Suggestion

22. Weaker Relaxation sx tx s t sy ty • For convenience, we consider 2D decomposed model. [Shekhovtsov et al., CVPR 2007] The decomposed model represent each interacting pair (ls, lt) as a four tuple (lsx , lsy , ltx , lty). It is easy to see that there exists a four tuple which cannot be represented as a valid marginal distribution μst (ls, lt) over the four tuple. Energy of the decomposed model can be lower than that of the original model since the later contains less possible configurations •  Low energy bound of the proposed model is less tight!

23. TRW-S Modification • Although we successfully changed the factor nodes into pairwise interactions, we cannot avoid to include the auxiliary nodes which have vast amount of label space. Due to the min~{u} operator in message passing, the auxiliary node can induce O(|D|6) of time complexity as well as O(|D|4) of memory space to store the message. However, the storage for Ψaiactually is not needed since it never adds a value on a message. We can ignore all summations if zi != xi. Furthermore, as the auxiliary node should follow the configurations of nodes s and t when they are fixed, the search space of min{a} operator is reduced to O(|D|) which is same as the ordinary nodes. Thus, the complexity becomes O(|D|2) as like general pairwise TRW-S, by just modifying logics in the implementation. a