Retinal Vascular Graph Separation for Arteries and Veins: Method and Results

1. Separation of the Retinal Vascular Graphin Arteries and Veins Speaker: Kai RothausCo-authors: P. Rhiem, X. Jiang CVPR Group, University of Münster Homepage: cvpr.uni-muenster.de

2. Outline • Introduction • Medical purpose • Image-processing • Method • SAT-problem specification (vessel labelling) • Operations for graph manipulation (edge labelling) • Solving Conflicts • Results • Conclusions and further work Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

3. Medical Purpose • Why retinal vessel are of interest? • Vessels of retina and brain are conjuct • Only on retina vessels are visible directly • Conclusions on diseases are possible • Anatomy of the eye • Vessels enter the eyeball at the optic disc • Vessels only branch (no reconnection) • Capillars are invisible • Differences of two vessel types on retina: Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

4. Vessel segmentation • Input: Retinal Image • Output: Binary vessel image • Many segmentation algorithms, based on • Matched-filter • Tracking • Intensity riges or (1st moment deviations) • Curvature (2nd moment deviations) • Special difficulties • Handling of bifurcations and crossings • Central-light reflex • Different vessel width • Wide intensity spectrum • Pathological objects nearby • Mainly, we use hand-segmented images Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

5. binary vessel image skeleton image vasculature graph Graph-based representation of the vasculature • Input: Binary vessel image • Output: Vasculature graph • Compute the skeleton of the vasculature • Classify skeleton pixel in • End pixel (form vertices of degree 1) • Connection pixel (form edges) • Branching pixel (form vertices of degree 3) • Crossing pixel (form vertices of degree 4) • Construct graph-based representation • Arising Problems: • Segmentation errors could lead to small cycles • Discontinuous segmentation leads to an over-fragmented graph representation • Skeleton of a crossing could lead to two branches Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

6. a v v a a v a v a v a v v a SAT-Problem Specification (vessel labelling) • Problem: Classify each vessel as artery (a) or vein (v) • Mainly recent approaches are based on local features • Colour, cross-profile, thickness, etc. • Work only good for thick vessels nearby the optic disc • We propose a structure-based approach (on vasculature graph) • Label each vessel segment vi as artery (Li=a) or vein (Li=v) • Formalise anatomical properties of the vasculature: • At branches only edges of the same labelling are involved • At crossings an artery crossing a vein • Construct logical clauses that describe the properties • Cumulate above rules for all vertices and formulate the SAT-problem • Solve this as a CSP (Constraint Search Problem) with AC-3 Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

7. conflict conflict manuallabel manuallabel The labelling process (AC-3*) • Add the incident vertices of few manually labelled vessel segments in the process queue Q • While Q is not empty • Take the first vertex and corresponding logical rule • Reduce set of labels of the incident vessels consistent to the rule • If there is a conflict try to solve it (details later) • Otherwise add the new vertices to Q • Order of processing the vertices (rules) is important Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

8. conflict Q={ v6 } Q={ v3,v8 } Q={ v4,v8 } Q={ v8,v7 } Improvement: Introduce an intelligent initial edge labelling to detect split crossings Q={ v7 } Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

9. vasculature graph edge labelling vessel labelling Operations for graph manipulation (edge labelling) • Segmentation or skeleton errors lead to unsolvable SAT-problem • Graph structure has to be manipulated slightly • Allowed operations should handle: • Split crossings(instead of 1 deg. 4 vertex 2 adjacent deg. 3 vertices) • Missing segments(crossing degenerated to vertex of degree 3) • Falsely detected branches • Falsely detected segments • Instead of manipulating the graph directly we introduce a second order labelling (edge labelling): Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

10. Steering the labelling process (Belief propagation) • Plausibility weights [0,1] for each vertex • Assign crossing vertex the plausibility 1 - P1(d) • Assign branch vertex the plausibility (with β =max αi) P1(d)+P2(β) - P1(d)P2(β) • Plausibility weights [0,1] for each a/v-labelled vessel • Assign hand-labelled vessels plausibility 1 • During AC-3* algorithm use a multiplicative propagation scheme (with weights of corresponding vertex and edge) • Use weights as heuristic to order Q as priority-queue • Use the average vessel weights to rate labelling results P1(d) P2(β) Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

11. Initial edge labelling • Decide on plausibility measures P1(d) and P2(β) if a connection edge between to branches is probably a crossing • No false c-label should be introduced • Label edge with c-label iff [ d<3 ]or[ P1(d)<0.75 andP2(β)<P2(30°) ] Confusion matrix on 10 training images Accuracy of >96 % Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

12. Solving Conflicts • Conflicts cannot been avoided (even not with initial labelling) • Conflicts are basically introduced by cycles in the vascular graph • Topology is responsible for conflicts • Solving-strategy: • Search cycle (vertex set V’), where all vessel labels are defined • Establish edge candidate set E’={ e | e incident to a v in V’ } • Choose a “suitable” n-labelled edge of E’, with minimum weight and change edge label to c (crossing) • Otherwise label the conflict edge with e (end-segment) • Restart the AC-3* algorithm Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

13. artery (auto.) vene (auto.) artery (man.) vene (man.) Original image Binary image Interactive labelling tool • Requirement: binary vessel image • Physician mark single vessel segments as arteries an veins • Propagation of the manual labelling as far as possible • Solve logical conflicts automatically • If the result is not good enough for the observer, more vessel label could be manually added • Presenting results in two different ways: Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

14. Results on manually segmented images • STARE data set of A. Hoover et al. image im0082 Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

15. Discussion results on manual segmentations • Most conflicts could be solved by introducing c-label • Only few conflicts could not been solved • Problematic regions are even hard to been labelled by experts • Normally few hand-labels are necessary Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

16. Results on automatic segmentations • Method of Soares et al. and test database DRIVE of Staal • High demands on segmentation algorithm:Different vessel width, no gaps in segmentation, low false positive rate, etc. • Some segmentations leads to poorly connected graphs (less rules) Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

17. Summary and Conclusions • We have developed a method for propagating vessel classification • Requirement is a binary vessel image • Problem is formulated as Constraint Search Problem • Arising conflicts are solved by manipulating graph structure • Interactive environment is developed for physicians • Methods works good for tested image databases • Quality depends strongly on segmentation result • Further works • Statistical foundation of plausibility function • Realise initial labelling with Bayesian classifier • Justify method by comparison with ground-truth data • Enhance conflict solver • Classify strong vessel automatically as artery or vein • Integrate method in a framework for vascular structure analysis Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

18. Final slide Thank you for your attention! Are there any questions? Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins