1 / 45

IMAGERET Detection and Decision-Support Diagnosis of Diabetic Retinopathy Using Machine Vision

IMAGERET Detection and Decision-Support Diagnosis of Diabetic Retinopathy Using Machine Vision. Professor Heikki Kälviäinen et al. Machine Vision and Pattern Recognition Laboratory (MVPR) Department of Information Technology Faculty of Technology Management

coby
Download Presentation

IMAGERET Detection and Decision-Support Diagnosis of Diabetic Retinopathy Using Machine Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IMAGERETDetection and Decision-Support Diagnosis of Diabetic Retinopathy Using Machine Vision Professor Heikki Kälviäinen et al. Machine Vision and Pattern Recognition Laboratory(MVPR) Department of Information Technology Faculty of Technology Management Lappeenranta University of Technology (LUT), FINLAND Heikki.Kalviainen@lut.fi http://www.lut.fi/~kalviai http://www.it.lut.fi/ip/research/mvpr/ Prof. Heikki Kälviäinen et al., ImageRet Project

  2. Lappeenranta University of Technology (LUT) FINLAND Lappeenranta Helsinki Oslo St.Petersburg Stockholm Tallinn Moscow London Berlin Prof. Heikki Kälviäinen et al., ImageRet Project

  3. Outline • Machine Vision and Pattern Recognition Laboratory. • Diabetes and retina. • ImageRet project and the consortium. • Objectives and results. • On-going research and future challenges. Prof. Heikki Kälviäinen et al., ImageRet Project

  4. LUT Information Technology (LUT IT) • Leader: Prof. Heikki Kälviäinen. • 6 Professors, 70 members, 850 B.Sc./M.Sc./Ph.D. students in total, 60-70 masters and 4-5 doctors per year. • Laboratories: • Machine Vision and Pattern Recognition (MVPR). • LUT Center of Excellence in Research. • Software Engineering (SWE). • Communications Software (CS). LUT IT: http://www.it.lut.fi MVPR: http://www.it.lut.fi/ip/research/mvpr/ Prof. Heikki Kälviäinen et al., ImageRet Project

  5. MVPR Laboratory: Research Profile Prof. Heikki Kälviäinen et al., ImageRet Project

  6. Machine Vision and Pattern Recognition Laboratory (MVPR) • Leader: Prof. Heikki Kälviäinen. • 2nd largest computer vision research group in Finland. • Center of Excellence in Research in LUT. • 24 members: • 3 Professors + 3 Post docs + 2 Visiting doctors + 11 PhD students + undergraduate students + industry coordinator. • Co-operation with 14 international universities and research institutes. • Results: 18 Ph.D. degrees (and 3 externally produced), over 400 scientific publications, 40 research projects, and spin-off companies. • Objectives: 2 PhDs/year. • Annual external project funding 700.000 EUR, basic funding 300.000 EUR, total 1.0 million EUR. http://www.it.lut.fi/ip/research/mvpr/ Prof. Heikki Kälviäinen et al., ImageRet Project

  7. Diabetes • Diabetes is a metabolic disorder where the blood glucose level has been increased. • Two types: • Type 1: insulin dependent (mostly children and young persons). • Type 2: non-insulin dependent (mostly middle-aged and elderly people). • Diabetes is a serious disease: • When left untreated, diabetes can lead to serious medical complications of the kidneys, the peripheral nervous system, the eyes, and it can also cause cardiovascular diseases. • Diabetes is a common and rapidly increasing disease: • For example, 500 000 Finns (10 % of the population) are having Type 1 (in Finland the most common in the world) or Type 2 (In Finland the most common in the Nordic countries), with the increase of the number of Type 2 diabetics by 70 % in the next 10 years! • Diabetes is an expensive disease: • 12 % of total health service costs in Finland. • Approx. 15% of all health care expenses in EU go to the treatment of diabetes and its complications. Prof. Heikki Kälviäinen et al., ImageRet Project

  8. Diabetic Retinopathy • Diabetic retinopathy is a disease of the retina (the tissue responsible for vision in the eye) caused by diabetes. • Without proper treatment it can lead to the loss of vision or even blindness (the leading cause of the blindness in the working age population). • Early detection of the retinal complications is crucial. • An ophthalmic fundus camera can be used to monitor the condition of the retina => fundus photoghaphy. Prof. Heikki Kälviäinen et al., ImageRet Project

  9. Fundus Image Acquisition The eye: Besides the vision system, an useful peephole inside a human being to see what is happening. Zeiss Fundus Camera:1500 x 1152 pixels 24 bits per pixel. Prof. Heikki Kälviäinen et al., ImageRet Project

  10. Prof. Heikki Kälviäinen et al., ImageRet Project

  11. Challenges and Objectives • Challenges: robust screening needed. • 500.000 diabetes patients in Finland and the number is increasing. • How to monitor the known patients and find the new ones? • Not enough medical experts nor funding for applying current practices. => We must find robust automatic or semiautomatic solutions for two tasks: • To decide whether the eye is healthy or not (the disease present or not). • To find reliably abnormalities in the eye, if the eye is considered to be not healthy. Prof. Heikki Kälviäinen et al., ImageRet Project

  12. ImageRet Project • FinnWell technology program was established by Finnish National Agency for Technology and Innovation (TEKES). • The project called Optimal Detection and Decision-support Diagnosis of Diabetic Retinopathy (ImageRet) was established to develop reliable and accurate image processing and pattern recognition methods for automatic fundus analysis. • Project of 38 months in 2006-2009 with the several partners (750.945 EUR): • Lappeenranta University of Technology (LUT), Finland: project coordination, machine vision and pattern recognition. • University of Kuopio, Finland (UKU): ophthalmology. • University of Joensuu, Finland (UJO): spectral imaging. • Mikkeli Polytechnics, Finland (MAMK): databases and metadata. • University of Bristol, UK (UB): optic disk detection. • Companies: Kuomed Oy, Mawell Oy ,Perimetria Oy, Pfizer Oy, Santen Oy, VAS Oy. http://www.it.lut.fi/project/imageret/ Prof. Heikki Kälviäinen et al., ImageRet Project

  13. ImageRet: Acknowledgements Intensive co-operation of many researchers Pauli Fält (UJO), Jari Forström (MAMK), Pertti Harju (MAMK), Dr. Jouni Hiltunen (UJO), Dr. Markku Hauta-Kasari (UJO), Valentina Kalesnykiene (UKU), Tomi Kauppi (LUT), Markku Kuivalainen (LUT), Prof. Joni Kämäräinen (LUT), Prof. Heikki Kälviäinen (LUT), Dr. Lasse Lensu (LUT), Mika Letonsaari (MAMK), Prof. Majid Mirmehdi (UB), Pekka Nikula (LUT), Prof. Jussi Parkkinen (UJO), Dr. Juhani Pietilä (Perimetria), Markku Rossi (MAMK), Prof. Iiris Sorri (UKU), Prof.Hannu Uusitalo (UKU), etc. & many company representatives. Prof. Heikki Kälviäinen et al., ImageRet Project

  14. ImageRet: Objectives • Image annotation tool for medical expert annotation. • Medical experts can save and compare their diagnoses with the tool. • Fundus image databases. • Expert annotations collected as ground truth in public databases. • Testing protocols for benchmarking between different methods. • Private patient databases (including temporal changes in the eye). • Evaluation framework. • A solid basis for the image analysis system development and comparison. • Image-based and pixel-based methods. • Image-based: Is there a healthy eye or not in an image? • Pixel-based: Detection of abnormalities related to diabetic retinopathy: hard exudate, soft exudate, hemorrhage, microaneurysm, and neovascularization. • Spectral imaging. • How much more it can be “seen” using spectral imaging? Prof. Heikki Kälviäinen et al., ImageRet Project

  15. 2. 1. 3. Normal Fundus Image • Papilla (optic disk). • Blood vessels. • Macula (the area of the sharp vision). Prof. Heikki Kälviäinen et al., ImageRet Project

  16. Hard Exudate • Hard exudate consists of blood plasma and lipids leaked from blood vessels. • It is one of the most commonly occurring lesion in diabetic retinopathy. • Yellow-white lesions. • Sharp margins. Prof. Heikki Kälviäinen et al., ImageRet Project

  17. Soft Exudate • Soft exudate is a micro-infarct occurring in an eye. • Yellowish lesions. • Fuzzy margins. Prof. Heikki Kälviäinen et al., ImageRet Project

  18. Hemorrhage • Hemorrhage consists of blood leaked from vessels. • Dark red lesions. • The color is quite similar as in vessels. Prof. Heikki Kälviäinen et al., ImageRet Project

  19. Microaneurysm • Out-pouching of capillary. • Visible as a tiny red dot. • The first observable type of lesion in retinopathy. • Quite difficult to notice in a color fundus image. Prof. Heikki Kälviäinen et al., ImageRet Project

  20. Neovascularization • Abnormal vessels growing to satisfy the lack of oxygen in a retinopathic eye. • Can cause severe problems. Prof. Heikki Kälviäinen et al., ImageRet Project

  21. Medical Expert Annotations Digital fundus image. Medical expert annotations. Prof. Heikki Kälviäinen et al., ImageRet Project

  22. Image Annotation Tool Prof. Heikki Kälviäinen et al., ImageRet Project

  23. Fundus Image Databases Databases • DIARETDB0 • DIARETDB1 • DIARETDB1 V2.1 publicly available at http://www.it.lut.fi/project/imageret/ Prof. Heikki Kälviäinen et al., ImageRet Project

  24. Diabetic Retinopathy Database Images (89): Train images 28 Test Images 61 Med. experts 4 Findings: Haemorrhages (Ha) Microaneurysms (Ma) Hard exudates (He) Soft exudates (Se) Prof. Heikki Kälviäinen et al., ImageRet Project

  25. Evaluation Framework Prof. Heikki Kälviäinen et al., ImageRet Project

  26. Evaluation Framework – Training • Colour. • Texture. • Shape. • Uneven illumination. • Colour distortions. • Imaging related. • Eye related. • Imaging noise. Prof. Heikki Kälviäinen et al., ImageRet Project

  27. Fusing Multiple Medical Expert Annotations (b) (a) (c) Annotation fusion approaches*: a) weighted area intersection b) representative point neighbourhood c) representative point neighbourhood masked. * Tomi Kauppi, Joni-Kristian Kämäräinen, Lasse Lensu, Valentina Kalesnykiene, Iiris Sorri, Heikki Kälviäinen, Hannu Uusitalo, Juhani Pietilä, Proc. of the 16th Scandinavian Conference on Image (SCIA2009), Fusion of multiple expert annotations and overall score selection for medical diagnosis. Prof. Heikki Kälviäinen et al., ImageRet Project

  28. Feature Extraction - Colour as Feature Diabetic retinopathy colour distributions Prof. Heikki Kälviäinen et al., ImageRet Project

  29. Estimating Colour Distributions: Learning Estimating colour distributions with a Gaussian mixture model using the unsupervised Figueiredo-Jain algorithm. Prof. Heikki Kälviäinen et al., ImageRet Project

  30. Evaluation Framework – Analysis Prof. Heikki Kälviäinen et al., ImageRet Project

  31. Analysis – Classification (Colour) 1/2 a) b) Pixel-wise likelihood for hard exudates: a) original image; b) probability density map (likelihood) for colour (RGB). Prof. Heikki Kälviäinen et al., ImageRet Project

  32. Analysis – Classification 2/2 • The pixel-wise probability of diabetic finding, p(finding), for image is the combination of the selected probability density maps: p(finding) = p('colour')p('texture')p('reliability' )... Prof. Heikki Kälviäinen et al., ImageRet Project

  33. Analysis – Overall Score Fusion • Test hypothesis = the disease present (positive) or not (negative). • Overall score = a test outcome indicator for an image (a higher value increases the certainty of the positive outcome). • Overall score fusion strategies*: max, summax, mean, product. * Tomi Kauppi, Joni-Kristian Kämäräinen, Lasse Lensu, Valentina Kalesnykiene, Iiris Sorri, Heikki Kälviäinen, Hannu Uusitalo, Juhani Pietilä, Proc. of the 16th Scandinavian Conference on Image (SCIA2009), Fusion of multiple expert annotations and overall score selection for medical diagnosis. Prof. Heikki Kälviäinen et al., ImageRet Project

  34. Evaluation Framework – Evaluation • Sensitivity. • Specificity. * Tomi Kauppi, Valentina Kalesnykiene, Joni-Kristian Kämäräinen, Lasse Lensu, Iiris Sorri, Asta Raninen, Raija Voutilainen, Hannu Uusitalo, Heikki Kälviäinen, Juhani Pietilä, Proc. of the British Machine Vision Conference (BMVC2007), The DIARETDB1 diabetic retinopathy database and evaluation protocol, pp. 252-261, Vol. 1. Prof. Heikki Kälviäinen et al., ImageRet Project

  35. Evaluation – Receiver Operating Curve (ROC) Parameters Sensitivity = T_P/(T_P+F_N) Specificity = T_N/(T_N+F_P) T_P = true positives (abnormal) T_N = true negatives (normal) F_P = false positives (normal as abnormal) F_N = false negatives (abnormal as normal) Prof. Heikki Kälviäinen et al., ImageRet Project

  36. Evaluation – ROC Curves = Equal error rate (EER) = Weighted error rate (WER) Prof. Heikki Kälviäinen et al., ImageRet Project

  37. Method Development So Far • Automatic image analysis in human supervision is possible. • Possible applications. • Medical diagnosis assistance – screening. • Fundus image sorting according to severity/certainty of the disease. • Semi-automatic tool to aid remote diagnosis. • Quality control of diagnosis work. • Patient specific image databases. Prof. Heikki Kälviäinen et al., ImageRet Project

  38. Spectral Imaging: Significantly New Information about Diabetic Retinopathy ? • Grayscale images: 1 channel (e.g. fluorescein angiography). • RGB images: 3 channels (colour photographs). • Spectral images: Tens or hundreds of separate colour channels. => Contain significantly more colour information than RGB images. Grayscale RGB Spectral image R/G/B Spatial Spectrum Spatial Spatial Prof. Heikki Kälviäinen et al., ImageRet Project

  39. Article in the 16th Scandinavian Conference on Image Analysis (SCIA 2009), Oslo, Norway, June 15-19, 2009:Extending Diabetic Retinopathy Imaging from Color to Spectra Pauli Fält1, Jouni Hiltunen1, Markku Hauta-Kasari1, Iiris Sorri2, Valentina Kalesnykiene2, and Hannu Uusitalo2,31InFotonics Center Joensuu, University of Joensuu, Joensuu, Finland2Department of Ophthalmology, Kuopio University Hospital and University of Kuopio, Kuopio, Finland3Department of Ophthalmology, Tampere University Hospital, Tampere, Finland

  40. Spectral Fundus Camera • Built by Color Vision Group, University of Joensuu, Finland based on Canon CR-45NM fundus camera. • Spectral separation by 30 narrow bandpass interference filters. • 400 – 700 nm at approx. 10 nm steps. • A digital grayscale image for each filter separately => spectral image. Prof. Heikki Kälviäinen et al., ImageRet Project

  41. Human Subjects 66 volunteers: 54 diabetic patients + 12 control subjects. The clinical trials were conducted in the Department of Ophthalmology of the Kuopio University Hospital, Finland. A corresponding spectral database will be published soon. Prof. Heikki Kälviäinen et al., ImageRet Project

  42. Optimal Colour Channels • RGB • 580 / 540 / 500 nm Prof. Heikki Kälviäinen et al., ImageRet Project

  43. Can We “See” More? • RGB • 580 / 540 / 500 nm Prof. Heikki Kälviäinen et al., ImageRet Project

  44. Yes? Prof. Heikki Kälviäinen et al., ImageRet Project

  45. Summary of Results and Future Challenges • Image annotation tool for medical expert annotation. • Done. • Fundus image databases. • Done as defined in the objectives. • More expert annotations to verify ground truth and a new release, if needed. • First patient databases as a function of time collected. • Spectral database to be published. • Evaluation framework. • Done. • Image-based and pixel-based methods. • “Semiautomatic” solution done (image-based screening). • More method development needed: spatial prior information, texture analysis, shape analysis, spectral colour information. • Other diseases than diabetes. • Spectral imaging. • Images taken and preliminary expert annotations marked (more needed). • Feature selection to be studied and related methods to be developed. Prof. Heikki Kälviäinen et al., ImageRet Project

More Related