Spectral Imaging of the Retina

1. Spectral Imaging of the Retina Andy R.Harvey, Ied Abboud, Alistair Gorman, Andy I.McNaught* School of Engineering & Physical Sciences, Heriot Watt University, Edinburgh, UK *Eye Unit, Cheltenham General Hospital, Cheltenham, UK

2. Outline • What is spectral imaging? • Spectral retinal imaging • Why? • Spectral time-sequential spectral imaging • For flexibility and research • 2D snapshot • For real-time, high throughput screening • Conclusions

3. Conventional Spectral Imaging Dysplastic cell Superficial squamous cell Lymphocyte Intermediate cell PMN Spectrally classified image Classification spectra RGB Image Courtesy CRI

4. Imaging the eye Light Sclera Lens Choroid Iris Cornea Macula Optic nerve Anterior chamber (full of aqueous humour) Retina Optic disc Vitreous humour Posterior chamber

5. The Role of Spectral Retinal Imaging • By 2020 there will be 200 million visually-impaired people world wide • Glaucoma, diabetic retinopathy, ARMD • 80% of those cases are preventable or treatable • Screening and early detection are crucial • Can spectral imaging offer enhancements to current screening techniques ? • Spectral imaging is non-invasive and safe • cf. fluorescein angiogram • Spectral imaging can enable imaging of • Retinal biochemistry • Blood oximetry • Diabetic retinopathy, glaucoma • Lipofuscin etc • Age-related macula degeneration

6. Spectral Imaging:Traditional approaches Nl(t) Ny Nx Nl Ny(t) Nx Time-sequential spectral multiplex Time-sequential spatial multiplex • Limitations • Optically inefficient • 2D time-varying scenes • 2D snapshot • required for: • Retinal imaging • in vitro, in vivo imaging And Fourier-transform equivalents

7. Spectral Fundus Camera • Source filtering by LCTF incorporated into COTS fundus camera • 10 nm spectral width • 20 msec random access • Images captured using a cooled, low-noise CCD camera

8. Coregistered Spectral Images of a Healthy Retina Isobestic point Images translationally and rotationally coregistered

9. Spectral angle map of healthy and diabetic retina Normal Retina Diabetic Retina • Shading indicates similarity of each pixel spectrum with artery and vein spectra • Qualitative oxymetry

10. Supervised spectral classifiaction • Implicit calibration based on spectral signatures within the eye • Classification possible without absolute calibration • Clear distinction between veins/arteries, on/off optic disc • Spectra depends on local environment • Inversion of data to calculate biochemical concentrations (eg oxygenation) requires a model of light propagation and scattering in the retina to remove environmental effects • Monte Carlo, Kubelka Monk, Transfer equation

11. Requirements for a snapshot technique: retinal imaging PC15 • Improved calibration • Patient patience • Remove imperfect coregistration • due to Variations in imaging distortion between images • Similar issues with other in vivo applications • Imaging internal epithelial cancers • Eg gastrointestinal

12. Image Replication Imaging Spectrometer:IRIS F F F F F F F F F Snapshot image • zero temporal misregistration • ‘100%’ optical efficiency • World’s only snapshot, 2D spectral imager (almost !) • Conceptually related to Lyot filter Large format detector Spectral Demultiplexor

13. Lyot filter: principle of operation Waveplate Polariser

14. IRIS snapshot spectral imager: principle of operation • Wollaston prism polarisers replicate images • Each Wollaston prism-waveplate pair provides both cos2 and sin2 responses • All possible products of spectral responses are formed at detector

15. Spectral transmission cos2(pnD)cos2(2pnD) cos2(pnD)sin2(2pnD) sin2(pnD)cos2(2pnD) cos2pnD sin2(pnD)sin2(2pnD) sin2pnD cos2(pnD)sin2(2pnD)cos2(4pnD) cos2(pnD)cos2(2pnD)cos2(4pnD) cos2(pnD)cos2(2pnD)sin2(4pnD) cos2(pnD)sin2(pnD)sin2(4pnD) sin2(pnD)cos2(2pnD)cos2(4pnD) sin2(pnD)sin2(2pnD)cos2(4pnD) sin2(pnD)cos2(2pnD)sin2(4pnD) sin2(pnD)sin2(pnD)sin2(4pnD) Wollaston/waveplate assembly

16. Spectral responses • 32 channel, visible-band system • 520nm 720nm • 5 Quartz retarders • 8 channel visible-band system • 520nm820m • 3 Quartz retarders • Bands are overlapping bell shapes • Choose cost function to minimise sidelobes • Small (~5%) reduction in spectral separation • Cut-off filters used to define spectral range

17. Optical scaling laws Polariser, retarders & Wollaston prisms (index matched) Field stop Camera Bandpass filter Imaging lens Collimating lens Primary lens Hamamatsu ORCA-ER Outputs: Field stop size Collimating lens rear element diameter Splitting angles, apertures & depths of prisms Apertures of retarders, polarisers and filters Imaging lens focal length & front element diameter Inputs: FoV Sub image size on CCD CCD pixel size Primary lens magnification & F# Collimating lens back focal distance, focal length & front element diameter Prism birefringence

18. Components & Assembly • 8 channel system • 520nm to 820nm • 3 Quartz retarders • 3 Calcite Wollaston prisms

19. Spectral Retinal Imaging Canon CR4-45NM • Difficult imaging conditions render application of traditional HSI techniques problematic • IRIS enables real-time and snapshot spectral imaging

20. Blood oximetry 40 20 • Optimal spectral band for retinal oximetry • Vessel thickness ~ optical depth • 570-615 nm • Eight bands approximately equally spaced 80

21. Video sequence recorded with bandpass filtered inspection lamp

22. Retinal image recorded with flash illumination

23. 574 581 592 585 607 595 603 613 Coregistered and PCA images PC1 & PC2 PC2 PC1

24. Summary • Spectral imaging of the retina shows promise for non-invasive detection of retinal disease • Clinical trials on-going • LCTF-based, time-sequential spectral filtering enables rapid and flexible 2D spectral retinal imaging • Flexible data acquisition • Pulse and other motion artefacts limit accuracy • Snapshot spectral imaging in 2D (IRIS) promises high-performance real-time multi-spectral imaging • Ideal for in vivo imaging • No temporal misregistration • Absolute, quantitative data requires a model of light interaction within the retina

25. IRIS snapshot spectral imager • Wollaston prism polarisers replicate images • Each Wollaston prism-waveplate pair provides both cos2 and sin2 responses • All possible products of spectral responses are formed at detector

26. Measured & predicted spectral responses

27. Absolute total transmission Absolute response curves in polarised light 50 Response (%) 25 0 • Bandpass filter & polariser dominate losses • Improved system: T>80% • Theoretical throughput is 2n times higher than for spatial/spectral multiplexed techniques!

28. Application to microscopy:Imaging of multiple fluorophors • IRIS fitted to conventional epi-fluorescence microscope • Germinating spores of Neurospora crassa stained with • GFP – nucleii fluoresce at 510 nm • FM4-64 – membranes fluoresce at >580 nm 50 Response (%) 25 0

29. Principle component decomposition PC1 PC3 • Artery structure is a pulse artefact • Very difficult to co-register by image processing means • Snapshot technique desirable PC15

30. Conclusions • IRIS is a new spectral imaging technique that enables snapshot spectral imaging in 2D • No rejection of light • No data inversion • Highest-possible signal-to-noise ratios • Simple logistics • Inherently compact and robust • Simply fitted to conventional imaging systems • Birefringent materials exist for applications from 0.2m to 12 m • Applications • In vivo, in vitro imaging • Retinal imaging • Microscopy • Multiple fluorophors • Quantum dots • Surveillance • Remote sensing • Etc.

31. Spectral Characteristics of the Retina Isobestic point Red Green Blue • Optical depth of Hb & HbO2 dominates variation of penetration with l • Tissues vary between highly turbid and transparent • Blue light images retinal surface • Light at ~600 nm enables spectral oximetry within retinal blood vessels • optical depth of HbO2 > vessel thickness so vessels translucent • optical depth of Hb< vessel thickness so vessels are opaque • Light > 640 nm penetrates to coroid

32. Issues for Spectral Retinal Imaging ± 100 pixels ±2º • Spectral imaging of static scenes is relatively ‘easy’ • Spectral imaging of the retina encounters • Imaging through an erratically moving, low-quality f/6 eye-lens system • Calibration • Components of interest within acomplex turbid medium • Patient tolerance • Using current technology, time-sequentialspectral bandpass offers • Optimal SNR • Reduced light intensity at the retina • Agile selection of spectral bands (data efficient) • Issues • Coregistration • Calibration Solutions: 2D snapshot spectral imaging

33. The End

34. Direct Imaging Spectrometry (Fourier) Transform Imaging Spectrometry Scanning mirror Ns Fixed mirror ND(t) FT Nl(t) Ny Nx Ny Ny Detector array Nx Nx Ns Nl ND FT FT Ny(t) Ny(t) Ny(t) Nx Nx Nx 1D image x path difference D Temporally scanned Snapshot/fully staring

35. Why another spectral imaging technique? • Traditional approaches • Time sequential spectral multiplex • Monochromatic two-dimensional image in snapshot • Time sequential spatial multiplex • One-dimensional spectral image in a snapshot • (and Fourier-transform equivalents) • Problems • Cannot record two-dimensional spectral images of time-varying scenes • Optically inefficient • Time-resolved (snapshot) spectral imaging is required for • Dynamic scenes • In vitro, in vivo imaging and microsocopy • Combustion dynamics, surveillance… • Irregular motion between scene and imager • In vivo imaging • Ophthalmology • Remote sensing, airborne surveillance, industrial inspection…