Acceleration of the Retinal Vascular Tracing Algorithm using FPGAs

1. Acceleration of the Retinal Vascular Tracing Algorithm using FPGAs Miriam Leeser Shawn Miller Smart Camera: Provides Host PC with image data along with image processing results at frame rate with low latency Framegrabber Results Image Data Data Packing Design MEMORY 2 MEMORY 0 Memory Switching Design Host BlockRAM Image Data Results Direction Filter Design MEMORY 3 MEMORY 1 FPGA PCI Bus Memory Switching Design FIREBIRD BOARD

2. Retinal Vascular Tracing Application Goal: Detection and enhancement of the vascular structure of a patient’s retina from a live video feed Latency and throughput requirements of real-time image processing cannot be provided by software on a general-purpose processor

3. Timing Issues 0 2 4 1 3 6 8 5 7 9 10 12 14 11 13 16 18 15 17 19 20 22 24 21 23 • Return results for each pixel at frame rate of camera • Very Low Latency • Surgical laser must be shut off immediately after detecting that it is aimed incorrectly • Cannot tolerate 1 frame delay (33msec at 30 frames/sec) • Complex memory management required to achieve minimum latency Storage of a 5x5 pixel image Memory 0 Memory 1 Latency currently on the order of 100msec

4. Miriam Leeser Shawn Miller Framegrabber (Dillon Eng.) Image Data Data Packing Design Image Data MEMORY 2 FPGA Memory Switching Design Memory Switching Design Host BlockRAM Image Data MEMORY 0 Direction Filter Design PCI Bus MEMORY 3 Image Data MEMORY 1 FIREBIRD BOARD Clock Input Memory 0 1 3 Input Memory 1 0 2 4 5 9 6 8 7 Time Writing data from camera 13 11 10 12 14 Reading data to be processed 15 17 19 16 18 Inactive • Reconfigurable Hardware • Firebird reconfigurable computing engine from Annapolis Micro Systems • 1 Xilinx VIRTEX E (XCV2000E) FPGA • 5 Memory banks (4 x 64-bit, 1 x 32-bit) • 5.4 Gbytes/sec of memory bandwidth • 66Mhz/64-bit PCI interface to host 23 20 22 24 21 Results Conclusions Original Image Each pixel is passed through the design unaltered. • Stand-alone camera outputs only image data. • Our design outputs not only image data, but directional template responses as well. • The cost of additional image processing is a latency on the order of 10-4 seconds. This is a low cost when considering that at 30 frames/sec, a new frame of image data is introduced every • 33 msec. • The application for this project is Retinal Vascular Tracing, but the same method can be applied to any problem that requires real-time image processing. More Information In proceedings: Rapid Automated Tracing and Feature Extraction from Retinal Fundus Images Using Direct Exploratory Algorithms A. Can, H. Shen, J.N. Turner, H.L. Tanenbaum and B. Roysam, IEEE Transactions on Information Technology in Biomedicine, June 99. On the web: http://www.ece.neu.edu/groups/rpl/projects/retinaltracing Direction The direction template with the maximum response for each pixel. The direction is represented by a value between 0 and 15. Response The maximum response that led to the direction decision for each pixel. This work was supported in part by CenSSIS, the Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award Number EEC-9986821). Acceleration of the Retinal Vascular Tracing Algorithm using FPGAs Badrinath Roysam Charles Stewart Ken Fritsche • Algorithm • What does the algorithm do? • Retinal vascular tracing: detection of blood vessels in images of the retina • The algorithm finds blood vessels and traces out their structure • Where is the algorithm used? • Processing live video of the patient retina during laser retinal surgery • Highlighting the vascular structure helps the surgeon avoid damage • Why do we need to accelerate it? • Current implementation: software on a general-purpose processor • Images are 512x512 pixels, and need to be processed at frame rate Objective To accelerate retinal vascular tracing by implementing computation of template responses in reconfigurable hardware. Direction Templates • Implementation • Hardware Acceleration • Template responses are calculated in hardware in parallel • All pixels in the image are processed • Camera connected directly to the board (no host interaction) • Only the results are sent to the host after they become available, and while new results are being calculated • Very Low Latency • Surgical laser must be shut off immediately if we detect that it is aimed incorrectly • Cannot tolerate a one frame delay • Complex memory management scheme must be introduced • Memory Management • Problems • Must continuously write new data to input memory from the camera and be reading the data to be processed • Cannot read and write from one memory on the same clock cycle • The image is stored row-wise, but must be read column-wise • Solution • Store the image in “checkerboard” fashion in two memories. Every other pixel is stored in a different memory • A column of data is read by alternating between the two memories on every clock cycle Checkerboard storage of a 5x5 image Memory 0 Memory 1