Development of a Remotely Reconfigurable Integrated Image Processing Platform ([RIP] 2 )

1. VG02-184 Development of a Remotely Reconfigurable Integrated Image Processing Platform ([RIP]2) Robin Coxe, Harry Kindle, Bill Marinelli Physical Sciences Inc. Erik Widding, Brad Lichtenstein, David Dean Birger Engineering Inc. Creigh Gordon AFRL/VSSE 2002 MAPLD International Conference Presentation D6 11 September 2002

2. Outline VG02-184 • The Big Picture • System Overview • Program Overview • Lessons Learned

3. Target Application VG02-184 • Goal: Enable the exploitation of hyperspectral imagery (HSI) for military, environmental, and homeland defense applications in near-real time (NRT) • Proposed Solution: A compact, flexible, and low-power FPGA-based reconfigurable computing device • Challenge: Next generation of hyperspectral imagers will have data rates of >100 Mbits/s

4. Y A i r / S p a c e b o r n e S e n s o r P l a t f o r m F P G A D o w n l i n k X P r o c e s s o r N R T H y p e r s p e c t r a l U p d a t e I m a g e r R a t e F i e l d C o m m a n d e r ’ s L a p t o p Real-Time HSI Processing VG02-184 • Provide end user with customized hyperspectral data products at real-time video rates E E V V I I L L N o x i o u s C l o u d o f G a s F - 5 6 7 9

5. R a w I m a g e r y D a t a ( A D C C o u n t s ) G a i n & O f f s e t C o r r e c t i o n s C a l i b r a t e d A t m o s p h e r i c I m a g e r y C o r r e c t i o n s I m a g e C l a s s i f i c a t i o n O p t i o n a l L o s s l e s s P r o c e s s e d D a t a C o m p r e s s i o n P r o d u c t s a n d / o r E n c r y p t i o n F - 2 7 6 1 Hyperspectral Image Processing VG02-184 • Processing of imagery from hyperspectral remote sensors is currently extremely time-consuming • sheer quantity of data • glacially slow I/O • size/weight/volume/power constraints • Delays are unacceptable for many military applications • Must bust the bottleneck in order to exploit HSI for applications with strict timeliness requirements

6. Release point Release point Visible image of chemical plume release stack Broadband IR image w/automated detection Spectrum of representative pixel w/DMMP detect Chemical Plume Visualization VG02-184 • Release above is DMMP (~1800 ppmv, 160C) • 8 frames/wavelength, tint = 1.44 ms • 1.3 km stand-off • False color overlay on broadband IR (red = probable detect, yellow = possible detect)

7. Program Milestones VG02-184 • Phase I Small Business Innovation Research Contract: • Topic AF02-035 (AFRL/VSSE), POP: 1 May 2002-28 February 2003 • demonstrate radiometric calibration of archived data from the 36-band PSI AIRIS hyperspectral imager at a frame rate of 30 Hz • detail two alternative concepts for COTS prototype capable of executing two benchmark hyperspectral image processing algorithms at real-time video rates • Phase II Goals: • hardware-in-the-loop demonstrations of a real-time image processor prototype with multiple sensor configurations • demonstration of remote run-time reconfiguration of FPGAs • space qualification plan • Phase III Goals: • engineering model of a reconfigurable image processing unit for a space sensor

8. PSI’s [RIP]2 Concept VG02-184 • An integrated sensor and image processing platform usable by mere mortals • Located on sensor platform: decrease data volume  faster downlink • In-mission reconfigurable processing • adapt to mission changes and sensor performance • use embedded microprocessors to orchestrate reconfiguration of FPGA • Capable of robust operation in hostile environments • Develop and debug applications on conventional PCs • implement DSP functions in H/W using Matlab/Simulink/System Generator tools (OO H/W design) • port C applications into microprocessors embedded in FPGAs • store configuration bitstreams in FLASH memory (retain IP rights) • Scaleable to parallel applications

9. Radiometric Calibration Demo VG02-184 • Read in 2 calibration datacubes (64x64 pixels x 36 ): low temperature blackbody and high temperature blackbody • Given the blackbody radiance L(T, ) in W/(cm2 sr m), calculate the gain and offset coefficients at each pixel at each wavelength • Convert the number of ADC counts associated with each pixel at each wavelength for each image cube into a spectral radiance using the gain and offset coefficients: • Steps #1 and #2 performed in advance and do not occur in real time

10. H e a d e r C o n n e c t o r f o r E n a b l e F u t u r e c o n n e c t i o n t o P S I A I R I S c a m e r a C o n f i g u r a t i o n C o n f i g u r a t i o n R S - 4 2 2 I n p u t R S - 4 2 2 O u t p u t C a b l e E E P R O M C o n n e c t o r C o n n e c t o r 2 3 t U A R T I / F M o d u l e R S - 4 2 2 I / F M o d u l e r 2 O s c i l l a t o r o - S P R H S I D a t a E m b e d d e d 3 2 - S i m u l a t o r M o d u l e t r b i t R I S C C P U o S R A M P C o r e l e R a d i o m e t r i c l l a C a l i b r a t i o n r a V G A d i s p l a y I / F M o d u l e P X i l i n x V i r t e x 2 F P G A B i r g e r E n g i n e e r Radiometric Calibration Demo Hardware VG02-184 t F c / a I p h m s a o l • Image datacubes stored in Compact Flash clocked into radiometric calibration pipeline at 9 MB/s by HSI data simulator module to ensure datacube processing rate of 30 Hz • Selected calibrated frames are displayed on VGA monitor C F T r i p l e 8 - b i t D A C i n g I n c . V i r t e x - I I D e v e l o p m e n t B o a r d F - 5 6 8 0

11. Prototype System Design VG02-184 • Goal: fully calibrate and classify HSI at 30 Hz/datacube (RT video) • Benchmark Algorithms: spectral angle mapper (simple) and a PSI proprietary multi-band correlation analysis (complicated and >O(1) more computationally intensive) • Evaluating performance for three system options: • Virtex-II FPGAs with embedded 125 MHz MicroBlaze “soft core” RISC processor(s) [Phase I demo] • Virtex-II Pro FPGA: Virtex-II FPGAs with up to four embedded 300 MHz+ PowerPC 405 “hard cores” per chip and 3.125 Gbps serial interconnects [Phase II prototype] • Texas Instruments TMS320C6701 floating-point DSP with Virtex FPGA accelerator

12. U s e r C a l c u l a t e C o s i n e o f S p e c t r a l A n g l e R e a d i n R e f e r e n c e f o r E a c h P i x e l : S p e c t r u m Y k a c o s ( ) i j Y e s N l S > T h r e s h o l d R Y i j k k R e a d i n C a l i b r a t e d ? k = 1 I m a g e r y R F r o m a c o s ( ) = i j k i j S S 2 2 R a d . C a l . M o d u l e R Y i j k k N o Performance Benchmark: Spectral Angle Mapper VG02-184 F a l s e C o l o r D i s p l a y O v e r l a y f o r S e l e c t e d P i x e l s F - 4 8 7 8 • Simple algorithm (500K +, 500K x, 5K square roots, 5K divides, 5 K comparisons) • Benchmark using GNU GCC cross-compiler and PSIM PowerPC 405 Instruction Set Simulator • Initial results indicate near-real time throughput is possible with ~4 PPC 405’s @ 300 MHz

13. M A T L A B E n v i r o n m e n t L i b r a r y ( I n c l u d i n g S i m u l i n k S y s t e m M o d e l X i l i n x B l o c k s e t ) I n p u t O u t p u t S i m u l a t i o n - 1 Z + k S i m u l a t i o n D a t a I n c l u d i n g S - f u n c t i o n s S y n t h e s i s S y s t e m G e n e r a t o r C o d e G e n e r a t i o n S o f t w a r e E N T I T Y m u l t I S • M a p t o I P L i b r a r i e s G e n e r i c ( w : • C o n t r o l S i g n a l s P o r t ( a , b : I N • V H D L D e s i g n P o r t ( y : O U T • H D L T e s t b e n c h E N D E N T I T Y • C o n s t r a i n t s • S i m u l a t i o n S c r i p t s , P r o j e c t F i l e s S y n t h e s i s C O C o m p l i e r User-Friendly System Design Approach VG02-184 GUI • Matlab and Simulink: • drag n’ drop icon-based FPGA design • straightforward assessment of fixed-point quantization errors • Xilinx System Generator automatically translates DSP algorithms modeled with Simulink into VHDL • can accommodate custom VHDL modules • System Generator produces bit- and cycle-accurate simulations of FPGA designs and a VHDL testbench • simulation and verification at early design stages saves time and money • Write applications targeted to embedded -processors in C X i l i n x C o r e D e s i g n T o o l s P a r a m e t e r s V H D L E n v i r o n m e n t R E G e n e r a t o r T e s t V e c t o r s E D I F E D I F & T i m i n g F P G A L o g i c P l a c e & R o u t e S i m u l a t o r P a s s / F a i l B i t s t r e a m F - 5 5 5 3 “Under the Hood”

14. Lessons Learned VG02-184 • Embedded RISC processors (MicroBlaze, PowerPC 405) do not have hardware floating point units • must implement floating-point math in software, compromising performance • the MicroBlaze math.h library only contains opcode for the 4 arithmetic operations • Cannot rely solely on C-compilers to optimize algorithms: • H/W efficient arithmetic routines or direct implementation in H/W • fixed point math • loop unrolling/parallel data pipelines • Potential end users are very concerned about radiation effects. • A low power system design will be a challenge for high-throughput, highly parallel applications • dynamic power consumption scales with % of logic resources utilized and # of signal transitions per clock cycle