GPU RAYTRACING FOR REAL-TIME SENSOR-BAND PHENOMENOLOGY MODELING JRM Technologies, Inc.

1. WIDA 2012 GPU RAYTRACING FOR REAL-TIME SENSOR-BAND PHENOMENOLOGY MODELING JRM Technologies, Inc. Spectral Sciences, Inc. Army Research Laboratory

2. JRM Christopher E. Fink, Ph.D. Daniel Bybee, BSCS Joseph Russ Moulton, Jr., MSEE Karl Leodler, BSAE SSI Dave Robertson, Ph.D. ARL Richard Shirkey, Ph.D. Personnel

3. Goal : A fast, radiometrically-correct sensor-band scene renderer. Typical Application : NVG target-to-background contrast assessments in highly-cluttered urban scenes. Could provide input to TAWS. Solution : GPU-based raytracing Start with nVidia Scenix/Optix engines (CUDA language) Add On-the-Fly Scene Geometry Generator Support loading OpenFlight models & textures Ephemeris & Natural Irradiance Prediction (solar, lunar, stellar, airglow, etc.) Planckian & Gas Discharge Local Lighting Modtran Atmospherics & Local Atmospheric Scattering Sandford-Robertson BRDF Reflection with measured material data Sensor Effects Processing (optics, detector, electronics) OVERVIEW

4. Why GPU Raytracing? Wireframe/Polygonal Raytracing allows for : High spatial resolution Local and area sources Specularity Refraction/Transmission Shadowing Multiple reflections Atmospheric Scattering GPU processing brings speed. Raster Graphics Raytraced

5. Back-Tracing Pros: Rays only generated where they contribute to viewpoint. Cons: Poor sampling of sources. Have to regenerate rays for every viewpoint change. Forward-Tracing Pros: Good source sampling Cons: Extra bookkeeping needed to direct rays to observer. Photon Mapping Hybrid Forward to deposit photon energy from sources onto surfaces, scatter, and repeat. Produces global illumination solution. Backward to sample the distribution from multiple arbitrary viewpoints, without recalculating global solution. “Backward” “Forward” Hybrid : Photon Map Backtracing vs. Forward-Tracing

6. Additional Radiometric Optimizations Atmospheric Photons : Allowing photons to “stick” in atmosphere, not just on surfaces, allows prediction of atmospheric scattering, without expensive volumetric gridding. Separation of “Scene-External” and “Scene-Internal” Atmospherics: Outside SkyDome : Offline pre-processing of multi-layer/multi-path models to form hemispherical radiance map for inward raycasts. Inside SkyDome : Atmospheric photon-map based scattering & propagation. Importance Sampling : Reduces number of raycasts required for sufficient sampling of BRDF, BSDF, and SkyDome Irradiance functions. Bounding Volume Hierarchy (BVH) : Advanced photon-map storage/retrieval technique. Progressive Refinement : Iterative photon buffering & re-use technique.

7. Grid Spec Road Spec Building Spec Wall Spec Story Spec Window Spec MMLS Spec Scene Graph / Geometry Loading Option 1 : User-defined on-the-fly geometry creation Examples UrbanSceneSimple UrbanScenePhaseI 1 ROAD_ROW 1 ROAD 1 BUILDING_ROW 1 BUILDING 1 STORY 1 ROAD_ROW 1 ROAD 2 BUILDING_ROWs 2 BUILDINGs per row 1 STORY per building

8. Scene Graph / Geometry Loading • Option 2 : Pre-generated, reusable 3D CAD models • OpenFlight 3D terrain and entity models & associated material-encoded textures (MCMs, Emat fractions) • CSG CMMW terrains (height field + material code + RF sigma0) • Wavefront OBJ • Collada • Use of material-encoded textures (rather than just polygons) allows for higher spatial resolution. • These all required creation of database format converters to feed Nvidia’s Scenix v.7.2 scenegraph traverser.

9. Ephemeris model & stellar data Atmospherics model & input specification or data Irradiance model Man-Made local sources & spectral power density data Thermal modeling & material data Reflection modeling & surface BRDF data Sensor Effects (optics, detector, electronics) Sensor-Band Requirements

10. Atmospheric Radiance, Scattering & Transmission Loss Seamless Modtran/Radtran Integration • Scene-External Contributions • Direct Lunar • Diffuse Lunar (single + multiply-scattered) • Diffuse Skyshine (thermal) • Diffuse (aggregate) stellar • Airglow/Aurorae (via SAMM/SHARC model) • Nearby Cityglow • Scene-Internal Data • Extinction Coefficient • Scattering Albedo • Henyey-Greenstein Parameter • Thermal Emission per unit volume parameter

11. NVG-band Backtraced Example Scenes Default VIS-band Sensor-specific NVG band Note shadows, transmission, refraction, local lights, and sensor effects.

12. Forward-Traced Multiple Surface Scattering Example Forward-traced Photon mapping combined with material properties and local irradiance. Two reflections : Note appearance of human threat in the corner! Single-reflection-only case

13. Atmospheric Scattering & Transmission Loss • Note : • shadows • direct local radiance • surface reflection • atmospheric scattering • atmospheric transmission loss

14. User Interface OSV/RT GUI Scenario/Sensor Control Image/Video Output OSV Physics-enabled Raster Graphics Optix/JRM/SSI Physics-enabled Raytraced Graphics Materially-encoded 3D OpenFlight Database • SigSim • Pre-processing • Atmospherics • Thermal • MMLS Scenix Scenegraph Converter OTF Scene Geometry Spec

15. User Interface Optics Parameters Waveband & Resolution / FOV

16. User Interface Detector Params Electronics Params

17. User Interface Environmental Params Raytrace Control

18. Performance BACKTRACING 640x480 @ 80 Hz for : 27000 polys & 95 textures 6 secondary raycasts per pixel FORWARD-TRACED PHOTON MAPPING 640x480 @ 23Hz for : 27000 polys & 95 textures 1024 x 1024 photons reflecting 3 times each

19. Validation against SSI’s MC-Scene and DARPA field data Extension to Infrared regime Extension to RF regime Addition of localized clouds, dust, smokes/obscurants GUI additions, e.g. for On-the-fly Geometry Generator TAWS Integration Goals for Follow-Up Funding

20. P Validation • Options • Analytical Calculations for simple scenes • SSI’s MC-Scene NRT CPU Backtracer • DARPA field data MCScene Simulation Tool : SSI’s Monte Carlo Based Scene Simulation • UV to LWIR • 3D atmospheres & surfaces • Molecular absorption • Rayleigh scattering • Aerosol absorption & scattering • Multiple scattering • Thermal emission • Reflections from topographic terrain • Scattering, emission, and transmission by 3D clouds

21. Thermal-band Backtraced Implementation MWIR 4pm Internal heat generation Horizon & earthshine-loading on vertical surfaces LWIR 11pm MWIR 4pm Frictional BC on treads Diffraction blur Thermal noise

22. RF-Implementation : Coherence & Polarization Each ray now carries polarized, complex components. RCS reflection is now a complex Jones matrix multiplication : Each ray path now also has to carry a propagation phase factor : SAR with horizontal field SAR with vertical field

23. RF Implementation : Correlated Local Clutter Maps Improving RF clutter maps by including effects of multiple scattering Original Uncorrelated Map Single-angle Correlated Multiply-Scattered Correlated with 4x4 angle-averaging Multiply-Scattered Correlated with 18x72 angle-averaging

24. RF Implementation : Bistatic Scatter Center Sets Bistatic scatter centers compress the “internal” multiple reflections into a small set of localized transfer functions for later composition into a scene, thus saving a lot of run-time processing / raycasting.