1 / 34

NVL Sensor Fusion Test Bed

NVL Sensor Fusion Test Bed. March 18, 2004. Introduction. US Army Night Vision & Electronic Sensors Directorate (NVESD) Network of acoustic and image sensors Visible and IR Classification of civilian targets. Motivation & Background. Military targets ~98%

nami
Download Presentation

NVL Sensor Fusion Test Bed

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NVL Sensor Fusion Test Bed March 18, 2004

  2. Introduction • US Army Night Vision & Electronic Sensors Directorate (NVESD) • Network of acoustic and image sensors • Visible and IR • Classification of civilian targets

  3. Motivation & Background • Military targets ~98% • 3 levels of fusion process, acoustic and seismic data • Current civilian classification ~80% • Improve accuracy rating • Both civilian targets and personnel classification • Image data

  4. Objectives • Generation of data set of image and acoustic data • Development and fusion of moving target ATR algorithms • Establish methods to collect data and its “ground truth”

  5. SFTB Setup SFTB Base Station Wireless Ethernet connection Commands through DOS Scripts Node 1 IR sensor Node 2  Visual sensor Met station & GPS Node 1  Acoustic sensor Node 2  Acoustic sensor GPS Node 1  Acoustic sensor Node 2  Acoustic sensor Node 3  IR sensor Node 3  Acoustic sensor GPS Node 3  Acoustic sensor GPS

  6. Sensors • Indigo Alpha Thermal camera • Pulnix TMC-7DSP Color camera • Knowles BL-1994 Microphone

  7. General Test Conditions (1) • 3 nodes each with hexagonal acoustical array of 7 microphones and imaging sensor • Nodes 1 & 3 have uncooled IR camera • Node 2 has visible color camera • Nodes gather information simultaneously for 3 minutes • Acoustic sensor turns on imaging sensors • MUSIC algorithm for DOA estimation

  8. General Test Conditions (2) • Targets moving on gravel & asphalt roads • Fully exposed • Trees or other vehicles occasionally in the way • License plates on the targets are not readable • Stationary sensors • Daylight operation (9:30am to 3:30pm)

  9. General Test Conditions (3) • Target motion • Constant speed • Stops midway • Constant acceleration, deceleration • Stops for count of ten • Each target traverses at 5, 10, 15, 20 mph • Start and stop outside FoV of nodes • Creation of different scenarios

  10. SFTB Operation (1) • Attended mode • Short term data collection / Demo mode • Collects 4 types of data • Surveillance, directed, pan scanning • SFTB_Base.exe, FullSim.exe • Data collection mode • Pure data collection • Collects 4 types of data • Acquire.exe (Video and acoustic), MetEffects.exe

  11. SFTB Operation (2) • Collected data • Acoustic  .dat • Image  .arf • Ground Truth  .agt • Filenames depends on sensor, node, scenario and targets

  12. Numbering System • SSSN00000_0000 • SSS = camera name • IN1 = Indigo IR camera 1 • IN2 = Indigo IR camera 2 • PX1 = Polinex Visible camera 1 • AC1 = Acoustic number 1 • N = node number (1-3) • 00000 = scenario number • _0000 = number indicating vehicle number (1-7) • Can have multiple numbers  multiple targets

  13. Number System Example • in1200004_0003 • In1  indigo thermal camera #1 • 2  node 2 • 00004  scenario number 4 • _0003  target 3 • ac1300001_0056 • ac1  acoustic array #1 • 3  node 3 • 00001  scenario number 1 • _0056  both targets 5 and 6

  14. AGT Format • Very similar to a class in a high level programming language • Agt{ PrjSect {…} SenSect { SenUpd {…} } TgtSec { TgtUpd {…} }}

  15. AGT format

  16. PrjSect • Name = “sftb” • Scenario = “00001_0001” • Site = “nvis0306”

  17. SenSect • Denotes the sensor section of the AGT • Contains all of the SenUpd • Name = “in11” • Denotes which sensor being used

  18. SenUpd • LatLong • Elevation • Keyword “Frame #1” • Keyword “AcousticAz: 0” • Keyword “Nodeld: 1” • Azimuth 75.9077301025 • Time 2003 160 16 22 34 587 • Fov 40.0 30.0

  19. TgtSect & TgtUpd • TgtSect is the sector that contains all the target updates • TgtUpd • Keyword “Frame: #1” • Time 2003 160 16 22 34 587 • Tgt • Range 48.0 • TgtType “MAN” • Aspect 46.0 • PixLoc 40 63

  20. ARF Info • Automatic target recognition working group Raster Format • Contains header, sub headers, footers, 1 or more images • Supports multiple frames • Supports 16 different image types in same file

  21. ARF Info • Rows, cols, version, type, # frames, offset… • Colormap, comments • Image file

  22. ImageJ • NVL used a plugin for converting image from .arf to other formats • Image processing and analysis in java • Formats – dicom, pgm, jpg, bmp, tiff, raw… • Operations – FFT, convolution, fractal box count, morphological…

  23. Acoustic Data • Raw data in acoustic .dat file • Contains header information for system time (similar to AGT), node #, longitude/latitude (0’s), and bearing (0’s) • make_wave.py to convert from .dat to .wav • Changes to specify output file needed within the python script to make it work properly • Script drops .dat extension and adds .wav

  24. Targets • Honda CRX (target 1) • Chevy Cavalier (target 2)

  25. Targets • Toyota pickup (target 3) • GMC pickup (target 4)

  26. Targets • Vehicle (?) (target 5) • Names obtained from AGT files that would eventually contain a TgtType indicating the target • Toyota 4Runner (target 6) • Stake body light truck (target 7) • Dave Rankins

  27. Example Data • Convert arf files into raw using ImageJ • Modify raw image into PGM • Switch endianness • Apply image processing techniques to the image • Very hard to distinguish objects due to having a dark image

  28. Example Sound Clips • Normalized data using shareware program • Target 1 • Node 1 • Node 2 • Node 3 • Target 6 • Node 1 • Node 2 • Node 3

  29. Example Images (1) • Sensor placement • FoV of a sensor covers atleast half of total FoV

  30. Example Images (2)

  31. Example Images (3)

  32. Example Images (4)

  33. Future Work • MatLab acoustical analysis • Segmentation & shape analysis • Feature selection & extraction • Fusion • Target Recognition algorithms

  34. Questions?

More Related