Sensor network based vehicle classification and license plate identification system

1. Jan Frigo, Vinod Kulathumani Ed Rosten, Eric Raby Sean Brennan Sensor network based vehicle classification and license plate identification system

2. Scenario • Facility monitoring • detect suspicious vehicles entering secure area • deployed at key access points / check posts or along length of a road • Vehicle classes • Personal such as car, SUV • Heavy loads such as pickup trucks, • Military vehicles such as ATV, hummers and huge log trucks • Platforms • Mica2 motes • ARM processor stargates

3. Objectives • Classify vehicles with • High reliability • low latency • low energy • Extract license plate image

4. Challenges • Small scale deployment at each access point (< 10 units) • Vehicles last in influence region for a short time (1-2 seconds) • Spectral signature of vehicles changes with time • Resource constrained devices • Vehicles moving at variable speeds • Environment • Temperature • Physical barriers (trees, winding road, environmental) • System Power

5. Seismic Acoutic Node Architecture Network 2 GHz 900 MHz Mica2 Stargate Geophone Microphone

6. Seismic-Acoustic Field Experiment

7. Frequency characteristics of seismic detection • Geophone placed 50 ft away from road to avoid acoustic interference

8. Seismic detection • Sample at 100 Hz • 16 bits samples using MDS320 board • Estimate ‘energy’ of 12-25Hz band • Haar wavelets up to level 2 • Energy = average of coefficients of band 2 • Haar wavelet computed on 128 samples every 10 ms • 10 new samples each round • 118 samples from previous round • Compute variance of the energy on moving window of size 20 • Use variance threshold to detect vehicle

9. Seismic detection performance • Seismic detection triggers acoustic sampling and / or processing • Energy efficient • Person walking 2 ft does not trigger detection • Person thumping feet (running) < 10 ft away triggers detection • Can be isolated using temporal characteristics

10. Acoustic classification • FFT used to obtain spectral characteristics • Fixed point FFT implemented on stargate • Classifier trained using FFTs computed on stargate • Identify best feature vector characteristics to distinguish between vehicle classes • Use Fisher linear discriminant analysis (FLDV) for classification • Pairwise classifier • Select order of classification that maximizes accuracy • Input obtained vectors into stargate for classification

11. Acoustic classification • FFT computed every 1/8 of a second • 512 samples FFT • 12 samples from previous round • 8Hz resolution • Consider frequencies > 64 Hz • Mic response varies at < 60 Hz • Temporal variation in response < 60 Hz (probably due to wind) • Closest 1.5 seconds of data used as training samples

12. Mean FFT coefficients for Vehicle Classes

13. Acoustic classification • Classification order that maximizes accuracy • Presence of vehicle • Hummer vs car and truck • Car vs truck • Presence • Use average ‘energy’ of 200-360 Hz band • Moving window variance (size 20) based detection • 200-360 Hz band less sensitive to high frequency chirp and wind noise

14. Classification using FLDV • Hummer vs car / truck • Feature vector 1: ratio of energies of 80 – 112 Hz and 350 – 500 Hz bands • Feature vector 2: ratio of energies of 250 – 300 Hz and 350-500 Hz bands • Ratios less sensitive to mic response and distance from road • FLDV uses training samples to compute best projection vector

15. Acoustic classification • Integrating output • Approaching vehicle characteristics differ from closest point • Classifier operates for ~10 seconds as the vehicle approaches and passes node • Classifier designed such that • Low probability of car being classified as truck or hummer at any instant • Low probability of truck being classified as hummer at any instant • > 5 consecutive truck classifications within a test run-> vehicle is truck • > 5 consecutive hummer classifications within a test run -> vehicle is hummer

16. License Plate Recognition Classify Pixels Find bounding boxes Extract and resample plates Find and filter regions Send image • Sending only license plates over the network requires very little bandwidth. • Resampling license plate to fixed size reduces network usage when vehicles are close. • Computationally expensive OCR is run on remote host • Algorithms use integer arithmetic only

17. Performance • Vehicle classification • Within 2 seconds of object passing zone • Reliability > 90% • 2.26 watts power consumption [when computing] • Will last about 12 hours if continuously computing on 4.8V 4200mAH cell • License plate recognition • Detection accuracy > 95% • Suppresses 90% of image content • Latency 5.1 seconds [mainly for image capture] • Higher power consumption [can perform 5582 trials on 4.8V 4200maH cell]

18. License plate detection algorithm • Combination of • Viola jones detector [object detection] • Decision tree classifier [license plate segmentation] • Analyse using Haar Wavlet like features • Efficient to compute using an integral image • Integral image is also required for resampling license plate • Computational time is independent of feature size • Train decision tree classifier to recognize license plate pixels • Decision trees are very efficient: • Only integer arithmetic required for evaluation • Tune the tree to rapidly reject most pixels very quickly

19. Ongoing and Future Work • Increase power efficiency • Embedded FPGA implementations for in situ computing • Estimated 5 – 100x power savings and 30 – 100x speed up in run-time performance over COTS • Node Architecture combination of • new ARM processor technology on next generation mezzanine board • Igloo FPGA on sensor board • Low power analog acoustic circuitry • Design of cooperative analog-digital signal processing systems • Upto 300X power savings • Identify optimal balance between nodal computation, in-network processing and central computation