Robust Discrimination of Human vs. Animal Footsteps Using Seismic Signals

1. VG11-079 Robust Discrimination of Human vs.Animal Footsteps Using Seismic Signals SPIE Defense, Security and Sensing 2011 Unattended Ground, Sea, and Air Sensor Technologies and Applications Aram Faghfouri Michael Frish Physical Sciences Inc. E-mail: faghfouri@psicorp.com frish@psicorp.com April 28, 2011 Acknowledgement of Support and Disclaimer This material is based upon work supported by the Department of Homeland Security under Department of Interior Contract Number N10PC20011. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the position or the policy of the Government and no official endorsement should be inferred.

2. Agenda VG11-079 • Motivation • Overview • Intruder Detection Based on Seismic Signals • Seismic Sources and Signals • Signal Acquisition System • Signals • Test Grid Layout • Types of Signals • Technical Approach • Statistical Pattern Recognition • Feature Extraction • Classification • Decision Making • Results • Summary • Acknowledgement

3. Motivation VG11-079 • Secure Border Initiative (SBI) • Program created by the Department of Homeland Security for organizing operating components of border security • Improve the border security through • Detecting potential illegal border crossers • Detecting entries when they occur • Identifying the type of entry (human, vehicle, animal) • Challenges of the southern border • Long length: 1,969 miles • Most frequently crossed international border in the world (350,000,000 crossings/year) • A considerable portion of the border is in the desert with wild and domestic animals • Recent technological response • SBInet • Failed due to unacceptable probability of detection and false alarm

4. Overview:Intruder Detection Based on Seismic Signals VG11-079 • Seismic/acoustic sensors • How they work • Transform mechanical waves into electrical voltage or current (e.g., piezoelectric transducers) • Application • Detect seismic activities caused by different mechanical sources (e.g., humans, vehicles,animals, mechanical tools) • Advantages • Relatively lower prices • Easy to hide • Indirect sensing (do not need line of sight) • Can work individually or as a network • Disadvantages • Relatively short detection range (less than 50 m for footsteps) • It is difficult to discriminate different sources of seismic activities based on acquired signals

5. Seismic Sources and Signals Different foot shapes and gaits generate different seismic waves Lower frequencies of the seismic sources travel farther than higher frequencies Single steps of humans and animals are hard to discriminate A proper discrimination method should be able to extract properties of the seismic waves that can be detected even for longer ranges VG11-079 General technical approach is applicable to diverse signals (e.g., acoustic, seismic.)

6. Signal Acquisition Scenarios

7. Signal Acquisition Signal 8192 Hz, 16 bit resolution, low-pass filtered at 10kHz Location US-Mexico border zone (Arizona) 113 distinct event files, each file several tens of seconds, including: Human 1, 2, or 4 people; 5’ 7” - 6’ 3”; 165 - 215 lb Walking slowly, normally, or running As a group or randomly Over different paths Animal Rat Rabbit Coyote Cow Horse Walks with a person Walks or trots with a rider Vehicle A truck passes; 25 or 35 MPH Background No seismic activity in the scene VG11-079

8. Test Grid Layout VG11-079

9. Paths of the Seismic Sources VG11-079 • Node 1 (blue, left) and Node 2 (red, right) are 25-meters apart • Human targets start 50 meters from closest sensor, end 50 meters on other side of grid. • No people or motion within 100 meters of test grid during event data collection • Vehicle targets start with engines off, 200 meters outside test grid; end with engines off, 200 meters other side of grid.

10. Types of Signals NATURAL Sources Wild animals observed predominately between sunset and sunrise Camera images manually post-processed to provide: event date, start time, end time target class (type of animal or target if possible) proximity of the event to the sensor node(s) path through the test grid movement features/notes (fast, slow, hopping, stomping, digging, etc.). NO-ACTIVITY (Background) During these intervals, no alarms were recorded and no significant activity was noted in real-time inspection of the video. VG11-079

11. Technical Approach

12. Statistical Approach VG11-079 • Statistical pattern recognition • Considerable uncertainty in source, medium, and propagation of the seismic signals makes this problem a proper candidate for statistical pattern recognition • Stages of statistical pattern recognition • Feature extraction • Classification • Investigation of the performance

13. Statistical Pattern Recognition Segment the utilized seismic signal into 1 sec windows For each window, extract signal features and form a feature array Group feature arrays into classes Each class represents a physical phenomenon or pattern Training with known data correlates classes with phenomena VG11-079 Human Vehicle Large Animal Background

14. Feature Extraction VG11-079 • Statistical Features • Standard deviation • Skewness • Kurtosis • … Feature Extraction Feature Extraction Color Color Feature Array Shape Shape Size Size Smell Smell Taste Taste Are these “feature arrays” Similar?

15. Feature Extraction and Feature Arrays VG11-079 • Feature extraction • A form of extracting the most important information from the signal • Each feature is a quantified signal specification • The collection of these feature quantities forms a “feature array” xn Feature Extraction Other Physics-based Features Signal Pattern (Statistical Properties) Spectral Analysis

16. Select training sets Training a Supervised Classifier (1) VG11-079 • Process: • Select a training set • Train a classifier • Test if the classifier can separate classes properly Hyper-plane Defined by the Classifier Train the Classifier Feature Space

17. Training a Supervised Classifier (2) VG11-079 • Selection of training set • A random set of feature arrays is selected. This is called the “training set” • Usually, the size of this set should not exceed 40% of the entire number of samples • The rest of the feature arrays are used for “testing” and “evaluation” of the classifier • We use only 10% of the data for training • Learning phase • The classifier is presented with the feature arrays of the training set and their labels • The classifier adjusts its weights or support vectors (learns) for optimal separation of the feature arrays from one another • Testing phase • The feature arrays that were not used for training are used for testing • The test set is given to the classifier and based on its outcome (class labels) the number of true positive, true negative, false positive and false negative are calculated • A proper classifier is expected to have high true positive and true negative, and low false negative and false positive • Robustness • Repeat the above stages multiple times and investigate if the test results remain within the same ranges over multiple tests (at least 10 times)

18. Classifier: A Specific Type of Neural Networks VG11-079 • Neural networks • How it works • Given different input samples (x), determine weights such that E([f(x)-y]2) is minimized over all training samples • Training • A set of examples of feature arrays are fed as the input • Internal layers of weights are adjusted such that they can output the desired class (or label). • Once the weights are adjusted, the classifier is capable of determining the labels for arbitrary input feature arrays. • Testing • Use the samples that were not used for training and assess the performance of the classifier Weights f(x)

19. Sensor Decision Making VG11-079 • In addition to detection of individual time-segments, their ensemble behavior can be used to enhance the detection rate • It takes some time for the seismic source to pass through the detection area • Multiple consecutive feature arrays similar to one another are generated • The classifier detects a specific source for multiple consecutive times • Confidence level • If the classifier detects different sources, the confidence level for the source S can be defined as(the number of times that S is observed/total number of observations) • Decision maker • The source with the highest confidence level is selected as the “occurred source”

20. Results

21. Output Examples VG11-079 Each colored point represents a one second data segment 28 features were extracted from each segment Trained classifier with 10% of data, tested remaining 90% False positives generally due to moving vehicles False negatives generally due to humans walking with large animals

22. Results VG11-079 • Overall • True Positive: 98.3% • True Negative: 99.3% • False Positive: 0.7% • False Negative: 1.7% • The worst confidence level: 67.3%

23. All Outputs113 Event Files, 17977 Feature Arrays VG11-079 Results: Pd > 98% Pfa < 0.7%

24. Blind Data Set Results VG11-079 • Blind Data Set Classification • Background Events • 22/30 - 73% (All 8 missed backgrounds classified as small animal) • In our classification, small animals, medium animals, and background are clustered in the same group (30/30: 100%) • Scripted • 93% (54/58: 3 humans as animals, 1 vehicle as human) • Natural • 97% (48/49: 1 rabbit classified as human) • Human vs. Non Human • Probability of detection: 93% (54/58) • False alarm: 0.73% (1/137)

25. Summary VG11-079 • Supervised classification can discriminate seismic signals of humans from animals and vehicles very effectively • We achieved and exceeded all of the technical objectives • Achieved Pd>98% (Goal: Pd>90% ) • Achieved Pfa<0.7% (Goal: Pfa<1%) • Performed the testing on the blind data set and achieved • Probability of detection: 93% (54/58) • False alarm: 0.73% (1/137)

26. Acknowledgement VG11-079 • PSI would like to thank Ms. Leslie Shumway and the DHS for helping and supporting this project.