ForeSight : Mapping Vehicles in Visual Domain and Electronic Domain

1. ForeSight: Mapping Vehicles in Visual Domain and Electronic Domain Dong Li, Zhixue Lu, Tarun Bansal, Erik Schilling and PrasunSinha Department of Computer Science and Engineering The Ohio State University

2. Need for Targeted Communication OK, but who are you? What’s in front? Hey you at the back -- Your lights are off! Are you talking to me? I am overtaking you, don’t change lane!

3. Today’s Solutions • Unicast: Hand Gestures, Eye Contact • Requires parties to see each other • Broadcast: Honk, Shout • Disturbs others • Agitates/annoys both parties

4. Tomorrow’s Technology: One Possibility • Broadcast using Smartphone/DSRC • Honking/shouting in the electronic domain • Would cause sensory overload for drivers

5. Fundamental Problem in Targeted V2V Communication Who is the sender/receiver? • Sender: What is the receiver’s unique address? • Receiver: Which vehicle sent message to me?

6. Objective To match vehicles in visual and electronic domains. At the same time • Decrease matching time • Increase accuracy • Generate less network traffic EID: Electronic ID of the vehicle (e.g., IP/MAC address) VID: Visual ID assigned by camera (e.g., red/yellow/blue box)

7. Available Features A unique set of features known to both vehicles is desired.

8. Main Idea • If single feature is unreliable, can we use multiple features to do matching? • System requirement: Camera, GPS Receiver and Radio • Radio: communication Smartphone, DSRC • Camera: identify vehicles Smartphone, Vehicle Security Driving Recorder Camera

9. Challenges • Feature Inaccuracy • E.g., A blue vehicle might be observed as black. • Heterogeneous Capability • Vehicles may not have smartphone, camera, radio, or may not be running our solution. • Distributed in Nature • Each vehicle only knows limited information

10. Vehicle Matching Process

11. Visual Matrix(from Video-Analysis) Vehicles Observed through Camera VID : Visual ID (camera assigns visual IDs to the observed vehicles) VID only has local meaning (cannot be used by neighbors)

12. Electronic Matrix(from Electronic Messages) IDs received through WiFi/DSRC EID: Electronic ID (IP address, MAC address, etc.)

13. Create Similarity Matrix Visual Matrix V Electronic Matrix E S = V  ET Similarity Matrix S

14. Adaptive Weight (AW) Algorithm • The Problem: How to combine different features to get the similarity value between two cars? • The Intuition: Features with diversity values are important. • E.g., color provides no information if the cars have the same color • The Solution: • Define Feature Distinguishability: the probability that any two observed vehicles are different based on this feature • Similarity of two vehicles: weighted mean of the feature distinguishability values.

15. Matching with Similarity Matrix Electronic Domain Visual Domain • Steps • Assign VIDs • Receive EIDs • Calc. Similarity • Remove low similarity links different lane, different color 0.01 Ea V2 0.5 same lane, different color different lane, similar color 0.5 0.99 V1 Eb same lane, similar color me

16. Matching with Similarity Matrix Weighted Bipartite Graph Matching Problem • Greedy Matching • Maximal Matching Ea Ea V2 V2 0.5 0.5 0.5 0.5 V1 Eb Eb V1 0.9 0.9 VIDs EIDs VIDs EIDs Greedy matching is preferred.

17. Clustering Vehicles • Global distinguishability not required • Nearby cars need to be distinguished • Cluster the cars into smaller groups based on feature distance. • Apply the AW algorithm within clusters Clustering

18. Experiment • Driving in freeway & local drive with 3 cars • Using smartphone to collect GPS, video • Experiment result • Vehicles with same color leads to low precision

19. Simulation • Using SUMO + NS3 • Modeled the visibility of neighboring cars • Modeled car detection prob., color detection accuracy, etc. 0.18 0.23 ForeSight significantly improves the matching performance!

20. Case Studies: Improve GPS • Each vehicle estimates its location with • Its own GPS measurement • Neighbors’ estimation of its location (assistance from Nbrs.) • Interesting Observations: • When a car’s GPSerror low, it is more likely to be matched by more neighbors. • The match error increases as the number of neighbors increases: dense traffic makes matching more unreliable. High vehicle density

21. Case Studies: Reduce Disturbance • Application: Send message to vehicles that are in front but has a slower speed • Compare Broadcast, GPS and ForeSight

22. Future Work: Conflict Resolving • Conflicts may Appear • Matching result computed by different vehicles • Matching result at different time • Possible Solution • Collaboration between neighbors E3 E3 E1 E1 Ea Ea Eb Eb E2 E2 E1 E1 EID VID