Machine Learning and Optimization For Traffic and Emergency Resource Management.

1. Machine Learning and Optimization For Traffic and Emergency Resource Management. Milos Hauskrecht Department of Computer Science University of Pittsburgh Students: Branislav Kveton, Tomas Singliar UPitt collaborators: Louise Comfort, JS Lin External: Eli Upfal (Brown), Carlos Guestrin (CMU)

2. S-CITI related projects • Modeling multivariate distributions of traffic variables • Optimization of (emergency) resources over unreliable transportation network • Traffic monitoring and traffic incident detection • Optimization of distributed systems with discrete and continuous variables: Traffic light control

3. S-CITI related projects • Modeling multivariate distributions of traffic variables • Optimization of (emergency) resources over unreliable transportation network • Traffic monitoring and traffic incident detection • Optimization of control of distributed systems with discrete and continuous variables: Traffic light control

4. Traffic network • Traffic network systems are • stochastic (things happen at random) • distributed (at many places concurrently) • Modeling and computational challenges • Very complex structure • Involved interactions • High dimensionality PITTSBURGH

5. Challenges • Modeling the behavior of a large stochastic system • Represent relations between traffic variables • Inference (Answer queries about model) • Estimate congestion in unobserved area using limited information • Useful for a variety of optimization tasks • Learning (Discovering the model automatically) • Interaction patterns not known • Expert knowledge difficult to elicit • Use Data Our solutions: probabilistic graphical models, statistical Machine learning methods

6. Road traffic data • We use PennDOT sensor network155 sensors for volume and speed every 5 minutes

7. Models of traffic data • Local interactions • Markov random field • Effects are circular • Solution: Break the cycles

8. The all-independent assumption • Unrealistic!

9. Mixture of trees • A tree structure retains many dependencies but still loses some • Have many trees to represent interactions

10. Latent variable model • A combination of latent factors represent interactions

11. Four projects • Modeling multivariate distributions of traffic variables • Optimization of (emergency) resources over unreliable transportation network • Traffic monitoring and traffic incident detection • Optimization of distributed systems with discrete and continuous variables: Traffic light control

12. Optimizations in unreliable transportation networks • Unreliable network – connections (or nodes) may fail • E.g. traffic congestion, power line failure

13. Optimizations in unreliable transportation networks • Unreliable network – connections (nodes) may fail • more than one connection may go down to

14. Optimizations in unreliable transportation networks • Unreliable network – connections (nodes) may fail • many connections may go down together

15. Optimizations in unreliable transportation networks • Unreliable network – connections (nodes) may fail • parts of the network may become disconnected

16. Optimizations of resources in unreliable transportation networks • Example: emergency system. Emergency vehicles use the network system to get from one location to the other

17. Optimizations of resources in unreliable transportation networks • One failure here won’t prevent us from reaching the target, though the path taken can be longer

18. Optimizations of resources in unreliable transportation networks • Two failures can get the two nodes disconnected

19. Optimizations of resources in unreliable transportation networks • Emergencies can occur at different locations and they can come with different priorities

20. Optimizations of resources in unreliable transportation networks • … considering all possible emergencies, it may be better to change the initial location of the vehicle to get a better coverage

21. Optimizations of resources in unreliable transportation networks • … If emergencies are concurrent and/or some connections are very unreliable it may be better to use two vehicles …

22. Optimizations of resources in unreliable transportation networks • where to place the vehicles and how many of them to achieve the coverage with the best expected cost-benefit tradeoff ? ? ? ? ? ? ? ? ? ?

23. Solving the problem A two stage stochastic program with recourse Problem stages: • Find optimal allocations of resources (em. vehicles) • Match (repeatedly) emergency demands with allocated vehicles after failures occur Curse of dimensionality: many possible failure configurations in the second stage Our solution: Stochastic (MC) approximations (UAI-2001, UAI-2003) Current: adapt to continuous random quantities (congestion rates,traffic flows and their relations)

24. Four projects • Modeling multivariate distributions of traffic variables • Optimization of (emergency) resources over unreliable transportation network • Traffic monitoring and traffic incident detection • Optimization of distributed systems with discrete and continuous variables: Traffic light control

25. Incident detection on dynamic data incident no incident incident

26. Incident detection algorithms • Incidents detected indirectly through caused congestion • State of the art: California 2 algorithm • If OCC(up) – OCC(down) > T1, next step • If [OCC(up) – OCC(down)]/ OCC(up) > T2, next step • If [OCC(up) – OCC(down)]/ OCC(down) > T3, possible accident • If previous condition persists for another time step, sound alarm • Hand-calibrated for the specific section of the road Occupancy spikes Occupancy falls

27. Incident detection algorithms Machine Learning approach (ICML 2006) • Use a set of simple feature detectors and learn the classifier from the data • Improved performance SVM based model California 2

28. Four projects • Modeling multivariate distributions of traffic variables • Optimization of (emergency) resources over unreliable transportation network • Traffic monitoring and traffic incident detection • Optimization of control of distributed systems with discrete and continuous variables: Traffic light control

29. Dynamic traffic management • A set of intersections • A set of connection (roads)in between intersections • Traffic lights regulating the traffic flow on roads • Traffic lights are controlled independently • Objective: coordinate traffic lights to minimize congestions and maximize the throughput

30. Solutions • Problems: • how to model the dynamic behavior of the system • how to optimize the plans • Our solutions (NIPS 03,ICAPS 04, UAI 04, IJCAI 05, ICAPS 06, AAAI 06) • Model: Factored hybrid Markov decision processes • continuous and discrete variables • Optimization: • Hybrid Approximate Linear Programming • optimizations over 30 dimensional continuous state spaces and 25 dimensional action spaces • Goals: hundreds of state and action variables

31. Thank you • Questions