Problem Space Analysis for Plan Library Generation and Algorithm Selection in Real-time Systems

1. Problem Space Analysis for Plan Library Generation and Algorithm Selection in Real-time Systems Robert H. Holder, III Dissertation Proposal Defense August 26, 2009 Committee: Dr. Tim Finin, Chair Dr. Marie desJardins Dr. Tim Oates Dr. R. Scott Cost

2. -Scott Adams, “Dilbert” comic, June 1, 2008 ``...it was not obvious what you had to do next so you had to think two, six, ten moves ahead. Scenario planning is about having twelve plans, so if one does not work you go to the next. The fun is figuring out a backup for whatever could go wrong.'‘ -Seth Godin, entrepreneur, describing his strategy for success in games and entrepreneurship. From The Red Rubber Ball At Work by Kevin Carroll

3. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary

4. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary

5. Motivation • Real-time Planning Problems • Shipboard Computing Resource Allocation • Mobile Sensor Scheduling • Unmanned Vehicle Routing • Wireless Sensor Network (WSN) Reconfiguration

6. Motivation A plan library is a means of rapidly adapting to a new environment. For planning problems in a real-time environment, Hypothesis: Plan Library: Problem space analysis caninform the efficient creation of a plan library. Problem Space Analysis: A sampling of problem instances and their solutions can lend insight into the underlying structure of the domain.

7. Motivation Additional benefits of problem space analysis

8. Contributions • Framework to model and reason about the topological structure of the solutions of related plans • Techniques to predict and leverage the effect of problem instance characteristics and attributes on the topological structure of the solution space • Novel algorithms that exploit solution space structure to generate plan libraries, select & configure algorithms, and decompose large problems • Evaluation of these algorithms by comparison to competing techniques for a set of sample problems

9. Dynamic Vehicle Routing Problem Three vehicles to visit twelve destinations

10. Dynamic Vehicle Routing Problem Static solution for known destinations

11. Dynamic Vehicle Routing Problem Consideration for potential additional destination

12. Dynamic Vehicle Routing Problem X X X Contingency solution for additional destination

13. Dynamic Vehicle Routing Problem Implementation of contingency solution

14. Dynamic Vehicle Routing Problem How to plan contingencies for arbitrary possibilities?

15. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary

16. Related Work –Technique Comparison • Plan Library generation • case-based reasoning (created after runtime) • reactive planning (subset of input, localized) • conditional & contingency planning (localized) • decision-theoretic (relies on probabilities) • Real-time Planning • plan repair (runtime, works for small changes) • anytime & contract algorithms (runtime)

17. Related Work – Case-Based Reasoning Smyth & McKenna (2001)

18. Related Work – Domain Space Analysis • State Space Analysis • Bulka (2009) – Backbone planning • Kondaris (2008) – Automated skill learning • Hoffman (2001) – State space “benches” and “exits” • Solution Space Analysis • Miner (2009) – Solution gradient lines • Rosen, et. al. (2005) – Medical plan comparison • Gopal & Starkschall (2002) – Medical plan topology • Plan-Space Planning • Trinquart (2003) – Plan space reachability • Hoffman & Nebel (2001) – Uses plan space structure to estimate distance to goal state

19. Related Work – Gopal & Starkschall (2002)

20. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary

21. Approach Overview

22. Traveling Salesperson Problem 4 1 2 3 • Four fixed cities • Central starting point • Unknown fifth city

23. Approach: Problem-Solution Map 4 1 2 3 Each point represents a potential location of the fifth city. The color of the point represents the optimal solution for the resulting 5-city TSP.

24. Approach:Problem-Solution Map 4 0-1-3-2-5-4 0-1-5-4-2-3 0-1-3-2-4-5 1 2 0-1-3-5-2-4 3 0-1-5-3-2-4 0-5-1-3-2-4 0-3-2-4-1-5 0-1-4-2-3-5 Each point represents a potential location of the fifth city. The color of the point represents the optimal solution for the resulting 5-city TSP.

25. Approach • Problem-Solution Map (PS Map) • contiguous regions – only need to store one solution per region • complexity – higher interaction of regions indicates a more complex space

26. Approach: Solution-Problem-Utility Map Optimal library requires 8/120 = 6.7% of possible solutions. Is this too many?

27. Approach: Solution-Problem-Utility Map Tolerating solution degradation and extending the scope of neighboring solutions can reduce library size

28. Approach: Solution-Problem- Utility Map Discrete and continuous SPU Maps showing global competency of one solution

29. Approach • Solution Problem Utility Map (SPU Map) • reduce solution regions (and thus library size) by tolerating utility degradation • gradient of solution degradation

30. Approach:Solution-Similarity Map 0-1-3-2-5-4 0-1-5-4-2-3 0-1-3-2-4-5 0-1-3-5-2-4 0-1-5-3-2-4 0-5-1-3-2-4 0-3-2-4-1-5 0-1-4-2-3-5 high similarity low similarity

31. Approach • Solution Similarity Map (SS Map) • reduce library size by • relying on run-time adaptation • creating parameterized solutions • suggests regions where regular world assumption does not hold

32. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary

33. Algorithm Selection & Configuration Initial solution – use fast, aggressive heuristics

34. Algorithm Selection & Configuration Initial solution – use more precise heuristics, emphasize exploration over exploitation, use less aggressive hill-climbing Adaptation – if solutions are similar, use hill-climbing, else use genetic algorithm

35. Informed Problem Decomposition • To decompose a DVRP or DTSP, a system can fix one of the unknown city locations. • Choosing cities such that the subproblem yields a more contiguous map will be advantageous for planning. more contiguous less contiguous

36. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Sampling • Interpolation • Domain-Based Hints • Research Directions • Summary

37. Map Generation • 5-city TSP (1 unknown city) • small problem with two degrees of freedom • 12k problem instances • naive solver • fast runtime • 5-city TSP (2 unknown cities) • larger problem with four degrees of freedom • 311k problem instances • naive solver • ~20 minutes runtime

38. Map Generation • Complete map generation is not feasible • (would make algorithm selection irrelevant) • how can we approximate the map efficiently? • Map approximation • sampling • interpolation • domain-based hints • example: fixed city locations lie on regional borders

39. Map Generation Initial view of solution space topology

40. Map Generation Ideal view of solution space topology

41. Map Generation Preliminary experiment with random sampling and nearest neighbor classification

42. Map Generation Ideal PS Map Approximated PS Map Preliminary experiment with random sampling and nearest neighbor classification

43. Map Generation Preliminary experiment with random sampling and nearest neighbor classification demonstrates 75% accuracy from a 0.5% sample rate.

44. Sampling & Classification • Sampling • uniform/random/NOLHS/Rapidly expanding Random Trees (RRT) • strategic sampling (active learning) • schemes biased by domain hints • Classification • Nearest neighbor • k nearest neighbors vs. radius of nearest neighbors • weighting neighbors by distance • Support Vector Machine (linear, non-linear) • Bayesian Network • Neural Network

45. Domain-Based Hints All the fixed points lie on regional boundaries

46. Domain-Based Hints All the fixed points lie on regional boundaries

47. SPU and SS Map Generation • SPU Map • calculate utility of solution for sample of problem instances • regression to find limits of solution competence • does optimal region shape inform tolerated region shape? • SS Map • for each solution, find similarity to each neighbor • look for similarities to non-neighbors? • algorithm selection - can we characterize how quickly solution is changing?

48. Regression • Regression depends on a function form, i.e. a “kernel” • Can we determine the appropriate kernel based on problem characteristics? • Piecemeal regression may allow local customization of regression • Support Vector Regression Machines (Drucker, 1996)

49. Approach Overview

50. Outline • Motivation • Related Work • Approach • Problem-Solution (PS) Map • Solution-Problem-Utility (SPU) Map • Solution Similarity (SS) Map • Map Utilization • Plan Library Generation • Algorithm Selection and Configuration • Informed Problem Decomposition • Map Generation • Domain-Based Hints • Sampling • Interpolation • Research Directions • Summary