1 / 14

Dynamic State-Space Partitioning in External-Memory Graph Search

Dynamic State-Space Partitioning in External-Memory Graph Search. Rong Zhou † and Eric A. Hansen ‡ † Palo Alto Research Center ‡ Mississippi State University. External-memory graph search. Internal memory vs. External memory. 1 ~ 4 GB. 160 GB ~ 1.5 TB.

viveca
Download Presentation

Dynamic State-Space Partitioning in External-Memory Graph Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic State-Space Partitioning in External-Memory Graph Search Rong Zhou† and Eric A. Hansen‡ †Palo Alto Research Center ‡Mississippi State University

  2. External-memory graph search Internal memory vs. External memory 1 ~ 4 GB 160 GB ~ 1.5 TB • External memory is cheap and almost inexhaustible • But random access of external memory (e.g., for duplicate detection) is 105 ~ 106 times slower than internal memory

  3. Previous work • Hash-based delayed duplicate detection [Korf & Schultze AAAI-05; Korf JACM-08] • Structured duplicate detection [Zhou & Hansen AAAI-04, 06] • Both use state-space abstraction to … • partition nodes into buckets or disk files • leverage graph local structure to save RAM or disk space

  4. Structured duplicate detection [Zhou & Hansen AAAI-04] • Localizes memory references in duplicate detection by exploiting graph structure revealed by a state-space projection function • Example of projection function ? ? ? ? ? ? ? ? ? ? … ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? … blank pos. = 1 0 15

  5. Abstract state-space graph • Created by state-space projection function • Example B0 B1 B2 B3 2 1 3 B5 B6 B7 B4 5 6 7 4 8 9 10 11 B8 B9 B10 B11 13 14 12 15 B12 B13 B14 B15 > 10 trillion states 16 abstract states

  6. B2 B3 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 B2 B3 B5 B8 B7 B6 B4 B1 … B0 B0 B15 B14 Duplicate-detection scope A set of blocks (of stored nodes) that is guaranteed to contain all stored successor nodes of the currently-expanding node B0 B1 B1 B2 B3 B5 B6 B7 B4 B4 B8 B9 B10 B11 B12 B13 B14 B15

  7. B4 B1 B2 B2 B3 B3 B2 B3 B5 B8 B7 B6 B5 B5 B6 B6 B7 B7 B4 B4 B1 … B0 B0 B0 B15 B14 B8 B8 B9 B9 B10 B10 B11 B11 B12 B12 B13 B13 B14 B14 B15 B15 Edge Partitioning Reduces duplicate-detection scope to one block of stored nodes – Guaranteed! B1 B0 B1 B1 B2 B3 B4 B5 B6 B7 B4 B4 B8 B9 B10 B11 B12 B13 B14 B15

  8. What is a good abstraction? • Capture local structure • DDD: Interleaving expansion and merging • SDD: Fewer incremental expansions • Distribute nodes evenly into buckets • Make sure largest bucket fits in RAM • Not too many buckets • Achieving both is challenging, especially for static abstraction

  9. 2 1 3 5 6 7 4 8 9 10 11 6 4 3 5 13 14 12 15 1 8 7 9 10 11 2 13 14 12 15 A pathological example Degenerative state-space projection function 2 6 4 3 5 1 3 1 8 7 5 6 7 4 9 10 11 8 9 10 11 2 13 14 13 14 12 15 12 15 Start Goal In theory, there are 518,918,400 buckets. But most (> 99.99%) of them are empty!

  10. Greedy abstraction algorithm • Starts with a “blank” abstraction • Mark all state variables as unselected • While ( size of abstract graph  M) • Find an unselected variable Vi s.t. adding it to current abstraction minimizes largest bucket size • Add Vi into set of abstraction variables • Mark Vi as selected • Update current abstraction • Move nodes to their new buckets

  11. Example Nodes 1st Iteration 2nd Iteration

  12. Computational results • Planning results on 15 Puzzle • First planner to optimally solve all 100 of Korf’s 15 Puzzle instances (93 for previous best solver) • <20 MB of RAM for the hardest instance #88 (static partitioning needs >5x RAM for #88) • Uses only Manhattan-Distance heuristic • STRIPS planning (6 domains from IPC) • Peak RAM reduced by up to ~19x • Better time-space tradeoff • Improves with accuracy of heuristic function

  13. Bucket size histogram for instance #88

  14. Conclusion and future work • Not all abstractions are created equal – even for the ones with the same resolution! • Largest bucket depends on starting state • Static abstraction ineffective for heuristic search • Future work • Sampling approach • Parallel search

More Related