Efficient Stochastic Local Search for MPE Solving

1. Efficient Stochastic Local Search for MPE Solving Frank HutterThe University of British Columbia (UBC), Vancouver, Canada Joint work with Holger Hoos (UBC) and Thomas Stützle (Darmstadt University of Technology, Germany)

2. SLS: general algorithmic framework for solving combinatorial problems

3. MPE in graphical models: many applications

4. Outline • Most probable explanation (MPE) problem • Problem definition • Previous work • SLS algorithms for MPE • Illustration • Previous SLS algorithms • Guided Local Search (GLS) in detail • From Guided Local Search to GLS+ • Modifications • Performance gains • Comparison to state-of-the-art

5. 2 1 0 0 1 0 X1 1 X4 X5 X6 8 X1 X2 X3 X4 X5 X6 0 0 0 0 0 0.1 1 21.2 0 0 1 0.9 2 0.9 0 1 0 3.2 0 1 1 100 1 2 3 4 5 6 7 8 1 0 0 0 1 0 1 0 1 1 0 23.2 1 1 1 13.7 MPE - problem definition(in most general representation: factor graphs) • Given a factor graph • Discrete Variables X = {X1, ..., Xn} • Factors  = {1,...,m} over subsets of X • A factor i over variables Vi µ X assigns a non-negative number to every complete instantiation vi of Vi • Find • Complete instantiation {x1,...,xn} maximizing i=1mi[x1,...,xn] • NP-hard (simple reduction from SAT) • Also known as Max-product or Maximum a posteriori (MAP)

6. Previous approaches for solving MPE • Variable elimination / Junction tree • Exponential in the graphical model´s induced width • Approximation with loopy belief propagation and its generalizations [Yedidia, Freeman, Weiss ´02] • Approximation with Mini Buckets (MB) [Dechter & Rish ´97] ! also gives lower & upper bound • Search algorithms • Local Search • Branch and Bound with various MB heuristics [Dechter´s group, ´99 - 05]UAI ´03: B&B with MB heuristic shown to be state-of-the-art

7. Motivation for our work • B&B clearly outperforms best SLS algorithm so far, even on random problem instances [Marinescu, Kask, Dechter, UAI ´03] • MPE is closely related to weighted Max-SAT [Park ´02] • For Max-SAT, SLS is state-of-the-art(at the very least for random problems) • Why is SLS not state-of-the-art for MPE ? • Additional problem structure inside the factors • But for completely random problems ? • SLS algos should be much better than they currently are • We took the best SLS algorithm so far (GLS) and improved it

8. Outline • Most probable explanation (MPE) problem • Problem definition • Previous work • SLS algorithms for MPE • Illustration • Previous SLS algorithms • Guided Local Search (GLS) in detail • From Guided Local Search to GLS+ • Modifications • Performance gains • Comparison to state-of-the-art

9. SLS for MPE – illustration 2 Instantiation: 1 0 0 X1 X2 X3 X4 1 2 3 4 5 i=1Mi[2,1,0,0] = 0.1 * 0.2 * 2.7 * 0.9 * 33.2

10. SLS for MPE – illustration 2 Instantiation: 0 0 1!0 X1 X2 X3 X4 1 2 3 4 5 i=1Mi[2,0,0,0] = i=1Mi[2,1,0,0] * 0.9/0.2 *10/33.2

11. Previous SLS algorithms for MPE • Iterative Conditional Modes [Besag, ´86] • Just greedy hill climbing • Stochastic Simulation • Sampling algorithm, very poor for optimization • Greedy + Stochastic Simulation [Kask & Dechter, ´99] • Outperforms the above & simulated annealing by orders of magnitude • Guided Local Search (GLS) [Park ´02] • (Iterated Local Search (ILS) [Hutter ´04]) • Outperforms Greedy + Stochastic Simulation by orders of magnitude

12. . . . . Guided Local Search (GLS) [Voudouris 1997] • Subclass of Dynamic Local Search[Hoos & Stützle, 2004]:Iteratively:1) Local search ! local optimum2) Modify evaluation function • In local optima: penalize some solution features • Solution features for MPE are partial assigments • Evaluation fct. = Objective fct. - sum of respective penalties • Penalty update rule experimentally designed • Performs very well across many problem classes

13. GLS for MPE [Park 2002] • Initialize penalties to 0 • Evaluation function: • Obj. function - sum of penalties of current instantiation • i=1mi[x1,...,xn] - i=1pi[x1,...,xn] • In local optimum: • Choose partial instantiations (according to GLS update rule) • Increment their penalty by 1 • Every N local optima • Smooth all penalties by multiplying them with  < 1 • Important to eventually optimize the original objective function

14. Outline • Most probable explanation (MPE) problem • Problem definition • Previous work • SLS algorithms for MPE • Illustration • Previous SLS algorithms • Guided Local Search (GLS) in detail • From Guided Local Search to GLS+ • Modifications • Performance gains • Comparison to state-of-the-art

15. GLS ! GLS+:Overview of modified components • Modified evaluation function • Pay more attention to the actual objective function • Improved caching of evaluation function • Straightforward adaption from SAT caching schemes • Tuning of smoothing parameter  • Over two orders of magnitude improvement ! • Initialization with Mini-Buckets instead of random • Was shown to perform better by [Kask & Dechter, 1999]

16. GLS ! GLS+ (1)Modified evaluation function • GLS • i=1mi[x1,...,xn] - i=1pi[x1,...,xn] • Product of entries minus sum of penalties¼ zero minus sum of penaltiesAlmost neglecting objective function • GLS+ • i=1m log(i[x1,...,xn]) - i=1pi[x1,...,xn] • Use logarithmic objective function • Very simple, but much better results • Penalties are now just new temporary factors that decay over time! • Could be improved by dynamic weighting of the penalties

17. GLS+ GLS+ GLS GLS GLS ! GLS+ (1) Modified evaluation function • Much faster in early stages of the search • Speedups of about 1 order of magnitude

18. A A A A GLS ! GLS+ (2)Speedups by caching • Time complexity for a single best-improvement step: • Previously best caching: (|V| £ |DV| £V) • Improved caching: (|Vimproving|£ |DV|)

19.  = 0.99 = 0.999 = 1  = 0.9  = 0.99 & =0.999  = 0.8  = 1  = 0.7 GLS ! GLS+ (3)Tuning the smoothing factor  • [Park ´02] stated GLS to have ``no parameters´´ • Changing  from Park`s setting 0.8 to 0.99 • Sometimes from unsolvable to milliseconds • Effect increases for large instances

20. GLS ! GLS+ (4)Initialization with Mini-Buckets • Sometimes a bit worse, sometimes much better • Particularly helps for some structured instances

21. Outline • Most probable explanation (MPE) problem • Problem definition • Previous work • SLS algorithms for MPE • Illustration • Previous SLS algorithms • Guided Local Search (GLS) in detail • From Guided Local Search to GLS+ • Modifications • Performance gains • Comparison to state-of-the-art

22. Comparison based on [Marinescu, Kask, Dechter, UAI ´03] • Branch & Bound with MB heuristic was state-of-the-art for MPE, even for random instances! • Scales better than original GLS with • Number of variables • Domain size • Both as anytime algorithm and in terms of time needed to find optimum • On the same problem instances, we show that our new GLS+ scales better than their implementation with • Number of variables • Domain size • Density • Induced width

23. Benchmark instances • Randomly generated Bayes nets • Graph structure: completely random/grid networks • Controlled number of variables & domain size • Random networks with controlled induced width • Bayesian networks from Bayes net repository

24. Original GLS vs. B&B with MB heuristic :relative solution quality after 100 secondsfor random grid networks of size NxN A Large Small Medium A

25. GLS+ vs. GLS and B&B with MB heuristic :relative solution quality after 100 secondsfor random grid networks of size NxN Large Small Medium

26. GLS+ vs. B&B with MB heuristic : Solution time with increasing domain size on random networks Large Medium Small

27. Solution times with increasing induced width on random networks d-BBMB s-BBMB Orig GLS GLS+ A

28. Results for Bayes net repository • GLS+ shows overall best performance • Only algorithm to solve Link network (in 1 second!) • Problems for Barley and especially Diabetes • Preprocessing with partial variable elimination helps a lot • Can reduce #(variables) dramatically

29. Conclusions • SLS algorithms are competitive for MPE solving • Scale very well, especially with induced width • But they need careful design, analysis & parameter tuning • SLS and Machine Learning (ML) people should talk • SLS can perform very well for some traditional ML problems • Our C source code is online • Please use it  • There‘s also a Matlab interface

30. Extensions in progress • Real problem domains • MRFs for stereo vision • CRFs for sketch recognition • Domain-dependent extensions • Hierarchical SLS for problems in computer vision • Automated parameter tuning • Use Machine Learning to predict runtime for different settings of algorithm parameters • Use parameter setting with lowest predicted runtime

31. The End • Thanks to • Holger Hoos & Thomas Stützle • Radu Marinescu for their B&B code • You for your attention 