Local search and Optimisation

1. Local search and Optimisation Study of the key local search techniques Introduction: global versus local Concluding Remarks

2. Introduction: Global versus Local search

3. Global versus Local search • Global search: • interest: find a pathto a goal • properties: • search through partial paths • in a systematic way (consider all paths – completeness) • opportunity to detect loops

4. Global versus Local search (2) • Local search: • interest: find a goal or a statethat maximizes/minimizes some objective function. • or a state that maximizes/minimizes some objective function. • 4-queens example: • interest in the solutions • not in the way we find them

5. Global versus Local search (3) • Local search: • interest: find a goal or a statethat maximizes/minimizes some objective function. • rostering: • objective function: estimates quality of roster • optimize the objective function

6. Is the path relevant or not ? The 8-puzzle: path relevant Chess: path relevant Water jugs: path relevant Traveling sales person: path relevant * Symbolic integrals: could be both Blocks planning: path relevant q-queens puzzle: not relevant rostering: not relevant

7. Traveling sales person: • Representation is a partial sequence: (New York, Boston) Global search ! • Representation is a potential solution: (New York, Boston, Miami, SanFran, Dallas, New York) Local search! - the path is encoded in every state - just find a good/optimal state

8. General observations on Local Search: • Applicable if path to solution is not important • but see comment on TSP • Keeps only 1 (or a fixed k) state(s). • k for local beam search and genetic algorithms • Most often, does not systematically investigate all possibilities • and as a result, may be incomplete or suboptimal • Does not avoid loops • unless explicitly designed to (e.g. Tabu search) or included in the state representation • Is often for optimization of an objective function

9. Local Search Algorithms: Hill Climbing (3) (local version) Simulated Annealing Local k-Beam Search Genetic Algorithms Tabu Search Heuristics and Metaheuristics

10. Hill-Climbing (3)or Greedy local search The really “local-search” variant of Hill Climbing

11. Hill Climbing (3) algorithm: • Let h be the objective function State:= S; STOP := False; WHILEnot STOP DONeighbors := successors(State); IFmax(h(Neighbors)) > h(State) State:= maximal_h_neighbor; ElseSTOP:= True; ReturnState Hill Climbing 2, but without paths or, minimization (see 8-queens)

12. Foothills: Plateaus Local maximum Ridges The problems:

13. More properties: • Termination ? If h is bounded (in relevant direction) and there is a minimal step in h-function. • Completeness ? No !

14. Case study: 8-queens n1 n2 n3 n4 n5 n6 n7 n8 h = 17 Minimization ! • h = the number of pairs of queens attacking each other in the state

15. Neighbors of (n1, n2,…., n8): • obtained by changing only 1 ni 8-queens (cont.) h = 1 56 neighbors a local minimum

16. For 8-queens: How well does it work? But how to improve the success rate ?

17. At plateau: allow to move to equal-h neighbor Danger: Non-termination ! Allow only a maximum number of consecutive sideway moves (say: 100) Plateau’s: sideway moves Result: • Success rate: • 94% : success 6 % : local minimum

18. Variants on HC (3) • Stochastic Hill Climbing: Move to a random neighbor with a better h Gets more solutions / but is slower • First-choice Hill Climbing: Move to the first-found neighbor with a better h Useful if there are VERY many neighbors

19. Garanteed completeness:Random-Restart Hill Climbing If HC terminates without producing a solution: Then restart HC with a random new initial state If there are only finitely many states, and If each HC terminates, then: RRCH is complete (with probability 1)

20. Analysis of RRHC: Pure RRHC: If HC has a probability p of reaching success, then we need (average) 1/p restarts in RRHC. For 8-queens: - p= 0.14 1/p ≈ 7 iterations - Cost (average) ? (6 (failures) * 3) + (1 (success) * 4) = 22 steps With sideway moves added: For 8-queens: - p= 0.94 1/p = 1.06 iterations of which (1-p)/p = 0.06/0.94 fail - Cost (average) ? (0.06/0.94 (failures) * 64) + (1 (success) * 21) ≈ 25 steps 1/p-1

21. Conclusion ? • Local Search is replacing more and more other solvers in _many_ domainscontinuously • including optimization problems in ML.

22. Simulated AnnealingKirkpatrick et al. 1983 Simulate the process of annealing from metallurgy

23. 1) HC (3): best moves fast, but stuck in local optima Stochastic HC: random moves slow, but complete Motivations: Combine ! 2) RRHV: resart after failure why wait until failure? include ‘jumps’ during the process ! At high ‘temperature’ : frequent big jumps At low ‘temperature’ : few smaller ones 3) Get ping-pong ball to deepest hole, by rolling ball and shaking surface

24. The algorithm: State:= S; FORTime = 1 to ∞DO Temp:= DecreaseFunction(Time); IFTemp = 0 ThenReturnState; Else Next:= random_neighborg(State); Δh:= h(Next) – h(State); IFΔh > 0 Then State:= Next; Else State:= Next with probability e^(Δh/Temp) End_FOR For slowly deceasing temperature, will reach global optimum (with probablity 1)

25. Local k-Beam Search Beam search, without keeping partial paths

26. Local k-Beam Search • ≠ k parallel HC(3) searches The k new states are the k best of ALL the neighbors Stochastic Beam Search

27. Genetic AlgorithmsHolland 1975 Search inspired by evolution theory

28. General Context • Similar to stochastic k-beam search • keeps track of k States • Different: • generation of new states is “sexual” • In addition has: selection and mutation • States must be represented as strings over some alphabet • e.g. 0/1 bits or decimal numbers Crossover • Objective function is called fitness function

29. State representation: 8-string of numbers [1,8] • Population: set of k states -- here: k=4 8-queens example:

30. Step 1: Selection: • Fitness function applied to population • Probability of being selected: • proportional to fitness • Select: k/2 pairs of states 8-queens (cont.)

31. Step 2: Crossover: • Select random crossover point • here: 3 for pair one, 5 for pair two • Crossover applied to the strings 8-queens (cont.)

32. Step 3: Mutation: • With a small probability: • change a string member to a random value 8-queens (cont.)

33. The algorithm: Given: Fit (a fitness function) Pop:= the set of k initial states; REPEAT New_Pop:= {}; FORi=1,k Do x:= RandomSelect(Pop, Fit); y:= RandomSelect(Pop, Fit); child:= crossover(x,y); IF (small_random_probability) Then child:= mutate(child); New_Pop:= New_Pop U {child}; End_For Pop:= New_Pop; Until a member of Pop fit enough or time is up Different from the example, where both crossovers were used

34. Comments on GA • Very many variants – this is only one instance! • keep part of Pop, different types of crossover, … • What is added value? • If the encoding is well-constructed: substrings may represent useful building blocks Ex: (246*****) is a usefull pattern for 8-queens Ex Circuit design: some substring may represent a useful subcircuit • Then crossover may produce more useful states ! • In general: advantages of GA are not well understood

35. Interpretation of crossover: If we change our representation from 8 decimal numbers to 24 binary digits, how does the interpretation change?

36. Tabu SearchGlover 1986 Another way to get HC out of local minima

37. TabuList: Where you are forbidden to go next. In order to get HC out of a local maximum: Naïve idea: Allow one/some moves downhill Tabu = forbidden Problem: When switching back to HC, it will just move back!

38. Keep a list TabuList with information on which new states are not allowed The Tabu search idea: n3: 2 6 n3 n3 • Add to TabuList: (n3, 6, 2) : don’t make the opposite move or (n3, 2) : don’t place n3 back on 2 or (n3) : don’t move n3

39. Hoped effect: visualized • TabuList determines an area where NOT to move back. • TabuList is kept short: only recent history determines it.

40. The algorithm: Given: Fit (a fitness function) State:= S; Best:= S; TabuList:= {}; WHILE not(StopCondition) Do Candidates:= {}; FOR every Child in Neighbors(State) Do IFnot_forbidden(Child, TabuList) then Candidates:= Candidates U {Child}; Succ:= Maximal_Fit(Candidates); State:= Succ; IfFit(Succ) > Fit(Best) then TabuList:= TabuList U {ExcludeCondition(Succ, Best)}; Best:= Succ; Eliminate_old(TabuList); Return Best;

41. Example PersonelRostering: • Initialise to satisfy the required amounts • Allow only vertical swaps (neighbors) • If a swap has influenced a certain region of the timetable, do not allow any other swap to influence this region for a specified number of moves (Tabu list, Tabu attributes)

42. Heuristics and Meta-heuristics More differences with Global search Examples of problems and heuristics Meta-heuristics

43. In Global search: h: States N Heuristics In Local search: • How to represent a State? • How to define the Neighbors? • How to define the objective or Fit function? Are all heuristic choices that influence the search VERY much.

44. Finding routes: • Given a weighted graph, (V,E), and two vertices, ‘source’ and ‘destination’, find the path from source to destination with the smallest accumulated weight. • Dijkstra: O(|E|2 + |V|) (for sparse graphs O(|E|log|V|))

45. The objective function: In general: many different functions possible OR

46. Stock cutting: X 10000 Out of rectangular sheets with fixed dimensions • NP-complete, even in one dimension (pipe-cutting)

47. Objective function? Minimize waste!

48. Personelrostering • Consists of assignments of employees to working shifts while satisfying all constraints Constraints: • Shifts have start times and end times • Employees have a qualification • A required capacity per shift is given per qualification • Employees can work subject to specific regulations • … • Constraints: • Shifts have start times and end times • Employees have a qualification • A required capacity per shift is given per qualification • Employees can work subject to specific regulations • …

49. Objective function? • Just solve the problem (CP) • Number of constraints violated • Weighted number of constraints violated • Amount under assignment • Amount over assignment • This may lead to the definition of a goal function (representing a lot of domain information)

50. Neighborsfor Rostering • One can easily think of • Swaps • Removals • Insertions • ‘Large Swaps’ • These ‘easy’ options do depend on the domain • They define ‘steps’ in a ‘solution space’ with an associated change in the goal function. • One obvious heuristic is a hill-climber based on a selection of these possible steps.