Combinatorial Algorithms

1. Combinatorial Algorithms

2. Jessica INtroduction

3. What are Combinatorial Algorithms? • Algorithms to investigate combinatorial structures • Generation: Construct all the combinatorial structures of a particular type • Ex: subsets, permutations, partitions, trees, Catalan families • Enumeration: Compute the number of different structures of a particular type • Ex: find number of k-subsets of an n element set • Search: Find at least one example of a structure of particular type (if it exists) • Ex: find a clique of a specified size in a given graph

4. Optimization Problems • Suppose we are trying to solve an optimization problem • There exist backtracking solutions for this reason

5. Backtracking Algorithms • Generate all possible solutions in a certain lexicographic order • Advantages: • Useful for finding one optimal solution and for counting or enumerating all optimal solutions • Disadvantages: • May be inefficient for finding just one solution • For optimal verification, may need to examine large part of the state space tree, even with pruning • Sometimes need to “back up” several levels when partial solution cannot be extended

6. Heuristic Algorithm • Definition: A heuristic algorithm tries to find a certain combinatorial structure or solve an optimization problem by the use of heuristics. • Heuristic: “serving or helping to find out or discover; proceeding by trial and error” • Types of problems can be applied to: • Find 1 optimal solution • Find “close to” optimal solution

7. Heuristic Characteristics and Methods Characteristics Methods Hill-climbing Simulated annealing Tabu search Genetic algorithms • The state space is not fully explored • Randomization is often employed • There is a concept of neighborhood search

8. General framework for heuristic search Generic Optimization Problem (maximization): Instance: A finite set X An objective function P : X  Z m feasibility functions g : Xj Z, 1 ≤ j ≤ m Find: the maximum value of P(X) subject to Xj X and g (X) ≥ 0, for 1 ≤ j ≤ m

9. Designing a heuristic search • Define a neighborhood functionN : X  2X • Ex: N(X) = {X1 , X2 , X3 , X4 , X5 } • Design a neighborhood search: • Algorithm that finds a feasible solution on the neighborhood of a feasible solution X. • 2 types of neighborhood searches: • Exhaustive (chooses best profit among neighbor points) • Randomized (picks a random point among the neighbor points)

10. Defining a neighborhood function N : X  2X So, N(X) is a subset of X. • N(X) should contain elements that are similar or “close to” X. • N(X) may contain infeasible elements of X. • To be useful, want to get to Xopt from X0 via a number of applications of N(▪) • ie. Graph G with V(G) = X and E(G) = {{X, Y} : Y N(X)} should ideally be connected, or at least have one optimal solution in each of its connected components • Computing N(X) should be fast, and in particular |N(X)| shouldn’t be too large

11. Sumanaruban, Yogesh and Lahiru Hill Climbing Algorithm

12. Hill-Climbing • Idea: Go up the hill continuously, stop when stuck.

13. Hill Climbing Algorithm • Pick a random point in the search space • Consider all the neighbors of the current state • Choose the neighbor with the best quality and move to that state • Repeat 2 thru 4 until all the neighboring states are of lower quality • Return the current state as the solution state

14. Problem: it can get stuck in a local optimum • Simple (often effective) solution • Multiple random restarts

15. Uniform graph partition • Instance A complete graph on 2n vertices, A cost function, cost : • Find the minimum value of

16. Example • n = 4; • cost(1, 2) = 1, cost(1, 3) = 2, • cost(1, 4) = 5, cost(2, 3) = 0, • cost(2, 4) = 5, cost(3, 4) = 1 1 X2 x1 2 5 0 5 x4 x3 1

17. Only 3 feasible solutions: • X0 = {1, 2}, X1 = {3, 4} • C([X0, X1]) = 12 1 X2 x1 2 5 0 5 x4 x3 1

18. X0 = {1, 4}, X1 = {2, 3} • C([X0, X1]) = 9 1 X2 x1 2 5 0 5 x4 x3 1

19. X0 = {1, 3}, X1 = {2, 4} • C([X0, X1]) = 7 (optimal) 1 X2 x1 2 5 0 5 x4 x3 1

20. Neighbourhood function: exchange x ∈ X0 and y ∈ X1. Algorithm UGP(Cmax) X = [X0, X1] ← SelectRandomPartition c ← 1 while (c ≤ Cmax) do [Y0, Y1] ← Ascend(X) if not fail then {X0 ← Y0; X1 ← Y1; } else return c ← c + 1

21. Algorithm Ascend([X0, X1]) g ← 0 for each i ∈ X0 do for each j ∈ X1 do t ← G[X0,X1](i, j) (gain obtained in exchange) if (t > g) {x ← i; y ← j; g ← t} if (g > 0) then Y0 ← (X0 ∪ {y}) \ {x} Y0 ← (X1 ∪ {x}) \ {y} fail ← false return [Y0, Y1] else {fail ← true; return [X0, X1]}

22. local maximum plateau ridge Image from: http://classes.yale.edu/fractals/CA/GA/Fitness/Fitness.html Other drawbacks • Ridge = sequence of local maxima difficult for greedy algorithms to navigate • Plateau = an area of the state space where the evaluation function is flat.

23. Hill-climbing variations • How do we make hill climbing less greedy? • Stochastic hill-climbing • Random selection among the better neighbors. • The selection probability can vary with the steepness of the uphill move. • What if the neighborhood is too large to enumerate? • First-choice hill-climbing • Randomly generate neighbors, one at a time • If better, take the move • Random-restart hill-climbing • Tries to avoid getting stuck in local maxima.

24. Hill-Climbing Algorithm for Steiner Triple Systems Stinson's Algorithm

25. Steiner Triple System

26. Examples • STS(3), v = 3 (trivial case) • V = {1,2,3} • B = {123}

27. Examples • STS(3), v = 3 (trivial case) • V = {1,2,3} • B = {123} • STS(7), v = 7 • V = {1,2,3,4,5,6,7} • B = {1,2,3}, {1,4,5}, {1,6,7}, {2,4,6}, {2,5,7}, {3,4,7}, {3,5,6}

28. Examples • STS(3), v = 3 (trivial case) • V = {1,2,3} • B = {123} • STS(7), v = 7 • V = {1,2,3,4,5,6,7} • B = {1,2,3}, {1,4,5}, {1,6,7}, {2,4,6}, {2,5,7}, {3,4,7}, {3,5,6} • STS(9), v = 9 • V = {1,2,3,4,5,6,7,8,9} • B = {1,2,3}, {4,5,6}, {7,8,9}, {1,4,7}, {2,5,8}, {3,6,9}, {1,5,9}, {2,6,7}, {3,4,8}, {1,6,8}, {2,4,9}, {3,5,7}.

29. A Graph Theoretic View Another way to look at Steiner Triple Systems Image source - http://mathworld.wolfram.com/SteinerTripleSystem.html

30. A Graph Theoretic View Another way to look at Steiner Triple Systems Consider the complete graph on v vertices, Kv. A decomposition of Kvinto edge disjoint triangles (K3's) is equivalent to a Steiner Triple System. Image source - http://mathworld.wolfram.com/SteinerTripleSystem.html

31. Fano Plane

32. Fano Plane The Fano plane is an STS(7) Steiner triple system. The blocks are the 7 lines, each containing 3 points. Every pair of points belongs to a unique line.

33. Fano Plane The Fano plane is an STS(7) Steiner triple system. The blocks are the 7 lines, each containing 3 points. Every pair of points belongs to a unique line. Has application like factoring integers via quadratic forms

34. Kirkman's Schoolgirl Problem A problem in combinatorics proposed by Rev. Thomas PenyngtonKirkman in 1850 as Query VI in The Lady's and Gentleman's Diary (pg.48). Image source - http://delphiforfun.org/programs/kirkman.htm

35. Kirkman's Schoolgirl Problem A problem in combinatorics proposed by Rev. Thomas PenyngtonKirkman in 1850 as Query VI in The Lady's and Gentleman's Diary (pg.48). Fifteen young ladies in a school walk out three abreast for seven days in succession: it is required to arrange them daily so that no two shall walk twice abreast. Image source - http://delphiforfun.org/programs/kirkman.htm

36. Kirkman Systems The solution to Kirkman’s 15 schoolgirl problem is a resolution of a block design on 15 points with blocks of size 3 Steiner Triple System on 15 points

37. Kirkman Systems The solution to Kirkman’s 15 schoolgirl problem is a resolution of a block design on 15 points with blocks of size 3 Steiner Triple System on 15 points Solution not unique

38. Kirkman Systems The solution to Kirkman’s 15 schoolgirl problem is a resolution of a block design on 15 points with blocks of size 3 Steiner Triple System on 15 points Solution not unique 7 possible solutions for STS(15)

39. 1 If an STS(n) exists, so too does an STS(2n + 1). 2 If an STS(n) exists, so too does an STS(2n + 7). 3 1

40. 1 PROOF: 1.1 r = number of blocks in which any point appear v = total number of points

41. 1 PROOF: 1.1 r = number of blocks in which any point appear v = total number of points • Any point x must appear in some block with each of all other (v - 1) points

42. 1 PROOF: 1.1 r = number of blocks in which any point appear v = total number of points • Any point x must appear in some block with each of all other (v - 1) points • Point x occurs with 2 other points in each of the rblocks it appears in

43. 1 PROOF: 1.1 r = number of blocks in which any point appear v = total number of points • Any point x must appear in some block with each of all other (v - 1) points • Point x occurs with 2 other points in each of the rblocks it appears in • Therefore,

44. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points

45. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points • Let b = |B| be the total number of blocks

46. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points • Let b = |B| be the total number of blocks • We count T, the number of points with their replications appearing on B, in two ways:

47. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points • Let b = |B| be the total number of blocks • We count T, the number of points with their replications appearing on B, in two ways: • T = 3 X b

48. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points • Let b = |B| be the total number of blocks • We count T, the number of points with their replications appearing on B, in two ways: • T = 3 X b • T = v X r (r = (v-1)/2 using lemma 1.1)

49. 1 PROOF: 1.2 |B| = number of blocks in set B v = total number of points • Let b = |B| be the total number of blocks • We count T, the number of points with their replications appearing on B, in two ways: • T = 3 X b • T = v X r (r = (v-1)/2 using lemma 1.1) • Therefore,

50. If an STS(n) exists, so too does an STS(2n + 1). 2 If an STS(n) exists, so too does an STS(2n + 7). 3 Reference - http://www.math.mun.ca/~davidm/4341/sts.pdf