Theory of Algorithms: Brute Force

1. Theory of Algorithms:Brute Force James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004

2. Objectives • To introduce the brute force mind set • To show a variety of brute-force solutions: • Brute-Force String Matching • Polynomial Evaluation • Closest-Pair and Convex-Hull by Brute Force • Exhaustive Search • To discuss the strengths and weaknesses of a brute force strategy

3. Brute Force • A straightforward approach usually directly based on problem statement and definitions • Motto: Just do it! • Crude but often effective • Examples already encountered: • Computing an (a > 0, n a nonnegative integer) by multiplying a together n times • Computation of n! using recursion • Multiplying two n by n matrices • Selection sort

4. Brute Force String Matching • Problem: Find a substring in some text that matches a pattern • Pattern: a string of m characters to search for • Text: a (long) string of n characters to search in • Algorithm: • Align pattern at beginning of text • Moving left to right, compare each character of pattern to the corresponding character in text UNTIL • All characters are found to match (successful search); or • A mismatch is detected • WHILE pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat step 2.

5. Understanding String Matching • Example: • Pattern: AKA • Text: ABRAKADABRA • Trace: AKA • AKA • AKA • AKA • Number of Comparisons: • In the worst case, m comparisons before shifting, for each of n-m+1 tries • Efficiency:(nm)

6. Brute Force Polynomial Evaluation • Problem: Find the value of polynomial p(x) = anxn+ an-1xn-1 +… + a1x1 + a0 at a point x = x0 • Algorithm: • Efficiency? • p 0.0 • for in down to 0 do • power  1 • for j 1 to i do • power  power * x • p p + a[i] * power • return p

7. Brute Force Closest Pair • Problem: • Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), …, Pn = (xn, yn) • Using Cartesian coordinates and Euclidean distance • Algorithm: • Efficiency:(n2) • dmin ∞ • for i1 to n-1 do • for j  i+1 to n do • d  sqrt((xi - xj)2 + (yi - yj)2) • if d < dmin • dmin  d; index1  i; index2  j • return index1, index2

8. The Convex Hull Problem • Problem: Find the convex hull enclosing n 2-D points • Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S • Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set Convex Non-Convex

9. Brute Force Convex Hull P4 P9 • Algorithm: • For each pair of points p1 and p2 • Determine whether all other points lie to the same side of the straight line through p1 and p2 • Efficiency: • for n(n-1)/2 point pairs, check sidedness of (n-2) others • O(n3) P2 P8 P7 P3 P5 P1 P6

10. Pros and Cons of Brute Force • Strengths: • Wide applicability • Simplicity • Yields reasonable algorithms for some important problems and standard algorithms for simple computational tasks • A good yardstick for better algorithms • Sometimes doing better is not worth the bother • Weaknesses: • Rarely produces efficient algorithms • Some brute force algorithms infeasibly slow • Not as creative as some other design techniques

11. Exhaustive Search • Definition: • A brute force solution to the search for an element with a special property, • Usually among combinatorial objects such a permutations, combinations, or subsets of a set • Method: • Construct a way of listing all potential solutions to the problem in a systematic manner • All solutions are eventually listed • No solution is repeated • Evaluate solutions one by one (disqualifying infeasible ones) keeping track of the best one found so far • When search ends, announce the winner

12. Travelling Salesman Problem • Problem: • Given n cities with known distances between each pair • Find the shortest tour that passes through all the cities exactly once before returning to the starting city • Alternatively: • Find shortest Hamiltonian Circuit in a weighted connected graph • Example: 2 a b 5 3 4 8 c d 7

13. Travelling Salesman by Exhaustive Search Tour Cost . abcda 2+3+7+5 = 17 abdca 2+4+7+8 = 21 acbda 8+3+4+5 = 20 acdba 8+7+4+2 = 21 adbca 5+4+3+8 = 20 adcba 5+7+3+2 = 17 • Improvements: • Start and end at one particular city • Remove tours that differ only in direction • Efficiency: (n-1)!/2 = (n!)

14. Knapsack Problem • Problem: Given n items • weights: w1 w2 … wn • values: v1 v2 … vn • a knapsack of capacity W • Find the most valuable subset of the items that fit into the knapsack • Example (W = 16):

15. Knapsack by Exhaustive Search • Efficiency:Ω(2n)

16. Generating Combinatorial Objects: Subsets • Combinatorics uses Decrease (by one) and Conquer Algorithms • Subsets: generate all 2n subsets of A = {a1, …, an} • Divide into subsets of {a1, …, an-1} that contain an and those that don’t • Sneaky Solution: establish a correspondence between bit strings and subsets. Bit n denotes presence (1) or absence (0) of element an • Generate numbers from 0 to 2n-1 convert to bit strings  interpret as subsets • Examples: 000 = Ø , 010 = {a2} , 110 = {a1, a2}

17. Generating Combinatorial Objects: Permutations • Permutations: generate all n! reorderings of {1, …, n} • Generate all (n-1)! permutations of {1, …, n-1} • Insert n into each possible position (starting from the right or left, alternately) • Implemented by the Johnson-Trotter algorithm • Satisfies Minimal-Change requirement • Next permutation obtained by swapping two elements of previous • Useful for updating style algorithms • Example: • Start: 1 • Insert 2: 12 21 • Insert 3: 123 132 312 321 231 213

18. Final Comments on Exhaustive Search • Exhaustive search algorithms run in a realistic amount of time only on very small instances • In many cases there are much better alternatives! • Euler circuits • Shortest paths • Minimum spanning tree • Assignment problem • In some cases exhaustive search (or variation) is the only known solution