Outline

1. Outline • General Design and Problem Solving Strategies • More about Dynamic Programming • Example: Edit Distance • Backtracking (if there is time) • Another Strategy for the Knapsack Problem:

2. Design Strategies • Dynamic Programming Design Strategy • Solve an “easy” sub-problem • Store the solution • Use stored solution to solve a more difficult sub-problem. • Repeat until you solve the “big” hard problem • Other Strategies • Divide and Conquer • Brute Force • Greedy

3. Design Strategies • Dynamic Programming is not divide and conquer. • Consider Floyd’s algorithm • At no point did we break the input into two parts • What is Floyd’s algorithm really doing?

4. Design Strategies • What is Floyd’s algorithm really doing? • STEP 1: Find all the shortest paths allowing a hop through vertex A, store the shortest paths • STEP 2: Now, use that answer to find the shortest paths allowing a hop through vertex B • The algorithm exploits what was already computed, so STEP 2 really finds the shortest paths allowing hops through vertex A and B.

5. More General Design Strategies • Top Down • See the big picture first • Break it into parts • Analyze each part • Continue breaking down sub-parts into solvable tasks • Quicksort is a classic example.

6. More General Design Strategies • Top Down - Quicksort • See the big picture first • Need to put items in the correct sorted position • Break it into parts • Put the pivot in the correct position and partition the list into two parts • Analyze each part • Pick a pivot for each part… • Continue breaking down sub-parts into solvable tasks • Continue recursively until sub-parts are lists of size 1

7. More General Design Strategies • Bottom Up • Use the solution to larger and larger problems to solve the BIG problem and see the big picture • Use solution to small tasks to solve larger problems • Identify easily solvable tasks • Mergesort is a classic example

8. More General Design Strategies • Bottom Up - Mergesort • Use the solution to larger and larger problems to solve the BIG problem and see the big picture • Merging the final two sorted list • Use solution to small tasks to solve larger problems • Merging sorted lists • Identify easily solvable tasks • Sorting lists of size 2

9. More General Design Strategies • Is Floyd’s Algorithm Top Down or Bottom Up?

10. More General Design Strategies • Divide and Conquer can be both Top Down or Bottom Up • Dynamic Programming tends to only be Bottom Up.

11. More General Design Strategies • Consider Bottom-up Strategies • Divide and Conquer usually merges two smaller sub-problems into a large problem • (N/2 + N/2)  N • Dynamic Programming usually extends the solution in some way • N-2  N-1 • N_simple_version  N_harder_version

12. More about Dynamic Programming • How does Floyd’s Algorithm extend the solution? • N-2  N-1 • Does it consider a smaller graph and then extends the solution to a larger graph? • N_simple_version  N_harder_version • Does it consider a simpler shortest path problem and extend it to a more complex shortes path problem?

13. More about Dynamic Programming • In a graph, it is really easy to solve the shortest path problem if you do not allow any hops (intermediate vertices) • The adjacency matrix stores all the shortest paths (direct hop)

14. More about Dynamic Programming • It is also easy to solve the problem if you only allow a hop through vertex x • if (M[a][x] + M[x][b] < M[a][b]) then • update the distance • O(N2) is required to update all the cells • Then, just repeat this process N times; one for each intermediate vertex. O(N3) total time

15. Top Down vs. Bottom Up • Top Down • Rethinking the design of existing ideas/inventions • Managing projects that are underway • Works really good in a Utopian world • Bottom Up • Designing totally new ideas • Putting together projects from scratch • Seen more often in the real world.

16. Bottom-up Design • Top Down • Lets build a flying carriage; what are the parts? • Lift, propulsion, steering, etc. • Lets build a steering mechanism; what are the parts? • We need a steering control • Umm? Wait, we need to know how the other parts work first. • Lets build a lift mechanism; how do we do this? • ??? • Lets build a propulsion mechanism

17. Bottom-up Design • Bottom UP • Discoveries: • This shape produces lift • A spinning propeller creates propulsion in the air • Canvas with a wood frame is light enough • Next Step: Perhaps we can build an stable, controllable flying thing.

18. Bottom-up Design • Before we can analyze the big picture • We have to • Look at some of the initial smaller problems • See how they were solved • See how they led to new discoveries

19. Another Dynamic Programming Algorithm • Problem: • Find The Edit Distance Between Two Strings • Solutions: • Brute Force – O(KN) • Greedy – No Optimal Algorithms yet • Divide & Conquer – None discovered yet • Dynamic Programming – O(N2)

20. Edit Distance • How many edits are needed to exactly match the Target with the Pattern • Target: TCGACGTCA • Pattern: TGACGTGC

21. Edit Distance • How many edits are needed to exactly match the Target with the Pattern • Target: TCGACGT CA • Pattern: T GACGTGC • Three: • By Deleting C and A from the target, and by Deleting G from the Pattern

22. Edit Distance • Applications: • Approximate String Matching • Spell checking • Google – finding similar word variations • DNA sequence comparison • Pattern Recognition

23. T C G A C G T C A 0 1 2 3 4 5 6 7 8 9 T 1 2 1 2 1 2 3 4 3 4 5 5 6 7 6 7 8 G 2 A 3 6 3 4 2 7 5 3 4 6 3 2 5 2 2 3 5 4 3 6 1 5 2 4 3 2 3 2 5 4 1 3 3 2 1 5 2 4 3 3 2 2 1 2 2 3 3 5 4 5 3 4 4 6 3 C 4 G 5 T 6 G 7 C 8 Edit Distance – Dynamic Programming Optimal edit distance forTG and TCG 0 Optimal edit distance for TG and TCGA 1 Optimal edit distance forTGA and TCGA Optimal edit distance forTGA and TCG Final Answer

24. 0 1 2 3 4 5 6 7 8 9 1 0 2 1 1 2 2 3 4 3 5 4 5 6 7 6 8 7 2 1 3 5 6 4 7 2 3 6 3 4 2 5 3 2 3 4 5 2 3 3 5 4 1 2 6 3 2 5 4 1 2 2 3 1 5 3 2 2 2 4 3 3 1 3 2 4 3 2 5 3 6 5 4 3 4 4 5 6 7 8 Edit Distance int matrix[n+1][m+1]; for (x = 0; x <= n; x++) matrix[x][0] = x; for (y = 1; y <= m; y++) matrix [0][y] = y; for (x = 1; x <= n; x++) for (y = 1; y <= m; y++) if (seq1[x] == seq2[y]) matrix[x][y] = matrix[x-1][y-1]; else matrix[x][y] = max(matrix[x][y-1] + 1, matrix[x-1][y] + 1); return matrix[n][m];

25. How many times is this assignment performed? Edit Distance How many times is this assignment performed? How many times is this assignment performed? int matrix[n+1][m+1]; for (x = 0; x <= n; x++) matrix[x][0] = x; for (y = 0; y <= m; y++) matrix [0][y] = y; for (x = 1; x <= n; x++) for (y = 1; y <= m; y++) if (seq1[x] == seq2[y]) matrix[x][y] = matrix[x-1][y-1]; else matrix[x][y] = max(matrix[x][y-1] + 1, matrix[x-1][y] + 1); return matrix[n][m]; How many times is this comparison performed?

26. T C G A C G T C A 0 1 2 3 4 5 6 7 8 9 T 1 2 1 2 1 2 3 4 3 4 5 5 6 7 6 7 8 G 2 A 3 6 3 4 2 7 5 3 4 6 3 2 5 2 2 3 5 4 3 6 1 3 4 2 5 4 1 3 5 2 2 2 2 1 5 3 3 2 2 1 3 4 3 3 2 4 5 2 3 4 4 3 5 3 6 C 4 n=8 G 5 T 6 G 7 C 8 Edit Distance – Dynamic Programming To derive this value 5,we need to know thatwe can match two G’s after already matching two C’s and previously matching two T’s In the worst case,this may take n comparisons 0 1 To derive the value 7,we need to know that we can match two T’s To derive the value 6,we need to know that we can match two C’s after matching two T’s

27. T C G A C G T C A 0 1 2 3 4 5 6 7 8 9 T 1 1 2 2 1 3 2 4 3 5 4 6 5 7 6 7 8 G 2 A 3 2 3 4 6 7 5 5 4 2 3 3 6 5 3 3 4 2 2 3 5 1 6 4 2 3 5 4 2 1 2 3 2 5 2 1 3 1 3 4 2 2 3 2 5 3 2 4 3 4 3 6 4 5 3 C 4 G 5 T 6 G 7 C 8 Edit Distance – Dynamic Programming 0 Luckily, we can match these two C’sBut now we’ve matched the last symbolWe can’t do any more matching (period!) 1 Given our previous matches,there is no way we can match two A’sThus, the edit distance is increased

28. Lesson to learn • There is no way to compute the optimal (minimum) edit distance without considering all possible matching combinations. • The only way to do that is to consider all possible sub-problems. • This is the reason the entire table must be considered. • If you can compute the optimal (minimum) edit distance using less than O(nm) computations. • Then you will be renown!