Do We Teach the Right Algorithm Design Techniques ?

1. Do We Teach the Right Algorithm Design Techniques ? AnanyLevitin ACM SIGCSE 1999

2. Outline • Introduction • Four General Design Techniques • A Test of Generality • Further Refinements • Conclusion

3. Introduction-1 • According to many textbooks, a consensus seems to have evolved as to which approaches qualify as major techniques for designing algorithms. • This is includes: divide-and-conquer, greedy approach, dynamic programming, backtracking, and branch-and-bound.

4. Introduction-2 • However, this widely accepted taxonomy has serious shortcoming. • First, it includes techniques of different levels of generality. For example, it seems obvious that divide-and-conquer is more general than greedy approach and branch-and-bound. • Second, it fails to distinguish divide-and-conquer and decrease-and-conquer. • Third, it fails to include brute force and transform-and-conquer.

5. Introduction-3 • Fourth, its linear, as opposed to hierarchical, structure fails to reflect important special cases of techniques. • Finally, it fails to classify many classical algorithms (e.g., Euclid’s algorithm, heapsort, search trees, hashing, etc.) • This paper seeks to rectify these shortcomings and presents new taxonomy.

6. Four General Design Techniques-1 • Brute Force • It usually based on the problem’s statement and definitions of concepts involved • This technique can not be overlooked by the following reasons • Applicable to a very wide variety of problems, e.g., computing the sum of n numbers, adding two matrices ... • Useful for solving small-size instances of a problem • Serving an important theoretical or educational purpose, e.g., NP-hard problem, as a yardstick for more efficient alternatives for solving a problem

7. Four General Design Techniques-2 • Divide-and-conquer • It based on partitioning a problem into a number of smaller subproblems, usually of the same kind and ideally of about the same size • The subproblems are then solved and their solutions combined to get a solution to the original problem. • Divide-before-processing: the bulk of the work is done while combining solutions to smaller subproblems, e.g., mergesort. • Process-before-dividing: processing before a partition into subproblems, e.g., quicksort.

8. Four General Design Techniques-3 • Decrease-and-conquer • This technique is solving a problem by reducing its instance to a smaller one, solving the latter, and then extending the obtained solution to get a solution to the original instance • decrease by a constant: insertion sort • decrease by a constant factor(a.k.a. prune-and-search): binary search • variable size decrease: Euclid’s algorithm

9. Four General Design Techniques-4 • Transform-and-conquer • This technique is based on the idea of transformation • simplification: solves a problem by first transforming its instance to another instance of the same problem which makes the problem easier to solve, e.g., Gaussian elimination, heapsort ... • representation change: it is based on a transformation of a problem’s input to a different representation, e.g., hashing, heapsort …

10. Four General Design Techniques-4 cont. • preprossing: The idea is to process a part of the input or the entire input to get some auxiliary information which speeds up solving the problem, e.g., KMP algorithms. • reduction: An instance of a problem is transformed to an instance of a different problem altogether, e.g, NP-hard problems.

11. A Test of Generality-1 • We partition design techniques into two categories: more general and less general techniques. • How to make such a determinate ? We would like to suggest the following test. In order to qualify for inclusion in the category of most general approaches, a technique must yield reasonable algorithms for the two problems: sorting and searching.

12. A Test of Generality-2 The others - greedy approach, dynamic programming, backtracking, and branch-and-bound - fail to qualify as the most general design techniques.

13. Further Refinements-1 • Local search techniques • Greedy methods: it builds solutions piece by piece … the choice selected is that which produces the largest immediate gain while maintaining feasibility, e.g., Prim’s algorithm. • Iterative methods: it start with any feasible solution and proceed to improve upon the solution by repeated applications of a simple step, e.g., Ford-Fulkerson algorithm.

14. Further Refinements-2 • Dynamic programming • Bottom-up: a table of solutions to subproblems is filled starting with the problem’s smallest subproblems. A solution to the original instance of the problem is then obtained from the table constructed. • Top-down: memory function

15. Further Refinements-3 • State-space-tree techniques • Backtracking: take coloring problem as an example: 1 2 Use three colors to color the vertices of this graph. How many different ways ? 4 3

16. Further Refinements-3 cont. S 1 2 3 V1 V2 1 2 3 1 2 3 V3

17. Further Refinements-3 cont. • Branch-and-Bound: take TSP problem as an example. If there are four cities, all feasible solutions are

18. Further Refinements-3 cont. • Consider the traveling costs between any two cities: =3+3+5+4+1

19. Further Refinements-3 cont. • Start to branch • Choose 12 • Otherwise

20. Further Refinements-3 cont. • The current branch:

21. Further Refinements-3 cont. • The difference between them lies in that backtracking is not limited to optimization problems, while branch-and-bound is not restricted to a specific way of traversing the problem’s space tree.

22. Conclusion • This paper gives a new taxonomy of algorithm design techniques. • No matter how many general design techniques are recognized, there will always be algorithms that cannot be naturally interpreted as an application of those techniques. • Some algorithms can be interpreted as an application of different techniques, e.g., selection sort, as a brute-force algorithm and as a decrease-and-conquer method. • Some algorithms may incorporate ideas of several techniques, e.g. Fourier transform takes advantage of both the transform and divide-and-conquer ideas.