Algorithms +

1. Algorithms + L. Grewe

2. Algorithms and Programs • Algorithm: a method or a process followed to solve a problem. • A recipe. • An algorithm takes the input to a problem (function) and transforms it to the output. • A mapping of input to output. • A problem can have many algorithms.

3. Algorithm Properties • An algorithm should (ideally) possess the following properties: • It must be correct. • It must be composed of a series of concrete steps. • There can be no ambiguity as to which step will be performed next. • It must be composed of a finite number of steps. • It must terminate. • A computer program is an instance, or concrete representation, for an algorithm in some programming language.

4. Algorithm Efficiency • There are often many approaches (algorithms) to solve a problem. How do we choose between them? • At the heart of computer program design are two (sometimes conflicting) goals. • To design an algorithm that is easy to understand, code, debug. • To design an algorithm that makes efficient use of the computer’s resources.

5. Algorithms • Any problem can have a large number of algorithms that could be used to solve it. • ……but there are some algorithms that are effecitve in solving many problems.

6. Common Algorithms / Algorithm Methods • Greedy Algorithms • Divide and Conquer • Dynamic Programming • Backtracking • Branch and Bound

7. Other (common) Algorithms / Algorithm Methods • Linear Programming • Integer Programming • Neural Networks • Genetic Algorithms • Simulated Annealing • Typically these are covered in application specific courses that use them (e.g. Artificial Intelligence)

8. Algorithms specific to a data structure • Also algorithms can be specifically designed for common operations on a particular data structure. • Example - Graph Algorithms • Graph matching • Find shortest path(s)

9. Defining the Problem • Before even trying to design or reuse an existing algorithm, you must define your problem. • ……many problems can be defined as an optimization problem.

10. Optimization Problem • Problem = Function to X + Constraints • Function to X • This is where you describe the problem as a formula/function. Example, “find the shortest path” can be stated as Sum (distances) is minimum. Here the X is to “minimize”. The Function is the “Sum(distances)”. • Sometimes these are called “COST functions” • Constraints • In our shortest path problem this might be to never visit the same node in the path twice.

11. One Solution to any Problem…..The Brute Force Algorithm • Exponential Time, because exponentially many • This is WHY we discuss algorithms!!!! Try every solution!

12. Optimization Problem Elements • Instances: The possible inputs to the problem. • Solutions for Instance: Each instance has an exponentially large set of solutions. • Cost of Solution: Each solution has an easy to compute cost or value. • Specification • <preCond>: The input is one instance. • <postCond>: An valid solution with optimal cost. (minimum or maximum)

13. Class of algorithms that solve Optimization Problems in a “greedy way” Some greedy algorithms will generate an optimal solution, others only a “good (enough)” solution. Greedy Method = at each point in the algorithm a decision is make that is best at that point. Decisions made are not changed at a later point. Greedy Criterion = criterion used to make a greedy decision at each point. Greedy Algorithms

14. Divide and Conquer • Problem = Set of Several Independent (smaller) sub-problems. • Divide problem into several independent sub-problems and solve each sub-problem. Combine solutions to derive final solution. • Can work well on parallel computers. • Many times the sub-problems are the same problem and need to only develop one algorithm to solve them and then the algorithm to combine the results.

15. Divide and Conquer • Example: • You are given a bag with 16 coins and told one is counterfeit and lighter than the others. Problem = determine if bag contains the counterfeit coin. You have a machine that compares the weight of two sets of coins and tells you which is lighter or if they are the same. • Starting Thoughts: • You could start with comparing coin 1 and 2. If one is lighter you are done. You can then compare coin 3 and 4 and so on. If you have N coins this can take N/2 times. • The Divide and Conquer Way: • If you have N coins divide into two N/2 groups. Weigh them. If one is lighter then we are done and the bag does contain a counterfeit coin. If they are the same, there is no counterfeit coin. This takes ONLY 1 operation. • What happens if you want to find the counterfeit coin?

16. Dynamic Programming • dynamic programming is a method of solving complex problems by breaking them down into simpler steps. • Bottom-up dynamic programming simply means storing the results of certain calculations, which are then re-used later because the same calculation is a sub-problem in a larger calculation. • Top-down dynamic programming involves formulating a complex calculation as a recursive series of simpler calculations.

17. More on Dynamic Programming • Some programming languages can automatically memorize the result of a function call with a particular set of arguments • Some languages make it possible portably (e.g. Scheme, Common Lisp or Perl), some need special extensions (e.g. C++, see [2]). Some languages have automatic memoization built in. In any case, this is only possible for a referentially transparent function.

18. Backtracking • Backtracking is a general algorithm for finding all (or some) solutions to some computational problem, that incrementally builds candidates to the solutions, and abandons each partial candidate c ("backtracks") as soon as it determines that c cannot possibly be completed to a valid solution • problems which admit the concept of a "partial candidate solution“ • and a relatively quick test of whether it can possibly be completed to a valid solution.

19. Branch and Bound • Divides a problem to be solved into a number of subproblems, similar to the strategy backtracking. • Systematic enumeration of all candidate solutions, where large subsets of fruitless candidates are discarded en masse, by using upper and lower estimated bounds of the quantity being optimized • Efficiency of the method depends strongly on the node-splitting procedure and on the upper and lower bound estimators. All other things being equal, it is best to choose a splitting method that provides non-overlapping subsets.