CS375 – Introduction to Algorithms

1. CS375 – Introduction to Algorithms • Problems • Strategy • Efficiency • Analysis • Order

2. This class - Introduction • Goals • Methods • Examples

3. Goals of the course: Prepare students for: Future technical challenges Using critical thinking for problem solving Implementing algorithms efficiently and correctly Arguing correctness Analyzing time complexity Presenting common algorithms Learning to design using well known methods Comparing algorithms Introduce: The theory of NP-completeness

4. Basic Concepts • Algorithm: - Applying a technique to a problem results in a step-by-step procedure for solving problem. The step-by-step procedure is called an algorithm for the problem. • Example: - Sort a list S of n numbers in non-decreasing order. The answer is the numbers in sorted sequence. - Determine whether the number x is in the list S of n numbers. The answer is yes if x is in S, and no if it is not - solution: Sequential search; Binary search - Add array members - Matrix multiplication

5. Importance of Algorithm Efficiency • Time • Storage • Example - Sequential search versus binary search Basic operation: comparison Number of comparisons is grown in different rate - nth Fibonacci sequence Recursive versus iterative

6. Example: search strategy • Sequential search vs. binary search Problem: determine whether x is in the sorted array S of n keys Inputs: positive integer n, sorted (non-decreasing order) array of keys S indexed from 1 to n, a key x Output: location, the location of x in S (0 if x is not in S)

7. Example: search strategy • Sequential search: Basic operation: comparison Void Seqsearch(int n, const keytype S[], keytype x, index& location) { location=1; while(location<=n && S[location] != x) location++; if(location > n) location = 0; }

8. Example: search strategy • Binary search: Basic operation: comparison Void Binsearch(int n, const keytype S[], keytype x, index& location) { index low, high, mid; low = 1; high =n; location=0; while(low<=high && location ==0) { mid = floor((low+high)/2); if(x==S[mid]) location = mid; else if (x< S[mid]) high = mid -1; else(low = mid +1); } }

9. Example: number of comparisons • Sequential search: n 32 128 1024 1,048,576 • Binary search: lg(n) + 1 6 8 11 21 Eg: S[1],…, S[16],…, S[24], S[28], S[30], S[31], S[32] (1st) (2nd) (3rd) (4th) (5th) (6th)

10. Analysis of Algorithm Complexity • Input size • Basic operation • Time complexity - T(n) : Every-case time complexity - W(n): Worst-case time complexity - A(n): Average-case time complexity - B(n): Best-case time complexity n is the input size • T(n) example - Add array members; Matrix multiplication; Exchange sort T(n) = n-1; n*n*n n(n-1)/2

11. Math preparation • Induction • Logarithm • Sets • Permutation and combination • Limits • Series • Asymptotic growth functions and recurrence • Probability theory

12. Programming preparation • Data structure • C • C++

13. Presenting Commonly used algorithms • Search (sequential, binary) • Sort (mergesort, heapsort, quicksort, etc.) • Traversal algorithms (breadth, depth, etc.) • Shortest path (Floyd, Dijkstra) • Spanning tree (Prim, Kruskal) • Knapsack • Traveling salesman • Bin packing

14. Well known problem • Problem: Given a map of North America, find the best route from New York to Orlando? • Well known problem: what is it? • Many efficient algorithms • Choose appropriate one (e.g., Floyd’s algorithm for shortest paths using dynamic programming)

15. Another well known problem • Problem: You got a job as a paper person. You want to find the shortest tour from your home to every person on your list? • Well known problem: what is it? • One solution to traveling salesperson problem: dynamic programming

16. Another well known problem (continued) • No efficient algorithm to general problem • Many heuristic and approximation algorithms (e.g., greedy heuristic) • Choose appropriate one

17. Another well known problem (continued) • Computer Graphics • Scan conversion • Flood Filling • Ray tracing, …

18. Another well known problem (continued) • Computational Geometry • Find a convex hull • Delaunay triangulation • Voronoi Diagram • Choose an efficient one

19. Design Methods or Strategies • Divide and conquer • Greedy • Dynamic programming • Backtrack • Branch and bound

20. Not addressed (Advanced algorithms) • Genetic algorithms • Neural net algorithms • Algebraic methods

21. The theory of NP completeness • Many common problems are NP-complete ( traveling salesperson, knapsack,...) (NP: non-deterministic polynomial) • Fast algorithms for solving NP-complete problems probably don’t exist • Techniques such as approximation algorithms are used (e.g., minimum spanning tree prim’s algorithm with triangle inequality)

22. Hardware Software Economics Biomedicine Computational geometry (graphics) Decision making ….. Are algorithms useful? “Great algorithms are the poetry of computation”

23. HardwareDesign • VLSI design • Multiplication • Search • Sort networks 3 7 8 6 2 Selection sort for A[1], …, A[n-1] 2 3 6 7 8

24. Software • Text processing • String matching, spell checking, and pretty print,… • Networks • Minimum spanning trees, routing algorithms, … • Data bases • Compilers

25. Text processing – pretty print I want ¶ this page ¶ to look good ¶ I want this page ¶ to look good ¶ Method: Dynamic programming

26. Engineering • Optimization problem (e.g., finite element, energy minimization, dynamic simulation). • Best feature selection for object representation and biometrics recognition (e.g., Genetic Algorithm) • Mathematics: geometric simulation and approximation (e.g., algorithm approximation) • Graphics visualization (e.g., area filling and scan conversion, 3D key framing). • Signal analysis: FFT, Huffman coding…

27. Economics • Transportation problems • Shortest paths, traveling salesman, knapsack, bin packing • Scheduling problems • Location problems (e.g., Voronoi Diagram) • Manufacturing decisions

28. Social decisions • The matching problem • Assigning residents to hospitals • Matching residents to medical program

29. Using critical thinking for problem solving • Considering different approaches for solving a problem (for example dynamic programming and greedy algorithm) • Analyzing the merits of each • Considering different implementations for a chosen approach (for example Prim’s algorithm) • Analyzing the merit of the different implementation