1 / 32

i206: Lecture 7: Analysis of Algorithms, continued

i206: Lecture 7: Analysis of Algorithms, continued. Marti Hearst Spring 2012. Algorithms. A clearly defined procedure for solving a problem. A clearly described procedure is made up of simple clearly defined steps. The procedure can be complex, but the steps must be simple and unambiguous.

tallis
Download Presentation

i206: Lecture 7: Analysis of Algorithms, continued

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. i206: Lecture 7:Analysis of Algorithms, continued Marti Hearst Spring 2012

  2. Algorithms • A clearly defined procedure for solving a problem. • A clearly described procedure is made up of simple clearly defined steps. • The procedure can be complex, but the steps must be simple and unambiguous.

  3. Problem Solving Strategies • Try to work the problem backwards • Reverse-engineer • Once you know it can be done, it is much easier to do • What are some examples? • Stepwise Refinement • Break the problem into several sub-problems • Solve each sub-problem separately • Produces a modular structure • Look for a related problem that has been solved before • Software design patterns

  4. Example of Stepwise RefinementSpiderman • Peter Parker’s goal: Make Mary Jane fall in love with him • To accomplish this goal: • Get a cool car • To accomplish this goal: • Get $3000 • To accomplish this goal: • Appear in a wrestling match … • Each goal requires completing just one subgoal

  5. Example of Stepwise RefinementStar Wars Episode IV • Luke Skywalker’s goal: Make Princess Leia fall in love with him (they weren’t siblings yet) • To accomplish this goal: • Rescue her from the death star • To accomplish this goal: • Land on Death Star • Remove her from her Prison Cell • Disable the Tractor Beam • Get her onto the Millennium Falcon • To accomplish subgoal (2) • Obtain Storm Trooper uniforms • Have Wookie pose as arrested wild animal • Find Location of Cell • Have R2D2 communicate coordinates • To accomplish subgoal (3) • Have last living Jedi walk across catwalks • To accomplish subgoal (4) • Run down hall • Survive in garbage shoot • Fight garbage monster • Have R2D2 stop compaction …

  6. “Divide and Conquer” Algorithms • Break the problem into smaller pieces • Recursively solve the sub-problems • Combine the results into a full solution • If structured properly, this can lead to a much faster solution.

  7. “Divide and Conquer” Algorithms • Problem: Cut up the licorice into 64 pieces!.

  8. “Divide and Conquer” Algorithms • Problem: Cut up the licorice into 64 pieces! • How do we do this in: • O(n) • O(log n) • O(sqrt n)

  9. Santa’s Socks • Problem: • The elves have packed all 1024 boxes with skates • But Santa’s dirty smelly socks fell into one of them. • We have to find the box with the dirty socks! But it’s almost midnight!!!!

  10. Santa’s Socks • http://www.youtube.com/watch?v=wVPCT1VjySA

  11. Analysis of Algorithms • Characterizing the running times of algorithms and data structure operations • Secondarily, characterizing the space usage of algorithms and data structures

  12. Algorithm Complexity • Worst Case Complexity: • the function defined by the maximum number of steps taken on any instance of size n • Best Case Complexity: • the function defined by the minimum number of steps taken on any instance of size n • Average Case Complexity: • the function defined by the average number of steps taken on any instance of size n Adapted from http://www.cs.sunysb.edu/~algorith/lectures-good/node1.html

  13. Best, Worst, and Average Case Complexity Number of steps Worst Case Complexity Average Case Complexity Best Case Complexity N (input size) Adapted from http://www.cs.sunysb.edu/~algorith/lectures-good/node1.html

  14. Monotonicity • mon·o·tone (m n -t n ) • A succession of sounds or words uttered in a single tone of voice • mon·o·ton·ic (m n -t n k) Mathematics. Designating sequences, the successive members of which either consistently increase or decrease but do not oscillate in relative value. • A function is monotonically increasing if whenever a>b then f(a)>=f(b). • A function is strictly increasing if whenever a>b then f(a)>f(b). • Similar for decreasing • Nonmonotonic is the opposite

  15. Monotonic and Nonmonotonic • https://en.wikipedia.org/wiki/Monotonic_function

  16. The Seven Common Functions • Constant: f(n) = c • E.g., adding two numbers, assigning a value to some variable, comparing two numbers, and other basic operations • Useful free web tool: Wolfram’s Alpha

  17. The Seven Common Functions Linear: f(n) = n • Do a single basic operation for each of n elements, e.g., reading n elements into computer memory, comparing a number x to each element of an array of size n

  18. The Seven Common Functions • Quadratic: f(n) = n2 • E.g., nested loops with inner and outer loops Brookshear Figure 5.19

  19. The Seven Common Functions Cubic: f(n) = n3

  20. The Seven Common Functions • More generally, all of the above are polynomial functions: • f(n) = a0 + a1n + a2n2 + a3n3 + … + annd • where the ai’s are constants, called the coefficients of the polynomial. Brookshear Figure 5.19

  21. The Seven Common Functions Logarithmic: f(n) = logbn • Intuition: number of times we can divide n by b until we get a number less than or equal to 1 • The base is omitted for the case of b=2, which is the most common in computing: • log n = log2n Brookshear Figure 5.20

  22. The Seven Common Functions n-log-n function: f(n) = n log n • Product of linear and logarithmic

  23. Comparing Growth Rates • 1 < log n < n1/2 < n < n log n < n2 < n3 < bn http://www.cs.pomona.edu/~marshall/courses/2002/spring/cs50/BigO/ John Chuang

  24. Polynomials Only the dominant terms of a polynomial matter in the long run.  Lower-order terms fade to insignificance as the problem size increases. http://www.cs.pomona.edu/~marshall/courses/2002/spring/cs50/BigO/ John Chuang

  25. Definition of Big-Oh A running time is O(g(n)) if there exist constants n0 > 0 and c > 0 such that for all problem sizes n > n0, the running time for a problem of size n is at most c(g(n)). In other words, c(g(n)) is an upper bound on the running time for sufficiently large n. c g(n) Example: the function f(n)=8n–2 is O(n) The running time of algorithm arrayMax is O(n) http://www.cs.dartmouth.edu/~farid/teaching/cs15/cs5/lectures/0519/0519.html

  26. More formally • Let f(n) and g(n) be functions mapping nonnegative integers to real numbers. • f(n) is (g(n)) if there exist positive constants n0 and c such that for all n>=n0,f(n) <= c*g(n) • Other ways to say this: f(n) is orderg(n) f(n) is big-Oh ofg(n) f(n) is Oh ofg(n) f(n)  O(g(n)) (set notation)

  27. Comparing Running Times Adapted from Goodrich & Tamassia

  28. Analysis Example: Phonebook • Given: • A physical phone book • Organized in alphabetical order • A name you want to look up • An algorithm in which you search through the book sequentially, from first page to last • What is the order of: • The best case running time? • The worst case running time? • The average case running time? • What is: • A better algorithm? • The worst case running time for this algorithm?

  29. Analysis Example (Phonebook) • This better algorithm is called Binary Search • What is its running time? • First you look in the middle of n elements • Then you look in the middle of n/2 = ½*n elements • Then you look in the middle of ½ * ½*n elements • … • Continue until there is only 1 element left • Say you did this m times: ½ * ½ * ½* …*n • Then the number of repetitions is the smallest integer m such that

  30. Analyzing Binary Search • In the worst case, the number of repetitions is the smallest integer m such that • We can rewrite this as follows: Multiply both sides by Take the log of both sides Since m is the worst case time, the algorithm is O(logn)

  31. Hint for Homework Problem 1 • Hint 1: • Write a program that computes the number of gifts for each value of N just to get a feeling for how it works. • Hint 2: • Last time we showed how triangles of dots prove: •     1 + 2 + 3 + ... + n   =   n(n + 1)/2 • Similarly, to add up the number of gifts, a closed form version of the total numbers is: • (n/6)(n+1)(n+2)

  32. Summary: Analysis of Algorithms • A method for determining, in an abstract way, the asymptotic running time of an algorithm • Here asymptotic means as n gets very large • Useful for comparing algorithms • Useful also for determing tractability • Meaning, a way to determine if the problem is intractable (impossible) or not • Exponential time algorithms are usually intractable. • We’ll revisit these ideas throughout the rest of the course.

More Related