1 / 27

Analysis & Design of Algorithms (CSCE 321)

Analysis & Design of Algorithms (CSCE 321). Prof. Amr Goneid Department of Computer Science, AUC Part 2. Types of Complexities. Types of Complexities. Types of Complexities. Rules for Upper Bound Comparing Complexities Types of Complexities. 1. Rules for Upper Bound.

hedy
Download Presentation

Analysis & Design of Algorithms (CSCE 321)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Analysis & Design of Algorithms(CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 2. Types of Complexities Prof. Amr Goneid, AUC

  2. Types of Complexities Prof. Amr Goneid, AUC

  3. Types of Complexities • Rules for Upper Bound • Comparing Complexities • Types of Complexities Prof. Amr Goneid, AUC

  4. 1. Rules for Upper Bound If (k) is a constant, then: • O(k) O(n) • O(k f(n)) = O(f(n)) O(kn) T(n) O(n) O(k) n Prof. Amr Goneid, AUC

  5. Rules for Big O • O(f(n)) + O(g(n)) = max(O(f(n)) ,O(g(n))) • e.g. f(n) = 2n = O(n), g(n) = 0.1 n3 = O(n3) T(n) = max(O(n) , O(n3)) = O(n3) 2n O(n3) 0.1 n3 Prof. Amr Goneid, AUC

  6. Rules for Big O • O(f(n)) * O(g(n)) = O(f(n) * g(n)) e.g. f(n) = n, g(n) = n2 T(n) = n * n2 = O(n3) Repeat n times O(n2) O(n3) Prof. Amr Goneid, AUC

  7. Rules for Big O • O(log n)  O(n)Prove! Claim: for all n ≥ 1, log n ≤ n. The proof is by induction on n. The claim is trivially true for n = 1, since 0 < 1. Now suppose n ≥ 1 and log n ≤ n. Then, log(n + 1) ≤ log 2n = logn + 1 ≤ n + 1 (by induction hypothesis) Prof. Amr Goneid, AUC

  8. Rules for Big O • For a polynomial of degree m, • Prove! • O(nm-1)  O(nm) follows from above Prof. Amr Goneid, AUC

  9. Rules for Big O Prof. Amr Goneid, AUC

  10. Summary of Rules for Big-O Prof. Amr Goneid, AUC

  11. 2. Comparing Complexities • Dominance: If lim(n->) f(n)/g(n) =  then f(n) dominates (i.e. grows faster), but if lim(n->) f(n)/g(n) = 0 then g(n) dominates. In the latter case, we say that f(n) = o(g(n)) little oh Prof. Amr Goneid, AUC

  12. Comparing Complexities Examples: • if a > b then na dominates nb since lim(n->) nb / na = 0 • n2 dominates (3n+2) since lim(n->) (3n+2) / n2 = 0 • n2 dominates (n log n) since lim(n->) (n log n) / n2 = lim(n->) log n / n = 0 Prof. Amr Goneid, AUC

  13. Using l’Hopital’s Rule (1696) Example: Show that f(n) = n(k+α) + nk (log n)2 and g(n) = k n(k+α) grow at the same rate. Prof. Amr Goneid, AUC

  14. Comparing Complexities • Which grows faster: f(n) = n3 or g(n) = n log n f(n) = n 0.001 or g(n) = log n f(n) = 2n+1 or g(n) = 2n f(n) = 2n or g(n) = 22n Prof. Amr Goneid, AUC

  15. Exercises Which function has smaller complexity ? • f = 100 n4 g = n5 • f = log(log n3) g = log n • f = n2 g = n log n • f = 50 n5 + n2 + n g = n5 • f = en g = n! Prof. Amr Goneid, AUC

  16. 3. Types of Complexities • Constant Complexity • T(n) = constant independent of (n) • Runs in constant amount of time  O(1) • Example: cout << a[0][0] • Polynomial Complexity • T(n)=amnm+…+ a2n2 + a1n1 + a0 • If m=1, then O(a1n+a0)  O(n) • If m > 1, then  O(nm) as nm dominates Prof. Amr Goneid, AUC

  17. Polynomials Complexities O(n3) O(n2) Log T(n) O(n) n Prof. Amr Goneid, AUC

  18. Types of Complexities • Logarithmic Complexity • Log2n=m is equivalent to n=2m • Reduces the problem to half  O(log2n) • Example: Binary Search T(n) = O(log2n) Much faster than Linear Search which has T(n) = O(n) Prof. Amr Goneid, AUC

  19. Linear vs Logarithmic Complexities T(n) O(n) O(log2n) n Prof. Amr Goneid, AUC

  20. Types of Complexities • Exponential • Example: List all the subsets of a set of n elements {a,b,c} {a,b,c}, {a,b},{a,c},{b,c},{a},{b},{c},{} • Number of operations T(n) = O(2n) • Exponential expansion of the problem  O(an) where a is a constant greater than 1 Prof. Amr Goneid, AUC

  21. Exponential Vs Polynomial O(2n) Log T(n) O(n3) O(n) n Prof. Amr Goneid, AUC

  22. Types of Complexities • Factorial time Algorithms • Example: • Traveling salesperson problem (TSP): Find the best route to take in visiting n cities away from home. What are the number of possible routes? For 3 cities: (A,B,C) Prof. Amr Goneid, AUC

  23. Possible routes in a TSP • Number of operations = 3!=6, Hence T(n) = n! • Expansion of the problem  O(n!) Prof. Amr Goneid, AUC

  24. Exponential Vs Factorial O(nn) O(n!) Log T(n) O(2n) n Prof. Amr Goneid, AUC

  25. Execution Time Example • Example: • For the exponential algorithm of listing all subsets of a given set, assume the set size to be of 1024 elements • Number of operations is 21024 about 1.8*10308 • If we can list a subset every nanosecond the process will take 5.7 * 10291 yr!!! Prof. Amr Goneid, AUC

  26. P and NP – Times • P (Polynomial) Times: O(1), O(log n), O(log n)2, O(n), O(n log n), O(n2), O(n3), …. • NP (Non-Polynomial) Times: O(2n) , O(en) , O(n!) , O(nn) , ….. Prof. Amr Goneid, AUC

  27. P and NP – Times • Polynomial time is GOOD Try to reduce the polynomial power • NP (e.g. Exponential) Time is BAD Prof. Amr Goneid, AUC

More Related