1 / 27

Administrivia, Lecture 4

Administrivia, Lecture 4. HW #2 assigned this weekend, due Thurs Week 4 HWs will be due Thurs of Weeks 2, 4, 6, 7, 9, 10 HW #1 solutions should be posted tonight (TA’s have compiled them)

colette-garner
Download Presentation

Administrivia, Lecture 4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Administrivia, Lecture 4 • HW #2 assigned this weekend, due Thurs Week 4 • HWs will be due Thurs of Weeks 2, 4, 6, 7, 9, 10 • HW #1 solutions should be posted tonight (TA’s have compiled them) • Reading for next week: Chapter 9.2-3 (linear-time selection), Chapter 33.3-4 (convex hull, closest-pair), Chapter 23 (minimum spanning trees)

  2. Divide and Conquer for Sorting (2.3/1.3) • Divide (into two equal parts) • Conquer (solve for each part separately) • Combine separate solutions • Mergesort • Divide into two equal parts • Sort each part using Mergesort (recursion!!!) • Merge two sorted subsequences

  3. Merging Two Subsequences x[1]-x[2]- … - x[K] y[1]-y[2]- … - y[L] if y[i] > x[j]  y[i+1] > x[j] < K+L-1 edges = # (comparisons) = linear time

  4. 179 310 351 652 285 179 179 310 310 351 351 652 652 423 423 423 861 861 861 254 254 254 450 450 450 520 520 520 179 179 310 285 285 285 310 179 310 351 652 351 652 351 423 423 423 423 423 450 861 861 861 861 520 254 254 254 254 652 450 450 450 450 861 520 520 520 520 179 179 179 254 285 285 310 310 310 351 351 351 652 652 652 423 423 423 861 861 861 254 254 254 450 450 450 520 520 520 179 285 285 285 310 351 652 285 254 450 520 423 861 254 450 520 254 450 520 310 179 179 652 652 351 351 423 423 423 861 861 861 254 450 520 179 285 310 179 351 652 351 652 423 861 285 179 652 351 423 254 861 423 450 520 861 254 285 285 254 450 520 310 285 310 310 450 520 285 285 310 310 285 423 861 254 285 310 310 179 351 652 179 179 179 351 652 652 351 351 450 520 652 Merge Sort Execution Example

  5. 1 2 3 4 5 6 7 8 1 3 5 8 2 4 6 7 1 5 3 8 4 7 2 6 5 1 3 8 7 4 6 2 Recursion Tree log n • n comparisons per level • log n levels • total runtime = n log n

  6. Master Method (4. 3) Recurrence T(n) = aT(n/b) + f(n) 1) If for some  > 0 then 2) If then 3) If for some  > 0 and a f(n/b) c f(n) for some c < 1 then

  7. Master Method Examples • Mergesort T(n) = 2T(n/2) + (n) • Strassen (28.2) T(n) = 7T(n/2) + (n2)

  8. Quicksort (7.1-7.2/8.1-8.2) • Sorts in placelike insertion sort, unlike merge sort • Divideinto two parts such that • elements of left part < elements of right part • Conquer: recursively solve for each part separately • Combine: trivial - do not do anything • Quicksort(A,p,r) • if p < r • then q  Partition (A,p,r) • Quicksort (A,p,q) • Quicksort (A,q+1,r) //divide //conquer left //conquer right

  9. Divide = Partition PARTITION(A,p,r) //Partition array from A[p] to A[r] with pivot A[p] //Result: All elements  original A[p] have index  i x = A[p] i = p - 1 j = r + 1 repeat forever repeat j = j - 1 until A[j]  x repeat i = i +1 until A[i]  x if i < j then exchange A[i]  A[j] else return j

  10. 9 9 7 7 6 6 15 15 16 16 5 5 10 10 11 11 i i j j 9 7 6 15 16 5 10 11 5 7 6 15 16 9 10 11 9 7 6 15 16 5 10 11 left left left left left right right right right right i i j 5* 7 6 15 16 j 9* 10 11 9 7 6 15 16 5 10 11 i 5 7 6 15 16 9 10 j 11 9 7 6 15 16 5 10 11 5 5 5 7 7 7 6 6 6 15 15 15 16 16 16 9 9 9 10 10 10 11 11 11 5 5 7 7 6 6 15 15 16 16 9 9 10 10 11 11 5 7 6 15 16 9 10 11 5 5 5 i 7 7 7 6 6 6 15 15 15 16 16 16 9 9 9 10 10 10 11 11 11 j i left right j i i j j i i 5 7 6 j j 15 16 j 9 10 11 left right i i 5 5 5 5 5 9 5 7 7 7 7 7 7 7 6 6 6 6 6 6 6 15 15 15 15 15 15 15 j 16 16 16 16 16 16 16 9 9 9 5 9 9 9 j 10 10 10 10 10 10 10 11 11 11 11 11 11 11 j i i i left j j j right 9 7 6 15 16 5 10 11 left right 9 9 left 7 7 6 6 15 15 16 16 5 5 right 10 10 11 11 i j left left right right i i j i j left right i i j left right 5 7 6 15 16 9 10 11 i i left right left right j j left left left right right right left right 5 6 7 15 16 9 10 11 5 6 7 11* 16 9 10 15* 5 left left 7 6 15 16 right right 9 10 11 5 6 7 11 10 9 16 15 j j 5 5 7 6 6 7 15 15 16 16 9 9 10 10 11 11 i j 5 6 7 11 10 9 16 15 left right 5 6 7 15 16 9 10 11 i 5 j 7 6 15 16 9 10 11 5 6 7 11 10 9 16 15 5 5 5 7 6 6 6 7 7 15 9 11 16 10 16 9 11 9 10 15 10 11 16 15 i 5 5 6 6 7 7 j 9 9 10 10 11 11 16 15 15 16 j i i j i j 5 5 6 6 7 7 15 11 16 16 9 9 10 10 15 11 i j j i j 5 6 7 15 16 9 10 11 i i j j i How It Works

  11. 0123456789 0 123456789 n 89 8 9 Runtime of Quicksort • Worst case: • every time nothing to move • pivot = left (right) end of subarray • O(n2) Recursion Tree of QSort

  12. Runtime of Quicksort • Best case: • every time partition in (almost) equal parts • no worse than in given proportion • O(n log n) • Average case = ?

  13. What is the DQ Recurrence for QSort? • t(n) = Q(n) + 1/nåj = 1 to n(t(j-1) + t(n-j)) pivotspivots equiprobablelengths of subproblems = Q(n) + 2/n åk = 0 to n-1 t(k) • Guess t(n) Î O(n log n) • t(n) £Q(n) + 2/n [åi=2 to n-1 c i log i] £ c n log n – cn/2

  14. Another QSort Analysis (Shift / Cancel) • (1) T(n) = n-1 + 2/n åj = 1 to n-1T(i), n ³ 2, T(1) = 0 • (2) T(n+1) = n+1 - 1 + 2/(n+1) åj = 1 to nT(i) (1)  (3) nT(n) = n(n-1) + 2 åj = 1 to n-1T(i) (2)  (4) (n+1)T(n+1) = (n+1)n + 2 åj = 1 to nT(i) (4) - (3)  (n+1)T(n+1) – nT(n) = 2n + 2T(n)  T(n+1) = (n+2)/(n+1) T(n) + 2n/(n+1)  T(n+1) £ (n+2)/(n+1) T(n) + 2

  15. Shift / Cancel (cont.) Unroll: Hn = 1 + ½ + 1/3 + …+ 1/n = ln n + + O(1/n) g= Euler’s constant = 0.577…  T(n) £ 2(n+1) (ln n + g– 3/2) + O(1)  T(n) Î O(n log n)

  16. Randomized Analysis of QSort • Probability space • Set W of elementary events º experiment outcomes • Family F of subsets of W, called events • Probability measure Pr, real-valued function on members of F • where A ÍW, A Î F  AcÎ F F closed under union, intersection For all A Î F, 0 £ Pr(A) £ 1 Pr(W) = 1 For disjoint events A1, A2, ... Pr(U Ai) = åPr(Ai) • Random variable is a function from elements of W into  • e.g., event (X = x) º set of elements of W for which X assumes the fixed value x • For integer-valued r.v. X, expectation E[X] = åi Pr(X=i)

  17. Randomized Analysis of QSort (cont.) • At each level, we compare each element to the splitter in its partition • Def. A successful pivot p satisfies n/8 < p < 7n/8 • Three facts • (1) E[S Xi] = S E[Xi] linearity of expectation • (2) If Pr(Heads) is q, expected # tosses up to and including first Head is 1/q // family size puzzle • (3) Pr(Ui Ai) £Si Pr(Ai) prob of union £ sum of probs • Given that Pr(successful pivot) = 3/4 , a single element ei participates in at most how many successful partitioning steps? • Ans: log8/7 n

  18. Randomized Analysis of QSort (cont.) • What is number of partition steps expected between kth , (k+1)st successful pivots? • (2) Þ 4/3 steps • (1) Þ 4/3 log8/7 n • Each element expects to participate in O(log n) pivots (comparisons) • Define r.v.’s which give # comparisons for each element • (1) Þ get O(n log n) expected comparisons • So, we have another analysis that gives us the O(n log n) expected complexity of QSort

  19. How Badly Can We Deviate From Expectation? • E.g., what is Pr (ith element sees 20 log n pivots)? • How many successes possible? • Know £ log8/7 n • Pivots are independent (Bernoulli trials) • Pr(success) = 3/4, Pr(failure) = 1/4 • To see 20 log n pivots, need 20 log n - log n failures in 20 log n tries • Can use (3) to bound probability of so many “bad” events, since in general events may not be independent • Can get: Pr(£ 20 n log n) is 1 - O(n-6)

  20. Selection • Best pivot: median • exercise: analyze complexity when bad pivots chosen • too expensive Þ random splitter • This leads to SELECTION: • select (L, k) returns kth - smallest element of L • e.g., k = |L|/2 Þ median • What is an efficient algorithm? • O(n log n)? • sorting • D/Q Recursion: • N.B.: This if RSelect, below • recall pivot from QSort • if i<k, look in right part for (k-i)th smallest • if i>k, look in left part for kth smallest

  21. Randomized Selection • Worst case: • W(n2) as in QSort analysis • Suppose can guarantee “good” pivot • e.g., n/4 £ i £ 3n/4 • Þ subproblem size £ 3n/4 • Let s(n) º time to find good pivot • Þ t(n) £ s(n) + cn + t([3n/4]) • find pivot • pivot, make subproblem • solve subproblem

  22. Randomized Selection • Suppose further: S(n) £ dn for some d; t(n) £ (c+d)n + t([3n/4]) • Claim: t(n) £ kn for some k would follow • Constructive induction or “substitution” • Ind. Hyp.: t(m) £ km for m £ n-1 • Ind. Step: t(n) £ (c+d)n + k(3n/4) = (c+d+3k/4)n £ kn which we want to be equivalent to t(n) £ kn • But this is true if k ³ 4(c+d)

  23. Break • Celebrity Problem: Given n people and a “knows” relation, is there a celebrity? • Notation: Directed graph (and, let’s say that there can be 0, 1 or 2 directed edges between any two vertices (== people)) • What is obvious algorithm? • Test each person’s celebrityhood • Induction • Hyp: Can tell whether there is a celebrity among the first n-1 people • Induction Step: • Have a celebrity among first n-1 people  two queries needed to verify whether known by nth person • No celebrity among first n-1 people  check whether nth person is a celebrity (2(n-1) queries needed) • (Else no celebrity) • (n2) queries

  24. Break (cont.) • Celebrity Problem: Can you do better? • Hint: (n-1) + 2(n-1) queries suffice • Why are we focusing on identifying the celebrity? • Key idea: eliminate a non-celebrity with each query • Kij = 0 j not celebrity • Kij = 1 i not celebrity • (We’ve seen this “complement” idea in DQ-MAXMIN) • Another question: Given the adjacency matrix of an undirected graph, describe a fast algorithm that finds all triangles in the graph. • Q: Why am I asking this now?

  25. D/Q for Arithmetic • Multiplying Large Integers • A = a0 + r1a1 + ... + rn-1an-1, r º radix • “classic” approach ÞQ(n2) work • Can we apply D/Q? • Let n = 2s, r = 10 º radix • AB = xz + 10s(wz + xy) + 102swy • T(n) = 4T(n/2) + q(n) Þ a = 4, b = 2 in Master Method • T(n) ÎQ(n2) • Need to reduce # subproblems, i.e., want a < 4 • Observation: r’ = (w+x)(y+z) = wy + (wz+xy) + xz • r’ ¬ (w+x)(y+z) • p ¬ wy • q ¬ xz • return 102sp + 10s(r’-p-q) + q • T(n) Î O(n log23) = O(n 1.59)

  26. Matrix Multiplication • A = […] , B = […] are n x n matrices • a11, a12, etc are n/2 x n/2 submatrices • M = AB = […] • where m11 = a11b11 + a12b21 etc. • Evaluation requires 8 multiplies, 4 adds • T(n) = 8T(n/2) + O(n) ÎQ(n3) • Strassen: • p1 = (a21 + a22 - a11)(b22 - b12 + b11) • p2 = a11b11 • p3 = a12b21 • p4 = (a11-a21)(b22-b12) • p5 = (a21+a22)(b12 - b11) • p6 = (a12-a21+a11-a22)b22 • p7 = a22(b11+b22-b12-b21)

  27. Strassen’s Matrix Multiplication p1 = (a21 + a22 - a11)(b22 - b12 + b11) p2 = a11b11 p3 = a12b21 p4 = (a11-a21)(b22-b12) p5 = (a21+a22)(b12 - b11) p6 = (a12-a21+a11-a22)b22 p7 = a22(b11+b22-b12-b21) AB11 = p2 + p3 AB12 = p1 + p2 + p5 + p6 AB21 = p1 + p2 + p4 + p7 AB22 = p1 + p2 + p4 + p5 • T(n) ÎQ(n2.81) // 7 multiplies, 24 adds • Can get to 15 adds

More Related