1 / 62

A simple example

A simple example. finding the maximum of a set S of n numbers. Time complexity. Time complexity: Calculation of T(n): Assume n = 2 k , T(n) = 2T(n/2)+1 = 2(2T(n/4)+1)+1 = 4T(n/4)+2+1 : =2 k-1 T(2)+2 k-2 + … +4+2+1 =2 k-1 +2 k-2 + … +4+2+1 =2 k -1 = n-1.

nasim-wong
Download Presentation

A simple example

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A simple example • finding the maximum of a set S of n numbers

  2. Time complexity • Time complexity: • Calculation of T(n): Assume n = 2k, T(n) = 2T(n/2)+1 = 2(2T(n/4)+1)+1 = 4T(n/4)+2+1 : =2k-1T(2)+2k-2+…+4+2+1 =2k-1+2k-2+…+4+2+1 =2k-1 = n-1

  3. A general divide-and-conquer algorithm Step 1: If the problem size is small, solve this problem directly; otherwise, split the original problem into 2 sub-problems with equal sizes. Step 2: Recursively solve these 2 sub-problems by applying this algorithm. Step 3: Merge the solutions of the 2 sub- problems into a solution of the original problem.

  4. Time complexity of the general algorithm • Time complexity: where S(n) : time for splitting M(n) : time for merging b : a constant c : a constant • e.g. Binary search • e.g. quick sort • e.g. merge sort

  5. Divide and Conquer • Divide-and-conquer method for algorithm design: • Divide: if the input size is too large to deal with in a straightforward manner, divide the problem into two or more disjoint subproblems • Conquer: use divide and conquer recursively to solve the subproblems • Combine: take the solutions to the subproblems and “merge” these solutions into a solution for the original problem

  6. Recurrences • Running times of algorithms with Recursive calls can be described using recurrences • A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs • Example: Merge Sort

  7. Solving Recurrences • Repeated substitution method • Expanding the recurrence by substitution and noticing patterns • Substitution method • guessing the solutions • verifying the solution by the mathematical induction • Recursion-trees • Master method • templates for different classes of recurrences

  8. Repeated Substitution Method • Let’s find the running time of merge sort (let’s assume that n=2b, for some b).

  9. Repeated Substitution Method • The procedure is straightforward: • Substitute • Expand • Substitute • Expand • … • Observe a pattern and write how your expression looks after the i-th substitution • Find out what the value of i (e.g., lgn) should be to get the base case of the recurrence (say T(1)) • Insert the value of T(1) and the expression of i into your expression

  10. Integer Multiplication • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • We can then define I*J by multiplying the parts and adding: • So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). • But that is no better than the algorithm we learned in grade school.

  11. An Improved Integer Multiplication Algorithm • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • Observe that there is a different way to multiply parts: • So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by the Master Theorem. • Thus, T(n) is O(n1.585).

  12. Matrix multiplication • Let A, B and C be n  n matrices C = AB C(i, j) = A(i, k)B(k, j) • The straightforward method to perform a matrix multiplication requires O(n3) time.

  13. Divide-and-conquer approach • C = AB C11 = A11 B11 + A12 B21 C12 = A11B12 + A12 B22 C21 = A21 B11 + A22 B21 C22 = A21 B12 + A22 B22 • Time complexity: (# of additions : n2) We get T(n) = O(n3)

  14. Strassen’s matrix multiplicaiton • P = (A11 + A22)(B11 + B22) Q = (A21 + A22)B11 R = A11(B12 - B22) S = A22(B21 - B11) T = (A11 + A12)B22 U = (A21 - A11)(B11 + B12) V = (A12 - A22)(B21 + B22). • C11 = P + S - T + V C12 = R + T C21 = Q + S C22 = P + R - Q + U

  15. Time complexity • 7 multiplications and 18 additions or subtractions • Time complexity: T(n) = an2 + 7T(n/2) = an2 + 7(a(n/2)2 + 7T(n/4)) = an2 + (7/4)an2 + 72T(n/4) = … : = an2(1 + 7/4 + (7/4)2+…+(7/4)k-1+7kT(1))  cn2(7/4)log2n+7log2n, c is a constant = cnlog24+log27-log24 +nlog27 = O(nlog27)  O(n2.81)

  16. Median Finding • Given a set of "n" unordered numbers we want to find the "k th" smallest number. (k is an integer between 1 and n).

  17. A Simple Solution • A simple sorting algorithm like heapsort will take Order of O(nlg2n) time. Step Running Time Sort n elements using heapsort O(nlog2n) Return the kth smallest element O(1) Total running time O(nlog2n)

  18. Linear Time selection algorithm • Also called Median Finding Algorithm. • Find k th smallest element in O (n) time in worst case. • Uses Divide and Conquer strategy. • Uses elimination in order to cut down the running time substantially.

  19. Steps to solve the problem • Step 1: If n is small, for example n<6, just sort and return the kth smallest number in constant time i.e; O(1) time. • Step 2: Group the given number in subsets of 5 in O(n) time.

  20. Step3: Sort each of the group in O (n) time. Find median of each group. • Given a set (……..2,5,9,19,24,54,5,87,9,10,44,32,21,13,24,18,26,16,19,25,39,47,56,71,91,61,44,28………) having n elements.

  21. Arrange the numbers in groups of five 2 54 44 4 25 ……………….. ……………….. ……………….. 5 32 18 39 ……………….. 5 ……………….. 47 21 26 9 87 ……………….. 13 16 56 19 9 ……………….. ……………….. 2 19 71 ……………….. 24 10 ………………..

  22. Find median of N/5 groups 2 5 2 4 25 ……………….. ……………….. ……………….. 5 13 16 39 ……………….. 9 ……………….. 9 10 18 47 21 ……………….. 32 19 56 19 54 ……………….. ……………….. 44 26 71 ……………….. 24 87 ……………….. Median of each group

  23. Find the Median of each group 2 5 2 4 25 ……………….. ……………….. 3.n/10 ……………….. 5 13 16 39 ……………….. 9 ……………….. 47 18 21 9 10 ……………….. 32 19 56 19 54 ……………….. ……………….. 44 26 71 ……………….. 24 87 ……………….. Find m ,the median of medians

  24. Find the sets L and R • Compare the n-1 elements with the median m and find two sets L and R such that every element in L is smaller than m and every element in R is greater than m. m L R 3n/10<L<7n/10 3n/10<R<7n/10

  25. Description of the Algorithm step • If n is small, for example n<6, just sort and return the k the smallest number.(time- 7) • If n>5, then partition the numbers into groups of 5.(time n) • Sort the numbers within each group. Select the middle elements (the medians). (time- 7n/5) • Call your "Selection" routine recursively to find the median of n/5 medians and call it m. (time-Tn/5) • Compare all n-1 elements with the median of medians m and determine the sets L and R, where L contains all elements <m, and R contains all elements >m. Clearly, the rank of m is r=|L|+1 (|L| is the size of L) (time- n)

  26. Contd…. • If k=r, then return m • If k<r, then return k th smallest of the set L .(time T7n/10) • If k>r, then return k-r th smallest of the set R.

  27. Recursive formula • T (n)=O(n) + T(n/5) +T(7n/10) We will solve this equation in order to get the complexity. We assume that T (n)< C*n T (n)= a*n + T (n/5) + T (7n/10) <= C*n/5+ C*7*n/10 + a*n <= C*n if 9*C/10 +a <= C Which implies C >= 10*a Hence 10*a*n satisfies the recurrence and so T(n) is O(n)

  28. 2-D ranking finding • Def: Let A = (a1,a2), B = (b1,b2). A dominates B iff a1> b1 and a2 > b2 • Def: Given a set S of n points, the rank of a point x is the number of points dominated by x. D B C A E rank(A)= 0 rank(B) = 1 rank(C) = 1 rank(D) = 3 rank(E) = 0

  29. Straightforward algorithm: compare all pairs of points : O(n2)

  30. Divide-and-conquer 2-D ranking finding Input: A set S of planar points P1,P2,…,Pn Output: The rank of every point in S Step 1: (Split the points along the median line L into A and B.) a. If S contains only one point, return its rank as 0. b. Else, choose a cut line L perpendicular to the x-axis such that n/2 points of S have x-values L (call this set of points A) and the remainder points have x-values L(call this set B). Note that L is a median x-value of this set. Step 2: Find ranks of points in A and ranks of points in B, recursively. Step 3: Sort points in A and B according to their y-values. Scan these points sequentially and determine, for each point in B, the number of points in A whose y-values are less than its y-value. The rank of this point is equal to the rank of this point among points in B, plus the number of points in A whose y-values are less than its y-value.

  31. 2-D maxima finding problem • Def : A point (x1, y1) dominates (x2, y2) if x1 > x2 and y1 > y2. A point is called a maxima if no other point dominates it • Straightforward method : Compare every pair of points. Time complexity: O(n2)

  32. Divide-and-conquer for maxima finding The maximal points of SL and SR

  33. The algorithm: • Input: A set of n planar points. • Output: The maximal points of S. Step 1: If S contains only one point, return it as the maxima. Otherwise, find a line L perpendicular to the X-axis which separates the set of points into two subsets SLand SR , each of which consisting of n/2 points. Step 2: Recursively find the maximal points of SL and SR. Step 3: Find the largest y-value of SR. Project the maximal points of SL onto L. Discard each of the maximal points of SL if its y-value is less than the largest y-value of SR.

  34. Time complexity: T(n) Step 1: O(n) Step 2: 2T(n/2) Step 3: O(n) Assume n = 2k T(n) = O(n log n)

  35. The closest pair problem • Given a set S of n points, find a pair of points which are closest together. • 1-D version : Solved by sorting Time complexity : O(n log n) 2-D version

  36. at most 6 points in area A:

  37. The algorithm: • Input: A set of n planar points. • Output: The distance between two closest points. Step 1: Sort points in S according to their y-values and x-values. Step 2:If S contains only two points, return infinity as their distance. Step 3:Find a median line L perpendicular to the X-axis to divide S into two subsets, with equal sizes, SL and SR. Step 4:Recursively apply Step 2 and Step 3 to solve the closest pair problems of SL and SR. Let dL(dR) denote the distance between the closest pair in SL (SR). Let d = min(dL, dR).

  38. Step 5: For a point P in the half-slab bounded by L-d and L, let its y-value by denoted as yP . For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within yP +d and yP -d. If the distance dbetween P and a point in the other half-slab is less than d, let d=d . The final value of d is the answer. • Time complexity: O(n log n) Step 1: O(n log n) Steps 2~5: T(n) = O(n log n)

  39. concave polygon: convex polygon: The convex hull problem • The convex hull of a set of planar points is the smallest convex polygon containing all of the points.

  40. The divide-and-conquer strategy to solve the problem:

  41. The merging procedure: • Select an interior point p. • There are 3 sequences of points which have increasing polar angles with respect to p. (1) g, h, i, j, k (2) a, b, c, d (3) f, e • Merge these 3 sequences into 1 sequence: g, h, a, b, f, c, e, d, i, j, k. • Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles.

  42. e.g. points b and f need to be deleted. Final result:

  43. Divide-and-conquer for convex hull • Input : A set S of planar points • Output : A convex hull for S Step 1: If S contains no more than five points, use exhaustive searching to find the convex hull and return. Step 2: Find a median line perpendicular to the X-axis which divides S into SL and SR; SL lies to the left of SR. Step 3: Recursively construct convex hulls for SL and SR. Denote these convex hulls by Hull(SL) and Hull(SR) respectively.

  44. Step 4:Apply the merging procedure to merge Hull(SL) and Hull(SR) together to form a convex hull. • Time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

  45. The Fast Fourier Transform

  46. Outline • Polynomial Multiplication Problem • Primitive Roots of Unity • The Discrete Fourier Transform • The FFT Algorithm

  47. Polynomials • Polynomial: • In general,

  48. Polynomial Evaluation • Horner’s Rule: • Given coefficients (a0,a1,a2,…,an-1), defining polynomial • Given x, we can evaluate p(x) in O(n) time using the equation • Eval(A,x): [Where A=(a0,a1,a2,…,an-1)] • If n=1, then return a0 • Else, • Let A’=(a1,a2,…,an-1) [assume this can be done in constant time] • return a0+x*Eval(A’,x)

  49. Polynomial Multiplication • Given coefficients (a0,a1,a2,…,an-1) and (b0,b1,b2,…,bn-1) defining two polynomials, p() and q(), and number x, compute p(x)q(x). • Horner’s rule doesn’t help, since where • A straightforward evaluation would take O(n2) time. The “magical” FFT will do it in O(n log n) time.

More Related