1 / 88

CS420 lecture four Recurrence Relations

CS420 lecture four Recurrence Relations. wim bohm, cs.colostate. Complexity. Divide a program in basic blocks and count the order of magnitude number of basic block executions basic block : code fragment that takes constant time to execute

airell
Download Presentation

CS420 lecture four Recurrence Relations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS420 lecture fourRecurrence Relations wim bohm, cs.colostate

  2. Complexity • Divide a program in basic blocks and count the order of magnitude number of basic block executions • basic block: code fragment that takes constant time to execute • careful: some language constructs may take more than constant time • contains in Java • + in APL / C++ • pattern matching in SNOBOL / Python

  3. Recurrence Relations • The complexity of an algorithm can often be expressed in a recurrence relation with the input size as parameter. Eg (merge sort) T(n) = 2T(n/2)+n T(1) = 1

  4. Repeated substitution • Simple recurrence relations (one recurrent term in the rhs) can sometimes be solved using repeated substitution • Two types: Linear and DivCo • Linear F(n) = aF(n-d)+g(n), base: F(1)=v1 • DivcoF(n)= aF(n/d)+g(n), base: F(1)=v1 • Two question: • what is the pattern • how often is it applied until we hit the base case

  5. Linear Example Hanoi: M(n)=2M(n-1)+1, M(1)=1 M(n) = 2M(n-1)+1 = 2(2M(n-2)+1)+1 = 4M(n-2)+2+1 = 4(2M(n-3)+1)+2+1= 8M(n-3)+4+2+1= ... 2kM(n-k)+2k-1+2k-2+...+2+1= hit base for k = n-1: = 2n-1M(1)+2n-1+2n-2+...+2+1 = 2n-1

  6. DivCo example Merge sort: T(n) = 2T(n/2) + n, T(1)=1 n = 2k T(n)=2(2(T(n/4)+n/2)+n = 4T(n/4) + 2n = 8T(n/8) + 3n ... = 2kT(n/2k)+kn hit base for k = logn = 2kT(n/2k)+kn =n+kn = O(nlogn)

  7. Another one: binary search Let n = 2k and f(1)=1 and f(n) = f(n/2)+c f(n)=f(n/2)+c = f(n/4)+2c = f(n/8)+3c = f(n/2k)+kc = hit base for k=lgn: f(1)+lgn.c = O(lgn)

  8. DivCo repeated substitution pattern n = bk f(n)= af(n/b)+g(n) = a(af(n/b2)+g(n/b))+g(n) = a2f(n/b2)+ag(n)+g(n) = a2(af(n/b2)+g(n)+ag(n)+g(n) = a3f(n/b3)+a2g(n/b2)+ag(n/b)+g(n) = ...

  9. Linear repeated substitution pattern n = kb+1 f(n)= af(n-b)+g(n) = a(f(n-2b)+g(n-b)+g(n)= a2f(n-2b)+ag(n-b)+g(n) =a2(af(n-3b)+g(n))+ag(n-b)+g(n)= a3f(n-3b)+a2g(n-2b)+ag(n-b)+g(n) = ...

  10. Master Method Cookbook approach to solution, recognizing the form. Cormenet.al.'s method is more complex than Rosen: An = CAn/d+knp • An = O(np) if C < dpeg An = 3An/2+n2 • An = O(nplog(n)) if C = dpeg An = 2An/2+n • An = O(nlogdc) if C > dpeg An = 3An/2+n Proof based on repeated substitution plus induction

  11. Let's prove a simpler version Let f(n)=af(n/b)+c, n=bk, then proof:

  12. App: multiplying two n digit numbers A x B = an-1an-2..a1a0 X bn-1bn-2..b1b0 • high school method: O(n2) • divide and conquer approach A = A110n/2+A2 B = B110n/2+B2 AxB = A1B110n+(A1B2+A2B1)10n/2+ A2B2 A1B2+A2B1 = (A1+A2)(B1+B2)-A1B1-A2B2 so we only need 3 n/2 digit multiplications: A1B2, A2B1 and (A1+A2)(B1+B2)

  13. Complexity of DivCon digit multiply Tn=3Tn/2+O(n) Look up in master table:

  14. Complexity of DivCon digit multiply Tn=3Tn/2+O(n) Look up in master table: Tn=O(nlg3)~O(n1.6)

  15. Similar trick for matrix multiply C = AxB, simple O(n3) code for i=1 to n for j = 1 to n C[i,j]=0 for k = 1 to nC[i,j]+=A[i,k]*B[k,j]

  16. Block matrix multiply A1,1 A1,2 B1,1 B1,2 * = A2,1 A2,2 B2,1 B2,2 A1,1 B1,1 + A1,2 B2,1 A1,1 B1,2 + A1,2 B2,2 A2,1 B1,1 + A2,2 B2,1 A2,1 B1,2 + A2,2 B2,2

  17. naive block matmult T(n) = 8T(n/2) + O(n2) master table:

  18. naive block matrix multiply T(n) = 8T(n/2) + O(n2) master table: T(n) = O(nlog8)=O(n3)

  19. Strassen Strassen shows that we only need 7 block multiplies (plus O(n2) work: matrix adds/subtracts ) see notes T(n) = 7T(n/2) + O(n2) master table: T(n) = O(nlg7)~O(n2.81)

  20. Linear homogeneous recurrence relations (source: Rosen chapter 7) • an = c1.an-1 + c2.an-2 + .. + cr.an-r a0 = v0, .. , ar-1 = vr-1 : initial values • Guess individual solutions of form n • give rise to characteristic equation • solve to obtain characteristic roots • General solution • linear combination of roots • Specific solution • find coefficients of general solution using initial values

  21. Example: Fibonacci Fn=Fn-1+Fn-2, F0=F1=1 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, ... Exponential growth, guess nand substitute n =n-1 +n-2 2 = +1 characteristic eq: 2 -  - 1 = 0 quadratic equation: ax2+bx+c = 0 solution: x1,2=(-b±sqrt(b2-4ac))/2a

  22. General Solution • Roots of 2 -  - 1 = 0 1,2= (1±sqrt(1+4))/2 = (1±sqrt(5))/2 general solution: linear combination of the roots

  23. A1 and A2 forspecific Solution • Specific solution uses the initial values F0=F1=1 • two equations two unknowns, solve to find As

  24. Specific Solution

  25. Does it always work? Characteristic equation  roots r1 and r2. The general solution: A1r1n+A2r2n Initial values: f0=C0 and f1=C1 specific solution: f0= C0 = A1+A2 A2=C0-A1 f1= C1 = A1r1+A2r2 C1=A1r1+(C0-A1)r2=A1(r1-r2)+C0r2 only works if r1 ≠ r2

  26. Is it a problem? Not really. Equal roots r come from characteristic equation: (x-r)(x-r)=x2-2rx+r2=0 come from recurrence relation: fn=2rfn-1-r2fn-2 If our recurrence relation has only positive coefficients we cannot get equal roots. For Analysis of Algorithms we can restrict the recurrences to having only positive coefficients. We ADD the complexity of sub solutions together

  27. First order Linear Homogeneous Recurrence Relations Simple form Fn=cFn-1 F0=a Use repeated substitution to solve

  28. First order Linear Homogeneous Recurrence Relations Simple form Fn=cFn-1 F0=a Use repeated substitution to solve Fn=cFn-1 =c2Fn-2=c3Fn-3= ... = cnF0=cna

  29. 20% yearly interest on $1000 What is the value in year n Vn=1.2Vn-1 V0=1000

  30. 20% yearly interest on $1000 What is the value in year n Vn=1.2Vn-1 V0=1000 Vn=(1.2)n.1000

  31. Exercise • Determine the general and specific solution of Fn= Fn-1+2Fn-2 with F0=2 and F1=7

  32. Exercise 2 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Characteristic equation?

  33. Exercise 3 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Characteristic equation: x2-x-2=0

  34. Exercise 4 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Characteristic equation: x2-x-2=0 Roots? x1,2=(1±sqrt(1-(-8)))/2

  35. Exercise 5 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Characteristic equation: x2-x-2=0 Roots: r1=2 r2=-1

  36. Exercise 6 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 General Solution?

  37. Exercise 7 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 General Solution: Fn=A12n+A2(-1)n

  38. Exercise 8 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Specific Solution? Find A1 and A2 using initial values

  39. Exercise 9 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Specific Solution: F0=2=A1+A2 F1=7=A1.2+A2.(-1) A1=3 A2=-1 Fn=3.2n-(-1)n

  40. Exercise 10 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Specific Solution: F0=2=A1+A2 F1=7=A1.2+A2.(-1) A1=3 A2=-1 Fn=3.2n-(-1)n DONE???

  41. Exercise 11 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Specific Solution: F0=2=A1+A2 F1=7=A1.2+A2.(-1) A1=3 A2=-1 Fn=3.2n-(-1)n NO! Check your answer

  42. Exercise 11 • Fn= Fn-1+2Fn-2 with F0=2 and F1=7 Specific Solution: F0=2=A1+A2 F1=7=A1.2+A2.(-1) A1=3 A2=-1 Fn=3.2n-(-1)n Check F0 F1 F2 F3

  43. Linear INhomogeneous recurrences Find the homogeneous solution Suppose pn is a particular solution Then the general solution is constant*homogeneous solution + particular solution

  44. Linear Inhomogeneous Simple case an = c1.an-1 + f(n) happens most often • Very simple, special case: c1 = 1 • an = an-1 + f(n) = ... = a0 + f(1) +..+ f(n) • General case: • Solve homogeneous part: hn = A.c1n • if *n is a particular solution then an= A.c1n + *n is the general solution • Particular solution “mimics” f(n) f(n): c, d.n , d.n2 , dn *n: b, a.n + b, a.n2+b.n+c, b.dn or b.n.dn (dc1) or (d=c1)

  45. Some particular solutions for An = CAn-1 + f(n) linear → linear, quadratic → quadratic, etc. Let’s try it on Mn= 2 Mn-1 +1, M1= 1

  46. Example: Hanoi Mn=2Mn-1+1 M1=1 Homogeneous solution: A.2n Particular solution: B B obeys the recurrence relation so B=2B+1 -> B=-1 General solution: Mn=A.2n-1 M1=1 so A.2-1=1 -> A=1 Specific solution: Mn=2n-1

  47. Another example An=2An-1+n A0=0 A1=1 A2=4 A3=11 ... 1. homogeneous solution?

  48. Another example 1 An=2An-1+n A0=0 A1=1 A2=4 A3=11 A4=26 1. homogeneous solution: An=2An-1 An=2n

  49. Another example 2 An=2An-1+n A0=0 A1=1 A2=4 A3=11 A4=26 1. homogeneous solution: An=2An-1 An=2n 2. particular solution?

  50. Another example 3 An=2An-1+n A0=0 A1=1 A2=4 A3=11 A4=26 1. homogeneous solution: An=2An-1 An=2n 2. particular solution .... has form a.n+b and fits the recurrence

More Related