1 / 41

Recurrence Relation

Recurrence Relation. Outline. What is a recurrence relation ? Solving linear recurrence relations Divide-and-conquer algorithms and recurrence relations. What is a recurrence relation ?. Definition.

dreama
Download Presentation

Recurrence Relation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recurrence Relation

  2. Outline • What is a recurrence relation ? • Solving linear recurrence relations • Divide-and-conquer algorithms and recurrence relations

  3. What is a recurrence relation ?

  4. Definition A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or more of the previous terms of the sequence, namely a0, a1, …, an-1, for all integers n n0, where n0 is a non-negative integer. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation. Example: Given the recurrence relation an=2an-1-an-2. 3, 5, 7, 9, 11, … satisfies the recurrence relation. 2, 3, 4, 5, 6, … also satisfies the recurrence relation.

  5. Example: Compound interest Let Pn be the amount of money in the account after n years. If you initially deposit Pn=1000 in your account and the bank gives k % per year with interest compounded annually. Pn= (1+k/100) Pn-1 P1= (1+k/100) P0 = (1+k/100) P0 P2= (1+k/100) P1 = (1+k/100)2P0 P3= (1+k/100) P2 = (1+k/100)3P0 Pn= (1+k/100) Pn-1 = (1+k/100)nP0

  6. Example: Fibonacci numbers • A pair of rabbits is placed on an island. After 2 months old, a pair of rabbits produces another pair each month. How many rabbits are there after n months? fn =fn-1 + fn-2

  7. Tower of Hanoi

  8. Example: Tower of Hanoi H(1) = 1 H(n) = 2H(n-1) + 1 H(n) = 2(2H(n-2) +1) +1 = 4H(n-2) + 2 + 1 = 4(2(H(n-3) + 1) + 2 + 1 = 8H(n-3) + 4 + 2 + 1 = 8(2H(n-4) +1) + 4 + 2 + 1 = 16H(n-4) + 8 + 4 + 2 + 1 = 2n-1 + 2n-2 + 2n-3 + … + 1 = 2n - 1 Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1 x +1 = (2n-1 + 2n-2 + … + 4 + 2 + 1) +1 x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2 x + 1 = 2n-1 + 2n-2 + … + 4 + 4 x + 1 = 2n-1 + 2n-2 + … + 8 x + 1 = 2n x = 2n - 1

  9. Linear recurrence relation Homogeneous and non-homogeneous

  10. What is a linear recurrence relation ? Linear recurrence relation of degree k an= c1an-1+ c2an-2 +…+ ckan-k + F(n) • ciis a constant, for i= 1, 2, …, k • ck  0 Linear homogeneous recurrence relation • F(n) = 0 Linear non-homogeneous recurrence relation • F(n)  0

  11. Examples Linear homogeneous • an = 1.2 an-1 : degree 1 • fn = fn-1 + fn-2 : degree 2 • an = 3an-3 : degree 3 Non-linear homogeneous • an = a2n-1 + an-2 • an = nan-1 - 2an-2 Linear non-homogeneous • an = an-1 + 2n • hn = 2hn-1 + 1 • an = 3an-1 + n Non-linear non-homogeneous • an = a2n-1 + 2n • an = n2 an-1 + n

  12. Solving linear homogeneous recurrence relation

  13. Theorem • The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2iffan = 1rn1 + 2rn2 , where : • r1and r2are two distinct roots of r2 - c1r - c2 = 0, • a0 = 1 + 2 • a1 = 1r1 + 2r2 • [ 1 = (a1 – a0 r2)/(r1-r2 ) 2 = (a0 r1 –a1)/(r1-r2 ) ]

  14. Informal proof Recurrence relation: an = c1an-1 + c2an-2 Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0. Let  and  be constants. • Show if an = 1rn1 + 2rn2, the sequence {an} is a solution of the recurrence relation. • Show if the sequence {an} is a solution of the recurrence relation, then an = 1rn1 + 2rn2.

  15. Informal proof Show if an = 1rn1 + 2rn2, the sequence {an} is a solution of the recurrence relation. Let an = c1an-1 + c2an-2be a recurrence relation. Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0. Let  and  be constants. Since r1 and r2 are roots of r2 - c1r - c2= 0, r12 - c1r1 - c2 = 0 (r12 = c1r1 + c2) andr22 - c1r2 -c2= 0 (r22 = c1r2 + c2).

  16. Informal proof r12 = c1r1 + c2andr22 = c1r2 + c2. c1an-1 + c2an-2 = c1(1r1n-1+2r2n-1) + c2(1r1n-2+2r2n-2) = c11r1n-1+ c1 2r2n-1 + c21r1n-2+ c2 2r2n-2 = c11r1n-1+ c21r1n-2+c1 2r2n-1 + c2 2r2n-2 = 1r1n-2(c1r1 + c2) + 2r2n-2 (c1r2 + c2) = 1r1n-2 r12 + 2r2n-2 r22= 1 r1n+ 2 r2n =an

  17. Informal proof • Show if the sequence {an} is a solution of the recurrence relation, then there are 1and2 such that an = 1rn1 + 2rn2. • From the initial conditions a0and a1, a0 = 1 + 2and a1 = 1r1 + 2r2 2 = a0 - 1 1r1 = a1 - 2r2 = a1 - (a0-1) r2 = a1 – a0r2 + 1r2 1 (r1 - r2 ) = a1 – a0r2 >> 1 = (a1 – a0r2 )/(r1 - r2 ) 2 =a0-1 =a0-(a1–a0r2 )/(r1-r2 ) = a0r1-a0r2 -a1+a0r2 /(r1-r2 ) 2 = (a0r1-a1) / (r1-r2 )

  18. Example What is the solution of an = an-1 + 2an-2 with a0=2 a1=7 ? We have c1 = 1 andc2 = 2. The characteristic equation is r2 - r - 2 = 0, with roots r1= 2 andr2= -1 . 1 = (a1 – a0r2 )/(r1 - r2 ) = (7+2)/(2+1) = 3 2 = (a0r1 – a1)/(r1- r2 ) = (4-7)/(2+1) = -1 an = 1rn1 + 2rn2 = 32n - (-1)n

  19. Example: Fibonacci number What is the solution of an = an-1 + an-2 with a0=0 a1=1 ? We have c1 = 1 andc2 = 1. The characteristic equation is r2 - r - 1 = 0, with roots r1= (1+5)/2 andr2= (1-5)/2 . 1 = (a1 – a0r2 )/(r1 - r2 ) = (1-0)/5 = 1/5 2 = (a0r1 – a1)/(r1- r2 ) = (0-1)/5 = -1/5 an = 1rn1 + 2rn2 = ((1+5)/2)n /5 - ((1-5)/2)n/5 an = ( (1+5)n - (1-5)n )/(52n )

  20. Theorem • The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2iffan = 1rn0 + 2nrn0 , where : • r0is the only root of r2 - c1r - c2 = 0, • a0 = 1 • a1 = 1r0 + 2r0 = r0(1 + 2) • [ 1 = a0 2 = a1/r0 – a0 ]

  21. Example What is the solution of an= 6an-1 -9an-2 with a0=1 a1=6 ? We have c1 = 6 andc2 = -9. The characteristic equation is r2 - 6r + 9 = 0, with roots r0= 3. 1 = a0= 1 2 = a1/r0 – a0 = 6/3-1 = 1 an = 1rn0 + 2nrn0 = 3n + n 3n = 3n (1+ n).

  22. Theorem • The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2+…+ ckan-kiffan = 1rn1 + 2rn2+…+ krnk, where : • r1, r2 ,…, rkare kdistinct roots of rk -c1rk-1- c2rk-2 -…- ck= 0, • a0 = 1 + 2 +…+ k • a1 = 1r1 + 2r2 +…+ krk • a2 = 1r21 + 2r22 +…+ kr2k • …

  23. Theorem • The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2+…+ ckan-kiffan = 1rn1 + 2rn2+…+ krnk, where : • r1, r2 ,…, rkare kdistinct roots of rk -c1rk-1- c2rk-2 -…- ck= 0, • a0 = 1 + 2 +…+ k • a1 = 1r1 + 2r2 +…+ krk • a2 = 1r21 + 2r22 +…+ kr2k • …

  24. Theorem • The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2+…+ ckan-kiff an = (1,0 + 1,1n + …+ 1,m1nm1-1)rn1 + (2,0 + 2,1n + …+ 2,m2nm2-1)rn2+…+ (t,0 + t,1n + …+ t,mtnmt-1)rnt, where : • r1, r2 ,…, rtare tdistinct roots of rk -c1rk-1- c2rk-2 -…- ck= 0, with multiplicities m1, m2 ,…, mt

  25. Example What is the solution of an=6an-1-11an-2+6an-3 with a0=2, a1=5, a2=15 ? We have c1 = 6, c2 = -11 andc3 = 6. The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with roots r1= 1,r2= 2 andr3= 3 . a0 = 1 + 2 + 3 a1 = 1r1 + 2r2 + 3r3 = 1 + 22 + 33 a2 = 1r12 + 2r22 + 3r32= 1 + 42 + 93 1 = 1, 2 = -1, 3 = 2 an = 1 – 2n + 23n

  26. Solving linear non-homogeneous recurrence relation

  27. Theorem If the sequence {an(p)} is a particular solution of the recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k+ F(n) then every solution is of the form {an(p)+ an(h)} , where {an(h)} is a solution of the associated homogeneous recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k.

  28. Example What are the solutions of an= 3an-1+ 2nwith a1=3 ? We have c1 = 3. The associated characteristic eqn is r-3=0, with root r = 3. Then,an(h) = 3n . Letpn = cn +d. Then, from an= 3an-1+ 2n, cn +d = 3(c(n-1)+d ) + 2n. Thus, c = -1 d = -3/2, and an(p) = -n -3/2 an= an(h) + an(p) = 3n -n -3/2. From a1= 3 = 3 -1 - 3/2,  = 11/6. The solution is an=(11/6)3n -n -3/2.

  29. Example: summation from 1 to n What are the solutions of an= an-1+ nwith a1=1 ? We have c1 = 1. The associated characteristic eqn is r-1=0, with root r = 1. Then,an(h) = 1n =  . Letpn = n(cn+d) = cn2+dn. Then, from an= an-1+ n, cn2 +dn= c(n-1)2+d (n-1) + n. Thus, cn2 +dn= cn2-2cn+c+d n - d + n , c-d + n(1-2c)=0. c-d =0 and 1-2c=0. That is, c=d=1/2. an= an(h) + an(p) =  + n(n+1)/2. From a1= 1 =  +1,  = 0. The solution is an=n(n+1)/2.

  30. Divide-and-conquer Algorithms

  31. Divide-and-Conquer Concept To solve a problem of size n, we solve a sub-problems of size n/b, and combine solutions from sub-problems. n/b n/b n/b n/b X X1 X2 X3 ... Xa n elements

  32. Binary Search • To search for an element a from a sorted list X of size n elements. • If a=m, then stop • If a<m, then search for a from X1. • If a>m, then search for a from X2. X X1 m X2

  33. Fast multiplication of large integers • a = 2nA1 + A0. b = 2nB1 + B0. A1A0B1B0 ab = 22nA1B1 + 2n(A0B1+ A1B0) +A0B0. ab = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0B0. ab = 22n A1B1 + 2nA1B1 + 2nA1 B0 –2nA0B0 +2nA0B1 – 2nA1B1 + 2n A0B0 + A0B0. 10010 00011 10000 10011

  34. Recurrence Relations In Divide-and-conquer algorithms

  35. Divide-and-Conquer Recurrence Relations Let f(n) be the number of operations required to solve a problem of size n. f(n) = a f(n/b) + g(n). n/b n/b n/b n/b X X1 X2 X3 ... Xa n elements

  36. Binary Search • To search for an element a from a sorted list X of size n elements. • If a=m, then stop • If a<m, then search for a from X1. • If a>m, then search for a from X2. Let f(n) be the # of comparison in binary search within n elements. f(n) = f(n/2) + 2. X X1 m X2

  37. Fast multiplication of large integers • a = 2nA1 + A0. b = 2nB1 + B0. A1A0B1B0 ab = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0B0. Let f(n) be the number of operations needed to multiply two n-bitintegers. f(2n) = 3f(n) + Cn. 10010 00011 10000 10011

  38. Theorem Let f be an increasing function that satisfies the recurrence relation f(n) = a f(n/b) + c whenever n is divisible by b, b is an integer greater than 1, a1, c is a positive real number. When n=bk, f(n) = t nlogba + s, for some t and s. When a>1, f(n) is O(nlogba ). When a=1, f(n) is O(log n).

  39. Examples Binary Search: f(n) = f(n/2) + 2. Because a=1, f(n) is O(log n). Find Max. : f(n) = 2f(n/2) + 2. Because a>1, f(n) is O(nlog11 ) = O(n) .

  40. Master Theorem Let f be an increasing function that satisfies the recurrence relation f(n) = a f(n/b) + cnd whenever n=bk for some positive integer k, b is an integer greater than 1, a1, c is a positive real number, d is a non-negative real number. When a<bd, f(n) is O(nd ). When a=bd, f(n) is O(nd log n). When a>bd, f(n) is O(nlogba).

  41. Examples Merge sort: f(n) = 2f(n/2) + n. a = 2, b = 2, c = 1, d = 1 Because a=bd, f(n) is O(nlog n). Integer multiplication: f(n) = 3f(n/2) + Cn a = 3, b = 2, c = C, d = 1 Because a>bd, f(n) is O(nlog23).

More Related