CS 2210 (22C:19) Discrete Structures Advanced Counting

1. CS 2210 (22C:19) Discrete StructuresAdvanced Counting Spring 2015 Sukumar Ghosh

2. Compound Interest A person deposits $10,000 in a savings account that yields 10% interest annually. How much will be there in the account after 30 years? Let Pn = account balance after n years. Then Pn = Pn-1 + 0.10 Pn-1 = 1.1Pn-1 Note that the definition is recursive. What is the solution Pn? Pn = Pn-1 + 0.10 Pn-1 = 1.1Pn-1is a recurrence relation By “solving” this, we get the non-recursive version of it.

3. Recurrence Relation Recursively defined sequences are also known as recurrence relations. The actual sequence is a solution of the recurrence relations. Consider the recurrence relation: an+1 = 2an(n > 0) [Given a1=1] The solution is: an = 2n-1 (The sequence is 1, 2, 4, 8, …) So, a30 = 229 Given any recurrence relation, can we “solve” it? Which are the ones that can be solved easily?

4. More examples of Recurrence Relations Fibonacci sequence: an = an-1 + an-2(n > 2) [Given a1 = 1, a2 = 1] What is the formula for an? 2. How many bit strings of length n that do not have two consecutive 0s. For n=1, the strings are 0 and 1 For n=2, the strings are 01, 10, 11 For n=3, the strings are 011, 111, 101, 010, 110 Do you see a pattern here?

5. Example of Recurrence Relations Let anbe the number of bit strings of length n that do not have two consecutive 0’s. This can be represented as an = an-1 + an-2 (why?) [bit string of length (n-1) without a 00 anywhere] 1 (an-1) and [bit string of length (n-2) without a 00 anywhere] 1 0 (an-2) an = an-1 + an-2 is a recurrence relation. Given this, can you find an?

6. Tower of Hanoi Transfer these disks from one peg to another. However, at no time, a disk should be placed on another disk of smaller size. Start with 64 disks. When you have finished transferring them one peg to another, the world will end.

7. Tower of Hanoi Let, Hn = number of moves to transfer n disks. Then Hn = 2Hn-1 +1 (why?) Can you solve this and compute H64? (H1 = 1)

8. Solving Linear Homogeneous Recurrence Relations A linear recurrence relation is of the form an = c1.an-1 + c2. an-2 + c3. an-3 + …+ ck. an-k (here c1, c2, …, cn are constants) Its solution is of the form an = rn (where r is a constant) if and only if r is a solution of rn = c1.rn-1 + c2. rn-2 + c3. rn-3 + …+ ck. rn-k This equation is known as the characteristic equation.

9. Example 1 Solve: an = an-1 + 2 an-2 (Given that a0 = 2 and a1 = 7) Its solution is of the “form” an = rn The characteristic equation is: r2 = r + 2,i.e. r2 - r - 2= 0. It has two roots r = 2, and r = -1 The sequence {an} is a solution to this recurrence relation iff an = α1 2n + α2 (-1)n a0 = 2 = α1 + α2 a1 = 7 = α1. 2 + α2.(-1) This leads to α1= 3 + α2 = -1 So, the solution is an = 3. 2n - (-1)n

10. Example 2: Fibonacci sequence Solve: fn = fn-1 + fn-2 (Given that f0 = 0 and f1 = 1) Its solution is of the form fn = rn The characteristic equation is: r2 - r - 1= 0. It has two roots r = ½(1 + √5) and ½(1 - √5) The sequence {an} is a solution to this recurrence relation iff fn = α1 (½(1 + √5))n + α2 (½(1 - √5))n (Now, compute α1 and α2 from the initial conditions): α1 = 1/√5 and α2 = -1/√5 The final solution is fn = 1/√5. (½(1 + √5))n - 1/√5.(½(1 - √5))n • 

11. Example 3: Case of equal roots If the characteristic equation has only one root r0 (*), then thesolution will be an = α1 r0n + α2 .nr0n See the example in the book. • 

12. Example 4: Characteristic equation with complex roots Solve: an = 2.an-1 -2.an-2 (Given that a0 = 0 and a1 = 2) The characteristic equation is: r2 - 2r + 2= 0. It has two roots (1 + i) and (1 - i) The sequence {an} is a solution to this recurrence relation iff an = α1 (1+i)n + α2 (1-i)n (Now, compute α1 and α2 from the initial conditions): α1 = - i and α2 = i The final solution is an = -i.(1+i)n + i.(1-i)n Check if it works!

13. Divide and Conquer Recurrence Relations • Some recursive algorithms divide a problem of size “n” into “b” sub-problems each of size “n/b”, and derive the solution by combining the results from these sub-problems. • This is known as the divide-and-conquer approach Example 1. Binary Search: If f(n) comparisons are needed to search an object from a list of size n, then f(n) = f(n/2) + 2

14. Divide and Conquer Recurrence Relations Example 2: Finding the maximum and minimum of a sequence f(n) = 2.f(n/2) + 2 Example 3. Merge Sort: Divide the list into two sublists, sort each of them and then merge. Here f(n) = 2.f(n/2) + n

15. Divide and Conquer Recurrence Relations Theorem. The solution to a recurrence relations of the form f(n) = a.f(n/b) + c (here b divides n, a ≥ 1, b >1, and c is a positive real number) is f(n) (if a=1) (if a >1) (See the complete derivation in page 530)

16. Divide and Conquer Recurrence Relations Proof outline. Given f(n) = a.f(n/b) + c Let n=bk. Then f(n) = a.[a.f(n/b2)+c] + c = a.[a.[a.f(n/b3)+c]+c]+ c and so on … = ak. f(n/bk) + c.(ak-1+ak-2+…+1) … (1) = ak.f(n/bk) + c.(ak-1)/(a-1) = ak.f(1) + c.(ak-1)/(a-1) … (2)

17. Divide and Conquer Recurrence Relations Proof outline. Given f(n) = a.f(n/b) + c When a=1, f(n) = f(1) + c.k (from 1) Note that n=bk, k = logbn, So f(n) = f(1) + c. logbn [Thus f(n) = O(logn)] When a>1, f(n) = ak.[f(1) + c/(a-1)] + c/(a-1) [ ]

18. Divide and Conquer Recurrence Relations What if n ≠ bk? The result still holds. Assume that bk< n <bk+1. So, f(n) < f(bk+1) f(bk+1) = f(1) + c.(k+1) = [f(1) + c] + c.k = [f(1) + c] + c.logbn Therefore, f(n) is O(logn)

19. Divide and Conquer Recurrence Relations Apply to binary search f(n) = f(n/2) + 2 The complexity of binary search f(n) (since a=1) What about finding the maximum or minimum of a sequence? f(n) = 2f(n/2) + 2 So, the complexity is f(n)

20. Master Theorem Note that there are four parameter: a, b, c, d