Recursion

1. Thinking about Algorithms Abstractly Multiplying Recurrence Relations Code Stack of Stack Frames Tree of Stack Frames Friends and Strong Induction Towers of Hanoi Check List Merge & Quick Sort Simple Recursion on Trees Generalizing the Problem Things not to do Heap Sort & Priority Queues Trees Representing Equations Pretty Print Parsing Recursive Images Ackermann's Function Recursion Jeff Edmonds York University Lecture3 COSC 3101

2. 52 88 14 14,23,25,30,31,52,62,79,88,98 31 98 25 30 23 62 79 Merge Sort Consider your input instance. You are assured that it meets the precondition. Technique: Divide and Conquer Your job is to produce an output that meets the post condition.

3. 52 88 14 1,2,3 31 98 25 30 23 62 79 Split Set into Two Merge Sort • You can have as many friends as you want. • You give each a sub instance. • Meets the precondition • Smaller • Trust them to give you the output for their instances. 79 cm 1 3 Trust your friendsDon’t micro manage them 2 By magic!

4. 14 25,31,52,88,98 30 23 62 79 Split Set into Two Merge Sort • You can have as many friends as you want. • You give each a sub instance. • Meets the precondition • Smaller • Trust them to give you the output for their instances. Get one friend to sort the first half. 79 cm 52 88 Trust your friendDon’t micro manage him. 31 98 25

5. 14 25,31,52,88,98 14,23,30,62,79 30 23 62 79 Split Set into Two Merge Sort • You can have as many friends as you want. • You give each a sub instance. • Meets the precondition • Smaller • Trust them to give you the output for their instances. Get another friend to sort the second half. 79 cm Trust your friendDon’t micro manage him.

6. 14,23,30,62,79 25,31,52,88,98 Split Set into Two Merge Sort Use this help to solve your own instance. Merge two sorted lists into one

7. Split Set into Two Merge Sort Use this help to solve your own instance. Merge two sorted lists into one 88, 25, 52, 31, 98 23, 62, 79 14, 30,

8. 14,23,25,30,31,52,62,79,88,98 Split Set into Two Merge Sort Use this help to solve your own instance. Merge two sorted lists into one

9. 0 i-1 i T+1 i i On Step At a Time It can be difficult to understand where computation go. I implored you to not worry about the entire computation. Strange(x,y): x1 = x/4; y1 = 3y; f1 = Strange( x1, y1 ); x2 = x - 3x1; y2 = y; f2 = Strange( x2, y2 ); return( f1+f2); Precondition Postcondition next (x-3x1)  y x/4  3y next

10. Friends & Strong Induction Recursive Algorithm: • Assume you have an algorithm that works. • Use it to write an algorithm that works. If I could get in, I could get the key. Then I could unlock the door so that I can get in. Circular Argument!

11. Friends & Strong Induction Recursive Algorithm: • Assume you have an algorithm that works. • Use it to write an algorithm that works. To get into my house I must get the key from a smaller house

12. Use brute force to get into the smallest house. Friends & Strong Induction Recursive Algorithm: • Assume you have an algorithm that works. • Use it to write an algorithm that works.

13. Use brute force to get into the smallest house. Friends & Strong Induction Recursive Algorithm: • Assume you have an algorithm that works. • Use it to write an algorithm that works. Advice: Do not trace it out. Trust your friend. Do your job.

14. MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) Representations of Recursive Algorithms Cons Views Pros • Code • Tracing out stack of stack frames • Tree of stack frames • Friends (strong induction) How implement for a computer How runs on computer View entire computation Worry about one step at a time I need a higher levelof understanding I am not a computerCrazy making Too much Must trust.

15. X = 33 Y = 12 MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) RecursionFriends & Strong Induction intsx,y product xy • Know Precond: Postcond: • Consider your input instance • If it is small enough solve it on your own. • Allocate work • Construct one or more sub-instances • It must be smaller • and meet the precondition • Assume by magic your friends give you the answer for these. • Use this help to solve your own instance. • Do not worry about anything else. • Micro-manage friends by tracing out what they and their friend’s friends do. • Who your boss is. ac = 31 bd = 32 (a+b)(c+d) =63 = 3 = 6 =18 XY = 396 X = 3 Y = 2 XY = 6 X = 3 Y = 1 XY = 3 X = 6 Y = 3 XY = 18

16. X = 33 Y = 12 MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) return( e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ) RecursionFriends & Strong Induction This technique is often referred to as Divide and Conquer ac = 31 bd = 32 (a+b)(c+d) =63 = 3 = 6 =18 XY = 396 X = 3 Y = 2 XY = 6 X = 3 Y = 1 XY = 3 X = 6 Y = 3 XY = 18

17. Complex Numbers • Remember how to multiply 2 complex numbers? • (a+bi)(c+di) = [ac –bd] + [ad + bc] i • Input: a,b,c,d Output: ac-bd, ad+bc • If a real multiplication costs 1 and an addition costs a penny. What is the cheapest way to obtain the output from the input? • Can you do better than 4.02?

18. Gauss’ $3.05 Method:Input: a,b,c,d Output: ac-bd, ad+bc • m1= ac • m2= bd • A1 = m1– m2= ac-bd • m3 = (a+b)(c+d) = ac + ad + bc + bd • A2 = m3 – m1 – m2= ad+bc

19. Question: • The Gauss “hack” saves one multiplication out of four. It requires 25% less work. • Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings?

20. Running Time Tom LehrerNew Math How to add 2 n-digit numbers. ** ** ** ** ** ** ** ** ** ** ** + Decimal Digitsor Binary BitsI don’t care.

21. Running Time How to add 2 n-digit numbers. ** ** ** ** ** ** ** ** ** * ** ** * +

22. Running Time How to add 2 n-digit numbers. ** ** ** ** ** ** ** ** * ** * ** * ** * +

23. Running Time How to add 2 n-digit numbers. ** ** ** ** ** ** ** * ** * ** * * ** * ** * +

24. Running Time How to add 2 n-digit numbers. ** ** ** ** ** ** * ** * ** * * ** * * ** * ** * +

25. Running Time How to add 2 n-digit numbers. * * * ** * * ** * * ** * * ** * * ** * *** * * ** * * ** * * ** * * ** * ** * +

26. Running Time * * * ** * * ** * * ** * * ** * * ** * * ** * * ** * * ** * * ** * * ** * ** * + How to add 2 n-digit numbers. On any reasonable computer adding 3 digits can be done in constant time. T(n) = The amount of time grade school addition uses to add two n-digit numbers ≤ θ(n) = linear time. Means some constant (eg 10) times n.

27. Running Time How to add 2 n-digit numbers. • QUESTION:Is there an algorithm to add two n-digit numbers whose time grows sub-linearly in n? • No! You must read the input to know the answer. And this takes time n!

28. Running Time How to add 2 n-digit numbers. So any algorithm for addition must use time at least linear in the size of the numbers. Grade school addition is essentially as good as it can be.

29. Running Time n2 How to multiply 2 n-bit numbers. * * * * * * * *  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

30. Running Time time # of bits in numbers Grade School Addition: Linear timeGrade School Multiplication: Quadratic time Neat! We have demonstrated that as things scale multiplication is a harder problem than addition. For big inputs, even 100000 n<<0.000001 n2

31. Running Time Grade School Addition: Linear timeGrade School Multiplication: Quadratic time Neat! We have demonstrated that as things scale multiplication is a harder problem than addition. Don’t jump to conclusions!We compared two algorithms. We did not prove that no possible multiplication algorithmruns in linear time. Is there a clever algorithm to multiply two numbers in less than quadratic time?

32. Running Time n2 a × b = a + a + a + ... + a b Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * T(n) = Time multiply T(n) = Time multiply * * * * * * * * = θ(n2) = quadratic time. = θ(b) = linear time. * * * * * * * * Which is faster? * * * * * * * * 92834765225674897 × 83883977590110394875 9 * * * * * * * * * * * * * * * * Minutes? Add a digit? Which would you use? Exponential? Centuries?

33. Running Time On Line Banking? Given a value N find N = a × b loop b = 2..N check if N/b T(n) = Time factor = θ(N) = linear time. 9 9283476522567489783883977590110394875 Minutes? Add a digit? Exponential? Centuries?

34. Size of paper # of bits # of digits Value - n = 2 in2 - n = 17 bits - n = 5 digits - n = 83920 5 83920 1’’ 2’’ Size of Input Instance • Intuitive • Formal • Reasonable • Unreasonable # of bits = log2(Value) Value = 2# of bits

35. Running Time n2 a × b = a + a + a + ... + a b Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * T(n) = Time multiply T(n) = Time multiply * * * * * * * * = θ(n2) = quadratic time. = θ(b) = linear time. * * * * * * * * Which is faster? * * * * * * * * 92834765225674897 × 8388397759011039475 * * * * * * * * * * * * * * * * b = value = 8388397759011039475 Time ≈ 8388397759011039475 n = # digits = 20 Time ≈ 202 ≈ 400

36. Running Time n2 a × b = a + a + a + ... + a b Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * T(n) = Time multiply T(n) = Time multiply * * * * * * * * = θ(n2) = quadratic time. = θ(b) = linear time. * * * * * * * * Which is faster? * * * * * * * * 92834765225674897 × 8388397759011039475 * * * * * * * * * * * * * * * * b = value ≈ 10n Time ≈ 10n ≈ exponential!!! n = # digits = 20 Time ≈ 202 ≈ 400

37. Running Time n2 a × b = a + a + a + ... + a b Grade School vs Kindergarten  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * T(n) = Time multiply T(n) = Time multiply * * * * * * * * = θ(n2) = quadratic time. = θ(b) = linear time. * * * * * * * * Which is faster? * * * * * * * * 9 92834765225674897 × 8388397759011039475 * * * * * * * * * * * * * * * * Adding a single digit multiplies the time by 10! n = # digits = 20 Time ≈ 202 ≈ 400

38. Grade School Addition: Linear timeGrade School Multiplication: Quadratic time time # of bits in numbers Kindergarten Multiplication: Exponential time For big inputs, n<< n2 <<2n Oops this was worse!

39. Divide And Conquer(an approach to faster algorithms) • DIVIDE my instance to the problem into smaller instances to the same problem. • Have a friend (recursively) solve them.Do not worry about it yourself. • GLUE the answers together so as to obtain the answer to your larger instance.

40. Multiplication of 2 n-bit numbers The input X can be thought of in two ways: • as an integer X = 2134 • as a string X = “2134” of n=4 digits. A recursive program gives each “friend” an smaller instance. Simply cut this string into two halves a and b. These can be thought of in two ways: • as strings a = “21” and b= “34”with n/2= 2 digits each. • as integers a = 21andb= 34. Note:X = 2134 = 21100 + 34 = a10n/2 + b X = Y = a c b d = a 2n/2 + b = c 2n/2 + d

41. Multiplication of 2 n-bit numbers X = a 2n/2 + b and Y = c 2n/2 + d X  Y= (a 2n/2 + b)  (c 2n/2 + d) = ac 2n+ (ad+bc) 2n/2 + bd Example:X = 23 and Y = 12 ac 10n+ ( ad + bc) 10n/2 + cd= 21 102 + (22+31) 101 + 32 X  Y= 23 12 46 230 276 23 12 6 40 30200276

42. Multiplication of 2 n-bit numbers a b c d • X = • Y = • XY = ac 2n + (ad+bc) 2n/2 + bd MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

43. Multiplication of 2 n-bit numbers • T(n) = time taken by MULT on two n-bit numbers • What is T(n)? What is its growth rate? • Is it θ(n2)? MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

44. Recurrence Relation • T(1) = k for some constant k • T(n) = 4 T(n/2) + k’ n + k’’ (for some constants k’ and k’’) MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

45. Recurrence Relation • Lets be concrete • T(1) = 1 • T(n) = 4 T(n/2) + n • How do we unravel T(n)? MULT(X,Y): If |X| = |Y| = 1 then return( XY ) Break X into a,b and Y into c,d return( MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d) )

46. Technique 1Guess and Verify • Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n • Guess: G(n) = 2n2 – n • Verify:

47. T(1) = 1 Technique 2: Decorate The Tree T(1) = 1 • T(n) = n + 4 T(n/2) T(n) T(n) n n = = T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2)

48. T(n) n = T(n/2) T(n/2) T(n/2) T(n/2)

49. n/2 T(n/4) T(n/4) T(n/4) T(n/4) T(n) n = T(n/2) T(n/2) T(n/2)

50. n/2 n/2 n/2 n/2 T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n) n =