CS 106 Computing Fundamentals II Chapter 79 “ Recursion ”

1. Herbert G. Mayer, PSU CS status 6/24/2013 Initial content copied verbatim from CS 106 material developed by CS professors: Cynthia Brown & Robert Martin CS 106Computing Fundamentals IIChapter 79“Recursion”

2. Syllabus • Recursion • Mimic a While Loop • Recursion • Step by Step • Recursion can be Clearer • Recursion Issues • Functional Programs • Sample: Fibonacci • Merging • Merge Sort Outline • Analysis • More ideas

3. Recursion • Recursion is a powerful technique for thinking about certain types of algorithms • It can be used to simulate a loop, or other kinds of applications • Simplistically speaking, a recursive function is a function that calls itself • More correctly and completely: A recursive function is a function that is partly defined in simpler versions of itself • As with a while loop, there is a danger of an infinite recursion –referred to as infinite regress-- so there has to be a test for stopping it, and something (usually a parameter) must change between calls

4. Mimic a While Loop • Here’s the code for the while loop to build a multiplication table in our Loop Multiplication demo: • j = 1 • Do While j <= MAX • lstAnswer.AddItem(strM & " X " & _ CStr(j) & " = " & _ CStr(numM * j)) • j = j + 1 • Loop

5. A Recursive Procedure • Procedure call to get things started: • RecurMult(numM, strM, 1) • Procedure code: • Sub RecurMult(ByVal numM As Integer, ByVal strM As String, ByVal j As Integer) • If j <= MAX Then • lstAnswer.Items.Add(strM & " X " & _ CStr(j) & " = " & _ CStr(numM * j)) • RecurMult(numM, strM, j + 1) • End If • End Sub

6. How does it work? • Note MAX is global. In the initial call, parameter j is 1. We print the line of the table with j = 1 and do the next call, with j = 2. • In the second call, we print the line of the table with j = 2, and do the call with j = 3. • This continues till we do a call with j > MAX. In that case we just return without initiating another call. • This triggers all the other calls, in reverse sequence, to return, and we’re done

7. It’s not an easy concept… • So now we’ll look at a simple example

8. Recursive Definition • Sometimes it makes sense to define a quantity recursively: for example, the sum of the first n numbers: • S(1) = 1 • S(n) = n + S(n-1) for n>1 • (As opposed to S(n) = 1 + 2 + … + n)

9. Questions for Recursive Definitions • If I know how to compute the answer for n-1, how do I compute it for n? (Potentially you can use any values for arguments smaller than n in the definition) • In our example, S(n) = S(n-1) + n • How does the recursion stop? What is the “bottom” argument and what is the value for that argument? • In our example, S(1) = 1

10. Recursive Computation • Private Function SumOfN(ByVal n As Integer) As Integer • If n <= 1 Then ‘expecting n = 1 but catch error • SumOfN = 1 • Else • SumOfN = n + SumOfN(n – 1)) • End If • End Function • X = SumOfN(5) returns 5 + 4 + 3 + 2 + 1 = 15

11. Step by Step • SumOfN(5) returns 5 + SumOfN(4) • SumOfN(4) returns 4 + SumOfN(3) • SumOfN(3) returns 3 + SumOfN(2) • SumOfN(2) returns 2 + SumOfN(1) • SumOfN(1) returns 1 • Once we “hit bottom,” Sum0fN(1)returns its value; then SumOfN(2) can compute and return its value, and so on.

12. Compare to While Loop: • j = n • sum = 0 • Do While j >= 1 • sum = sum + j • j = j – 1 • Loop

13. Recursion can be Clearer • Recursion directly implements the definition • It is plain to see that it computes the correct value • Coming up with the loop is not that easy for more complex recursive definitions, and its structure is quite different • Note there is also a closed form for this recursion: S(n) = n(n + 1) / 2 • Closed forms can be hard to find

14. Recursion Issues • As with a while loop, a recursion can be infinite if we do not include a way to stop it • The test to see if it is done should usually be the first thing a recursive function does • Recursion uses more resources than a loop and it may not be possible to do a very large recursion (depends on language implementation)

15. Functional Programming • This is a style of programming that replaces loops with recursions and assignment statements with parameter/argument associations • After you get the knack of doing it, it can result in very clear, concise programs • There are languages especially designed to support this style

16. Example: Fibonacci Numbers • A program that implements Fibonacci numbers several ways. Here’s the recursive definition: • Fib(0) = 0 • Fib(1) = 1 • Fib(n) = Fib(n – 1) + Fib(n – 2)

17. Demo: Fibonacci Numbers

18. Recursive Fib(5) Fib(1) Fib(2) Fib(3) Fib(0) Fib(1) Fib(4) Fib(1) Fib(2) Fib (0) Fib(5) Fib(1) Fib(2) Fib(0) Fib(3) Fib(1)

19. Recursive Fib(5): Order of Calls 4 Fib(1) 3 Fib(2) 2 Fib(3) Fib(0) 5 6 Fib(1) Fib(4) 8 7 1 Fib(1) Fib(2) Fib (0) 9 Fib(5) 12 10 Fib(1) 11 Fib(2) Fib(0) 13 Fib(3) Fib(1) 14

20. Array Fib(5) In the first picture, we add elements 0 and 1 to get element 2 (Fib(2)), giving the second picture. Next add elements 1 and 2 to get Fib(3). Etc. (We start indexing with zero.)

21. Loop Fib(5) fn = fnm1 + fnm2 fnm2 = fnm1 fnm1 = fn start step 3 step 1 step 4 step 2

22. Recursive Fib(5): Order of Calls 4 Fib(1) 3 Fib(2) 2 Fib(3) Fib(0) 5 6 Fib(1) Fib(4) 8 7 1 Fib(1) Fib(2) Fib (0) 9 Fib(5) 12 10 Fib(1) 11 Fib(2) Fib(0) 13 Fib(3) Fib(1) 14

23. Memoized Fib(5) 4 Fib(1) 3 Fib(2) 2 Fib(3) Fib(0) 5 6 Fib(1) Fib(4) 7 1 Fib(2) Fib(5) 8 Fib(3)

24. Using Recursion in Sorting • Motivation: to develop a fast sorting algorithm • Recall Selection Sort: it takes time proportional to n2, where we use the number of comparisons as a way to estimate the time, and n is the number of elements to be sorted • This is too slow for sorting large data sets, even on a very fast computer

25. Why Selection Sort is Slow • Selection sort and similar algorithms require us to do something similar to comparing every element to every other element • We can be clever and avoid some of the comparisons but the basic nature of the algorithm-- that it takes time proportional to n2-- remains the same unless we use a radically different approach • So in selection sort, instead of n + n +…+n, n times, = n*n, we have n + (n-1) + (n-2) + … + 2 + 1, which equals n(n-1)/2. Less than half as big, but still roughly proportional to n2, especially for large n

26. The Fundamental Idea • The fundamental idea is to divide the problem roughly in half each time, solve the subproblems, and then put them back together • If done cleverly, this can give us a time proportional to n log n. • One way to do this is based on merging sorted lists. Let’s first look at how that works.

27. Merging • Say we have two sorted lists that we want to combine to make one sorted list. For example: • List A: 1, 5, 8, 15, 19 • List B: 2, 5, 7, 20, 22 • Method: look at the first element in each list. Put the smaller one in the answer. If one list is empty, put all the elements from the other list in the answer

28. Step 1 • List A: 1, 5, 8, 15, 19 • List B: 2, 5, 7, 20, 22 • Answer: [empty] • List A: 5, 8, 15, 19 • List B: 2, 5, 7, 20, 22 • Answer: 1

29. Step 2 List A: 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: 1 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2

30. Step 3 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5

31. Step 4 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5

32. Step 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7

33. Step 6 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8

34. Step 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15

35. Step 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19

36. Step 9 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19 List A: List B: Answer: 1, 2, 5, 5, 7, 8, 15, 19, 20, 22

37. Comparisons for Merging • Until we empty one of the lists, it takes one comparison to get one element into the answer • So, roughly, the number of comparisons to merge two sorted lists is proportional to the total number of elements in the two lists.

38. Merge Sort Outline • Divide the list in half and sort each half • Merge the two sorted halves into a sorted list • How do we sort each half? Using MergeSort! • Huh? Isn’t this a circular definition or something?

39. Start at the Bottom • We’ll think about it this way: suppose we have an unsorted list of 8 elements. We are going to divide it into 8 tiny lists of one element each, and merge them in pairs • Here’s our example list. We’ll show it as an array so it is easy to talk about each element.

40. Merging One-Element Pairs

41. Merging Two-Element Pairs

42. Merging Four-Element Pairs

43. Analysis • We double the size of the pairs each time. The number of times we can double to reach size n, starting with 1, is log n. So there are log n stages. • The time for each stage is proportional to n, since the total elements being merged each time is n. • So the overall time is n log n

44. More Ideas • There are lots more recursive sorting algorithms • For example, in Quicksort, we divide the problem in half (in time n) by putting the elements bigger than some given element in the back and the smaller ones in the front. • Do the same to each piece till you get to size one: there are log n stages

45. Sorting Summary (1) • Selection Sort and Bubble Sort are not recursive. They both take time proportional to n2, though Bubble sort can be somewhat faster than Selection sort • Mergesort and Quicksort are both recursive in concept, though there are ways to avoid explicit recursions in the implementation. Mergesort is excellent especially for sorting data too big to all fit in memory. It always takes time proportional to n log n

46. Sorting Summary (2) • Quicksort is recursive and usually very fast, proportional to n log n. In the worst case (sorted data!) it can take time proportional to n2, though. Clever variations try to avoid this problem. • The best possible time to sort just based on comparisons is proportional to n log n. You can do a bit better if, for example, you know that the data will be numbers

47. Sorting Summary (3) • For small amounts of data just use a simple algorithm, or rely on the ones built into Excel or VBA • If your program handles huge amounts of data then writing your own fast sort is one thing to try for speeding it up • Our main purpose here was to give you a feeling for the vast variety of clever algorithms that can be developed to perform a task

48. The Sorting Sampler • The sorting sampler lets you play around with several algorithms and look at how they behave when sorting the same data.