Recursion, Complexity, and Sorting

1. Recursion, Complexity, and Sorting By Andrew Zeng

2. Table of Contents • Quicksort • Mergesort • Recursion • Algorithms and Complexity • Review Questions

3. Overview • In this slideshow we will cover: • The implementation of sorts • Speed of the sorts • The benefits and disadvantages of recursion • The analysis of algorithms • Big-O notation • Review problems

4. Algorithms and Complexity • Big-O notation • It is an abstract function used to represent time complexity of an algorithm or another function with parameter size n • Examples are O(n3),O(log n), and O(an). (Note than a can be different values, such as 2 and 3, and be different complexities because they differ by a non constant factor of O(1.5n). Also note n2 is the same as n2/100)

5. Algorithms and Complexity cont. • Big-O analysis • Simple assignment actions are O(1) • The running time of a function calls is the Big-O of its body. • Method parameters are O(1) • Logic tests such as in if/else statements are in O(1). The complexity of the if/else chain is the complexity of the worst case • Add together all of the complexities or multiply if in a loop to get the Big-O notation

6. Recursion • In computer science, recursion is when a method or function calls itself, much like a recursive series in mathematics. • This requires stack memory to keep track of all the methods that have been called • Recursion must have a base case, which will stop the function from infinitely calling itself. • For example, a selection sort could call itself to sort from 0  size – 1, then call itself to sort from 0  size – 2, then find the greatest and put that on the top of the sorted array

7. Pros and cons of recursion • Imagine the sort previously mentioned. • If the size was 1 billion, it would require 1 billion method calls, so the sort would run out of memory • Therefore it is almost always better to use iterative solutions whenever possible. • Recursion can be useful when • It is difficult to divide the problem into an iterative approach • It is as fast and as memory consuming as an iterative approach because it will probably make code clearer Now imagine a stack a millions times as big.

8. Example of recursion • Click to continue • Click this if you are bored of clicking Not having a base case is like falling forever without an edge to hang on to in Portal.

9. Quicksort • Its complexity is O(n2) in its worst case because, by choosing the worst pivot every time (the highest element value), it basically turns into a bubble sort • On average it is O(n log n)

10. Quicksort cont. • Generally what happens is you choose a pivot. Then all the values smaller than the pivot are moved to the left and the bigger ones to the right. • This is the most simple version. The algorithm can be modified also to sort items ascending or descending.

11. Merge Sort • Background • Developed by the mathematician John von Neumann in 1945, who invented the computer • Based on the divide and conquer algorithm idea • Its complexity is O(n log n) in both best and worst case, but it is a slower O(n log n) on average than that of quicksort. Sometimes, though, Mergesort can be tied with quicksort

12. Merge Sort cont. • The general code for the sort is • If the list size is less than 2, it is sorted and return the sublist • Otherwise divide the list in two half the size lists and recursively sort the sublists • Merge the sublists together and return the new list Here is a visual sample of Mergesort

13. Pros and Cons • It has good speed and is very consistent • Not really much, except that with a huge array, memory problems can arise from an overly large stack • Its total recursive calls is O(2n-1)compared with the quicksort n • It also makes fewer comparisons than quicksort except in rare cases

14. Summary • We have found in this presentation that: • Recursion should be used only when necessary. • The speed of the sorts Quicksort and Mergesort, among others. • How to analyze a algorithm, recursive or iterative.

15. Review Questions • #1 – Analyze the Big-O of the following code: A: O(n2 log n) B: O(n*n!) C: O(nn) D:O(n3)

16. Review Questions • #2 – Which of the following is the fastest algorithm? • A: Quicksorting an array of size 5n • B: Mergesorting an array of size 5n • C: Bubble sorting an array of size n • D: It depends • #3 – Analyze the result of the following code on the left for a = 3. (Spaces = line feeds) • A: c++ c++ c++ .net .net .net • B: c++ c++ c++ .net .net • C: c++ c++ c++ .net c++ c++ c++ .net .net • D: None of the above

17. Review Answers • #1 – Start with sort(), because that is the first method called, it calls inorder() and negates it which is O(1). • inorder() is clearly O(n), because there a single equality test done at most n times so its O(n * 1 = n). • shuffle() is O(n) because all of the action in the loop are O(1) and the shuffle is done n times. • Add loop overhead of inorder() and shuffle() and you get O(n) because the O(1) is of little consequence as n gets big. • Now analyze the while loop. The probability that all the elements are in order is 1/n * 1/(n-1) * … ½ * 1. So it has probability 1/n! it is right, so the expected amount of times it takes on average is n!, so the complexity is O(n*n!). In this case we do not consider the worst case because it may never randomly get to the answer.

18. Review Answers cont. • #2 – The answer is D: it depends. • One way to approach this is that for all cases, Quicksort on average faster than Mergesort, but can be slower. • On the other hand, bubble sort is slower for the same size array as the other two sorts, but the size is smaller so for small n it is faster.

19. Review Answers cont. • #3 – The answer is C. For a = 3, the method is recursively called for a = 2, which then calls a = 1, and then a = 0. • At a = 0, the method tests to see the value of a and breaks out of the recursive sequence. • Then at a = 1, c++ is printed once in the 1st loop and .net 0 times in the 2nd. • Then the method finishes and a = 2 executes, and acts almost like a = 1 except for the number of times the words are printed, which both increase by 1 as a goes up by 1. • This is also true for a = 3.

20. Code writing exercise • Write a recursive and iterative algorithm to create anagrams of a word. Imagine a word as collection or characters. Let n = size • Assume that there are no repeats of a letter. • Recursive: • Starting with the whole collection do this size of word times • Create anagrams of the sub-word (the whole word excluding the first letter, then display the word when if it has reached a size of 2 (the base case is size 1, because there are no permutations except itself) • Then rotate the word by moving the first letter to the end • Iterative: • Starting at element 0, swap adjacent elements until you reach the end • Repeat this process size! / (size -1) times.