Algorithm Design Techniques: Induction

1. Algorithm Design Techniques: Induction • Chapter 5 (Except Sections 5.6 and 5.7)

2. Induction • Main objective: Using induction or inductive reasoning as a “recursive” algorithm design technique • Why recursion • Concise algorithms for complex problems can be developed • The algorithms are easy to comprehend • Development time • Proof of correctness of the designed algorithm is usually simple.

3. How Does Induction Produce Recursive Algorithms? • Having a problem with input size n, it is sometimes easier to start with a solution to the problem with a smaller size and extend the solution to include the input size n.

4. Problems to Discuss • Selection sort • Insertion sort • Radix sort • Integer exponentiation • Evaluating polynomials

5. Recursive Selection Sort • Induction Hypothesis: We know how to sort A[2..n] • Inductive Reasoning: We sort A[1..n] as follows • Find the minimum A[j], 1  j  n • Swap(A[1],A[j]) • Sort A[2..n] // induction hypothesis • What is the base case? • What is the initial call to the sorting algorithm? • This kind of recursion is called tail-recursion.

6. Recursive Selection Sort Algorithm Algorithm SELECTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in nondecreasing order. Procedure sort(i) {Sort A[i..n]} • if i < n then Begin 2. k := i; 3. for j := i + 1 to n 4. if A[j] < A[k] then k := j; 5. if k != i then Swap(A[i], A[k]); 6. sort(i + 1); End

7. Complexity Analysis of Recursive Selection Sort • What is the recurrence relation that describes the number of comparisons carried out by the algorithm? • What is the solution to the recurrence?

8. Recursive Insertion Sort • Induction Hypothesis: We know how to sort A[1..n-1] • Inductive Reasoning: We sort A[1..n] as follows: • Sort A[1..n-1] // induction hypothesis • Insert A[n] in its proper position in A[1..n] • This may involve copying zero or more elements one position ahead in order to insert A[n]

9. Recursive Insertion Sort Algorithm Algorithm INSERTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in non-decreasing order. Procedure sort(i) {Sort A[1..i]} 1. if i >1 then 2. sort(i – 1) {Recursive Call} 3. x := A[i] 4. j := i – 1 5. while j > 0 and A[j] > x 6. A[j + 1] := A[j] 7. j := j – 1 end while 8. A[j + 1] := x end if

10. Complexity Analysis of Recursive Insertion Sort • Unlike selection sort, the analysis has to differentiate between the best case and the worst case. Why? • Recurrence of the best case: • Solution to the best case: • Recurrence of the worst case: • Solution to the worst case:

11. Radix Sort • Treats keys as numbers in a particular radix or base. • This is NOT an elements comparison-based sorting algorithm • It only works under the following assumption:

12. Radix Sort Derivation • Assume that our keys are of the form dkdk – 1 … d1 • Induction Hypothesis: Suppose that we know how to sort numbers lexicographically according to their least k – 1 digits, dk – 1, dk – 2, …, d1 , k > 1. • Inductive reasoning: We sort the numbers based on their first k digits as follows: • Use induction hypothesis to sort the numbers based on their 1..k-1 digits. • Sort the numbers based on their corresponding kth digits

13. Description of Radix Sort Algorithm • Distribute the input numbers into 10 sublists L0, L1,…, L9 according to the least significant digit, d1. • Form a new list, we denote by main list, by removing the input from the 10 lists, starting from L0, L1, …etc. in order. • Distribute the main list into the 10 sublists according to the second digit, d2, and then repeat step 2. • Repeat step 3 for the rest of the digits in order of d3, d4, …, dk.

14. Example • Sort the following numbers using radix sort: 467 1247 3275 6792 9187 9134 4675 39 7 6644

15. Radix Sort Algorithm Algorithm RADIXSORT Input: A linked list of numbers L = {a1, a2,…,an} and k, the maximum number of digits. Output: L sorted in nondecreasing order. 1. for j ←1 to k do 2. Prepare 10 empty lists L0, L1, …, L9. 3. while L is not empty 4. a ← next element in L. Delete a from L. 5. i ← jth digit in a. Append a to list Li. 6. end while 7. L ← L0 8. for i ←1 to 9 do 9. L ← L . Li {append list Li to L} 10. end for 11. end for 12. return L

16. Time and Space Complexity Analysis • Time Complexity • Space Complexity

17. Integer Exponentiation • What is the straightforward algorithm to raise x to the power n? • Such an algorithm is exponential in input size!!!!!!!!

18. Algorithm Derivation • Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2 • Induction Hypothesis: Assume that we know how to compute xn/2. • Inductive Reasoning: We raise x to the power n as follows: • Use induction hypothesis to compute y=xn/2 . xn/2 • If n is even, xn = y • If n is odd, xn = x . y

19. Integer Exponentiation Algorithm ALGORITHM EXPRECURSIVE Input: A real number x and a nonnegative integer n. Output:xn. /* the first call to the algorithm is power(x,n) */ Procedure power(x, m) {Compute xm} 1. if m=0then y← 1 2. else 3. y←power(x,m/2) 4. y←y2 5. if m is odd then y←x . y end if 7. return y

20. Complexity Analysis of the Algorithm • When n = 0, the number of multiplications is Therefore, T(0)= • Best Case Analysis: • When does the best case occur? • What is the recurrence equation? Solution? • Worst Case Analysis: • When does the worst case occur? • What is the recurrence equation? Solution?

21. Iterative Exponentiation Algorithm • Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2 • Starting with y = 1, scan the binary digits of the number n from left to right (j = k down to 0): • If bj = 0, square y • If bj = 1, square y and multiply it by x • Example: Compute 212 using the iterative exponentiation algorithm

22. Polynomial Evaluation • A polynomial of degree n is generally written as: where, a0, a1, …, anand x, is a sequence of n + 2 real numbers. Our objective here is to evaluate this general polynomial at the point x.

23. Two Straightforward Solutions • Evaluate each term aixi separately • What is its time complexity? • Compute xi by multiplying x by the previously computed value xi-1 • How many multiplications, assignments, and additions do we have?

24. Horner’s Rule

25. Horner’s Rule • Induction Hypothesis: Suppose we know how to evaluate the following: • Inductive Reasoning: We evaluate Pn(x) as follows: • Using induction hypothesis, we know how to evaluate Pn-1(x) • Pn(x) = x Pn-1(x) + a0 • What is the base step?

26. Horner’s Algorithm • Horner’s Algorithm Input: A sequence of n+2 real numbers a0, a1, …, an, x Output: Pn(x)=anxn+an-1xn-1+…+a1x+a0 p←an for j← 1 to n do p←x . p + an-j return p • How many multiplications, additions, and assignments do we have?