ICS 353: Design and Analysis of Algorithms

King Fahd University of Petroleum & Minerals Information & Computer Science Department ICS 353: Design and Analysis of Algorithms Induction

Reading Assignment • M. Alsuwaiyel, Introduction to Algorithms: Design Techniques and Analysis, World Scientific Publishing Co., Inc. 1999. • Chapter 5 Sections 1-5.

Outline • Introduction • Recursive Selection Sort and Recursive Insertion Sort • Radix Sort • Integer Exponentiation • Evaluating Polynomials (Horner's Rule)

Induction • Main objective: Using induction or inductive reasoning as a “recursive” algorithm design technique • Why recursion • Concise algorithms for complex problems can be developed • The algorithms are easy to comprehend • Development time • Proof of correctness of the designed algorithm is usually simple.

How Does Induction Produce Recursive Algorithms? • Having a problem with input size n, it is sometimes easier to start with a solution to the problem with a smaller size and extend the solution to include the input size n.

Problems to Discuss • Selection sort • Insertion sort • Radix sort • Integer exponentiation • Evaluating polynomials

Recursive Selection Sort • Induction Hypothesis: We know how to sort A[2..n] • Inductive Reasoning: We sort A[1..n] as follows • Find the minimum A[j], 1  j  n • Swap(A[1],A[j]) • Sort A[2..n] // induction hypothesis • What is the base case? • What is the initial call to the sorting algorithm? • This kind of recursion is called tail-recursion.

Algorithm SELECTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in nondecreasing order. Procedure sort(i) {Sort A[i..n]} • if i < n then Begin 2. k := i; 3. for j := i + 1 to n 4. if A[j] < A[k] then k := j; 5. if k != i then Swap(A[i], A[k]); 6. sort(i + 1); End

Complexity Analysis of Recursive Selection Sort • What is the recurrence relation that describes the number of comparisons carried out by the algorithm? • What is the solution to the recurrence?

Recursive Insertion Sort • Induction Hypothesis: We know how to sort A[1..n-1] • Inductive Reasoning: We sort A[1..n] as follows: • Sort A[1..n-1] // induction hypothesis • Insert A[n] in its proper position in A[1..n] • This may involve copying zero or more elements one position ahead in order to insert A[n]

Algorithm INSERTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in non-decreasing order. Procedure sort(i) {Sort A[1..i]} 1. if i >1 then 2. sort(i – 1) {Recursive Call} 3. x := A[i] 4. j := i – 1 5. while j > 0 and A[j] > x 6. A[j + 1] := A[j] 7. j := j – 1 end while 8. A[j + 1] := x end if

Complexity Analysis of Recursive Insertion Sort • Unlike selection sort, the analysis has to differentiate between the best case and the worst case. Why? • Recurrence of the best case: • Solution to the best case: • Recurrence of the worst case: • Solution to the worst case:

Radix Sort • Treats keys as numbers in a particular radix or base. • This is NOT an elements comparison-based sorting algorithm • It only works under the following assumption:

Radix Sort Derivation • Assume that our keys are of the form dkdk – 1 … d1 • Induction Hypothesis: Suppose that we know how to sort numbers lexicographically according to their least k – 1 digits, dk – 1, dk – 2, …, d1 , k > 1. • Inductive reasoning: We sort the numbers based on their first k digits as follows: • Use induction hypothesis to sort the numbers based on their 1..k-1 digits. • Sort the numbers based on their corresponding kth digits

Description of Radix Sort Algorithm • Distribute the input numbers into 10 sublists L0, L1,…, L9 according to the least significant digit, d1. • Form a new list, we denote by main list, by removing the input from the 10 lists, starting from L0, L1, …etc. in order. • Distribute the main list into the 10 sublists according to the second digit, d2, and then repeat step 2. • Repeat step 3 for the rest of the digits in order of d3, d4, …, dk.

Example • Sort the following numbers using radix sort: 467 1247 3275 6792 9187 9134 4675 39 7 6644

Radix Sort Algorithm Algorithm RADIXSORT Input: A linked list of numbers L = {a1, a2,…,an} and k, the maximum number of digits. Output: L sorted in nondecreasing order. 1. for j ←1 to k do 2. Prepare 10 empty lists L0, L1, …, L9. 3. while L is not empty 4. a ← next element in L. Delete a from L. 5. i ← jth digit in a. Append a to list Li. 6. end while 7. L ← L0 8. for i ←1 to 9 do 9. L ← L . Li {append list Li to L} 10. end for 11. end for 12. return L

Time and Space Complexity Analysis • Time Complexity • Space Complexity

Integer Exponentiation • What is the straightforward algorithm to raise x to the power n? • Such an algorithm is exponential in input size!!!!!!!!

Algorithm Derivation • Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2 • Induction Hypothesis: Assume that we know how to compute xn/2. • Inductive Reasoning: We raise x to the power n as follows: • Use induction hypothesis to compute y=xn/2 . xn/2 • If n is even, xn = y • If n is odd, xn = x . y

ALGORITHM EXPRECURSIVE Input: A real number x and a nonnegative integer n. Output:xn. /* the first call to the algorithm is power(x,n) */ Procedure power(x, m) {Compute xm} 1. if m=0then y← 1 2. else 3. y←power(x,m/2) 4. y←y2 5. if m is odd then y←x . y end if 7. return y

Complexity Analysis of the Algorithm • When n = 0, the number of multiplications is Therefore, T(0)= • Best Case Analysis: • When does the best case occur? • What is the recurrence equation? Solution? • Worst Case Analysis: • When does the worst case occur? • What is the recurrence equation? Solution?

Iterative Exponentiation Algorithm • Assume we want to raise x to the power n where n is an integer that is represented in binary as (bkbk-1…b1b0)2 • Starting with y = 1, scan the binary digits of the number n from left to right (j = k down to 0): • If bj = 0, square y • If bj = 1, square y and multiply it by x • Example: Compute 212 using the iterative exponentiation algorithm

Polynomial Evaluation • A polynomial of degree n is generally written as: where, a0, a1, …, anand x, is a sequence of n + 2 real numbers. Our objective here is to evaluate this general polynomial at the point x.

Two Straightforward Solutions • Evaluate each term aixi separately • What is its time complexity? • Compute xi by multiplying x by the previously computed value xi-1 • How many multiplications, assignments, and additions do we have?

Horner’s Rule

Horner’s Rule • Induction Hypothesis: Suppose we know how to evaluate the following: • Inductive Reasoning: We evaluate Pn(x) as follows: • Using induction hypothesis, we know how to evaluate Pn-1(x) • Pn(x) = x Pn-1(x) + a0 • What is the base step?

Horner’s Algorithm • Horner’s Algorithm Input: A sequence of n+2 real numbers a0, a1, …, an, x Output: Pn(x)=anxn+an-1xn-1+…+a1x+a0 p←an for j← 1 to n do p←x . p + an-j return p • How many multiplications, additions, and assignments do we have?

ICS 353: Design and Analysis of Algorithms