Discrete Mathematics 6 th edition, 2005

Discrete Mathematics6th edition, 2005 Chapter 4 Algorithms • Introduction • Examples of Algorithms • Analysis of Algorithms • Recursive Algorithms

4.1 Introduction • Analgorithm has the following characteristics: • Input: the algorithm receives input • Output: the algorithm produces output • Precision: the steps are precisely stated • Determinism: the intermediate results of each step of execution are unique and are determined only by the inputs and the results of the preceding steps • Finiteness: the algorithm terminates; i.e., it stops after finitely many instructions have been executed • Correctness: the output produced by the algorithm is correct; i.e., the algorithm correctly solves the problem • Generality: the algorithm applies to a sets of inputs

Example: Finding the Maximum of Three Numbers • Algorithm to find the largest of three numbers a, b, c. • Input : a, b, c • Output : large (the largest of a, b, and c) • max3(a,b,c) { • large = a • if (b>large) // if b is larger than large, update large • large = b • If (c > large) // if c is larger than large, update large • large = c • return large • }

4.2 Examples of Algorithms • Searching – Text Search Algorithm searches for an occurrence of the pattern p in text t. It returns the smallest index i such that p occurs in t starting at index i. If p does no occur in t, it returns 0. • Input: p (indexed from 1 to m), m, t (indexed 1 to n), n • Output: i text_search(p,m,t,n) { for i=1 to n-m+1 { j=1 // i is the index in t of the first character of the substring // to compare with p, and j is the index in p // the while loop comparesti…ti+m-1 andp1…pm while (ti+j-1 == pj) { j = j+1 if (j>m) return i } } return 0 }

Examples of Algorithms • Sorting – Insertion Sort • This algorithm sorts the sequence s1, …, sn in nondecreading order • Input: s, n • Output: s (sorted) insertion_sort(s,n) { for i=2 to n { val = si// save si so it can be inserted into the corrected place j = i-1 // if val < sj, move sj right to make room for si while (j1  val<sj) { sj+1 = sj j = j-1 } sj+1 = val// insert val } }

Randomized Algorithms • Relaxation the requirements of an algorithm • Finiteness • An operating system never terminates • Determinism • Algorithms for multiprocessor machine or distributed environment are rarely deterministic • Generality or Correctness • Many practical problems are too difficult to be solved efficiently

Randomized Algorithms • Randomized Algorithms • does not require that the intermediate results of each step of execution be uniquely defined • depend only on the inputs and resultsof the proceeding steps • at some points it makes random choice

Randomized Algorithms • major bridge tournaments use computer programs to shuffle the cards • shuffle algorithm • assume that a function rand(i,j) exists • returns a random integer x, i  x  j • Input: a, n • Output: a (shuffled) shuffle(a,n) { for i=1 to n-1 swap(ai, arand(i,n)) }

4.3 Analysis of Algorithms • Useless program for certain type of input • even though derived from a correct algorithm • the time needed to run the program is too great or • the space needed to hold the data is too great • Example • X = a set of n elements some elements are labeled ‘red’, some labeled ‘black’. Find the number of subsets of X that contain at least one red item. • Need to examine 2n subsets. • What if n is very big?

Complexity of algorithms • The time needed to execute an algorithm • is a function of the input • is difficult to obtain an explicit formula • instead of dealing directly with the input, we use parameters that characterize the size of the input • we estimate the time of an algorithm rather than computing its exact time

Complexity: the amount of time and/or space needed to execute the algorithm. Performance parameters of a computer program Computer that is being used to run the program The way the data are represented How the program is translated to machine instructions What kind of computer language is used Complexity of algorithms

Types of complexity • Best-case time • minimum time needed to execute the algorithm for inputs of size n • Worst-case time • maximum time needed to execute the algorithm for inputs of size n • Average-case time • average time needed

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} • f(n) = O(g(n)) • f(n) is of order at mostg(n) • if there exists a positive constant C1 such that |f(n)|  C1|g(n)| for all but finitely many n • f(n) is of order at most g(n) or f(n) is big oh of g(n) • asymptotic upper boundfor f • Example 1 • 60n2 + 5n + 1  60n2 +5n2 + n2 = 66n2 for all n1, • C1 = 66  60n2 + 5n + 1 = O(n2) • Example 2 • 2n + 3 lg n  2n +3n = 5n for all n1, • C1 = 5  2n + 3 lg n = O(n)

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} • f(n) = (g(n)) • f(n) is of order at leastg(n) • if there exists a positive constant C2 such that |f(n)| >C2|g(n)| for all but finitely many n • f(n) is of order at least g(n) or f(n) is omega of g(n) • asymptotic lower boundfor f • Example 1 • 60n2 + 5n + 1  60n2for all n1, • C1 = 60  60n2 + 5n + 1 = (n2) • Example 2 • 2n + 3 lg n  2nfor all n1, • C1 = 2  2n + 3 lg n = (n)

Order of an algorithm Let f and g be functions with domain Z+ = {1, 2, 3,…} • f(n) = (g(n)) • f(n) is or order g(n) if it is O(g(n)) and (g(n)). • f(n) is of order g(n) or f(n) is theta of g(n) • asymptotic tight bound for f • Example 1 • 60n2 + 5n + 1 = O(n2) and 60n2 + 5n + 1 = (n2)  60n2 + 5n + 1 = (n2) • Example 2 • 2n + 3 lg n = O(n) and 2n + 3 lg n = (n)  2n + 3 lg n = (n)

Order of an algorithm Theorem 4.3.4 • Let p(n) = aknk + ak-1nk-1 + … + a1n+ a0 be a polynomial innof degreek, where eachaiis nonnegative. • Thenp(n) = (nk) Proof 1. p(n) = O(nk) Let C1 = ak + ak-1 + … + a1+ a0. Then, for all n, p(n) = aknk + ak-1nk-1 + … + a1n+ a0  (ak + ak-1 + … + a1+ a0) nk =C1 nk Therefore, p(n) = O(nk) 2. p(n) =  (nk) For all n, p(n) = aknk + ak-1nk-1 + … + a1n+ a0  aknk =C2 nk Therefore, p(n) = (nk) 3. Sincep(n) = O(nk) andp(n) = (nk), p(n) = (nk)

Theta Form Name y=2n y 256 128 64 32 16 8 4 2 1 Constant Log log Log Linear n log n Quadratic Cubic Polynomial Exponential Factorial (1) (lg lg n) (lg n) (n) (n lg n) (n2) (n3) (nk), k1 (cn), c1 (n!) y=n2 y=nlgn y=n y=lgn y=1 1 2 3 4 5 6 7 8 9 10 11 12 n Growth of some common functions

Examples 1 + 2 + … + n • 1 + 2 + … + n  n + n + … + n = nn = n2for all n1 1 + 2 + … + n = O(n2) • 1 + 2 + … + n  n/2 + … + (n-1) + n  n/2 + … + n/2 + n/2 = (n+1)/2 n/2  (n/2)(n/2) = n2/4 for all n1 1 + 2 + … + n = (n2) • 1 + 2 + … + n = O(n2) and 1 + 2 + … + n = (n2) 1 + 2 + … + n = (n2)

Examples lg n! • lg n! = lg n + lg (n-1) + … + lg 2 + lg 1 for all n1  lg n + lg n + … + lg n + lg n = n lg nfor all n1 lg n! = O(n lg n) • lg n + lg (n-1) + … + lg 1  lg n + lg (n-1) + … + lg n/2  lg n/2 + … + lg n/2 = (n+1)/2 lg n/2  (n/2) lg (n/2) = (n/2) [lg n – lg 2] = (n/2) [(lg n)/2 + ((lg n)/2 -1)]  (n/2)(lg n)/2 = (n lg n)/4 for all n4 lg n! = (n lg n) • lg n! = O(n lg n) andn! = (n lg n) lg n! = (n lg n)

4.4 Recursive algorithms • A recursive procedure is a procedure that invokes itself • A recursive algorithm is an algorithm that contains a recursive procedure

Factorial of n • Definition • given a positive integer n, factorial of n is defined as the product of n by all numbers less than n and greater than 0. • Notation • n! = n(n-1)(n-2)…321 • Notes • n! = n(n-1)! = n(n-1)(n-2)!, etc.

Factorial of n • Algorithm 4.4.2 • Input: n, an integer greater than or equal to 0 • Output: n! • factorial (n) { • if (n == 0) • return 1 • return n * factorial(n-1) • }

Factorial of n Theorem 4.4.3 Algorithm 4.4.2 returns the value of n!, n0 Proof • 1. Basis Step (n=0) • If n=0, Algorithm 4.4.2 correctly returns the value of 0! (1) • 2. Inductive Step • Assume that Algorithm 4.4.2 correctly returns the value of (n-1)!, n>0. • Suppose that n is input to Algorithm 4.4.2. • Since n0, we proceed to line 4 of the algorithm. • By the inductive assumption, the function computes the value of (n-1)! • At line 4, the function correctly computes the value (n-1)!n = n! • 3. conclusion • Therefore, Algorithm 4.4.2 correctly returns the value of n!, for every integer n0.

Fibonacci sequence • Leonardo Fibonacci (Pisa, Italy, ca. 1170-1250) • Fibonacci sequence f1, f2,… defined recursively as follows: • f1 = 1 • f2 = 2 • fn = fn-1 + fn-2 for n> 3 • First terms of the sequence • 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597,…

Robot Working a2 an-1 a1 a2 = 1+1 a1 = 1 an-2 an= an-1 + an-2

Discrete Mathematics 6 th edition, 2005