461191 Discrete Mathematics Lecture 3: Algorithms, The Integers, and Matrices

1. 461191 Discrete MathematicsLecture 3: Algorithms, The Integers, and Matrices San Ratanasanya CS, KMUTNB Adapted from several sources: Michael P. Frank, University of Florida, University of Nevada, and University of Maryland at Baltimore

2. Today’s Topics • Review • Administrivia • Algorithms • Algorithmic and Problem Complexity • Introduction to Number Theory • Applications of Number Theory • Matrices

3. …from last week… • Sets and Set operations • Set is an unordered collection of objects • Operations on Set: union, intersection, difference, complement, symmetric difference • Venn Diagram and equivalence • Functions • Domain, Codomain, Range • One-to-one, onto, bijection, inverse, and composite functions • Sequences • Represents ordered list of elements • a1,a,2a3,…,an or {an} • It is convenient to describe Sequence using formula • an = 2an-1 • Summations

4. Example: Set • Prove that A  A = U is true A = {x | x  A} A = {x | x  A} A  A = {x | x  A  x  A} A  A = {x | x  U} A  A = U

5. Example: Function • Let f(x) = x2/3. Find f(S) if S = {x | xZ  |x|  3} f(3) = 32/3 = 3, f(2) = 22/3 = 1, f(1) = 12/3 = 0, f(0) = 0  f(S) = {0, 1, 3} • Suppose that g is a function from A to B and f is a function form B to C. Show that if both f and g are one-to-one functions, then fg is also one-to-one. We need to show that x A  y A (x  y  f(g(x))  f(g(y))). Or we need to show that x A  y A (x  y  g(x)  g(y))  x B  y B (x  y  f(x)  f(y)). Given that g is one-to-one, therefore g(x)  g(y) in B. Given that f is one-to-one, therefore f(x)  f(y) in C. We can now conclude that f(g(x))  f(g(y)).

6. Example: Sequence • Find a simple formula for an if the first 10 terms of the sequence {an} are 1, 7, 25, 79, 241, 727, 2185, 6559, 19681, 59047 a1 = 1 an = 3n - 2

7. Example: Summations

8. Administrivia • Homeworks due today • Homework 1.1 – 1.4 • Homework 2 • Programs used in this class cannot be downloaded from class webpage • they are TOO BIG… • There are only links to download those programs on class webpage • Prolog, ISETLW • Python, Scheme • Information on what to download is also on the webpage

9. Algorithms • The foundation of computer programming. • Most generally, an algorithm just means a definite procedure for performing some sort of task. • A computer program is simply a description of an algorithm, in a language precise enough for a computer to understand, requiring only operations that the computer already knows how to do. • We say that a program implements (or “is an implementation of”) its algorithm.

10. Algorithms You Already Know • Grade-school arithmetic algorithms: • How to add any two natural numbers written in decimal on paper, using carries. • Similar: Subtraction using borrowing. • Multiplication & long division. • Your favorite cooking recipe. • How to register for classes at KMUTNB.

11. Programming Languages • Some common programming languages: • Newer: Java, C, C++, C#, Visual Basic, JavaScript, Perl, Tcl, Pascal, many others… • Older: Fortran, Cobol, Lisp, Basic • Assembly languages, for low-level coding. • In this class we will use an informal, Pascal-like “pseudo-code” language. • You should know at least 1 real language!

12. Algorithm Example (English) • Task: Given a sequence {ai}=a1,…,an, aiN, say what its largest element is. • One algorithm for doing this, in English: • Set the value of a temporary variablev (largest element seen so far) to a1’s value. • Look at the next element ai in the sequence. • If ai>v, then re-assign v to the number ai. • Repeat then previous 2 steps until there are no more elements in the sequence, & return v.

13. Executing an Algorithm • When you start up a piece of software, we say the program or its algorithm are being run or executed by the computer. • Given a description of an algorithm, you can also execute it by hand, by working through all of its steps with pencil & paper. • Before ~1940, “computer” meant a person whose job was to execute algorithms!

14. Executing the Max algorithm • Let {ai}=7,12,3,15,8. Find its maximum… • Set v = a1 = 7. • Look at next element: a2 = 12. • Is a2>v? Yes, so change v to 12. • Look at next element: a2 = 3. • Is 3>12? No, leave v alone…. • Is 15>12? Yes, v=15…

15. Algorithm Characteristics Some important general features of algorithms: • Input. Information or data that comes in. • Output. Information or data that goes out. • Definiteness. Algorithm is precisely defined. • Correctness.Outputs correctly relate to inputs. • Finiteness. Won’t take forever to describe or run. • Effectiveness. Individual steps are all do-able. • Generality. Works for many possible inputs. • Efficiency.Takes little time & memory to run.

16. procedurename(argument: type) variable := expression informal statement beginstatementsend {comment} ifcondition then statement [else statement] for variable := initial value to final valuestatement whileconditionstatement procname(arguments) Not defined in book: returnexpression Declaration STATEMENTS Our Pseudocode Language

17. What is Pseudocode • Pseudocode is an intermediatestep between language description, normally English, of the steps of a procedure and a specification of this procedure using an actual programming language. • เป็นการอธิบายโปรแกรมด้วยภาษาที่ใช้กันทั่วไป ด้วยลักษณะที่คล้ายภาษาที่ใช้ในการเขียนโปรแกรม Human Languages Pseudocode Programming Languages

18. What is Program • Program = Algorithm + Data Structure • Each step of a program may make changes to variables • Trace = a sequence of changes in variables as each step of program is executed • Trace is a simple model of program executions • State = variable

19. procedure procname(arg: type) • Declares that the following text defines a procedure named procname that takes inputs (arguments) named arg which are data objects of the type type. • Example:proceduremaximum(L: list of integers) [statements defining maximum…]

20. variable := expression • An assignment statement evaluates the expression expression, then reassigns the variable variable to the value that results. • Example assignment statement:v := 3x+7 (If x is 2, changes v to 13.) • In pseudocode (but not real code), the expression might be informally stated: • x := the largest integer in the list L

21. Informal statement • Sometimes we may write a statement as an informal English imperative, if the meaning is still clear and precise: e.g.,“swap x and y” • Keep in mind that real programming languages never allow this. • When we ask for an algorithm to do so-and-so, writing “Do so-and-so” isn’t enough! • Break down algorithm into detailed steps.

22. Groups a sequence of statements together:beginstatement 1statement 2 …statement nend Allows the sequence to be used just like a single statement. Might be used: After a procedure declaration. In an if statement after then or else. In the body of a for or while loop. beginstatementsend Curly braces {} are used insteadin many languages.

23. {comment} • Not executed (does nothing). • Natural-language text explaining some aspect of the procedure to human readers. • Also called a remark in some real programming languages, e.g. BASIC. • Example, might appear in a max program: • {Note that v is the largest integer seen so far.}

24. Conditional Constructions • ifconditionthenstatement 1 • elsestatement 2 ifconditionthenstatement • ifconditionthen • begin • Block of statements • end • ifcondition 1thenstatement 1 • else ifcondition 2 thenstatement 2 • else ifcondition 2 thenstatement 2 • ……………………………………………………… • else ifcondition n thenstatement n • else statement n+1 • ifconditionthen • begin • statement 1 • end • else • begin • statement 2 • end

25. ifconditionthenstatement • Evaluate the propositional expression condition. • If the resulting truth value is True, then execute the statement statement; • otherwise, just skip on ahead to the next statement after the if statement. • Variant: ifcondthenstmt1elsestmt2 • Like before, but iff truth value is False, executes stmt2.

26. Loop Constructions for variable := initial value to final value statement for all element with certain property • for variable := initial value to final value • begin • Block of statements • end whilecondition statement whilecondition begin Block of statements 1 end sum := 0 whilen > 0 begin sum := sum + n n := n - 1 end sum := 0 fori := 1ton sum := sum + i

27. whileconditionstatement • Evaluate the propositional (Boolean) expression condition. • If the resulting value is True, then execute statement. • Continue repeating the above two actions over and over until finally the condition evaluates to False; then proceed to the next statement.

28. whileconditionstatement • Also equivalent to infinite nested ifs, like so: if conditionbeginstatement if conditionbeginstatement …(continue infinite nested if’s)endend

29. forvar := initial to finalstmt • Initial is an integer expression. • Final is another integer expression. • Semantics: Repeatedly execute stmt, first with variable var := initial, then with var := initial+1, then with var := initial+2, etc., then finally with var := final. • Question: What happens if stmt changes the value of var, or the value that initial or final evaluates to?

30. forvar := initial to finalstmt • For can be exactly defined in terms of while, like so: beginvar := initialwhilevar finalbeginstmtvar:=var + 1endend

31. procedure(argument) • A procedure call statement invokes the named procedure, giving it as its input the value of the argument expression. • Various real programming languages refer to procedures as functions (since the procedure call notation works similarly to function application f(x)), or as subroutines, subprograms, or methods.

32. Nested • Block of statements • Conditional • Loop • Procedure – a call to procedure within procedure Procedurestatistic (L:list of integers) begin max(L) min(L) average(L) ……….. end

33. Max procedure in pseudocode proceduremax(a1, a2, …, an: integers) v := a1{largest element so far} fori := 2 ton {go thru rest of elems} ifai > vthen v := ai {found bigger?} {at this point v’s value is the same as the largest integer in the list} returnv

34. Inventing an Algorithm • Requires a lot of creativity and intuition • Like writing proofs. • Unfortunately, we can’t give you an algorithm for inventing algorithms. • Just look at lots of examples… • And practice (preferably, on a computer) • And look at more examples… • And practice some more… etc., etc.

35. Algorithm-Inventing Example • Suppose we ask you to write an algorithm to compute the predicate: IsPrime:N→{T,F} • Computes whether a given natural number is a prime number. • First, start with a correct predicate-logic definition of the desired function: n: IsPrime(n)  ¬1<d<n: d|n Means d divides nevenly (without remainder)

36. IsPrime example, cont. • Notice that the negated exponential can be rewritten as a universal: ¬1<d<n: d|n  1<d<n: d | n 2≤ d ≤ n−1: d | n • This universal can then be translated directly into a corresponding for loop: ford := 2 to n−1 { Try all potential divisors >1 & <n } ifd|nthenreturn F{ n has divisor d; not prime }return T{ no divisors were found; n must be prime} Means d does notdivide n evenly (the remainder is ≠0)

37. Optimizing IsPrime • The IsPrime algorithm can be further optimized: ford := 2 to n1/2ifd|n then return Freturn T • This works because of this theorem: If n has any (integer) divisors, it must have one less than n1/2. Proof: Suppose n’s smallest divisor >1 is a, and let b :≡n/a. Then n = ab, but if a > n1/2 then b > n1/2 (since a is n’s smallest divisor) and so n = ab > (n1/2)2 = n, an absurdity. Note smaller range of search. Further optimizations are possible: - E.g., only try divisors that are primes less than n1/2.

38. Another example task • Problem of searching an ordered list. • Given a list L of n elements that are sorted into a definite order (e.g., numeric, alphabetical), • And given a particular element x, • Determine whether x appears in the list, • and if so, return its index (position) in the list. • Problem occurs often in many contexts. • Let’s find an efficient algorithm!

39. Search alg. #1: Linear Search procedurelinear search(x: integer, a1, a2, …, an: distinct integers)i := 1 {start at beginning of list}while (i n  x  ai) {not done, not found}i:=i + 1 {go to the next position}ifi n then location:=i{it was found}elselocation:= 0 {it wasn’t found}return location {index or 0 if not found}

40. Search alg. #2: Binary Search • Basic idea: On each step, look at the middle element of the remaining list to eliminate half of it, and quickly zero in on the desired element. <x <x <x >x

41. Search alg. #2: Binary Search procedurebinary search(x:integer, a1, a2, …, an: distinct integers)i := 1 {left endpoint of search interval}j := n{right endpoint of search interval}while i<j begin{while interval has >1 item}m := (i+j)/2 {midpoint}ifx>amtheni := m+1 else j := mendifx = aithenlocation:=ielselocation:= 0returnlocation

42. center element center element binary search for the letter ‘j’ search interval a c d f g h j l m o p r s u v x z search interval a c d f g h j l m o p r s u v x z

43. center element center element binary search for the letter ‘j’ search interval a c d f g h j l m o p r s u v x z search interval a c d f g h j l m o p r s u v x z

44. center element binary search for the letter ‘j’ search interval a c d f g h j l m o p r s u v x z found !

45. Practice exercises • Devise an algorithm that finds the sum of all the integers in a list. [2 min] • proceduresum(a1, a2, …, an: integers)s := 0 {sum of elems so far}fori := 1 ton {go thru all elems}s := s + ai {add current item}{at this point s is the sum of all items}returns

46. Sorting Algorithms • Sorting is a common operation in many applications. • E.g. spreadsheets and databases • It is also widely used as a subroutine in other data-processing algorithms. • Two sorting algorithms shown in textbook: • Bubble sort • Insertion sort However, these are notvery efficient, and you shouldnot use them on large data sets! We’ll see some more efficient algorithms later in the course.

47. 30 31 1 32 33 2 34 3 30 1 31 32 2 33 3 34 1 30 31 2 32 3 33 34 1 30 2 31 3 32 33 34 1 2 30 3 31 32 33 34 1 2 3 30 31 32 33 34 Bubble Sort • Smallest elements “float” up to the top of the list, like bubbles in a container of liquid.

48. Insertion Sort Algorithm • English description of algorithm: • For each item in the input list, • “Insert” it into the correct place in the sorted output list generated so far. Like so: • Use linear or binary search to find the location where the new item should be inserted. • Then, shift the items from that position onwards down by one position. • Put the new item in the hole remaining.

49. Pseudocodes Procedurebubblesort(a1,…,an: real numbers with n  2) for i := 1 to n-1 for j := 1 to n-1 if aj > aj+1then interchange aj and aj+1 {a1,…,an isin increasing order} Procedureinsertionsort(a1,…,an: real numbers with n  2) for j := 2 to n begin i := 1 while aj > ai i := i +1 m : = aj for k := 0 to j – i – 1 aj-k := aj-k-1 ai := am end {a1, a2,…,an are sorted}

50. Greedy Algorithms • Any algorithms selecting the ‘best’ choice at each step are called Greedy • Can solve optimization problem Procedurechange(c1, c2,…,cr: values of denomination of coins, where c1 > c2 >… > cr; n: a positive integer) for i := 1 to r while n  ci begin add a coin with value ci to the change n := n - ci end