Introduction

1. Introduction Thinking about Algorithms Abstractly Credits: Steve Rudich, Jeff Edmond, Ping Xuan

2. So you want to be a bioinformatician /mathematician /computer scientist? • What is an Algorithm ? • Grade school revisited: How to multiply two numbers

3. So you want to be a computer scientist?

4. Is your goal to be a mundane programmer?

5. Or a great leader and thinker?

6. Original Thinking

7. Boss assigns task: • Given today’s prices of pork, grain, sawdust, … • Given constraints on what constitutes a hotdog. • Make the cheapest hotdog. Everyday industry asks these questions.

8. Your answer: • Um? Tell me what to code. With more suffocated software engineering systems,the demand for mundane programmers will diminish.

9. Your answer: • I learned this great algorithm that will work. Soon all known algorithms will be available in libraries.

10. Your answer: • I can develop a new algorithm for you. Great thinkers will always be needed.

11. The future belongs to the computer scientist who has • Content: An up to date grasp of fundamental problems and solutions • Method: Principles and techniques to solve the vast array of unfamiliar problems that arise in a rapidly changing field

12. Course Content • A list of algoirthms. • Learn their code. • Trace them until you are convenced that they work. • Impliment them. class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--;} a[j] = B; }}

13. Course Content • A survey of algorithmic design techniques. • Abstract thinking. • How to develop new algorithms for any problem that may arise.

14. Study: • Many experienced programmers were asked to code up binary search.

15. Study: • Many experienced programmers were asked to code up binary search. 80% got it wrong Good thing is was not for a nuclear power plant.

16. What did they lack?

17. What did they lack? • Formal proof methods?

18. What did they lack? • Formal proof methods? Yes, likely Industry is starting to realize that formal methods are important. But even without formal methods …. ?

19. What did they lack? • Fundamental understanding of the algorithmic design techniques. • Abstract thinking.

20. Course Content Notations, analogies, and abstractions for developing, thinking about, and describing algorithms

21. Time Complexity t(n) = Q(n2) Recurrence Relations T(n) = a T(n/b) + f(n) You will see some Math …

22. What is an Algorithm? • A step-by-step description of procedures performing certain task. • Example: Sorting. Given a list of numbers, put them in increasing (non-decreasing) order. • Properties: Generality, Termination, Correctness, Feasibility. • Feasibility  Analysis of Algorithm Complexity theory

23. Analysis of Algorithm • Running time analysis • Input size N • Running time is a function of N: T(N) • Worst case, average case, … , etc • Time measured by number of computer operations • Each basic operation (add, load, write, etc) count as 1 • Actual clock time differs by a constant factor • Usually very complex • The use of asymptotic bounds • To study the rate of growth of T(N) compared to simpler functions f(N), such as N, N2, log(N), etc • A constant factor can be ignored

24. Some Definitions: Big O Notation • T(N) = O( f(N) ) • Exists constant c and n0 such that when N>n0, T(N) <= c * f(N) • Asymptotic Upper bound c f(N) T(N) n0 N

25. Some Definitions: Big Omega • T(N) = (g(N)) • Exists constant c and n0 such that when N>n0, T(N) >= c * g(N) • Asymptotic Lower bound T(N) c g(N) n0 N

26. Some Definitions: Big Theta • T(N) = ( h(N) ) • if and only if T(N)=O(h(N)) and T(N)= (h(N)) • tight bound c1 h(N) T(N) c2 h(N) N

27. c p(N) T(N) N Some Definitions: Little “o” • T(N) = o(p(N)) if lim N  ∞ = o. E.g. log(N) = o(N).

28. Example: Insertion Sort Algorithm • class InsertionSortAlgorithm extends SortAlgorithm { • void sort(int a[ ]) throws Exception { • for (int i = 1; i < a.length; i++) { • int j = i; • int B = a[i]; • while ((j > 0) && (a[j-1] > B)) { • a[j] = a[j-1]; • j--;} • a[j] = B; • }}

29. 0 i-1 i T+1 i i 9 km 5 km Iterative Algorithms <preCond> codeA loop exit when <exit Cond> codeB codeC <postCond> One step at a time Relay Race Code

30. 52 88 14 14,23,25,30,31,52,62,79,88,98 31 98 25 30 23 62 79 Problem Specification • Precondition: The input is a list of n values with the same value possibly repeated. • Post condition: The output is a list consisting of the same n values in non-decreasing order.

31. 30 14 25 23,31,52,88 98 79 Define Step • Select arbitrary element from side. • Insert it where it belongs. 30 14 62 25 98 79 23,31,52,62,88

32. Exit 79 km 75 km 6 elements to school 5 elements to school 30 14 25 23,31,52,88 98 79 Making progress while Maintaining the loop invariant 30 14 62 25 98 79 23,31,52,62,88 Sorted sub-list

33. 52 52 88 88 14 14 31 31 62 62 25 25 30 30 Exit Exit 0 km 23 23 98 98 79 79 n elements to school 0 elements to school 14,23,25,30,31,52,62,79,88,98 14,23,25,30,31,52,62,79,88,98 Beginning & Ending

34. 30 14 25 23,31,52,88 98 79 Running Time Inserting an element into a list of size i takes O(i) time in the worst case. Total = 1+2+3+…+n = n(n+1)/2 = (n2) 30 14 62 25 98 79 23,31,52,62,88

35. Explaining Insertion Sort We maintain a subset of elements sorted within alist. The remaining elements are off to the sidesomewhere. Initially,think of the first element in the array as a sorted list of lengthone. One at a time, we take one of the elements that is off to theside and we insert it into the sorted list where itbelongs. This gives a sorted list that is one element longer than itwas before. When the last element has been inserted, the array iscompletely sorted.

36. 23,25,31,52,88 52,23,88,31,25,30,98,62,14,79 23,31,52,88 Insertion Sort The input consists of an array of integers We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have by one for this larger prefix.

37. Insertion Sort Algorithm: Pseudo-Code • Input: an array (or list) a of numbers // data structure: a[0] …a[n-1] • Operations: • Leave the first element (location 0) untouched. • for each element from location 1 on insert it to the appropriate place in the (sorted) sub-array to its left. a 0 1 2 n-1

38. Operations in an Algorithm • Decision Making • branching operation • if … then … else … • Repetition • loops: for each element do { … } • conditional loops: while ( …. ) do { … }

39. Insertion Sort Algorithm: Code • class InsertionSortAlgorithm extends SortAlgorithm { • void sort(int a[ ]) throws Exception { • for (int i = 1; i < a.length; i++) { • int j = i; • int b = a[i]; • while ( (j > 0) && (a[j-1] > b) ) { • a[j] = a[j-1]; • j--;} • a[j] = B; • }}}

40. 2 X 2 = 5 Another Algorithm Grade School Revisited:How To Multiply Two Numbers

41. Complex Numbers • Remember how to multiply 2 complex numbers? • (a+bi)(c+di) = [ac –bd] + [ad + bc] i • Input: a,b,c,d Output: ac-bd, ad+bc • If a real multiplication costs $1 and an addition cost a penny. What is the cheapest way to obtain the output from the input? • Can you do better than $4.02?

42. Gauss’ $3.05 Method:Input: a,b,c,d Output: ac-bd, ad+bc • m1 = ac • m2 = bd • A1 = m1 – m2 = ac-bd • m3 = (a+b)(c+d) = ac + ad + bc + bd • A2 = m3 – m1 – m2 = ad+bc

43. Question: • The Gauss “hack” saves one multiplication out of four. It requires 25% less work. • Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings?

44. Odette Bonzo

45. How to add 2 n-bit numbers. ** ** ** ** ** ** ** ** ** ** ** +

46. How to add 2 n-bit numbers. ** ** ** ** ** ** ** ** ** * ** ** * +

47. How to add 2 n-bit numbers. ** ** ** ** ** ** ** ** * ** * ** * ** * +

48. How to add 2 n-bit numbers. ** ** ** ** ** ** ** * ** * ** * * ** * ** * +

49. How to add 2 n-bit numbers. ** ** ** ** ** ** * ** * ** * * ** * * ** * ** * +

50. How to add 2 n-bit numbers. * * * ** * * ** * * ** * * ** * * ** * *** * * ** * * ** * * ** * * ** * ** * +