Basics

1. Basics Prof. Hsin-Mu (Michael) Tsai (蔡欣穆) Department of Computer Science and Information Engineering National Taiwan University

2. 上課! • 醒了沒? :P • Homework 1 is out 5pm today, the due date is two weeks from yesterday (5pm). • Download the question sets from the course website • E-mail your question to dsa1@csie.ntu.edu.tw • Or, come to our office hours. data structure and algorithm I hsin-mu tsai, fall 2010

3. What is an algorithm? • A computable set of steps to achieve a desired result. • All algorithm must satisfy the following criteria: • Input • Output • Definiteness • Finiteness • Effectiveness data structure and algorithm I hsin-mu tsai, fall 2010

4. example • Statement 1:“Is n=2 the largest value of n for which there exist positive integers x, y, and z such that has a solution?” • Statement 2: “Store 5 divided by zero into x and go to statement ㄆ.” • Which criterion do they violate? • Input • Output • Definiteness • Finiteness • Effectiveness data structure and algorithm I hsin-mu tsai, fall 2010

5. What is a data structure? • An organization of information, usually in memory, for better algorithm efficiency. • Or, a way to store and organize data in order to facilitate access and modifications. data structure and algorithm I hsin-mu tsai, fall 2010

6. algorithm specification data structure and algorithm I hsin-mu tsai, fall 2010

7. How do we describe an algorithm? • Human language (English, Chinese, …) • Programming language • A mix of the above 菜瓜布曰: 出系館以後又轉到側門，過馬路以後到對面的Starbucks。 如果人不多的話就幫我買杯拿鐵。 如果人太多的話，就到旁邊的全家買一杯罐裝伯朗咖啡就好。 data structure and algorithm I hsin-mu tsai, fall 2010

8. Example: selection sort • Integers are stored in an array, list. The i-th integer is stored in list[i], 0<i<n. • Goal: Devise a program to sort a set of integers • Solution: From those integers that are currently unsorted, find the smallest and place it next in the sorted list. ㄅ ㄆ 1 1 ㄆ 2 ㄅ 2 1 ㄆ ㄅ data structure and algorithm I hsin-mu tsai, fall 2010

9. Example: selection sort • First attempt: for(i=0; i<n;++i){ Examine list[i] to list[n-1]and suppose that the smallest integer is at list[min]; Interchange list[i]and list[min]; } Task 1 Task 2 data structure and algorithm I hsin-mu tsai, fall 2010

10. Task 2 voidswap(int*x,int*y){ int temp =*x; *x=*y; *y=temp; } Or #define SWAP(x,y,t) ((t)=(x), (x)=(y), (y)=(t)) data structure and algorithm I hsin-mu tsai, fall 2010

11. min=i; for(j=i;j<n;++j) if(list[j]<list[min]) min=j; Task 1 Finally, see program 1.4 on p. 11 for a complete program of the selection sort we just develop. data structure and algorithm I hsin-mu tsai, fall 2010

12. How do we prove that it is correct? • Theorem 1.1: Function sort(list,n) correctly sorts a set of n>=1 integers. The result remains in list[0], …, list[n-1] such that . • Proof: When the outer for loop completes its iteration for i=q, we have . Further, on subsequent iterations, i>q and list[0] through list[q] are unchanged. Hence following the last iteration of the outer for loop (i.e., i=n-2), we have . data structure and algorithm I hsin-mu tsai, fall 2010

13. Example: binary search • Input: • searchnum: the number to be found • list: sorted array, size n, and • Output: • -1 if searchnum is not found in list • the index of searchnum in list if searchnum is found data structure and algorithm I hsin-mu tsai, fall 2010

14. Example: searchnum=13; 0 1 2 3 4 5 6 7 8 9 10 11 1 3 4 4 7 11 13 13 13 18 19 6 return middle; data structure and algorithm I hsin-mu tsai, fall 2010

15. Example: searchnum=13; 0 1 2 3 4 5 6 7 8 9 10 11 1 3 4 4 7 11 13 13 13 18 19 6 right left middle middle=(left+right)/2; left=middle+1; data structure and algorithm I hsin-mu tsai, fall 2010

16. Example: searchnum=5; 0 1 2 3 4 5 6 7 8 9 10 11 1 3 4 4 7 11 13 13 13 18 19 6 return -1; data structure and algorithm I hsin-mu tsai, fall 2010

17. intbinsearch(int list[], intsearchnum, int left, int right) { int middle; while(left<=right) { middle=(left+right)/2; switch(COMPARE(list[middle], searchnum)) { case -1: left=middle+1; break; case 0: return middle; case 1: right=middle-1; } } return -1; } data structure and algorithm I hsin-mu tsai, fall 2010

18. Recursive algorithms • What does “recursive” mean? • A function which calls itself (direct recursion), or • a function which calls other functions which call the calling function again (indirect recursion). • Any function that we can write using assignment, if-else, and while statements can be written recursively. data structure and algorithm I hsin-mu tsai, fall 2010

19. Recursive algorithms • Why do we want to use recursive functions (or algorithms)? • It allow us to express an otherwise complex process in very clear terms. • Often the recursive function is easier to understand than its iterative counterpart. data structure and algorithm I hsin-mu tsai, fall 2010

20. The story of recursive function ㄅ is too much for me. I’ll partition it, do part 1, and clone two copies of myself to do the rest. Hey man. Do the work ㄅ part 2. Function blah (clone 1) Yo. Do the work ㄅ. Function blah Hi. Do the work ㄅ part 3. Function blah (clone 2) I’m ㄅ. The work to be done. 2 1 3 ㄅ data structure and algorithm I hsin-mu tsai, fall 2010

21. example 1 • Function nchoosek (int n, int m) • How do we implement function nchoosek (int n, int m) recursively? • Boundary case • Do some work • Delegation data structure and algorithm I hsin-mu tsai, fall 2010

22. example 2 1. Termination condition intbinsearch(int list[], intsearchnum, int left, int right){ int middle; if (left<=right) { middle=(left+right)/2; switch (COMPARE(list[middle], searchnum)) { case -1: return binsearch(list, searchnum, middle+1, right); case 0: return middle; case 1: return binsearch(list, searchnum, left, middle-1); } } } 2. Recursive calls data structure and algorithm I hsin-mu tsai, fall 2010

23. example 3 • Output all permutations • Input: {a,b,c} • Output: (a,b,c),(a,c,b),(b,a,c),(b,c,a),(c,a,b),(c,b,a) • How do we write this program recursively? • Example: input={a,b,c,d}. Output = • a followed by all permutations of {b,c,d} • b followed by all permutations of {a,c,d} • c followed by all permutations of {a,b,d} • d followed by all permutations of {a,b,c} • “all permutations of {…}”  recursive calls! • Homework: read and understand program 1.9 data structure and algorithm I hsin-mu tsai, fall 2010

24. data abstraction data structure and algorithm I hsin-mu tsai, fall 2010

25. data type • What is a data type?A data type is a collection of objects and a set of operations that act on those objects. • Data types in C • char, int, float, long, double(unsigned, signed, …) • Array • Struct struct { int a; int b; char str[16]; int * iptr; } blah; intiarray[16]; data structure and algorithm I hsin-mu tsai, fall 2010

26. data type • Operations • +, -, *, /, %, == • =, +=, -= • ? : • sizeof, - (negative) • giligulu(int a, int b) data structure and algorithm I hsin-mu tsai, fall 2010

27. data type • Representation of the objects of the data type • Example: char • char blah=‘A’; (‘A’: ASCII code is 65(dec), or 0x41 (hex)) Q: The maximum number which can be represented with a char variable? A: 255. • Homework: How about char, int, long, float? 1 byte of memory: 01000001 data structure and algorithm I hsin-mu tsai, fall 2010

28. data type • Q: Do we need to know about the representation of a data type? • A: It depends. • We can usually write algorithms which are more efficient if we make use of the knowledge about the representation. • However!!! If the representation of the data type is changed, the program needs to be verified, revised, or completelyre-written. 囧 • Porting to a different platform (x86, ARM, embedded system, …) • Changing the specification of a program or a library(ex. 16-bit int 32-bit long) data structure and algorithm I hsin-mu tsai, fall 2010

29. Abstract data type • An “abstract data type” (ADT) is a data type that is organized in such a way that the specification of the objects and the specification of the operations on the objects is separated from the representation of the objects and the implementation of the operations. User Specification (Interface) Representation and Implementation data structure and algorithm I hsin-mu tsai, fall 2010

30. Abstract data type • Specifications: • Name of the function and the description of what the function does • The type of the argument(s) • The type of the result(s) (return value) • Function categories: • Creator/constructor • Transformers • Observer/reporter • Homework: Read Example 1.5 on p. 20 data structure and algorithm I hsin-mu tsai, fall 2010

31. performance analysis data structure and algorithm I hsin-mu tsai, fall 2010

32. how do you make evaluative judgment about program? • Does the program meet the original specifications of the task? • Does it work correctly? • Does the program contain documentation that shows how to use it and how it works? • Does the program effectively use functions to create logical units? • Is the program’s code readable? data structure and algorithm I hsin-mu tsai, fall 2010

33. how do you make evaluative judgment about program? • Does the program efficiently use primary and secondary storage? Primary storage: memory? Secondary storage: Hard drive, flash disk, etc. • Is the program’s running time acceptable for the task? Example: Network intrusion detection system (1) 99.8% detection rate, 50 minutes to finish analysis of a minute of traffic (2) 85% detection rate, 20 seconds to finish analysis of a minute of traffic data structure and algorithm I hsin-mu tsai, fall 2010

34. how do you make evaluative judgment about program? • Does the program efficiently use primary and secondary storage? • Is the program’s running time acceptable for the task? Space complexity Time complexity data structure and algorithm I hsin-mu tsai, fall 2010

35. space & time complexity • Space complexity of a program: The amount of memory that it needs to run to completion. • Time complexity of a program: The amount of computer time that it needs to run to completion. data structure and algorithm I hsin-mu tsai, fall 2010

36. space complexity • The space needed by a program: • Fixed space requirements • Do not depend on the number and size of the inputs/outputs. • Variable space requirements • Depends on some characteristics of the particular instance, I, of the problem being solved, P. • Depends on the additional space required when using recursion. data structure and algorithm I hsin-mu tsai, fall 2010

37. example float abc(float a, float b, float c) { return a+b+b*c+(a+b-c)/(a+b)+4.00; } • (variable space requirements) data structure and algorithm I hsin-mu tsai, fall 2010

38. example float sum(float list[], int n) { float tempsum=0; int i; for(i=0;i<n;++i) tempsum+=list[i]; return tempsum; } • (variable space requirements) data structure and algorithm I hsin-mu tsai, fall 2010

39. example function rsum(float list[], int n) { if (n) return rsum(list,n-1)+list[n-1]; return 0; } • (variable space requirements) data structure and algorithm I hsin-mu tsai, fall 2010

40. Probably not for this problem. Is it good to use?(recursion) data structure and algorithm I hsin-mu tsai, fall 2010

41. time complexity • Time taken by a program, P: • Compile time • Run (execution) time • Compile time: fixed. (Exceptions?) • C (and other compiled programming languages) One compilation  Multiple executions • Run time: • Depends on instance characteristics (input) data structure and algorithm I hsin-mu tsai, fall 2010

42. how to we determine ? • Method 1: • Count all instances of all operations in the program. Is it good to use?(method 1) data structure and algorithm I hsin-mu tsai, fall 2010

43. how to we determine ? • Method 2: • Separate the program into program steps whose execution time is independent of instance characteristics • Count the number of steps • Different program steps have different amounts of computing • a=2; • a=2*b+3*c/d-e+f/g/a/b/c; • Count the number of steps needs to solve a particular instance data structure and algorithm I hsin-mu tsai, fall 2010

44. example 1 float sum(float list[], int n) { float tempsum = 0; count++; //assignment int i; for (i=0;i<n;i++) { count++; // for loop tempsum+=list[i]; count++; //assignment } count++; //last iteration of for count++; return tempsum; } data structure and algorithm I hsin-mu tsai, fall 2010

45. example 1 float sum(float list[], int n) { float tempsum = 0; int i; for (i=0;i<n;i++) { count+=2; } count+=3; return tempsum; } count = 2n+3 (steps) data structure and algorithm I hsin-mu tsai, fall 2010

46. example 2 float rsum(float list[], int n) { count++; // if if (n) { count++; //return return rsum(list,n-1)+list[n-1]; } count++; //return return list[0]; } 2n+2 < 2n+3. Does this mean rsum is faster than sum ? No! count = 2n+2 (steps) data structure and algorithm I hsin-mu tsai, fall 2010

47. example 3 void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { inti,j; for(i=0; i<rows; ++i) for(j=0;j<cols;++j) c[i][j]=a[i][j]+b[i][j]; } data structure and algorithm I hsin-mu tsai, fall 2010

48. example 3 void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { inti,j; for(i=0; i<rows; ++i) { count++; for(j=0;j<cols;++j) { count++; c[i][j]=a[i][j]+b[i][j]; count++; } count++; } count++; } count=((2*cols)+2)*rows+1 =2cols*rows+2*rows+1 data structure and algorithm I hsin-mu tsai, fall 2010

49. example 1 revisited (tabular method) Please practice using the method for the other two examples. data structure and algorithm I hsin-mu tsai, fall 2010

50. summary • Instance characteristics? • The number of inputs • The number of outputs • The magnitude of the inputs • We then calculate the number of steps which is independent of the characteristics we selected. data structure and algorithm I hsin-mu tsai, fall 2010