Data Structures and Algorithms: Course Overview and Objectives

1. Data Structures and Algorithms Lecture 1

2. Course objectives: • Assess how the choice of data structures and algorithm design methods impacts the performance of programs. • Choose the appropriate data structure and algorithm design method for a specified application. • Solve problems using data structures such as linear lists, stacks, queues, hash tables,…. • Solve problems using algorithm design methods such as the greedy method, divide and and conquer.

3. Agenda • Administrative • Course objective and outline

4. Introduction • What is Algorithm? • a clearly specified set of simple instructions to be followed to solve a problem • Takes a set of values, as input and • produces a value, or set of values, as output • May be specified • In English • As a computer program • As a pseudo-code • Data structures • Methods of organizing data • Program = algorithms + data structures

5. Objectives • Analysis of algorithms • The theoretical study of the computer-program performance and resource usage. • Design of algorithms • data structures techniques

6. Expectation • Assumed Knowledge • Some experience in Java or other similar OO language • BEFORE each lecture • Download and printout the lecture slides • Read the related chapter (s) and lectures notes • Make sure you keep up with the progress • Consult course staff EARLY enough for difficulties and problems!

7. Our Textbook • Anany Levitin, Introduction to The Design & Analysis of Algorithms, 2rd edition.

8. Rule of Conducts • Okay to discuss ideas and problem approaches • All work must be your own creation • Turn off your mobile phone during the class • Be on time

9. Questions?

10. What is an algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. problem algorithm “computer” input output 9

11. Historical Perspective Muhammad ibn Musa al-Khwarizmi – 9th century mathematician www.lib.virginia.edu/science/parshall/khwariz.html 10

12. Example of computational problem: sorting Statement of problem: Input: A sequence of n numbers <a1, a2, …, an> Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤a´j whenever i < j Instance: The sequence <5, 3, 2, 8, 3> Algorithms: Selection sort Insertion sort Merge sort (many others) 11

13. Selection Sort Input: array a[1],…,a[n] Output: array a sorted in non-decreasing order Algorithm: for i=1 to n swap a[i] with smallest of a[i],…a[n] 12

14. Analysis of Algorithms How good is the algorithm? Correctness Time efficiency Space efficiency Does there exist a better algorithm? Lower bounds Optimality 13

15. What is an algorithm? Recipe, process, method, technique, procedure, routine,… with following requirements: Finiteness terminates after a finite number of steps Definiteness rigorously and unambiguously specified Input valid inputs are clearly specified Output can be proved to produce the correct output given a valid input Effectiveness steps are sufficiently simple and basic 14

16. Why study algorithms? Theoretical importance the core of computer science Practical importance A practitioner’s toolkit of known algorithms Framework for designing and analyzing algorithms for new problems 15

17. Analysis of algorithms How good is the algorithm? time efficiency space efficiency Does there exist a better algorithm? lower bounds optimality 16

18. Important problem types sorting searching string processing graph problems combinatorial problems geometric problems numerical problems 17

19. Fundamental data structures list array linked list string stack queue priority queue • graph • tree • set and dictionary 18

20. Questions? 19

21. Chapter Fundamentalsof theAnalysis ofAlgorithmEfficiency (1) 20

22. Introduction What is Algorithm? a clearly specified set of simple instructions to be followed to solve a problem Takes a set of values, as input and produces a value, or set of values, as output May be specified In English As a computer program As a pseudo-code Data structures Methods of organizing data Program = algorithms + data structures

23. Introduction Why need algorithm analysis ? writing a working program is not good enough The program may be inefficient! If the program is run on a large data set, then the running time becomes an issue

24. Algorithm Analysis… Factors affecting the running time computer compiler algorithm used input to the algorithm The content of the input affects the running time typically, the input size (number of items in the input) is the main consideration E.g. sorting problem  the number of items to be sorted E.g. multiply two matrices together  the total number of elements in the two matrices Machine model assumed Instructions are executed one after another, with no concurrent operations  Not parallel computers

25. Example Calculate Lines 1 and 4 count for one unit each Line 3: executed N times, each time four units Line 2: (For… 1 for initialization, N+1 for all the tests, N for all the increments) total 2N + 2 total cost: 6N + 4 1 2N+2 4N 1 1 2 3 4

26. Example: Selection Problem Given a list of N numbers, determine the kth largest, where k  N. Algorithm 1: (1)   Read N numbers into an array (2)   Sort the array in decreasing order by some simple algorithm (3)   Return the element in position k

27. Example: Selection Problem… Algorithm 2: (1)   Read the first k elements into an array and sort them in decreasing order (2)   Each remaining element is read one by one If smaller than the kth element, then it is ignored Otherwise, it is placed in its correct spot in the array, bumping one element out of the array. (3)   The element in the kth position is returned as the answer.

28. Example: Selection Problem… Which algorithm is better when N =100 and k = 100? N =100 and k = 1? What happens when N = 1,000,000 and k = 500,000? There exist better algorithms

29. Theoretical analysis of time efficiency input size running time Number of times basic operation is executed execution time for basic operation 28 Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size Basic operation: the operation that contributes most towards the running time of the algorithm T(n) ≈copC(n)

30. Best-case,average-case,worst-case 29 For some algorithms efficiency depends on form of input: • Worst case: Cworst(n) – maximum over inputs of size n • Best case: Cbest(n) – minimum over inputs of size n • Average case: Cavg(n) – “average” over inputs of size n • Number of times the basic operation will be executed on typical input • NOT the average of worst and best case • Expected number of basic operations considered as a random variable under some assumption about the probability distribution of all possible inputs

31. Input size and basic operation examples 30

32. Example:Sequentialsearch • Worst case • Best case • Average case 31

33. Example:Sequentialsearch (Count.) • Worst case The last number  n 32

34. Example:Sequentialsearch (Count.) • Worst case The last number  n 33

35. Binary Search Used with a sorted list First check the middle list element If the target matches the middle element, we are done If the target is less than the middle element, the key must be in the first half If the target is larger than the middle element, the key must be in the second half

36. Binary Search Example

37. Binary Search Algorithm start = 1 end = N while start ≤ end do middle = (start + end) / 2 switch (Compare(list[middle], target)) case -1: start = middle + 1 break case 0: return middle break case 1: end = middle – 1 break end select end while return 0

38. Algorithm Review Each comparison eliminates about half of the elements of the list from consideration If we begin with N = 2k – 1 elements in the list, there will be 2k–1 – 1 elements on the second pass, and 2k–2 – 1 elements on the third pass

39. Worst-Case Analysis In the worst case, we will either find the target on the last pass, or not find the target at all The last pass will have only one element left to compare, which happens when21-1 = 1 If N = 2k– 1, then there must be k = log(N+1) passes

40. Questions?