Programming for Engineers in Python

1. Programming for Engineers in Python Lecture 9: Sorting, Searching and Time Complexity Analysis Autumn 2011-12

2. Lecture 8: Highlights • Design a recursive algorithm by • 1. Solving big instances using the solution to smaller instances • 2. Solving directly the base cases • Recursive algorithms have • 1. Stopping criteria • 2. Recursive case(s) • 3. Construction of a solution using solution to smaller instances

3. Today • Information • Importance of quick access to information • How can it be done? • Preprocessing the data enables fast access to what we are interested in • Example: dictionary • The most basic data structure in Python: list • Sorting  preprocessing • Searching  fast access • Time complexity

4. Information There are about 20,000,000,000 web pages in the internet 4

5. 2 5 1 8 9 13 67 Sorting • A sorted array is an array whose values are in ascending/descending order • Very useful • Sorted array example: • Super easy in Python – the sorted function

6. Why is it Important to Sort Information? • To find a value, and fast! • Finding M values in a list of size N • Naive solution: given a query, traverse the list and find the value • Not efficient, average of N/2 operations per query • Better: sort the array once and than perform each query much faster

7. Why not Use Dictionaries? • Good idea! • Not appropriate for all applications: • Find the 5 most similar results to the query • Query's percentile • We will refer to array elements of the form (key, value)

8. Naïve Search in a General Array • Find location of a value in a given array

9. Binary Search (requires a sorted array) • Input: sorted array A, query k • Output: corresponding value / not found • Algorithm: • Check the middle element in A • If the corresponding key equals k return corresponding value • If k < middle find k in A[0:middle-1] • If k > middle find k in A[middle+1:end]

10. Example Searching for 56 4 5 index 7 8 0 1 2 3 6 9 value

11. Example Searching for 4 4 5 index 7 8 0 1 2 3 6 9 value

12. Code –Binary Search

13. Binary Search – 2nd Try

14. Time Complexity • Worst case: • Array size decreases with every recursive call • Every step is extremely fast (constant number of operations - c) • There are at most log2(n) steps • Total of approximately c*log2(n) • For n = 1,000,000 binary search will take 20 steps - much faster than the naive search

15. Time Complexityעל רגל אחת • Algorithms complexity is measured by run time and space (memory) • Ignoring quick operations that execute constant number of times (independent of input size) • Approximate time complexity in order of magnitude, denoted with O(http://en.wikipedia.org/wiki/Big_O_notation) • Example: n = 1,000,000 • O(n2) = constant * trillion (Tera) • O(n) = constant * million (Mega) • O(log2(n)) = constant * 20

16. Order of Magnitude 16

17. Graphical Comparison

18. Code – Iterative Binary Search

19. http://www.doughellmann.com/PyMOTW/timeit/ Testing EfficiencyPreparations (Tutorial for timeit: http://www.doughellmann.com/PyMOTW/timeit/)

20. Testing Efficiency

21. Results

22. Until now we assumed that the array is sorted… How to sort an array efficiently?

23. Bubble Sortמיון בועות • נסרוק את המערך ונשווה כל זוג ערכים שכנים • נחליף ביניהם אם הם בסדר הפוך • נחזור על התהליך עד שלא צריך לבצע יותר החלפות (המערך ממויין) • למה בועות? • האלגוריתם "מבעבע" בכל סריקה את האיבר הגדול ביותר למקומו הנכון בסוף המערך. 23

24. 7 2 2 2 2 2 2 2 2 2 2 2 2 7 7 7 7 7 5 2 4 5 4 4 7 5 5 5 5 8 8 4 7 5 5 5 4 5 8 5 7 7 8 5 4 5 7 4 7 4 7 4 8 8 8 8 4 8 8 4 8 4 4 8 8 2 5 4 7 8 Bubble Sort Example (done)

25. constant Code – Bubble Sort n iterations i iterations (n-1 + n-2 + n-3 + …. + 1) * const ~ ½ * n2

26. דוגמאות לחישוב סיבוכיות זמן ריצה • מצא ערך מקסימלי במערך לא ממויין • מצא ערך מקסימלי במערך ממויין • מצא את הערך החמישי הכי גדול במערך ממויין • מצא ערך מסויים במערך לא ממויין • מצא ערך מסויים במערך ממויין • ענה על n "שאלות פיבונאצ'י" • שאלת פיבונאצ'י: מהו הערך ה-K בסדרת פיבונאצ'י? • נניח ש-K מוגבל להיות קטן מ-MAX

27. Yes! – Merge Sort We showed that it is possible to sort in O(n2) Can we do it faster?

28. Comparing Bubble Sort with sorted

29. The Idea Behind Merge Sort • Sorting a short array is much faster than a long array • Two sorted arrays can be merged to a combined sorted array quite fast (O(n))

30. Generic Sorting • We would like to sort all kinds of data types • Ascending / descending order • What is the difference between the different cases? • Same algorithm! • Are we required to duplicate the same algorithm for each data type?

31. The Idea Behind Generic Sorting • Write a single function that will be able to sort all types in ascending and descending order • What are the parameters? • The list • Ascending/descending order • A comparative function