1 / 37

Algorithms

This article discusses the efficiency of searching and sorting algorithms in terms of time and storage. It explores how programs can be compared and optimized for execution time, considering factors such as computer speed and code optimization. It provides examples of linear search and binary search algorithms and explains how the number of operations and the order of complexity are measured. Overall, this article aims to help understand the importance of efficiency in computer programming.

gcobb
Download Presentation

Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithms Searching and SortingAlgorithm Efficiency

  2. Efficiency • A computer program should be totally correct, but it should also • execute as quickly as possible (time-efficiency) • use memory wisely (storage-efficiency) • How do we compare programs (or algorithms in general) with respect to execution time? • various computers run at different speeds due to different processors • compilers optimize code before execution • the same algorithm can be written differently depending on the programming language used

  3. Example: Linear Search • Input: • List A of n unique data values, indexed from 0 to n-1. (The data values are not in any specific order.) • The desired data value (target) we’re searching for. • Output: • The index of the location of the target in the list or -1 if the target is not found.

  4. Example: Linear Search • Algorithm: 1. Set index = 0. 2. While index < n and A[index] ≠ target, do the following: a. Add 1 to index 3. If index ≤ n-1, output index; Otherwise, output -1.

  5. Example: Linear Search index  0 Algorithm: 1. Set index = 0. 2. While index < n and A[index] ≠ target, do the following: a. Add 1 to index 3. If index ≤ n-1, output index; Otherwise, output -1. index < n andA[index] ≠ target no yes index  index + 1 yes index ≤ n-1 no index -1

  6. Example: Linear Search index  0 Algorithm: 1. Set index = 0. 2. While index < n and A[index] ≠ target, do the following: a. Add 1 to index 3. If index ≤ n-1, output index; Otherwise, output -1. index >= n or A[index] = target DeMorgan’s Law: not (A and B) = (not A) or (not B) yes no index  index + 1 yes index ≤ n-1 no index -1

  7. Some questions • What is the maximum number of data values we need to examine as a function of n? (i.e. worst case) • What is the minimum number of data values we need to examine as a function of n? (i.e. best case) • How much storage do we need to hold all of the information for use in this algorithm as a function of n?

  8. Linear Search • We measure time efficiency by counting the number of operations performed by the algorithm. • But what is an operation? 1. Set index = 0. 2. While index ≤ n-1 and A[index] ≠ target, do the following: a. Add 1 to index 3. If index ≤ n-1, output index; Otherwise, output -1.

  9. Counting Operations • We typically count operations that are a function of the amount of data (n) that we have to process. • Abstraction: • We don't count individual operators (+, -, ...). • We count more general operations: • assignments (← or "Set ... equal to ..."), or • comparisons (<, >, <,>, =, ≠) • For linear search, we might count comparisons. • How many comparisons does linear search require in the worst case? n+2

  10. Example: Binary Search • Input: • List A of n unique data values. NOTE: The data values must be sorted in increasing order. • The desired data value (target) we’re searching for. • Output: • The index of the location of the target in the array or -1 if the target is not found.

  11. Example: Binary Search • Algorithm: 1. Set i = 0 and j = n-1. 2. Repeat: a. mid = (i + j) / 2 (ignore any remainder). b. If (target < A[mid]), set j = mid-1. Otherwise if (target > A[mid]), set i = mid+1. until i > j or A[mid] = target. 3. If i <= j, output mid. Otherwise, output -1.

  12. Example: Binary Search i  0, j  n-1 Algorithm: 1. Set i = 0 and j = n-1. 2. Repeat: a. mid = (i + j) / 2 (ignore any remainder). b. If (target < A[mid]), set j = mid-1. Otherwise if (target > A[mid]), set i = mid+1. until i > j or A[mid] = target. 3. If i ≤ j, output mid. Otherwise, output -1. mid (i + j) div 2 yes no target < A[mid] yes target > A[mid] j  mid-1 i  mid+1 no no i > j or A[mid] = target yes yes no i ≤ j mid -1

  13. Counting Operations Again • For binary search, consider the worst-case scenario (target is not in list) • How many times can we split the list before the list becomes empty? • n=15: 15  7  3  1  0 (4 splits) • In general, we can split the list in half approximately log2n + 1 times before it becomes empty. • Recall the log function:logab = c is equivalent to ac = bExamples: log2128 = 7 log2n = 5 implies n = 32

  14. Order of Complexity • For very large n, we express the number of operations as the order of complexity. • Order of complexity for worst-case behavior is often expressed using Big-O notation: Number of operationsOrder of Complexity n O(n) n/2 + 6 O(n) 2n + 9 O(n) Usually doesn't matter what theconstants are...we are only concerned aboutthe highest powerof n.

  15. O(n) ("Linear") 2n + 9 n Number of Operations n/2 + 6 n (amount of data)

  16. O(n) Number of Operations n 30 10 20 10 10 10 20 30 n (amount of data)

  17. Order of Complexity Number of operationsOrder of Complexity n2 O(n2) 2n2 + 7 O(n2) n2/2 + 5n + 2 O(n2) Usually doesn't matter what theconstants are...we are only concerned aboutthe highest powerof n.

  18. O(n2) ("Quadratic") n2 n2/2 + 5n + 2 2n2 + 7 Number of Operations n (amount of data)

  19. O(n2) Number of Operations n2 900 500 400 300 100 10 20 30 n (amount of data)

  20. Order of Complexity Number of operationsOrder of Complexity log2n O(log n) log10n O(log n) 2(log2n) + 5 O(log n) The logarithm base is not written in big O notation since all that matters is that the function is logarithmic.

  21. O(log n) ("Logarithmic") 2(log2 n) + 5 Number of Operations log2 n log10 n n (amount of data)

  22. O(log n) Number of Operations log2 n 6 1 5 1 4 16 32 64 n (amount of data)

  23. O(n log n) Number of Operations n log2n = O(n log n) 384 (not drawn to scale) 224 160 96 64 16 32 64 n (amount of data)

  24. O(1) ("Constant") Number of Operations 15 = O(1) 15 n (amount of data) 10 20 40

  25. O(2n) 2n = O(2n) Number of Operations (definitely not drawn to scale) 32768 31744 1024 992 n (amount of data) 32 5 10 15

  26. Comparing Big O Functions O(2n) O(n2) O(n log n) Number of Operations O(n) O(log n) O(1) n (amount of data)

  27. Searches • Linear Search • Count comparisons as operations. • Worst Case: O(N) • Binary Search • Count comparisons as operations. • Worst Case: O(log N) • Which algorithm is better? • In general, for large values of N: log N < Nso binary search is better for large N. • BUT, binary search requires a sorted list.

  28. Selection Sort • Find the smallest data value in the list from index 0 to index n-1 and swap it into position 0 of the list. • Find the smallest data value in the list from index 1 to index n-1 and swap it into position 1 of the list. • Find the smallest data value in the list from index 2 to index n-1 and swap it into position 2 of the list. • …etc…

  29. Selection Sort • What is the worst-case order of complexity of selection sort on n data values? • Count the comparisons as operations. • First pass through: n-1 comparisons • Second pass through: n-2 comparisons • ... • Last (n-1st) pass through: 1 comparison • Order of complexity:(n-1)+(n-2)+...+1 = n(n-1)/2 = (n2-n)/2 = O(n2) • We call selection sort a "quadratic sort".

  30. MergeSort • Split the list into 2 halves.* • If the size of the half-list is not 1, • Sort each half using merge sort. (recursively) • Set up a new list. • Merge step: Move the smallest of each half-list into the new list. Repeat this step until one of the half-lists is empty. Then copy the remaining data in the non-empty half-list into the new list. *If the list has an odd number of elements, one list will have one extra element.

  31. MergeSort

  32. MergeSort

  33. Merge Sort • What is the order of complexity of merge sort? • Let C(n) = total number of operations performed for merge sort on n data values. • Operations include comparisons in the merge process. • C(1) = 0 • Maximum number of comparisons to merge 2 lists on size n/2 into one list of size n: n-1 comparisons. • C(n) = C(n/2) + C(n/2) + n-1 = 2C(n/2) + n – 1 • This is a recurrence relation (it’s recursive!). • Solution: C(n) = nlog2n - n + 1 = O(n log n).

  34. Order of Complexity • Examples: • O(logN) + O(N) = O(N) • O(N log N) + O(N) = O(N log N) • O(N logN) + O(N2) = O(N2) • O(2N) + O(N2) = O(2N) Algorithm O(A) Overall order of complexity of algorithm is max (O(A), O(B)). O(B)

  35. Order of Complexity • Examples: • O(logN) * O(N) = O(N log N) • O(N log N) * O(N) = O(N2 log N) • O(N) * O(1) = O(N) Algorithm O(A) Overall order of complexity of algorithm is O(A * B). Example:- Nested loops O(B) O(A) does not include complexity of part B of algorithm

  36. Searches Revisited • What if you wanted to use binary search on data that is not sorted? • Sort first: O(n log n) using Merge Sort • Then use binary search: O(log n) • Total runtime complexity:O(n log n) + O(log n) = O(n log n) • Now, how does this compare to linear search?

  37. Comparing Algorithms • Assume an algorithm processes n data values. If each operation takes 1 s to execute, how many s will it take to run the algorithm on 100 data values if the algorithm has the following number of computations? Number of ComputationsExecution Time n 100 s n • log2 n 665 s n2 10,000 s n3 1,000,000 s = 1 sec 2n > 1030s n! > 10160s

More Related