Chapter 3: The Efficiency of Algorithms

Chapter 3: The Efficiency of Algorithms Invitation to Computer Science, C++ Version, Third Edition

Objectives In this chapter, you will learn about: • Attributes of algorithms • Measuring efficiency • Analysis of algorithms • When things get out of hand Invitation to Computer Science, C++ Version, Third Edition

Introduction • Desirable characteristics in an algorithm • Correctness • Ease of understanding • Elegance • Efficiency Invitation to Computer Science, C++ Version, Third Edition

Attributes of Algorithms • Correctness • Does the algorithm solve the problem it is designed for? • An algorithm which correctly answers the wrong question is not “correct”. • Does the algorithm solve the problem correctly? • It must give the correct result for all possible inputs Invitation to Computer Science, C++ Version, Third Edition

Attributes of Algorithms (continued) • Correctness (cont’d) • Does it provide a result appropriate for the problem? • Ease of understanding • How easy is it to understand or alter an algorithm? • Important for program maintenance Invitation to Computer Science, C++ Version, Third Edition

Attributes of Algorithms (continued) • Elegance (Carl Friedrich Gauss (1777-1855) sum 1..100) • How clever or sophisticated is an algorithm? • Sometimes elegance and ease of understanding work at cross-purposes • Efficiency • How much time and/or space does an algorithm require when executed? ( time is related to work done) • Perhaps the most important desirable attribute Invitation to Computer Science, C++ Version, Third Edition

Measuring Efficiency • Analysis of algorithms - overview • Study of the efficiency of various algorithms • Efficiency measured as function relating time or space used to size of input • For one input size, best case, worst case, and average case behavior must be considered • The  notation captures the order of magnitude of the efficiency function Invitation to Computer Science, C++ Version, Third Edition

Sequential Search • Search for NAME among a list of n names • Start at the beginning and compare NAME to each entry until a match is foundHow much time (work) is required …How much space (memory) is used …for a given length, N, of the list? Best case? Worst case?A) count total steps done or,B) find a certain operation that is related to total steps and count number of times that operation is done Invitation to Computer Science, C++ Version, Third Edition

Figure 3.1 Sequential Search Algorithm Analysis (A) WC = 3n + 7 (8) stepsBC = 10 steps Invitation to Computer Science, C++ Version, Third Edition

Sequential Search Algorithm Analysis (B) • Comparison of the NAME being searched for against a name in the list • Central unit of work ( compare name on list to target – step 4) • Used for efficiency analysis • For lists with n entries: • Best case • Target NAME is the first name in the list • 1 comparison • (1) Invitation to Computer Science, C++ Version, Third Edition

Sequential Search Algorithm Analysis (B) • For lists with n entries: • Worst case • Target NAME is the last name on the list • Target NAME is not in the list • n comparisons required • (n) • Average case • Roughly n/2 comparisons • (n) Invitation to Computer Science, C++ Version, Third Edition

Sequential Search Algorithm Analysis (B)More detail – What is (n) ? Total Time (in worst case) = C1 +C2 + C5 + C6 +C8 + C9+ C10 + C3 (n+1) + C4(n) + C7(n) = (C3+C4+C7) x n + (C1+C2+C3+ C5 + C6 + C8 + C9 + C10) = C’ x n + C” Time = C’ x n + C” Is just the equation of a line. Time is linearly related to n, the amount of data being searched. Sequential Search is (n) Pronounced “Order Theta n” Invitation to Computer Science, C++ Version, Third Edition

Sequential Search (continued) • Space efficiency • Uses essentially no more memory storage than original input requires • Very space-efficient Invitation to Computer Science, C++ Version, Third Edition

Order of Magnitude: Order n • As n grows large, order of magnitude dominates running time, minimizing effect of coefficients and lower-order terms • All functions that have a linear shape are considered equivalent • Order of magnitude n • Written (n) • Functions vary as a constant times n Invitation to Computer Science, C++ Version, Third Edition

Figure 3.4 Work = cn for Various Values of c Invitation to Computer Science, C++ Version, Third Edition

Selection Sort • Sorting • Take a sequence of n values and rearrange them into order • Selection sort algorithm • Repeatedly searches for the largest value in a section of the data • Moves that value into its correct position in a sorted section of the list • Uses the Find Largest algorithm Invitation to Computer Science, C++ Version, Third Edition

Figure 3.6 Selection Sort Algorithm Invitation to Computer Science, C++ Version, Third Edition

Selection Sort (continued) • Count comparisons of largest so far against other values • Find Largest, given m values, does m-1 comparisons • Selection sort uses Find Largest n times, • Each time with a smaller list of values • Num Compares = n-1 + (n-2) + … + 2 + 1 = n(n-1)/2 Invitation to Computer Science, C++ Version, Third Edition

Selection Sort (continued) Invitation to Computer Science, C++ Version, Third Edition

Selection Sort (continued) • Time efficiency • Comparisons: n(n-1)/2 • Exchanges: n (swapping largest into place) • Total = n(n-1)/2 + n = ½ (n2 + n ) • Overall: (n2), best and worst cases • Space efficiency • Space for the input sequence, plus a constant number of local variables Invitation to Computer Science, C++ Version, Third Edition

Order of Magnitude – Order n2 • All functions with highest-order term cn2 have similar shape • An algorithm that does cn2 work for any constant c is order of magnitude n2, or (n2) Invitation to Computer Science, C++ Version, Third Edition

Order of Magnitude – Order n2 (continued) • Anything that is (n2) will eventually have larger values than anything that is (n), no matter what the constants are • An algorithm that runs in time (n) will outperform one that runs in (n2) Invitation to Computer Science, C++ Version, Third Edition

Figure 3.10 Work = cn2 for Various Values of c Invitation to Computer Science, C++ Version, Third Edition

Figure 3.11 A Comparison of n and n2 Invitation to Computer Science, C++ Version, Third Edition

Analysis of Algorithms • Multiple algorithms for one task may be compared for efficiency and other desirable attributes • Data cleanup problem • Search problem • Pattern matching Invitation to Computer Science, C++ Version, Third Edition

Data Cleanup Algorithms • Given a collection of numbers, find and remove all zeros • Possible algorithms • Shuffle-left • Copy-over • Converging-pointers Invitation to Computer Science, C++ Version, Third Edition

The Shuffle-Left Algorithm • Scan list from left to right • When a zero is found, shift all values to its right one slot to the left • Show example on board Invitation to Computer Science, C++ Version, Third Edition

Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup Invitation to Computer Science, C++ Version, Third Edition

The Shuffle-Left Algorithm (continued) • Time efficiency • Count examinations of list values and shifts • Best case • No shifts, n examinations • (n) • Worst case • Shift at each pass, n passes • n2 shifts plus n examinations • (n2) Invitation to Computer Science, C++ Version, Third Edition

The Shuffle-Left Algorithm (continued) • Space efficiency • n slots for n values, plus a few local variables • (n) Invitation to Computer Science, C++ Version, Third Edition

The Copy-Over Algorithm • Use a second list (show example on board) • Copy over each nonzero element in turn • Time efficiency • Count examinations and copies • Best case • All zeros • n examinations and 0 copies • (n) Invitation to Computer Science, C++ Version, Third Edition

Figure 3.15 The Copy-Over Algorithm for Data Cleanup Invitation to Computer Science, C++ Version, Third Edition

The Copy-Over Algorithm (continued) • Time efficiency (continued) • Worst case • No zeros • n examinations and n copies • (n) • Space efficiency • 2n slots for n values, plus a few extraneous variables Invitation to Computer Science, C++ Version, Third Edition

The Copy-Over Algorithm (continued) • Time/space tradeoff • Algorithms that solve the same problem offer a tradeoff: • One algorithm uses more time and less memory • Its alternative uses less time and more memory Invitation to Computer Science, C++ Version, Third Edition

The Converging-Pointers Algorithm • Swap zero values from left with values from right until pointers converge in the middle (show example) • Time efficiency • Count examinations and swaps • Best case • n examinations, no swaps • (n) Invitation to Computer Science, C++ Version, Third Edition

Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup Invitation to Computer Science, C++ Version, Third Edition

The Converging-Pointers Algorithm (continued) • Time efficiency (continued) • Worst case • n examinations, n swaps • (n) • Space efficiency • n slots for the values, plus a few extra variables Invitation to Computer Science, C++ Version, Third Edition

Figure 3.17 Analysis of Three Data Cleanup Algorithms Invitation to Computer Science, C++ Version, Third Edition

Binary Search • Given ordered data, • Search for NAME by comparing to middle element • If not a match, restrict search to either lower or upper half only • Each pass eliminates half the data • Show example. Show binary search tree. Invitation to Computer Science, C++ Version, Third Edition

Figure 3.18 Binary Search Algorithm (list must be sorted) Invitation to Computer Science, C++ Version, Third Edition

Binary Search (continued) • Efficiency • Best case • 1 comparison • (1) • Worst case • lg n comparisons ( show binary search tree ) • lg n: The number of times n may be divided by two before reaching 1 • (lg n) Invitation to Computer Science, C++ Version, Third Edition

Binary Search (continued) • Tradeoff • Sequential search • Slower, but works on unordered data • Binary search • Faster (much faster), but data must be sorted first Invitation to Computer Science, C++ Version, Third Edition

Figure 3.21 A Comparison of n and lg n Invitation to Computer Science, C++ Version, Third Edition

Pattern Matching • Analysis involves two measures of input size • m: length of pattern string • n: length of text string • Unit of work • Comparison of a pattern character with a text character • Show example Invitation to Computer Science, C++ Version, Third Edition

Pattern Matching (continued) • Efficiency • Best case • Pattern does not match at all • n - m + 1 comparisons • (n) • Worst case • Pattern almost matches at each point • (m -1)(n - m + 1) comparisons • (m x n) Invitation to Computer Science, C++ Version, Third Edition

Figure 3.22 Order-of-Magnitude Time Efficiency Summary Invitation to Computer Science, C++ Version, Third Edition

When Things Get Out of Hand • Polynomially bound algorithms • Work done is no worse than a constant multiple of n2 • Intractable algorithms • Run in worse than polynomial time • Examples • Hamiltonian circuit • Bin-packing Invitation to Computer Science, C++ Version, Third Edition

Hamiltonian Circuit Consider 4 cities joined as shown below. Is it possible to start at city A, visit each other city exactly once, and end up back at A? Yes. A-B-D-C-A and A-C-D-B-A. What if the number of cities and the routes connecting them were much greater? A computer would be needed. In general we call this collection of data a graph. A graph consists of nodes (cities) and edges ( roads). A Hamiltonian Circuit is a path through a graph, that begins and ends at the same node, and goes through all the other nodes exactly once. Invitation to Computer Science, C++ Version, Third Edition

Hamiltonian Circuit Problem: Find whether an arbitrary graph has a Hamiltonian circuit. Solution: Examine all possible paths that are the correct length and see whether any are Hamiltonian Circuits For a graph with 4 nodes, and 2 choices per node, there are 24 paths of length 4 to examine. In general, if we are solving the problem in a graph with n nodes and two choices per node, there would be 2n paths to examine. If we consider the examination of One path as a unit of work, the algorithm is O(2n). This type of algorithm is called a brute force algorithm. It beats the problem into submission by trying all possibilities. Invitation to Computer Science, C++ Version, Third Edition

Bin Packing Given an unlimited number of bins ofvolume 1 unit, and given n objects, allof volume between 0.0 and 1.0, findthe minimum number of bins neededto store all n objects.Applications:A manufacturer who ships sets of various items in fixed size cartons.A person wanting to store image files on a set of CDs in the most efficient way.Solutions: Try an approximate solution. Take objects in order, put first one infirst bin and the rest into the first bin that can hold it. Consider the example herewith a set of objects {0.3, 0.4, 0.5, 0.6} Invitation to Computer Science, C++ Version, Third Edition

Chapter 3: The Efficiency of Algorithms