Algorithms

1. Algorithms CSC 180 Lecture 6 Dr. Adam P. Anthony

2. Overview • What is an algorithm? • How are algorithms written? • Algorithms and Problem Solving • Basic Algorithmic Techniques • Iteration • Recursion • Evaluating Algorithms • Efficiency • Correctness

3. What is an Algorithm? • We’ve seen them before: • Executing Programs: • As long as there are more instructions: • Fetch the instruction identified by the program counter • Execute the instruction • Update the program counter • Recipes • Driving/Walking directions • Scratch Programs • More ideas?

4. So what is an Algorithm, Really? • Step-by-step instructions to achieve some (any!) task • Must be absolutely clear to the intended party, or it is worthless • If they can’t follow the directions, then they can’t achieve the task! • Each step must be do-able • “Travel back in time and stop WWII, then add whipped cream and sugar”

5. Hi! I’m Willis, and I’m new here! • I can: • Walk • Talk • Manipulate objects with my “arms” • Understand basic motions and movements • I don’t understand anything about this planet

6. Can you help me…? • Get to the student union from here? • Is any part of this re-usable? • Decide what to eat at the union? • What if I go to a different restaurant? • Make a left-hand turn in a car? • Is this a common pattern when driving? • Play Rock-Paper-Scissors? • How do we know when we are done?

7. Some Important Themes in Algorithm Design • Abstraction and reusability • Only need to give the ‘cross the street’ algorithm once, assuming thereafter that Willis can just use the same algorithm again • Working with lists • Many tasks are similar, just with different content • A ‘list’ is just a collection of information that we need to process before we make a decision • Making if/then decisions • Add complexity to our decision making process • Repetitive loops • Some tasks are repetitive. What language tricks did we use to express the repetitiveness? • How did we communicate when it was time to stop?

8. Hi, I’m Phillip, and I’m new here! • I can: • Add/Subtract/multiply/divide • JUMP • NOOP • LOAD/STORE • I truly know nothing else about this world, and only do what I’m told

9. Formal Definition of an Algorithm • “An algorithm is an ordered set of unambiguous, executable steps that defines a terminating process.”

10. An Algorithm Must Terminate • For theoretical reasons (read the book!), an algorithm must eventually stop running • Much debate about this since many algorithms run “forever” • Web servers, routers, etc. • My opinion: an algorithm must at least be interruptible • Traditional algorithms: all input is given at beginning, nothing new ever comes up. These should terminate. • On-line algorithms: input is given gradually over time, and the size of the input is typically irrelevant • Only valuable if we can interrupt it for a status check

11. An Algorithm is Ordered • Does not mean step-by-step • From dictionary.com, Definition of Ordered: • neatly or conveniently arranged; well-organized: an ordered office. • done according to specific principles or procedures: an ordered method of assembling the parts. • conducted according to certain precepts or rules: an ordered way of life. • Being ordered means that at any given point during the algorithm, you know exactly what to do next

12. Steps must be Unambiguous and Executable • A step is unambiguous if it’s painstakingly clear how that step should be carried out • Often depends on frame of reference • Executable effectively means ‘do-able’ • Count the number of states in the USA • “Make a list of all positive integers” • “Go back in time.”

13. Unambiguous/Executable/Both? GOOGLE MAPS finds shortest routes! Dictionary.com looks up words! File Browsers sort by name, date, type…

14. On Ambiguity • Despite its complexity, a processor can only really do a handful of things: • Arithmetic • Load/Store • Logic (=,>,<, AND, OR, etc.) • Jump/NO-OP • For the sake of simpler instruction, we’ll take for granted that the processor also knows how to print information to the screen: • Print “hey there!” • Then how does a computer do something like sort a list of numbers? • {4,1,2,3,5} becomes {1,2,3,4,5}

15. Key Abstraction #1: Variables • Have already discussed variables in many capacities. • Main Idea: save a value for later use • What unambiguous processor commands do we need to be able to assign values • X  5 !!!This tells us to ASSIGN the value 5 to x • Y  15 • Z  X + Y

16. Fundamental Algorithm: Self Assignment • Reassignment is the concept of taking a variable and giving it a new value that is based on itself: X  5 X  X + 1 !!!This says to SELF ASSIGN the value of X, by taking its old value and adding 1. At this point, X is 6

17. Fundamental Algorithm: Swap Assignments • If we have two variables, X and Y, how can we swap their values? X  5 Y  6 !!! How can we get X to be 6, and Y to be 5? Temp  X X  Y Y  Temp

18. Key Abstraction #2: Conditional Loops • We’ve seen repeat loops in Scratch: Repeat (10): print “hi!!” • A more useful variety is the conditional loop: while ( <CONDITION> ): DO WORK AGAIN • Useful because we can do more than just count: while ( response is not a number ): print “Incorrect response, please input a number” response  KEYBOARD_INPUT

19. Equivalence of Repeat and While Repeat (10) : print “hi!” Vs: X  10 While (X > 0): print “hi” X  X - 1 Vs: X  10 While (X > 0): print “hi for the “ + X + “th time!” X  X – 1 !!!!You can use the variable to your advantage!!!

20. Key Abstraction #3: Lists • Lists are just building on the usefulness of variables • Instead of storing one value at a location, we store several: • L = {6,12,45} • And we can access individual items, too! • L[1] is 6, L[2] is 12, L[3] is 45 • L[1]  15 updates the first item in the list: {15,12,45} • Discussion: • What types of processor commands do we need to manage lists? • Are these similar to variables in any way?

21. Fundamental Algorithm: Variables + Loops + Lists • A list uses the [], called the subscript operator, to access individual data items. • All you need to supply to [] is something that is a number • Could be a literal, constant or variable! L = {80, 90, 100} X  2 L[X]  95 !!!At this point, L is the list {80,95,100}!!!

22. Variables + Loops + Lists, cont. • How can we get the computer to print all the contents of a list to the screen? • Simple: count your way through the list! L = {6, 12, 45} i = 1 while ( i< 4 ) print L[i] i i + 1

23. BIG exercise • As a class, we are going to figure out how to sort a list of numbers using only the fundamental algorithms we’ve learned so far.

24. Abstraction, Procedures and Ambiguity • “Sort a list” is ambiguous to a processor • Doesn’t know what a list is • Doesn’t know what sorting is • Assume we write an algorithm that can sort a list L • Abstraction says that we can ignore the inside of a process (the algorithm) and treat the task as a single command • We NAME the algorithm SORT(L) such that we have a procedure in which L can be changed and reused • Then, whenever we need to sort, we don’t have to write out that whole algorithm again, but just use the procedure name!

25. Pseudo-Code • It’s tedious to build up abstractions from machine instructions each time we write an algorithm • Over the years the computing community has agreed on a number of ‘basic abstractions’ that we assume any decent computer has been preprogrammed to perform unambiguously • The collection of abstractions is called Pseudo-Code because it contains abstractions that programmers expect out of any decent Real Programming Language Code

26. Phillip’s Back! • Thanks to Pseudocode, I can now: • Add/Subtract/multiply/divide • Give names to things I computed in the past • Scan a list one item at a time • Understand if/then statements • Perform repetitive tasks automatically • Print text on my screen • I still know nothing else about this world, and only do what I’m told

27. Elements of Pseudo-Code • Variables • Save a value for use later on: interest_rate 0.05 • One important use: give values a meaningful name • “interest_gained balance * interest_rate” is clearer than • “interest_gained  150 * 0.05” (What does 150 represent?) • Because of above, always try to use words and clear abbreviations for variables: • X  Y + Z vs. • TotalFunds  CheckingBalance + SavingsBalance

28. In-Class Exercise • With a partner, find in the Scratch environment: • How to create a variable • How scratch ‘does’ the  operator • Write a scratch program that matches the following pseudocode: apples  5 oranges  3 apples  apples + 2 fruit  apples + oranges

29. Elements of Pseudo-Code • Lists: [1, 2, 3, 4, 5] • Variables are sometimes awkward: • Num_Apples 11 Num_Oranges  4 Num_Pears  7 • FruitCounts  [11,4,7] • Zero-Based vs. One-Based: • MANY languages say the first item in a list is the zeroth element: FruitCounts[0] = 11 • Humans tend to associate the first item with the number 1: FruitCounts[1] = 11 • Boundaries: • Zero-based: FruitCounts[3] = ???? (ERROR) • One-based: FruitCounts[3] = 7 but FruitCounts[0] = ???? (ERROR) • Safe to assume that the length of the list is unambiguously determined: • length(FruitCounts)

30. Elements of Pseudo-Code • If/Then/Else: • Centered around questions that have True/False answers • If structures can be nested: • You can start a new ‘if’ inside another • Else is optional • IF/THEN/ELSE-IF: • Allows you to test for exactly one condition • Order of conditions matters If <test> Then <do some work> Else <do this work if test failed>

31. Loops • While: • while <condition> do <action> • Repeat (do-while): • repeat <action> until <condition> • <condition> is anything that evaluates to True/False • <action> is any set of steps you wish to repeat • Difference between Wile/Repeat?

32. Procedures • A procedure is just a named algorithm: • procedure SORT(L) ... • RETURN: if a procedure has output, then RETURN says to stop working and give an answer procedure find_min(L) ... SORT(L) Return first item in L • A procedure’s name should be descriptive • Input Parameters: Variables that represent the data that the algorithm needs to run • procedure max(X,Y) • procedure greetings() • Procedures are how we formally introduce abstraction

33. Loop Example

34. Loops and Lists • We frequently wish to perform the same action to everything in a list: Procedure just_magnitude(L): i 0 while i < length(L) do: L[i]  abs(L[i]) i  i + 1 return L

35. Pseudocode Odds and Ends • Text: • Text is in quotation marks so it isn’t confused with code • Can be stored in variables: error_message “Bad input.” • Print data to the (imaginary) screen: • Print “Hello World!” • Read data from the user and store it in a variable • Response  READ() • Arithmetic (with variables and/or numbers) • X + Y • 0.5*X • Comparison (result is TRUE/FALSE) • X == Y • X != Y • X < 0.5 • X >= Z • Comments: • Begins with a special character (you can use //) • //SORT(L) is another algorithm

36. About Indentation Procedure Max ( X, Y ): if X > Y then RETURN X else RETURN Y Procedure Max ( X, Y ): if X > Y then RETURN X else RETURN Y • Which is easier to read?

37. A Better Example for(tie(vi, vi_end) = vertices(rd->attr_graph); vi != vi_end; vi++){ for(tie(vi2, vi2_end) = vertices(rd->attr_graph); vi2 != vi2_end; vi2++){if(*vi != *vi2){jacquardCoeff = jacquardCoefficient(rd, *vi, *vi2); if(jacquardCoeff > bestJacquardCoeff){ bestJacquardCoeff = jacquardCoeff; bestVertexNum1 = *vi; bestVertexNum2 = *vi2;}}}} for(tie(vi, vi_end) = vertices(rd->attr_graph); vi != vi_end; vi++) { for(tie(vi2, vi2_end) = vertices(rd->attr_graph); vi2 != vi2_end; vi2++) { if(*vi != *vi2) { jacquardCoeff = jacquardCoefficient(rd, *vi, *vi2); if(jacquardCoeff > bestJacquardCoeff) { bestJacquardCoeff = jacquardCoeff; bestVertexNum1 = *vi; bestVertexNum2 = *vi2; } } } }

38. The Modified procedure Greetings in pseudocode Procedure greetings(): count  4 while count > 0 do: if count is even : print “Hello” else: print “ there.” count  count – 1

39. Creating Algorithms • Algorithms are nothing more than a detailed solution to a specific problem • Problem Solving is an imprecise process • Basic Guidelines for Algorithm Development: • Understand the Problem • Devise a high level (English) plan for solving the problem • Translate the plan into pseudocode, while simultaneously identifying and clarifying ambiguous statements • Evaluate the pseudocode for accuracy and evaluate its potential as a general tool

40. Going Through the Motion • In Groups: • Create a pseudo-code algorithm that finds the maximum value in a list of integers WITHOUT sorting the list. • Anything that is not an element of pseudocode must be expanded within the algorithm, or defined as a secondary algorithmic procedure • A test case: [1, 3, 5, 46, 76, 23, 12, 0, 9, 95, 4, 3, 8, 1, 99, 105, 2345, 12, 90, 15, 45, 23, 18, 9990, 1540, 9, 8, 6, 7, 2, 3, 6, 66, 78, 45, 54, 60, 10000, 900, 12345, 180]

41. Common Algorithmic Forms • Iterative: • Any algorithm that uses a loop • Recursive: • Uses a special mathematical concept • Often leads to graceful algorithmic solutions

42. Problem: Sort • From very early in computing history, computers were used to sort • What are some methods humans use for sorting?

43. Iteration Example: Insertion Sort procedure insertion_sort(L): N  2 //use One-based list while N <= length(L)do: current  Nth item in L J  N – 1 while J > 0 andJth item in L > current do: copy Jth item into space J + 1 //when the while loop above stops, //J will be over the first item that //is less than current copy Nth item into space J + 1 N  N + 1

44. Understanding an Algorithm: Tracing

45. Recursion • A problem is recursive if: • It can be cut up into two or more pieces such that the smaller pieces are themselves instances of the problem, and • The solutions to the smaller pieces can be combined to find the solution to the larger problem • The brilliance of recursion is in part #2: • Repeatedly break problem into smaller pieces until the smallest, easiest problem remains • Most of the work resides in putting pieces back together • Efficiency is gained from part #1: • Cut problem in half, throw out portion that won’t help

46. Problem: Search • Searching is another common task: Where is 180Project.docx? • How do humans conduct a search in a list of values? • What if it is sorted (dictionary-style search)?

47. Recursion Example: Binary Search • Procedure BINARY_SEARCH(List, Value): • If List is empty, then the item is not in List, return FAIL • Compare Value to the middle entry in the list • If it is equal to Value, then return SUCCESS • If the middle entry is greater than Value THEN BINARY_SEARCH the left half of the list as if it is its own list • ELSE BINARY_SEARCH the right half of the list as if it is its own list

48. Evaluating Algorithms • A legitimate algorithm must always be CORRECT • Find the number 5 in a sorted list • Sort a list of integers • Find the lowest price flight • A good algorithm will also be as FAST and EFFICIENT as possible • Find the lowest price flight in less than 2 minutes • Sort a list of 300,000,000 SSN’s

49. Counting Steps • Time is not a good measure for speed (Why?) • Instead, we use the notion of ‘steps’ • One pseudo-code statement is a step • If a procedure is called within an algorithm (e.g. SORT(L) ), the steps to complete it must be determined before the main algorithm can be fully analyzed • Loops: number of repetitions TIMES the number of steps inside the loop • Sometimes you have to add each individual repetition (if not always the same) • A loop inside of another loop could be really costly! • Often measured in terms of the input values (size of list, value of numbers) How Many Steps? procedure mystery_algorithm (L, p): count  0 while count < length(L): reps  0 val  1 while reps < p: val = val * L[count] reps  reps + 1 print val count  count + 1

50. Complexity Classes • Compare: • f(x) = x2 • f(x) = x2 + 15 x + 8 • “Shape” of a curve determined by the largest term • Complexity classes determined only by the ‘shape’