1 / 138

Chapter 3: The Fundamentals: Algorithms, the Integers, and Matrices

Chapter 3: The Fundamentals: Algorithms, the Integers, and Matrices. § 3.1: Algorithms. § 3.1 – Algorithms. Algorithms. The foundation of computer programming. Most generally, an algorithm just means a definite procedure for performing some sort of task .

orlando-wolf
Download Presentation

Chapter 3: The Fundamentals: Algorithms, the Integers, and Matrices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 3:The Fundamentals: Algorithms, the Integers, and Matrices (c)2001-2002, Michael P. Frank

  2. §3.1: Algorithms § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  3. Algorithms • The foundation of computer programming. • Most generally, an algorithm just means a definite procedure for performing some sort of task. • A computer program is simply a description of an algorithm in a language precise enough for a computer to understand, requiring only operations the computer already knows how to do. • We say that a program implements (or “is an implementation of”) its algorithm. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  4. Programming Languages • Some common programming languages: • Newer:Java, C, C++, Visual Basic, JavaScript, Perl, Tcl, Pascal • Older: Fortran, Cobol, Lisp, Basic • Assembly languages, for low-level coding. • In this class we will use an informal, Pascal-like “pseudo-code” language. • You should know at least 1 real language! § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  5. Algorithm Example (English) • Task: Given a sequence {ai}=a1,…,an, aiN, say what its largest element is. • Set the value of a temporary variablev (largest element seen so far) to a1’s value. • Look at the next element ai in the sequence. • If ai>v, then re-assign v to the number ai. • Repeat previous 2 steps until there are no more elements in the sequence, & return v. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  6. Executing an Algorithm • When you start up a piece of software, we say the program or its algorithm are being run or executed by the computer. • Given a description of an algorithm, you can also execute it by hand, by working through all of its steps on paper. • Before ~WWII, “computer” meant a person whose job was to run algorithms! § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  7. Executing the Max algorithm • Let {ai}=7,12,3,15,8. Find its maximum… • Set v = a1 = 7. • Look at next element: a2 = 12. • Is a2>v? Yes, so change v to 12. • Look at next element: a2 = 3. • Is 3>12? No, leave v alone…. • Is 15>12? Yes, v=15… § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  8. Algorithm Characteristics Some important features of algorithms: • Input. Information or data that comes in. • Output. Information or data that goes out. • Definiteness. Precisely defined. • Correctness. Outputs correctly relate to inputs. • Finiteness. Won’t take forever to describe or run. • Effectiveness. Individual steps are all do-able. • Generality. Works for many possible inputs. • Efficiency. Takes little time & memory to run. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  9. procedurename(argument: type) variable:=expression informal statement beginstatementsend {comment} ifcondition then statement [else statement] for variable:=initial value to final valuestatement whileconditionstatement procname(arguments) Not defined in book: returnexpression Declaration STATEMENTS Our Pseudocode Language: §A2 § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  10. procedure procname(arg: type) • Declares that the following text defines a procedure named procname that takes inputs (arguments) named arg which are data objects of the type type. • Example:proceduremaximum(L: list of integers) [statements defining maximum…] § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  11. variable:=expression • An assignment statement evaluates the expression expression, then reassigns the variable variable to the value that results. • Example:v:= 3x+7 (If x is 2, changes v to 13.) • In pseudocode (but not real code), the expression might be informal: • x:= the largest integer in the list L § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  12. Informal statement • Sometimes we may write a statement as an informal English imperative, if the meaning is still clear and precise: “swap x and y” • Keep in mind that real programming languages never allow this. • When we ask for an algorithm to do so-and-so, writing “Do so-and-so” isn’t enough! • Break down algorithm into detailed steps. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  13. Groups a sequence of statements together:beginstatement 1statement 2 …statement nend Allows sequence to be used like a single statement. Might be used: After a procedure declaration. In an if statement after then or else. In the body of a for or while loop. beginstatementsend § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  14. {comment} • Not executed (does nothing). • Natural-language text explaining some aspect of the procedure to human readers. • Also called a remark in some real programming languages. • Example: • {Note that v is the largest integer seen so far.} § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  15. ifconditionthenstatement • Evaluate the propositional expression condition. • If the resulting truth value is true, then execute the statement statement; otherwise, just skip on ahead to the next statement. • Variant: ifcondthenstmt1elsestmt2Like before, but iff truth value is false, executes stmt2. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  16. whileconditionstatement • Evaluate the propositional expression condition. • If the resulting value is true, then execute statement. • Continue repeating the above two actions over and over until finally the condition evaluates to false; then go on to the next statement. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  17. whileconditionstatement • Also equivalent to infinite nested ifs, like so: if conditionbeginstatement if conditionbeginstatement …(continue infinite nested if’s)endend § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  18. forvar:=initial to finalstmt • Initial is an integer expression. • Final is another integer expression. • Repeatedly execute stmt, first with variable var :=initial, then with var :=initial+1, then with var :=initial+2, etc., then finally with var :=final. • What happens if stmt changes the value that initial or final evaluates to? § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  19. forvar:=initial to finalstmt • For can be exactly defined in terms of while, like so: beginvar:=initialwhilevar finalbeginstmtvar:=var + 1endend § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  20. procedure(argument) • A procedure call statement invokes the named procedure, giving it as its input the value of the argument expression. • Various real programming languages refer to procedures as functions (since the procedure call notation works similarly to function application f(x)), or as subroutines, subprograms, or methods. § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  21. Max procedure in pseudocode proceduremax(a1, a2, …, an: integers) v:=a1 {largest element so far} fori:= 2 ton {go thru rest of elems} ifai > vthen v:=ai {found bigger?} {at this point v’s value is the same as the largest integer in the list} returnv § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  22. Another example task • Problem of searching an ordered list. • Given a list L of n elements that are sorted into a definite order (e.g., numeric, alphabetical), • And given a particular element x, • Determine whether x appears in the list, • and if so, return its index (position) in the list. • Problem occurs often in many contexts. • Let’s find an efficient algorithm! § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  23. Search alg. #1: Linear Search procedurelinear search(x: integer, a1, a2, …, an: distinct integers)i:= 1while (i n  x  ai)i:=i + 1ifi n then location:=ielselocation:= 0return location {index or 0 if not found} § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  24. Search alg. #2: Binary Search • Basic idea: On each step, look at the middle element of the remaining list to eliminate half of it, and quickly zero in on the desired element. <x <x <x >x § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  25. Search alg. #2: Binary Search procedurebinary search(x:integer, a1, a2, …, an: distinct integers)i:= 1 {left endpoint of search interval}j:=n {right endpoint of search interval}while i<j begin {while interval has >1 item}m:=(i+j)/2 {midpoint}ifx>amtheni := m+1 else j := mendifx = aithenlocation:=ielselocation:= 0returnlocation § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  26. Sorting alg. : Bubble Sort procedurebubblesort(a1, a2, …, an)for i:= 1 ton-1for j:= 1 to n-i ifaj>aj+1then interchange aj and aj+1 {a1, a2, …, an is in increasing order} § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  27. Sorting alg. : Insertion Sort procedureinsertionsort(a1, a2, …, an) for j:= 2 ton begini:= 1 whileaj > ai i:=i+1 m:=aj for k:= 0 to j-i-1 aj-k:=aj-k-1 ai:=m end{a1, a2, …, an are sorted} § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  28. Greedy Change-Making Alg. procedurechange(c1, c2, …, cr : c1> c2> …> cr；n) for i:= 1 tor whilen  ci begin add a coin with value cito the change n:=n- ci end § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  29. Practice exercises • 3.1.3: Devise an algorithm that finds the sum of all the integers in a list. [2 min] • proceduresum(a1, a2, …, an: integers)s:= 0 {sum of elems so far}fori:= 1 ton {go thru all elems}s:=s + ai {add current item}{at this point s is the sum of all items}returns § 3.1 – Algorithms (c)2001-2002, Michael P. Frank

  30. §3.2: The Growth of Functions § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  31. Orders of Growth • For functions over numbers, we often need to know a rough measure of how fast a function grows. • If f(x) is faster growing than g(x), then f(x) always eventually becomes larger than g(x) in the limit (for large enough values of x). • Useful in engineering for showing that one design scales better or worse than another. § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  32. Orders of Growth - Motivation • Suppose you are designing a web site to process user data (e.g., financial records). • Suppose database program A takes fA(n)=30n+8 microseconds to process any n records, while program B takes fB(n)=n2+1 microseconds to process the n records. • Which program do you choose, knowing you’ll want to support millions of users? § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  33. Visualizing Orders of Growth • On a graph, asyou go to theright, a fastergrowingfunctioneventuallybecomeslarger... fA(n)=30n+8 Value of function  fB(n)=n2+1 Increasing n  § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  34. Concept of order of growth • We say fA(n)=30n+8is order n, or O(n). It is, at most, roughly proportional to n. • fB(n)=n2+1 is order n2, or O(n2). It is roughly proportional to n2. • Any O(n2) function is faster-growing than any O(n) function. • For large numbers of user records, the O(n2) function will always take more time. § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  35. Definition: O(g), at mostorder g Let g be any function RR. • Define “at most order g”, written O(g), to be: {f:RR | c,k: x>k: f(x)  cg(x)}. • “Beyond some point k, function f is at most a constant c times g (i.e., proportional to g).” • “f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that fO(g). • Sometimes the phrase “at most” is omitted. § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  36. Points about the definition • Note that f is O(g) so long as any values of c and k exist that satisfy the definition. • But: The particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work. • You are not required to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!) However, you should prove that the values you choose do work. § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  37. “Big-O” Proof Examples • Show that 30n+8 is O(n). • Show c, k: n>k:30n+8  cn. • Show that n2+1 is O(n2). • Show c, k: n>k: n2+1  cn2. § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  38. cn =31n n>k=8  Big-O example, graphically • Note 30n+8 isn’tless than nanywhere (n>0). • It isn’t evenless than 31neverywhere. • But it is less than31neverywhere tothe right of n=8. 30n+8 30n+8O(n) Value of function  n Increasing n  § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  39. Useful Facts about Big O • Big O, as a relation, is transitive: fO(g)  gO(h)  fO(h) • O with constant multiples, roots, and logs... f (in (1)) & constantsa,bR, with b0,af, f 1-b, and (logbf)aare all O(f). • Sums of functions:If gO(f) and hO(f), then g+hO(f). § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  40. More Big-O facts • c>0, O(cf)=O(f+c)=O(fc)=O(f) • f1O(g1)  f2O(g2)  • f1 f2 O(g1g2) • f1+f2 O(g1+g2) = O(max(g1,g2)) = O(g1) if g2O(g1) (Very useful!) § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  41. Orders of Growth (§3.2) - So Far • For any g:RR, “at most order g”,O(g)  {f:RR | c,k x>k |f(x)|  |cg(x)|}. • Often, one deals only with positive functions and can ignore absolute value symbols. • “fO(g)” often written “f is O(g)”or “f=O(g)”. • The latter form is an instance of a more general convention... § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  42. Order-of-Growth Expressions • “O(f)” when used as a term in an arithmetic expression means: “some function f such that fO(f)”. • E.g.: “x2+O(x)” means “x2 plus some function that is O(x)”. • Formally, you can think of any such expression as denoting a set of functions: “x2+O(x)” : {g | fO(x): g(x)= x2+f(x)} § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  43. Order of Growth Equations • Suppose E1 and E2 are order-of-growth expressions corresponding to the sets of functions S and T, respectively. • Then the “equation” E1=E2 really means fS, gT : f=gor simply ST. • Example: x2 + O(x) = O(x2) means fO(x): gO(x2): x2+f(x)=g(x) § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  44. Useful Facts about Big O •  f,g & constantsa,bR, with b0, • af = O(f); (e.g. 3x2 = O(x2)) • f+O(f) = O(f); (e.g. x2+x = O(x2)) • Also, if f=(1) (at least order 1), then: • |f|1-b = O(f); (e.g.x1 = O(x)) • (logb |f|)a= O(f). (e.g. log x = O(x)) • g=O(fg) (e.g.x = O(x log x)) • fg O(g) (e.g.x log x  O(x)) • a=O(f) (e.g. 3 = O(x)) § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  45. Definition: (g), exactly order g • If fO(g) and gO(f) then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g). • Another equivalent definition:(g)  {f:RR | c1c2k x>k: |c1g(x)||f(x)||c2g(x)| } • “Everywhere beyond some point k, f(x)lies in between two multiples of g(x).” § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  46. Rules for  • Mostly like rules for O( ), except: •  f,g>0 & constantsa,bR, with b>0,af (f), but  Same as with O.f  (fg) unless g=(1)  Unlike O.|f| 1-b (f), and  Unlike with O.(logb |f|)c (f).  Unlike with O. • The functions in the latter two cases we say are strictly of lower order than (f). § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  47.  example • Determine whether: • Solution: § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  48. Other Order-of-Growth Relations • (g) = {f | gO(f)}“The functions that are at least orderg.” • o(g) = {f | c>0 k x>k : |f(x)| < |cg(x)|}“The functions that are strictly lower order than g.” o(g)  O(g)  (g). • (g) = {f | c>0 k x>k : |cg(x)| < |f(x)|}“The functions that are strictly higher order than g.” (g)  (g)  (g). § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  49. Relations Between the Relations • Subset relations between order-of-growth sets. RR ( f ) O( f ) • f o( f ) ( f ) ( f ) § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

  50. Why o(f )O(x)(x) • A function that is O(x), but neither o(x) nor (x): § 3.2 – The Growth of Functions (c)2001-2002, Michael P. Frank

More Related