1 / 32

Analyzing Algorithms Growth of Functions

CSE 5350/7350 Introduction to Algorithms. Analyzing Algorithms Growth of Functions. (Chapters 1 & 2) Mihaela Iridon , Ph.D. mihaela@engr.smu.edu. What means analyzing algorithms?. Predicting the required resources What do we measure? Computational time Memory Communication bandwidth

minya
Download Presentation

Analyzing Algorithms Growth of Functions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSE 5350/7350 Introduction to Algorithms Analyzing AlgorithmsGrowth of Functions (Chapters 1 & 2) Mihaela Iridon, Ph.D. mihaela@engr.smu.edu Analyzing Algorithms Growth of Functions

  2. What means analyzing algorithms? • Predicting the required resources • What do we measure? • Computational time • Memory • Communication bandwidth • Other • Model constructs: • Technology • Resources (hardware & software) • Associated costs • Assumptions (1 processor, RAM-model: sequential operations) Analyzing Algorithms Growth of Functions

  3. Tools used for algorithm analysis • Mathematical tools: • Discrete combinatorics • Probability theory • Ability to single out the predominant operations (most significant terms in a formula) • Modeling and simulation tools • Software utilities, benchmarking programs Analyzing Algorithms Growth of Functions

  4. Terminology • Input Size • In general the time to execute a set of operations is dependent on the size of the input • Depends on the problem definition • = the number of items in the input • Could be more than one number (e.g. a graph) • Running Time • = the number of primitive operations (steps) executed • Machine-independent term Analyzing Algorithms Growth of Functions

  5. Example (1) – Insertion Sort (In Place) ///<summary> /// Sorts the input list of integers by using the Insertion Sorting algorithm /// (see Cormen textbook, Chapter 1.1) ///</summary> ///<param name="input">Input list of integers (to be sorted) – input listwill be modified</param> public static void InsertionSort(List<int> input) { if (input == null) return; //NULL input if (input.Count < 2) return; //one-element array; nothing to sort int i=0, key=0; for (int j = 1; j < input.Count; j++) { key = input[j]; //insert input[j] into the sorted sequence input[0..j-1] i = j-1; while (i > -1 && input[i] > key) { input[i+1] = input[i]; i--; } input[i+1] = key; } } Cost # of Times executed C1 n C2 n-1 0 n-1 C4 n-1 C5 Σ(j=1..n-1)tj C6 Σ(j=1..n-1)(tj – 1) C7 Σ(j=1..n-1)(tj – 1) C8 n-1 Analyzing Algorithms Growth of Functions

  6. Example (1) – Insertion Sort (In Place) • The running time depends on the input value • If the input is already sorted then the body of the while loop does not execute and the best case scenario/running time for insertion sort is: T(n) = c1 n + c2 (n-1) + c4 (n-1) + c5 (n-1) + c8 (n-1) = (c1 + c2 + c4 + c5 + c8) * n – (c2 + c4 + c5 + c8) = a * n + b  linear function of n • If the input is in reverse sorted order: (worst case scenario) T(n) = c1 n + c2 (n-1) + c4 (n-1) + c5 (n(n+1)/2 - 1) + c6 [n(n-1)/2] + c7 [n(n-1)/2] + c8 (n-1) = a * n2 + b * n + c  quadratic function of n Analyzing Algorithms Growth of Functions

  7. Worst-case & Average-case Analysis • Worst-case running time: • The longest running time for any input of size n (i.e. the longest path in the execution) • Upper bound on the running time for any input • Occurs fairly often • The average-case ~ the worst-case • Average-case running time: • Difficult to define what average input means • Example for Insertion Sort: On average, half the elements in an array A1 ... Aj-1 are less than an element Aj, and half are greater. Analyzing Algorithms Growth of Functions

  8. Order/Rate of Growth • Simplifying abstraction • Consider only the leading term • Ignore the leading term’s constant coefficient • Worst-case running time for Insertion Sort is Θ(n2) (theta of n-squared) • An algorithm with Θ(n2) will run faster than one with Θ(n3) Analyzing Algorithms Growth of Functions

  9. Divide-and-Conquer Algorithms • Incremental approach (e.g. Insertion Sort) • Divide-and-conquer approach (e.g. recursive algorithms such as Merge Sort) • [Divide] Break the problem into related or similar sub-problems of smaller size; • [Conquer] Solve the sub-problems • [Combine] Combine the solutions Analyzing Algorithms Growth of Functions

  10. Analyzing divide-and-conquer algorithms • Recursive approach  use recurrence equation (recurrence) to describe the running time • T(n) = running time on a problem of size n • If n= small (n <= c) then Θ(1) • Otherwise, divide problem in a sub-problems, each of size n / b • D(n) = time to divide • C(n) = time to combine • T(n) = a*T(n/b) + D(n) + C(n) (when n > c) Analyzing Algorithms Growth of Functions

  11. Merge Sort (1) public static void MergeSort2(List<int> input, int startIx, int endIx) { if (input == null) return; if (startIx == endIx)return; //stop condition int middle = (int)Math.Floor((endIx + startIx) / 2.0); MergeSort2(input, startIx, middle); MergeSort2(input, middle + 1, endIx); Combine2(input, startIx, middle, endIx); } Analyzing Algorithms Growth of Functions

  12. Merge Sort (2) public static void Combine2(List<int> input, int i1, int i2, int i3) { if (input == null || input.Count == 0) return; List<int> result = new List<int>(i3 - i1 + 1); //not 100% in-place int ix1 = i1, ix2 = i2+1; while (result.Count < i3 - i1 + 1) { while (ix1 < i2 + 1 && (ix2 == (i3 + 1) || input[ix1] < input[ix2])) result.Add(input[ix1++]); while (ix2 < i3 + 1 && (ix1 == (i2 + 1) || input[ix1] > input[ix2])) result.Add(input[ix2++]); } for (int j = i1; j <= i3; j++) input[j] = result[j - i1]; } Analyzing Algorithms Growth of Functions

  13. Analyzing Merge Sort • [Divide] D(n) = Θ(1) • [Conquer] 2 * T(n/2) • [Combine] C(n) = Θ(n) • T(n) = Θ(1) if n=1 2T(n/2) + Θ(n) if n≥1 = Θ(n * log2n)  better than Insertion Sort { Analyzing Algorithms Growth of Functions

  14. Growth of Functions • Algorithms efficiency • Compare relative performance of alternative algorithms • Analysis for large input size: e.g. n ∞ • Asymptotic efficiency of algorithms: • Input size is large enough to make only the order of growth of the running time relevant • How the running time increases with the size of the input in the limit, as the size of the input increases without bound Analyzing Algorithms Growth of Functions

  15. Asymptotic Notation • Notations used to describe the asymptotic running time of an algorithm • Are defined in terms of functions whose domains are the set N = {0, 1, 2, …} • Convenient for describing the worst-case running-time function T(n) • Notation  abused vs. misused Analyzing Algorithms Growth of Functions

  16. Θ-notation • Θ: asymptotically bounds a function from above and below f(n) = Θ(g(n)) indicates f(n) Θ(g(n)) or g(n) is an asymptotically tight bound for f(n) Analyzing Algorithms Growth of Functions

  17. O-notation • O: asymptotic upper bound f(n) = Θ(g(n))  f(n) = O(g(n)) • The Θ-notation is strongerthan the O-notation • Example: n = O(n2) • E.g.: worst-case for insertion sort = O(n2) • Notation abuse: The running time of insertion sort is O(n2). • The running time depends on the particular input of size n. • It is true only for the worst-case scenario (i.e. no matter what particular input of size n is chosen for each value of n) Analyzing Algorithms Growth of Functions

  18. Ω-notation • Ω: asymptotic lower bound • Theorem  f(n) and g(n), f(n) = Θ(g(n)) iff f(n) = O(g(n)) and f(n) = Ω(g(n)) • Used to bound the best-case running time • E.g.: best-case for insertion sort is Ω(n) • Worst-case for insertion sort is Ω(n2) Analyzing Algorithms Growth of Functions

  19. Graphical Comparison Analyzing Algorithms Growth of Functions

  20. o-notation • Upper bound that is not asymptotically tight • 2n2 = O(n2) is asymptotically tight • 2n = O(n2) is not asymptotically tight • 2n = o(n2), but 2n2 o(n2) Analyzing Algorithms Growth of Functions

  21. -notation • Lower bound that is not asymptotically tight • f(n)  (g(n)) iff g(n)  o(f(n)) • E.g.: n2/2 = (n), but n2/2 = (n2) Analyzing Algorithms Growth of Functions

  22. Comparison of Functions • Transitivity (for all five notations): If f(n) = X(g(n)) and g(n) = X(h(n))  f(n) = X(h(n)) • Reflexivity (for the big-X notations) f(n) = X(f(n)) • Symmetry: f(n) = Θ(g(n)) iff g(n) = Θ(f(n)) • Transpose symmetry: f(n) = O(g(n)) iff g(n) = Ω(n) f(n) = o(g(n)) iff g(n) = (n) Analyzing Algorithms Growth of Functions

  23. Analyzing Algorithms - Addendum Finding the largest clique in a graph Parsing an object model Analyzing Algorithms Growth of Functions

  24. Finding the largest clique • Graph: G(V,E) • A graph or undirected graphG is an ordered pair G: = (V,E) that is subject to the following conditions: • V is a set, whose elements are called vertices or nodes, • E is a set of pairs (unordered) of distinct vertices, called edges or lines. • A clique in an undirected graph G is a set of vertices V such that for every two vertices in V, there exists an edge connecting the two. (Complete sub-graph) Analyzing Algorithms Growth of Functions

  25. The Clique Problem • determining whether a graph contains a clique of at least a given size k. • Verification of actual clique : trivial • The clique problem is in NP (non-deterministic polynomial time). • NP-complete • The corresponding optimization problem, the maximum clique problem, is to find the largest clique in a graph. Analyzing Algorithms Growth of Functions

  26. Brute force approach to the clique problem • Examine each sub-graph with at least k vertices and check to see if it forms a clique • Number of cases to inspect: • A clique C=(vi1, vi2, .., vin) exists only when its n sub-cliques each of size n-1 exist. • Event-raising mechanism to increment a counter of sub-cliques using the threshold graph • Space required to build all cliques until G=completely connected: Analyzing Algorithms Growth of Functions

  27. Sorting an object model • Input: • A collection of property paths • Output: • A sorted collection of property paths Analyzing Algorithms Growth of Functions

  28. Input Sample (excerpt) • Company.Principals[0].References[0].ResidentialInfos[0].Address.AddressType • Company.Principals[0].References[0].ResidentialInfos[1].Address.AddressType • Company.Principals[0].References[1].ResidentialInfos[0].Address.AddressType • Company.Principals[0].References[1].ResidentialInfos[1].Address.AddressType • Company.Principals[1].References[0].ResidentialInfos[0].Address.AddressType • Company.Principals[1].References[0].ResidentialInfos[1].Address.AddressType • Company.Principals[1].References[1].ResidentialInfos[0].Address.AddressType • Company.Principals[1].References[1].ResidentialInfos[1].Address.AddressType • Company.Principals[0].References[0].ResidentialInfos[0].Address.StreetAddressLine1 • Company.Principals[0].References[0].ResidentialInfos[1].Address.StreetAddressLine1 • Company.Principals[0].References[1].ResidentialInfos[0].Address.StreetAddressLine1 • Company.Principals[0].References[1].ResidentialInfos[1].Address.StreetAddressLine1 • Company.Principals[1].References[0].ResidentialInfos[0].Address.StreetAddressLine1 • Company.Principals[1].References[0].ResidentialInfos[1].Address.StreetAddressLine1 • Company.Principals[1].References[1].ResidentialInfos[0].Address.StreetAddressLine1 • Company.Principals[1].References[1].ResidentialInfos[1].Address.StreetAddressLine1 • Company.Principals[0].References[0].ResidentialInfos[0].Address.StreetAddressLine2 • Company.Principals[0].References[0].ResidentialInfos[1].Address.StreetAddressLine2 • Company.Principals[0].References[1].ResidentialInfos[0].Address.StreetAddressLine2 • Company.Principals[0].References[1].ResidentialInfos[1].Address.StreetAddressLine2 • Company.Principals[1].References[0].ResidentialInfos[0].Address.StreetAddressLine2 • Company.Principals[1].References[0].ResidentialInfos[1].Address.StreetAddressLine2 • Company.Principals[1].References[1].ResidentialInfos[0].Address.StreetAddressLine2 • Company.Principals[1].References[1].ResidentialInfos[1].Address.StreetAddressLine2 Analyzing Algorithms Growth of Functions

  29. Object Model Principal EmploymentInfos[] IncomeInfos[] TradeLines[] References[] Assets[] IdentificationInfos[] ResidentialInfos[] ContactInfo PersonalInfo PrincipalType EmployedByCompany YearsAsOwner IndividualOrJointType PersonType 1..* 1 Company Principals[] FinancialStatements[] TradeLines[] Contacts[] Declarations[] Documents[] Addresses[] CompanyInfo BusinessAddress MailingAddress RelationshipSummary Address AddressType StreetAddressLine1 StreetAddressLine2 City County State Zipcode Country 1 1..* Analyzing Algorithms Growth of Functions

  30. Output sample (sorted object model) • Company.Principals[0].EmploymentInfos[0].EmployerAddress.AddressType • Company.Principals[0].EmploymentInfos[0].EmployerAddress.StreetAddressLine1 • Company.Principals[0].EmploymentInfos[0].EmployerAddress.StreetAddressLine2 • Company.Principals[0].EmploymentInfos[0].EmployerAddress.City • Company.Principals[0].EmploymentInfos[0].EmployerAddress.County • Company.Principals[0].EmploymentInfos[0].EmployerAddress.State • Company.Principals[0].EmploymentInfos[0].EmployerAddress.ZipCode • Company.Principals[0].EmploymentInfos[0].EmployerAddress.Country • Company.Principals[0].EmploymentInfos[0].EmployerName • Company.Principals[0].EmploymentInfos[0].EmploymentType • Company.Principals[0].EmploymentInfos[0].OccupationType • Company.Principals[0].EmploymentInfos[0].YearsOfEmployment • Company.Principals[0].EmploymentInfos[0].MonthsOfEmployment • Company.Principals[0].EmploymentInfos[0].Title • Company.Principals[0].EmploymentInfos[0].Department • Company.Principals[0].EmploymentInfos[1].EmployerAddress.AddressType • Company.Principals[0].EmploymentInfos[1].EmployerAddress.StreetAddressLine1 • Company.Principals[0].EmploymentInfos[1].EmployerAddress.StreetAddressLine2 • Company.Principals[0].EmploymentInfos[1].EmployerAddress.City • Company.Principals[0].EmploymentInfos[1].EmployerAddress.County • Company.Principals[0].EmploymentInfos[1].EmployerAddress.State • Company.Principals[0].EmploymentInfos[1].EmployerAddress.ZipCode • Company.Principals[0].EmploymentInfos[1].EmployerAddress.Country • Company.Principals[0].EmploymentInfos[1].EmployerName Analyzing Algorithms Growth of Functions

  31. Data Graph ROOT ApplicationNumber ConversationLogs Comment DateTimeStamp IncludeInUDR EmploymentInfos Company Principals IncomeInfos FinancialStatements TradeLines Analyzing Algorithms Growth of Functions

  32. Data Structures Node Text : string Count : int CrtIndex : int IsLast : bool NextNode : Node RightNode : Node Analyzing Algorithms Growth of Functions

More Related