1 / 36

Algorithms and data structures

Algorithms and data structures. Protected by http://creativecommons.org/licenses/by-nc-sa/3.0/hr/. Creative Commons. You are free to: share — copy and redistribute the material in any medium or format adapt — remix, transform, and build upon the material Under the following terms:

aletha
Download Presentation

Algorithms and data structures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithms and data structures Protected by http://creativecommons.org/licenses/by-nc-sa/3.0/hr/

  2. Creative Commons • You are free to: • share — copy and redistribute the material in any medium or format • adapt — remix, transform, and build upon the material • Under the following terms: • Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. • NonCommercial — You may not use the material for commercial purposes. • ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Notices: You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation. No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material. Text copiedfrom http://creativecommons.org/licenses/by-nc-sa/3.0/ Algorithms and data structures, FER

  3. Algorithms and complexity Algorithms Complexity of algorithms

  4. Abu Ja'far Muhammad ibn Musa Al-Khwarizmi (the sources differ regarding his full name and transcription!) أبو جعفر محمد بن موسى الخوارزمي Mohammad, father of Jafar, son of Musa from Khwarezm Born in Khwarezm, what is today Khiva, Uzbekistan, about year 780. Died in Baghdad about year 850. One of the 10 most appreciated mathematicians of all times. Abu Ja'far Muhammad ibn Musa Al-Khwarizmi Algorithms and data structures, FER

  5. Achievements of al Khwarizmi • Encourages using of Hindu/Arabic numbers – introduces zero • About year 825 he wrote the book “Kitab al-jabr w'al-muqabala” in Baghdad • الكتاب المختصر في حساب الجبر والمقابلة • The Compendious Book on Calculation by Completion and Balancing • jabr -  moving to the other side of an equation x - 2 = 12  x = 12 + 2 • muqabala - equal subtraction from both sides of an equation • x + y = y + 7  x = 7 • al-jabr  algebra - bone-setting Algorithms and data structures, FER

  6. From al Khwarizmito algorithms... • He believed that any mathematical problem can be split into steps, i.e. to a sequence of rules. • In the Latin translation of his book (12th century), in front of every rule there is: • Dixit Algorizmi - said Al Khwarizmi  • The algorithm is as follows... • The very first understanding of the word “algorithm” refers only to calculation rules related to numbers; later it also refers to rules for solving other tasks in mathematics • In the 20th century, with emerging of computers, the term is extended to computing, and later also to other fields • Rules to achieve a required goal Algorithms and data structures, FER

  7. What is an algorithm • A precisely described way to solve a problem • Uniquely defines what has to be done • Initial objects must be defined, belonging to a class of objects on which operations can be performed • As outcome appear final objects or results • Final number of steps; every step is described by an instruction • Execution is an algorithmic process • Applicable, if the result can be obtained in a finite time • Examples for unallowed instructions: • Calculate 5/0 • Increase x for 6 or 7 Algorithms and data structures, FER

  8. Algorithm • An algorithm must be effective: • The result should be achievable by using pencil and paper, in a finite amount time • Examples: • Addition of integers is effective • Addition of real numbers is not because a number with infinite amount of digits may appear • With programming skills and with understanding the nature of the problem to be solved, the student can write an effective algorithm • The goal of this course is to learn how to design and program an efficient algorithm. • Effective and efficient are not synonyms • effective = the one that brings effect, results • efficient = effective & successful (regarding the consumption of resources - time, processor, disk, memory) • Multiplication can be performed as repetitive addition - effective but not efficient! • Solving of a large set of linear equations using the Kramer’s rule - effective but not efficient! • Wikipedia: A simple way of distinguishing between efficiency and effectiveness is the saying, "Efficiency is doing things right, while Effectiveness is doing the right things. " Algorithms and data structures, FER

  9. Procedure • Procedure has all properties of an algorithm, except it does not necessarily terminate after a finite number of steps • In language C that may be a void function • Examples for procedure: • Operating system • Text editor • Execution time must be “reasonable” • Example: • An algorithm that would play chess choosing at each move all the possible consequences of it, would require billions of years even on a quickest conceivable computer Procedure Algorithm Algorithms and data structures, FER

  10. Analysing algorithms • Program - description of an algorithm in a programming language that uniquely defines what the computer should do • Programming – to learn the syntax of a procedural language and to acquire basic intuitive skills for algorithmisation of a problem being verbally described • Algorithms + data structures = programs (Wirth) • How do devise algorithms? • How to structure data? • How to express the algorithms? • How to verify the algorithm correctness? • How to analyse algorithms? • How to check (test) a program? • The algorithm design procedures are not unique and require creativity. Otherwise, algorithm generators would already exist. It means that (for the time being?) the content of this course cannot be completely algorithmised. We shall use the programming language C. For a compact description of complex algorithms pseudo-code may be used. Algorithms and data structures, FER

  11. Effective but not efficient? Important! Due to a new computer program, which has slowed down our work I beg the patients for patience! Thanks, Your Doctor Algorithms and data structures, FER

  12. Analysis of complexity "a priori" and "a posteriori“ analyses O-notation Asymptotic running time Examples

  13. Basics • Purpose • Intellectual amusement? • Prediction of execution time • Search for more efficient algorithms • Hypotheses: • Sequential single processor computer • Fixed time (or limited by an upper bound) for retrieval of contents of a memory location • Time for processing of an operation (arithmetic, logical, assignment, function call) is upper bounded by a constant Algorithms and data structures, FER

  14. “a priori” and “a posteriori” analyses • Selection of data sets for exhaustive algorithm testing: • best case scenario • worst case scenario • average (typical) behaviour • a priori • Duration of algorithm execution (worst case scenario) expressed as a function of some relevant arguments (e.g. amount of data) • a posteriori • Statistics achieved by measurements on computer Algorithms and data structures, FER

  15. “a priori” analysis • Estimation of execution time, independent of computer type, programming language, compiler... • examples: a) x += y;1 b) for(i = 1; i <= n; i++) {n x += y; } c) for(i = 1; i <= n; i++) {n2 for(j = 1; j <= n; j++) { x += y; } } Algorithms and data structures, FER

  16. Algortihm complexity • A tool is needed for comparison of algorithmsefficiency • Application of mathematics • Execution time is observed when the count of input data becomes sufficiently large • Example: execution of inversion of a n x n matrix, measured in seconds is in proportion to n3 • If the inversion of a 100 x 100 elements requires 1 minute, • then the inversion of a 200 x 200 matrix will take 8 minutes (23=8). • How to express the execution time as a function of n? • E.g. execution time grows with the matrix size as n3 Algorithms and data structures, FER

  17. O - notation • f(n) = O(g(n)) if there exist two positive constantscandn0such thatis valid f(n)cg(n)for allnn0 • We look for theminimum g(n)to satisfy the above requirement • In other words, the execution time is estimated as an order of magnitude of f(n), determined by the quantity of data n, multiplied by some constantc • examples: • n3+5n2+77n = O(n3) • How much work is required to move 1chair from the room A to the room B? • How much work is required to move nchairs from A to B? • How much work is required to move nchairs from A to B, whereby when each new chair is brought to B, all the present chairs there have to be shifted, supposing that for shifting of a single chair, the same effort is required like when bringing it from A to B? 1 + 2 + 3 + 4 + ...+ n = n(n+1)/2 = n2/2 + n/2 = O (n2) Algorithms and data structures, FER

  18. O - notation • By “a priori” analysis, the execution time O(g(n))of an algorithm is obtained • If the number of operations to be performed in an algorithm depends upon an input argument n in the form of a polynomial of mth degree, then the execution time for that algorithm equals O(nm). • Proof: If the execution time follows a polynomial dependency: A(n) = amnm + ... + a1n + a0 then it is valid: A(n)amnm + ... +a1n +a0 A(n)(am + am-1/n + ... +a1/nm-1+a0/nm)nm A(n) (am+ ... +a1 +a0)nm, for everyn1 • with: c = am+ ... +a1 +a0andn0= 1the statement is proved. Algorithms and data structures, FER

  19. O - notation • The value for ccan be any constant grater thenamifnis big enough: If an algorithm consists ofksegments with respective execution times: c1nm1, c2nm2,...,cknmk Then the total time is c1nm1 + c2nm2 +... + cknmk • This implies that for this algorithm the execution time is equal toO(nm), where m = max{mi}, i =1,...,k Algorithms and data structures, FER

  20. O - notation • For any nbig enough, it holds: O(1) < O(log n) < O(n) < O(nlog n) < O(n2) < O(n3) < ...< O(2n) < O(n!) • O(1) means that the execution time is upper bounded by a constant • Other values, until the penultimate, represent polynomial execution times • Each subsequent execution time is larger for an order of magnitude • Penultimate expression represents exponential execution time • No polynomial can be its upper bound because for n big enough, this function exceeds any polynomial • Algorithms requiring exponential time can be insolvable in a reasonable time, regardless to the processing speed of a sequential computer Algorithms and data structures, FER

  21.  -notation • f(n) = (g(n)) if there exist positive constants c and n0 such that is valid f(n)cg(n)for alln > n0 • We look for the maximum g(n) to satisfy the above requirement • Lower bound for an algorithm execution time • In no case the execution can be shorter than  • E.g. multiplication of two n x n matrices always requires time (n3) , O (n3) • Addition of n numbers always requires the time (n) = O (n) • Retrieval of a number among n unsorted numbers (1)  O (n) Algorithms and data structures, FER

  22. -notation • f(n) = (g(n)) if there exist positive constants c1,c2and n0 such that is valid c1g(n)f(n)c2g(n)for alln > n0 • f and g grow equally fast for largen • The ratio of f and gis betweenc1andc2 • In other words, the execution times are equal for the best and the worst case • Example: • Sort nwritten examinations lexicographically according to students’ names. The process starts by finding the first one, than in the remaining set it looks for the first one, and so on. • The process takes the same time regardless of possible ordering of the elements: O(n2) = (n2) =  (n2) Algorithms and data structures, FER

  23. Asymptotic execution time • f(n) ~ g(n) if • It is pronounced: "f(n)is asymptotically equal to the functiong(n)” • A more precise estimation of the execution timethan by O-notation • The leading member’s order of magnitude is known but also its constant multiplier • If, for example: f(n) = aknk+ ... + a0 then: f(n) =O(nk)andf(n) ~aknk • Examples: • 3*2x+7 logx+x ~ 3*2x • For the example from the previous slide (sorting of written examinations) • Due to linear decrement of the remaining set, where the first member is searched, asymptotic duration is:~n2/2 Algorithms and data structures, FER

  24. Asymptotic execution time • While counting repetitions in some algorithms, the summations such as follows often appear:  n2/2 • for (i = 1; i <= n; i++) • for (j = 0; j < i; i++) {...} n  i = n(n+1)/2 = O (n2) i=1 Algorithms and data structures, FER

  25. Different complexities in logarithmic scale Algorithms and data structures, FER

  26. O-notation - examples IspisiTrazi.c (PrintSearch.c) • print • The loop is always repeated n times (n) , O(n) , (n) • search • Execution time is O(n) • Lower bound is  (1) • In the best case, looked for array member is found in the first step • In the worst case, all the array members have to be examined Algorithms and data structures, FER

  27. “a posteriori” analysis • Real time necessary to execute an algorithm on a specific computer • #include <sys\timeb.h> // where it is declared • struct timeb { • time_t time; // elapsed seconds since midnight, • // 01.01.1970, UTC • unsigned short millitm; // milliseconds • short timezone; // difference to UTC, in minutes • short dstflag; // <>0 if daylight saving time is active • }; • void ftime(struct timeb *timeptr); Algorithms and data structures, FER

  28. “a posteriori” analysis • For time measurement, the program should include: • Universal Time Co-ordinated (UTC) • The old term wasGreenwich MeanTime (GMT) • struct timeb time1, time2; long durationms; • ftime (&time1); • ... • ftime (&time2); • durationms = 1000 * (time2.time - time1.time) + • time2.millitm - time1.millitm; Algorithms and data structures, FER

  29. Exercises • For the algorithm determine: a) a priori execution time b) asymptotic estimate for average execution time c) asymptotic estimate for the best case execution time d) asymptoticestimate for the worst case execution time e) for what value of the variable b happens the worst case? • for (i = 0; i < n; i++) { • if (a[i] == b) { • printf ("i = %d\n", i); • break; • } • } Algorithms and data structures, FER

  30. Exercises • Write the function to calculate the sum of digits for a given natural number N. What is the function complexity? • intsumDigits (int N) { • int sum = 0; • while (N > 0) { • sum += N % 10; • N /= 10; • } • return sum; • } Algorithms and data structures, FER

  31. Exercises Algorithms and data structures, FER

  32. Examples of dependency of execution time on the amount of data and complexity Algorithms and data structures, FER

  33. What can be solved in a given time Algorithms and data structures, FER

  34. Relative contribution of components in the expression: n4 + n3log n + 2 (n-1) O ( 2n ) Algorithms and data structures, FER

  35. Influence of the value of the constantK = 1 000 000 Algorithms and data structures, FER

  36. What does it mean “n big enough”, i.e. n > n0 • Source: Cormen,Leiserson & Rivest: Introduction to algorithms, 2/e,MIT Press, 2001 Algorithms and data structures, FER

More Related