Design & Analysis of Algoritms

Design & Analysis of Algoritms Ernastuti

Algorithm • The word algorithm comes from the name of a Persian author, Abu Ja’far Mohammed ibn Musa al Khowarizmi (825 A.D) who wrote a textbook on mathematics.

Algorithm • Algorithm is any special method of solving a certain kind of problem. • But this word has taken on a special significance in computer science, where algorithm has come to refer to a precise method useable by a computer for the solution of a problem.

This is what makes the notion of an algorithm different from words such as process, technique or method.

An Algorithm is composed of finite set of steps, each of which may require one or more operations. • Each operation must definite, meaning that it must be perfectly clear what should be done. • Directions such as “compute 5/0” or “add 6 or 7 to x” are not permitted because it is not clear what the result is or which of the two possibilities should be done.

Another important property each operation should have is that it be effective; each step must be such that it can, at least in principle, be done by a person using pencil and paper in a finite amount of time. • Performing arithmetic on integers is an example of an effective operation, but arithmetic with real numbers is not, since some values may be expressible only by an infinitely long decimal expansion. Adding two such numbers would violate the effectiveness property.

An algorithm produces one or more outputs and may have zero or more inputs which are externally supplied.

Another important criterion which will be assumed about algorithms is that they terminate after a finite number of operations.

There is another word for an algorithm which obeys all of the above properties (finite set of steps, definite, effective) except termination, and that is computational procedure. • One important example of a computational procedure is the operating system of a digital computer. • This procedure is designed to control the execution of jobs, such that when no jobs are available, it does not terminate, but continues in a waiting state until a new job is entered. • Though computational procedures include important examples such as this one, we will restrict our study to these computional procedures which always terminate.

Algorithm • Informally, an Algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. • An Algorithm is thus a squence of computational steps that transform the input into the output.

Program • A Program is the expression of an algorithm in a programming language.

In order to help us achieve the criterion of definiteness, algorithm will be written in a programming language. • Most of you have already programmed and run some algorithms on a computer. • This is desirable because before one studies a concept in general it helps if one has had some practical experience with it.

Perhaps you had some difficulty getting started in formulating an initial solution to a problem, or perhaps you were unable to decide which of two algorithms was better. • The goal of this course is to teach you how to make these decisions.

The study of algorithms includes many important and active areas of research. • There are perhaps five distinct areas of study one can identify : 1. How to devise algorithms 2. How to analyze algorithms 3. How to express algorithms 4. How to validate algorithms 5. How to test algorithms

1. How to devise algorithms • The act of creating an algorithm is an art which may never be fully automated. • A major goal of this course is to study various design techniques which have proven to be useful in that they have often yielded good algorithms.

Design of algorithm • A knowledge of design will certainly help one to create a good algorithms, yet without the tools of analysis there is no way to determine the quality of the result. • There are several design strategies/ techniques , such as : devide-and-conquer, the greedy method, dynamic programming, search- and-traversal, backtracking & branch-and-bound.

2. How to analyze algorithms • This field of study is called analysis algorithms. • As an algorithm is executed, it makes use of the computer’s central processing unit (CPU) to perform operations and it uses the memory to hold the program and its data. • Analysis of algorithms refers to the process of determining how much computing time and storage an algorithm will require. • This is a challenging one which sometimes requires great mathematical skill.

Analysis of algorithm • Analyzing even a simple algorithm can be a challenge. • The mathematical tools required may include discrete combinatorics, elementary probability theory, algebraic dexterity, and the ability to identify the most significant terms in a formula.

3. How to express algorithms • A Program is the expression of an algorithm in a programming language.

4. How to validate algorithms • Once an algorithm is devised it is necessary to show that it computes the correct answer for all possible legal inputs. • We refer to this process as algorithm validation. • The purpose of the validation is to assure us that this algorithm will work correctly independent of the issues concerning the programming language it will eventually be written in.

Once the validity of the method has been shown, a program can be written and a second phase begins. This phase is referred to as program verification. • A proof of correctness requires that the solution be stated in two forms. • One form is usually as a program which is annotated by a set of assertions about the input and output variables of the program. These assertions are often expressed in the predicate calculus. • The second form is called a specification and this may also be expressed in the predicate calculus. • A complete proof of program correctness requires that each statement of the programming language be pricisely defined and that all basic operations be proved correct. • All these details may cause a proof to be very much longer than the program.

5. How to test a program • Testing a program really consists of two phases Debugging and Profilling. • Debugging is the process of executing programs on sample data sets to determine if faulty results occur and, if so, to correct them ! • A proof of correctness is much more valuable than a thousand tests, ( if that proof is correct ), since it guarantees that the program will work correctly for all possible inputs.

Profilling is the process of executing a correct program on data sets and measuring the time and space it takes to compute the results. • This timing figures are useful in that they may confirm a previously done analysis and point out logical places to perform useful optimization.

As we can’t hope to cover all of these five categories completely, we will content ourselves with concentrating on design and analysis, spending less time on program construction and correctness. • One can see that the subject of algorithms is a very diverse and challenging one.

What are the prerequisites for understanding the D&AoA course • You should have some programming experience. In particular, you should understand recursive procedures and simple data structures as arrays and linked lists. • You should have some facility with proofs by mathematical induction.

Algorithm devised to solve the same problem often differ dramatically in their efficiency. • These differences can be much more significant than the difference between a personal computer and supercomputer. • As an example, let us pit a supercomputer running insertion sort againts a small personal computer running mergesort. • They each must sort an array of one million numbers.

Suppose the supercomputer executes 100 million instructions per second, while the personal computer executes only one million instructions per second. • To make the difference even more dramatic, suppose that the world’s craftiest programmer codes insertion sort in machine language for the supercomputer, and the resulting code requires 2n2 supercomputer instructions to sort n numbers. • Mergesort, on the other hand, is programmed for personal computer by an average programming using a high level language with an inefficient compiler, with the resulting code taking 50n log n .

To sort a million numbers : The supercomputer takes 2.(106)2 instructions / 108 instructions/seconds = 20,000 seconds  5.56 hours While the personal computer takes 50.(106) log 106 instr / 106 instructions/seconds  1,000 seconds  16.67 minutes

By using an algorithm whose running time has a lower order of growth, even with a poor compiler, the personal computer runs 20 times faster than the supercomputer. • This example shows that algorithms, like computer hardware, are a technology. • Total system performance depends on choosing efficient algorithms as much as on choosing fast hardware. • Just as rapid advances are being made in other computer technologies, they are being made in algorithms as well.

Design & Analysis of Algoritms