CS 575 Design and Analysis of Computer Algorithms Professor Michal Cutler Introduction

CS 575Design and Analysis of Computer AlgorithmsProfessor Michal CutlerIntroduction

This class • General Information • Web page for course http://bingweb.binghamton.edu/ ~cs575c/f07/index.html • Goals • Analysis • Problem size

Goals of the course: Prepare students for: Job interviews Future technical challenges Using critical thinking for problem solving Implementing algorithms efficiently and correctly Arguing correctness Analyzing time complexity Using common algorithms (building blocks) Learning to design using well known methods

Job interviews (1) • Job interviews contain many data structures and algorithm related questions • “Describe binary search” • “Describe an efficient algorithm that reads a file containing in random order all the numbers between 100 and 2035 excluding one, and determines the missing number. • The algorithm should be linear in time and use only constantspace”

Job interviews (2) • Often you will be asked to program your solutions • “Write a program to find the first character of a string that occurs only once” [A] • “Write a program to decide whether a linked list contains a cycle”

Job interviews (3) • Interviews also contain puzzles and brain teasers • The goal is to test: • Creative, logical and critical thinking • Basic knowledge

What you should do (1) • Repeat the question, and make sure that you understand it (ask questions if you don’t) • Try a simple example. For question [A] try the string “necessary sufficient” “a” is the answer for this example. • Consider alternative designs and data structures for the solution • Analyze the different computation times • Choose the best one • Often there is a very simple solution, and less obvious more efficient ones

What you should do (2) • Write the code making sure to deal with possible exceptions • Walk thru the code with a specific example • Make corrections if necessary

What should universities teach? From Communications ACM, September 2003, pp39 “A traditional university education is to provide the foundations for further learning. It provides just-in-case learning rather than just-in-time learning provided by on-the-job training”

Critical thinking Glaser’s definition: Critical thinking calls for a persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusion to which it tends (Glaser 1941)

Critical thinking Ennis’s definition: “Critical thinking is reasonable reflective thinking that is focused on deciding what to believe or do” (Norris and Ennis 1989)

Using critical thinking for problem solving • Considering different approaches for solving a problem (for example dynamic vectors) • Analyzing the merits of each • Considering different implementations for a chosen approach (for example Prim’s algorithm) • Analyzing the merit of the different implementation

Efficiency • The efficiency of an algorithm depends on the quantity of resources it requires • Usually we compare algorithms based on their time • Sometimes also based on the space resources they need. • The time required by an algorithm depends on the instance size, and its data

Instance Size • Formally: Size = number of (binary) bits needed to represent the instance on a computer. • We will usually be much less formal • For sort we assume that the size of a record is c or bound by c number of bits • Formally the size of an input to sort with n records of c bits is nc, informally we use just n • Why do we need the formal definition?

Number problems • Problems where input numbers can become increasingly large. • Examples: • Factorial of 10, 106, 1015 • Fibonacci numbers • Multiplying, adding, dividing big numbers • For these problems we should use the formal definition • Size for a single number with valuen is O(lg n) Best, average, worst?

int fib(int n) { if (n <=1) return n; prev=0; cur=1; for (i=2; i<=n; i++) { next = cur + prev; prev = cur; cur= next; } return next } Analysis. Number of operations c*n (?) Number bits for value n, s=floor(lgn) +1 floor(lgn)=s-1 n >= 2s -1 Fibonacci

Time Analysis • Best Case: The smallest amount of time needed to run any instance of a given size • Worst Case: The largest amount of time needed to run any instance of a given size • Average Case: the expected time required by an instance of a given size

Requirements for time analysis • Independent of (what?) • A priori (why?) • Large instances (why?) • Growth rate classes • What are they? • Why?

Growth Rate Classes Requirement • Time analysis should partition algorithms into: • ordered classes (algorithms in a class considered to have same efficiency), and • if class A “is better than“ class B then all algorithms that belong to A are considered more efficient than all algorithms in class B.

The classes • Time analysis partitions algorithms into general equivalence classes such as: • logarithmic, • square-root • linear, • linear*logarithmic (nlgn) • quadratic, cubic, … • polynomial (nk where k>=0 is a constant) • exponential, (such as cn) etc. • Reason for this kind of classification • Growth rate classes are derived from instruction counts and the derived constants are estimates

Instruction counts • Provide rough estimates of actual number of instructions executed • Depend on: • Language used to describe algorithm • Programmer style • Method used to derive count • May be quite different from actual counts • Algorithm with an estimated count = 2n, may not be faster than one with estimated count=5n (why?)

Method 1: Sum Line Counts • Derive a count for each line of code taking into account all loop nesting. • Compute total by adding line counts.

Method 2: Barometer Operation • A “barometer instruction” is selected • Count = number of times that barometer instruction is executed. • Search algorithms: • barometer instruction (x == L[j]?). • Sort algorithms: • barometer instruction (L[i] == L[j]?). • Should be chosen with care.

Next class • Asymptotic growth functions

CS 575 Design and Analysis of Computer Algorithms Professor Michal Cutler Introduction