Algorithms and Complexity 2: Complexity Notation

1. Algorithms and Complexity2: Complexity Notation John Levine John.Levine@cis.strath.ac.uk John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

2. Last time • What’s an algorithm then? • Intro and brief history • Sample use of algorithms • Sample algorithm - room count • Course overview • Lectures, practical, labs/tuts, the exam John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

3. The travelling seller • Mary, a computer seller, needs to visit 3 shops in a day (starting and finishing at the office): what’s the shortest route? • What if there’s 12 shops? 8km 5km 2km 12km 3km John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

4. Today • Size matters • Complexity notation • Calculating complexities 2 John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

5. Download time from internet • Say it takes 2s to establish a link and then we download at 5K/s, then if n is size of file in K: • time to download is n/5 + 2 • a linear function • dominant element as data size grows is n, so using big-oh notation = O(n) John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

6. String Search • Problem: • Find a string t in a longer string s • Simple approach i=0; found = false; while (!found) & (i<s.length) j = 0; oksofar = true; while (oksofar) & (j<t.length()) if (s.charAt(i+j)!=t.charAt(j)) oksofar = false; j++ found = oksofar; i++; John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

7. Rules for big-oh notation • Only record the highest, dominant, factor • Ignore constants and multipliers • Kind of like rounding • 1,000,001 is roughly 1,000,000 • so you say O(n2) and not O(12n2) • 4n3 + 3n2 + 1000n + 2000000 = O(n3) • 3n3 + 5 = O(n3) John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

8. Common classes of Big-Oh • In increasing complexity roughly: - logarithmic O(log n) - linear O(n) - lin-log O(n log n) - quadratic O(n2) - polynomial O(nk), k > 1 - exponential O(an), n > 1 - factorial O(n!), n > 1 John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

9. Common classes of Big-Oh John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

10. 2 3 n O(log n) O(n) O(n ) O(n ) O(n!) 1 10 sec 10 sec 10 sec 10 sec 10 sec 2 10 sec 10 sec 10 sec 10 sec 10 sec 4 10 sec 10 sec 10 sec 10 sec 10 sec 8 10 sec 10 sec 10 sec 11 sec 50 sec 16 10 sec 10 sec 10 sec 14 sec 663 years 32 10 sec 10 sec 11 sec 43 sec 8.34E+24 years 64 10 sec 10 sec 14 sec 5 min 4.02E+78 years 128 10 sec 10 sec 26 sec 35 min 1.22E+205 years 256 10 sec 10 sec 1 min 5 hours 512 10 sec 11 sec 5 min 37 hours 1,024 10 sec 11 sec 18 min 12 days 2,048 10 sec 12 sec 1 hours 3 months 4,096 10 sec 14 sec 5 hours 2 years 8,192 10 sec 18 sec 19 hours 17 years 16,384 10 sec 26 sec 3 days 139 years 32,768 10 sec 43 sec 12 days 1113 years 65,536 10 sec 1 min 2 months 8901 years 131,072 10 sec 2 min 6 months 71209 years 262,144 10 sec 5 min 2 years 569672 years 524,288 10 sec 9 min 9 years 4557377 years 1,048,576 10 sec 18 min 35 years 36 , 459 , 013 yrs Tyranny of complexity • 1000 items per sec plus 10 secs startup John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

11. <= = >= • Big-Oh says “as the numbers grow the dominant factor will be no worse (<=) than given” • e.g. an O(n log n) function grows no faster than another t = n log n • Big-Oh is used for the worst case - we will mostly talk worst, sometimes expected but never best nor average • Also Ω(n) and θ(n) for >= and = John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

12. Simple String Search Peter Piper picked a peck of pickled pepper;     A peck of pickled pepper Peter Piper picked.If Peter Piper picked a peck of pickled pepper,     Where’s the peck of pickled pepper Peter Piper picked? • Find pepper - lots of false starts • Can you do better? • Simple complexity is O(nm) John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

13. Better string search • Start search at end and keep a table peter_piper picked a peck of pickled pepper pepperpiper_picked a peck of pickled pepper peter pepperpicked a peck of pickled pepper peter piper pepper a peck of pickled pepper peter piper pickedpepperk of pickled pepper peter piper picked a pecpepperickled pepper peter piper picked a peck pepperkled pepper peter piper picked a peck of picpepperepper peter piper picked a peck of picklpepperper peter piper picked a peck of pickledpepperr peter piper picked a peck of pickled pepper 10 steps instead of 50 (I think) what about longer text? what about fewer letters - e.g. DNA coding? AGCCCGAACATTTACGCCGCTGGCGACTGCACCG John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

14. String search • Basic algorithm is O(nm) • Best algorithm is O(n) • BUT is much more complex • Have to think of when it matters • is data big enough for more complex soln • watch constants and multipliers John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/

15. Summary • Size matters • Complexity notation • Calculating complexities • Next time: start searching & sorting • Ask the class quiz John Levine, Computer and Information Sciences, Strathclyde Universityhttp://www.cis.strath.ac.uk/~johnl/