: Analysis of Algorithms

1. Welcome to : Analysis of Algorithms COP4531 CGS5427 Course Webpage http://www.compgeom.com/~piyush/teach/4531/ And Blackboard

2. The Course • Instructor: Piyush Kumar (piyush@acm.org) Office: Love 105B Phone: 850-645-2355 Office Hours: Tuesday 5:00pm - 6:00 pm; Thursday 4:00pm – 5:00pm; Or by appointment (use email) • Teaching Assistant : TBA (prob. None)

3. The Course • Grading Policy • Homework + Programming Assignment + Quiz : 30% = h score • Mid Terms + Finals: 70% = f score • To Pass: h >= 16 and f >= 40 • Letter Grades based on sorted (h+f) scores provided you pass.

4. The Course • Prerequisites: • COP 4530 w/ grade of C- or better • STA 4442 w/ grade of C- or better • Either MAD 3107 or MAD 3105

5. The Course • Format • Two lectures/week • Homework mostly biweekly • Problem sets • One programming assignment • 3 Surprise Short Quizzes • (Mostly will depend on what I did in the past week) • ( + One extra credit programming assignment ) • Two MidTerms (Sep 28th, Nov 4th) • FINAL EXAM is on DEC 10th, 3:00pm to 5:00pm. Venue: In Class, Love 301

6. Homework • Write problems beginning with a new page. • Only hard-copy (paper) submissions are allowed. • No late assignments • Look at the Course Information pages for more information

7. Homework Policy • If you ask to re-grade your homework please write out the basis of your request. • If the grader finds no basis for your complaint, then 10% points will be deducted from your original grade unless the grade is changed. • Note: This is not to discourage you from disputing your grade, but rather we encourage you to read and understand the posted solutions on the web before you ask your solutions to be re-graded

8. Homework Policy • Under no circumstances should you be copying others or modifying other people’s work. • It is fine to discuss problems with others, but all of the writing should be done without any collaboration. Make sure you read the Course Information webpage.

9. Homework Policy • You can work in a pair or alone • If you work in a pair, You are both supposed to write the solutions independently and staple before you submit. • Only one solution from a pair will be graded (The one on top).

10. Exam Policy • If you say “I don’t know” in any question in the exam/hw, you get 25% marks for that question/sub-question. • In case you don’t know the answer its better to leave it than filling the answer sheet with ‘crap’ because you might even loose that 25%

11. How to do well in this class? • Think • Keep a few minutes everyday for thinking about homework problems • Do all homework yourself, even if you have a partner. If you did not think on homework, be prepared to get burnt on exams.

12. How to do well in this class? • Attend Class Regularly. Be There or you will miss on • The explanations • The demos • In-Class Quizzes • Announcements • Hints on homework and exams • It’s not the same to have notes from the web or friends. Nothing can replace the experience of being there. • Read Before Class.

13. How to do well in Class? • Plan to spend a few hours every week for Reading Assignments and doing homework. ( 3 hours per class? ) • Check web-pages/blackboard frequently.

14. How to do well in Class? • Don’t be afraid of me! • Ask Questions in Class. • Attend office hours. • Feel free to visit me anytime (105b). • Feel free to email me. piyush@acm.org • Feel free to call me. 850-645-2355 • Again: Ask Questions in Class • (Don’t wait for others to ask them for you).

15. Web Pages • http://www.compgeom.com/~piyush/teach/4531/ : Course Calendar, Instructor Information, Scribing Material, Course Information, Grades. • Blackboard: Announcements, Homework, Discussion, Reading Assignments, Programming Projects. • Make Sure you check both these pages frequently. • Make Sure you check the email that is listed on the blackboard. • My email address for the course is piyush@acm.org

16. Algorithm: What is it? • An Algorithm is a well-defined computational procedure that transforms inputs into outputs, achieving the desired input-output relationship.

17. Course Objectives • design new algorithms. • analyze a given algorithm. • read and understand algorithms published in journals. • develop writing skills to present your own algorithms. • collaborate and work together with other people to design new algorithms. • Crack job interviews.

18. Algorithm Characteristics • Finiteness • Input • Output • Rigorous, Unambiguous and Sufficiently Basic at each step Correctness

19. A Cookbook Recipe • Add a dash of salt? • (Where? How much exactly?) • Toss lightly till the mixture is crumbly? • Wait till the water starts to boil? • (Remember: A watched pot never boils) • Resemblances: Input, Output, Finiteness.

20. What we want? • A ‘good’ Algorithm • In some loosely defined aesthetic sense • Time : Faster • Space : Smaller • Portability/Adaptability to different machines. • Simplicity and Elegance

21. Applications? • WWW and the Internet • Computational Biology • Scientific Simulation • VLSI Design • Security • Automated Vision/Image Processing • Compression of Data • Databases • Mathematical Optimization

22. The RAM Model • Analysis is performed with respect to a computational model • We will usually use a generic uniprocessor random-access machine (RAM) • All memory equally expensive to access • No concurrent operations • All reasonable instructions take unit time • Except, of course, function calls • Constant word size • Unless we are explicitly manipulating bits

23. Linear Search I • For I = 1 to N if (a[I] == x) return true; return false;

24. Linear Search II A[N+1] = x; I = 1; While A[I] != x ++I; if I == N+1 return false; Else return true;

25. A Rough Analysis • N = 10000

26. Sorting • Input:Array A[1...n], of elements in arbitrary order • Output: Array A[1...n] of the same elements, but in increasing order • Given a teacher find all his/her students. • Given a student find all his/her teachers.

27. Binary Search Initialize High < Low Get Midpoint Failure Adjust Low Adjust High Compare > < = Success

28. Binary Search Algorithm: Low= 1; High = n; while Low < High { m = floor( (Low+High)/2 ); if k <= A[m] then High = m - 1 else Low = m + 1 } if A[Low] = k then j = Low else j = 0

29. 15 29 19 27 31 5 An Illustration 23 Sorted Array: 5, 15, 19, 23, 27, 29, 31 Index : 1 2 3 4 5 6 7

30. Time and Space Complexity • Generally a function of the input size • E.g., sorting, multiplication • How we characterize input size depends: • Sorting: number of input items • Multiplication: total number of bits • Graph algorithms: number of nodes & edges • Etc

31. Running Time • Number of primitive steps that are executed • Except for time of executing a function call most statements roughly require the same amount of time • y = m * x + b • c = 5 / 9 * (t - 32 ) • z = f(x) + g(y) • We can be more exact if need be

32. Analysis • Worst case • Provides an upper bound on running time • An absolute guarantee • Average case • Provides the expected running time • Very useful, but treat with care: what is “average”? • Random (equally likely) inputs • Real-life inputs

33. Binary Search Analysis • Order Notation • Upper Bounds • Search Time = ?? • A better way to look at it, Binary Search Trees

34. Searching A bad king has a cellar of 1000 bottles of delightful and very expensive wine. a neighbouring queen plots to kill the bad king and sends a servant to poison the wine. (un)fortunately the bad king's guards catch the servant after he has only poisoned one bottle. alas, the guards don't know which bottle but know that the poison is so strong that even if diluted 1,000,000 times it would still kill the king. furthermore, it takes one month to have an effect. the bad king decides he will get some of the prisoners in his vast dungeons to drink the wine. being a clever bad king he knows he needs to murder no more than 10 prisoners - believing he can fob off such a low death rate - and will still be able to drink the rest of the wine at his anniversary party in 5 weeks time. Explain how...

35. Solution • Number each bottle in binary digits • Feed each prisoner one column of the list of the binary digits where 1 means the bottle is tasted and zero means its not • Convert the death of the 10 prisoners into a decimal number, That’s the bottle we are looking for.

36. Induction • Prove 1 + 2 + 3 + … + n = n(n+1) / 2 • Basis: • If n = 0, then 0 = 0(0+1) / 2 • Inductive hypothesis: • Assume 1 + 2 + 3 + … + n = n(n+1) / 2 • Step (show true for n+1): 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2

37. Induction: A Fine example

38. Practice Problem • Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) • Read MI from the web.

39. Homework problem: What’s wrong? • Problem: Prove that x^(k-1) = 1 • Proof: P(1) : x^(1-1) = 1 By Induction assume P(1),P(2)..P(n) are true. P(n+1): x^{(n+1)-1} = x^n = x^{n-1} * x^{n-1} / x^{n-2} = 1 * 1 / 1 = 1 !

40. Assignments • Go thru the slides for the first lecture. • Read chapter 1 of the text book • 'Suggested' Exercises: 1.1-5, 1.2-3 (Solutions not to be Submitted/Graded) • Read the course Information Sheet • Submit: Why the proof is wrong on the previous slide.

41. Next Time • In this course, we care most about asymptotic performance • How does the algorithm behave as the problem size gets very large? • Running time • Memory/storage requirements • Bandwidth/power requirements/logic gates/etc.