03/28/13

1. Big O and Algorithms (Part 2) 03/28/13 Discrete Structures (CS 173) Derek Hoiem, University of Illinois

2. Feedback: common comments • More practice questions • See prior terms: there are a few semesters’ worth • Write bigger, neater • I’ll try! • Hand out solutions to discussion sections • Not available but see all the prior term practice questions • Less overlap between lecture and reading • Balancing act but can try to vary more • Make homework due later • Done! • Make lectures/discussion sections shorter or longer • Explain general induction approach more • Today • Explain how to do proofs in more detail earlier • Good idea • Lectures/material is boring • Sorry! • Best class ever • Good! • Lots of comments about discussion sections • Will pass along • Change slides slower • Note that slides are also posted before lecture (almost always) and notes after

3. Last class • Big-O • Model algorithm efficiency as a function of the parameter size • Asymptotic analysis: how does it perform for large inputs? • Keep only dominant terms, ignoring coefficients • Applies to runtimes or memory usage or any other resources

4. This class • Midterm review: induction form and strategies • Analysis of algorithm complexity • For loops • While loops • Recursion

5. Induction • When to use it: You need to prove that something is true for all (an infinite set) of numbers, graphs, trees, etc. • The induction variable is always an integer (size of graph, number of puzzle pieces, etc.) • Key step is figuring out how the larger problem (number) can be formulated in terms of the smaller problem (number)

6. Induction: checklist • What is the induction variable ()? • Always an integer • What is the inductive hypothesis? • Almost always assuming the claim for first values of • Write out the full hypothesis, not just “Assume the claim is true.” • What is the inductive conclusion? • Almost always showing the claim holds for the next value of • Again, write out precisely what you need to show • What base case(s) do you need? • Sometimes this isn’t obvious until you write the induction step

7. Form of an induction proof Claim: A party with any positive number of monkeys will be fun.1 Proof: I will prove this by induction on integer , the number of monkeys at the party. Base: . A party with one monkey is fun because the monkey just doesn’t care. Induction: Assume that any party with monkeys is fun. I need to show that a party with monkeys is fun. Suppose there is a party with monkeys. That’s just a party with monkeys with one more monkey. By the inductive hypothesis, a party with monkeys is fun. Adding a monkey always makes a party even more fun, so a party with monkeys is also fun. QED. fun even more fun 1 disclaimer: this proof isn’t entirely rational

8. Choosing the induction variable • Algebraic proofs: the integer that the equation has to be proven for • Example: Prove for any positive integer . Induction on . • Puzzles: the number of puzzle pieces • Example: The tower of Hanoi puzzle can be solved for any number of disks. Induction on , the number of disks. • Graphs: the number of nodes in the graph • Example: Any fully connected graph with nodes has edges. Induction on , the number of nodes in the graph. • Trees: the height of the tree • Example: Any full and complete binary tree has nodes, where is the height of the tree. Induction on , the height of the tree.

9. Selecting the base case • Algebraic proofs: usually the smallest value(s) • Example: Prove for any positive integer . Base: =1. • Puzzles: • Example: The tower of Hanoi puzzle can be solved for any number of disks. Base: it works for one and two disks. • Graphs: • Example: Any fully connected graph with nodes has edges. Base: A graph with one node has 0 edges. • Trees: • Example: Any full and complete binary tree has nodes, where is the height of the tree. Base: A tree of height 0 has one node.

10. Induction strategy • Algebraic proofs: rewrite the equation for in terms of the the equation for . • Puzzles: figure out how solving the puzzle for pieces involves solving the puzzle for pieces. • Graphs: Start with a graph with nodes. Apply inductive hypothesis to a graph with one node removed, and note the difference caused by removing the node. • Trees: Start with a tree of height . Use the fact that the root’s subtrees have height of or less to show that the claim holds for the full tree.

11. Exam review • http://courses.engr.illinois.edu/cs173/sp2013/B-lecture/Exams/index.html • http://courses.engr.illinois.edu/cs173/sp2013/B-lecture/previous-terms.html • Discussion section

12. Algorithms • Practice analyzing runtime based on pseudocode

13. Key concept: instructions in the loops dominate

14. procedure bubbleSort(A : list of sortable items) repeat swapped = false for i = 1 to length(A) - 1 inclusive do: /* if this pair is out of order */ if A[i-1] > A[i] then /* swap them and remember something changed */ swap( A[i-1], A[i] ) swapped = true end if end for until not swapped end procedure Key concept: if the number of iterations of the loop is uncertain, determine the worst case Note: can also do best-case or worst-case analysis http://www.youtube.com/watch?v=lyZQPjUT5B4

15. Key concept: runtime can be in terms of multiple input parameters

16. Key concept: n/2 recursion

17. Comparison of sorting algorithms http://www.sorting-algorithms.com/random-initial-order Key concept: Be aware of best, worse, and average case

18. http://introcs.cs.princeton.edu/java/23recursion/hanoi/

19. Have a good weekend