Iterative Algorithms & Loop Invariants

1. Steps in an Iterative Algorithm Code from LI Fairy God Mother (Lake) More of Input (Insertion and DFA) More of Output (Selection & Blocks) Narrowing the search space Shrinking Instance (GCD) (Running Time) Jeff's result Iterative Algorithms & Loop Invariants Jeff Edmonds York University Lecture1 COSC 3101

2. vsA Sequence of Assertions A Sequence of Actions Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = bendif “assert: m is max in {a,b}” if( c>m ) m = cendif It is helpful to have different ways of looking at it. “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}”

3. Purpose of Assertions Useful for • thinking about algorithms • developing • describing • proving correctness

4. Definition of Assertions An assertion is a statement about the current state of the data structure that is either true or false. eg. the amount in your bank account is notnegative.

5. Definition of Assertions It is made at some particular point during the execution of an algorithm. It should be true independent ofthe path followed through the code and independent of the input. If it is false, then something has gone wrong in the logic of the algorithm.

6. Definition of Assertions An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader.

7. Definition of Assertions • The current state of the computation • Everything that is true at a particular instant in time. • Imagine flying in from Mars,what do you need to know about the present to be able to continue. • It does not say anything about the past,i.e. how the computation got here. • Eg <x=8, y=3, current line of code = line 84>

8. Definition of Assertions • The current state of the computation • An assertion, A • is a function • that takes as input the current state • and that outputs • EgA = “x is odd” • A(<x=5, y=3, line84>) = True • A(<x=8, y=3, line84> ) = False Trueor False  Computation is on the path  Something has gone wrong!

9. Algorithm Termination You need to define someMeasure of progressto prove that your algorithmeventually terminates. • An Measure of Progress, M • is a function • that takes as input the current state • and that outputs • EgMeasure = “value of y” • M(<x=5, y=3, line84> ) = 3 • We must prove that this number goes down (up) each iteration. a real number

10. Designing an Algorithm Is this sufficient? Exit Exit Exit 0 km Exit 79 km 75 km 79 km to school Exit

11. 52 88 14 14,23,25,30,31,52,62,79,88,98 31 98 25 30 23 62 79 Explaining Insertion Sort We maintain a subset of elements sorted within alist. The remaining elements are off to the sidesomewhere. Initially,think of the first element in the array as a sorted list of lengthone. One at a time, we take one of the elements that is off to theside and we insert it into the sorted list where itbelongs. This gives a sorted list that is one element longer than itwas before. When the last element has been inserted, the array iscompletely sorted.

12. Establishing Loop Invariant <preCond> codeA <loop-invariant> Maintaining Loop Invariant <loop-invariant> ¬<exit Cond> codeB <loop-invariant> Exit Exit Clean up loose ends <loop-invariant> <exit Cond> codeC <postCond> Partial Correctness Proves that IF the program terminates then it works <PreCond>&<code>Þ <PostCond>

13. Consider an Algorithm Exit 0 km Exit Exit Exit 79 km 75 km

14. Loop Invariant Maintained Exit

15. Computation can always proceed

16. Computation always makes progress Exit 79 km 75 km 79 km 75 km

17. Computation Terminates 0 km 0 km 0 km Exit Exit 79km 75 km

18. Computation Terminates Exit Exit

19. Consider an Algorithm Exit 0 km Exit Exit Exit 79 km 75 km This is sufficient!

20. Designing an Algorithm Exit Exit Exit 0 km Exit 79 km 75 km 79 km Exit

21. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • What is the loop invariant? • Each iteration add in a new object. Action not picture If you fight me, I will fail you

22. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • What is the loop invariant? • What do you want to be true in the middle of the computation? • To have a partial sum:s = 12 + 22 + 32 + … + i2 (and 1 ≤ i ≤ I)

23. Exit 79 km 75 km Exit Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (0 ≤ i ≤ I) • Measure of Progress • i= # of objects added. 79 km • Step to make progress • i = i+1 • Exit Condition • i = I

24. Exit 79 km 75 km Code from LI In order to make progress,we may head in one direction. • Step to make progress • i = i+1

25. Exit 79 km 75 km Code from LI In order to make progress,we may head in one direction. • Step to make progress • i = i+1

26. Exit Code from LI In order to maintain the loop invariant,we must adjust to come back to the path.

27. Code from LI This may be how we determine the step in then next iteration.

28. Maintaining Loop Invariant <loop-invariantt> ¬<exit Cond> codeB <loop-invariantt+1> Exit Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Partial Step: i = i+1 Let it and st be the values of i and s at the beginning of the iteration and let it+1 and st+1 be their after going around again.

29. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Partial Step: i = i+1 Use the fact that you already have the LI trueand take a small step. <loop-invariantt> ¬<exit Cond> Make the loop Invariant true!!! Solve the entire algorithm!!! Panic!! codeB: next <loop-invariantt+1>

30. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Partial Step: i = i+1 (codeB in loop) st = 12 + 22 + 32 + … + it2 <loop-invariantt> it< I ¬<exit Cond> it+1 = it + 1 codeB: (math) st+1 =12 + 22 + 32 + … + it+12 Write down LHS/RHSof what we must prove <loop-invariantt+1>

31. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Partial Step: i = i+1 st = 12 + 22 + 32 + … + it2 <loop-invariantt> it< I ¬<exit Cond> it+1 = it + 1 codeB: st+1 = st + it+12 st+1 =[12 + 22 + 32 + … + it2 ]+ it+12 =12 + 22 + 32 + … + it+12 = st + it+12 <loop-invariantt+1>

32. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Partial Step: i = i+1 it+1 = it + 1 s = s + i2 i = i + 1 st+1 = st + it+12 i = i + 1 s = s + i2

33. Exit Clean up loose ends <loop-invariant> <exit Cond> codeC <postCond> Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Step: i = i+1, s = s + i2

34. Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Step: i = i+1, s = s + i2 s = 12 + 22 + 32 + … + i2 <loop-invariant> I2 i = I <exit Cond> codeC: S = s S =12 + 22 + 32 + … + I2 <postCond>

35. Establishing Loop Invariant <preCond> codeA <loop-invariant> Code from LI • Precondition: The input is an integer I. • Postcondition: The output is S = 12 + 22 + 32 + … + I2 • Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I) • Step: i = i+1, s = s + i2 i = 0s = 0 0 0 1 1 2 1+4=5 3 1+4+9=14

36. Designing an Algorithm Exit Exit Exit 0 km Exit 79 km 75 km 79 km Exit

37. Typical Types of Loop Invariants • Fairy God Mother • More of the input • More of the output • Narowingthe search space • Shrinking Input Instance • Data Structure (System) Invariant

38. Typical Types of Loop Invariants • Fairy God Mother • I have arrived from Mars, some amount of progress has been made already and my Fairy God Mother has set the state of the computation to be just what I want so that I feel as comfortable as I could be given the amount of progress that has been made. • More of the input • More of the output • Narowingthe search space • Shrinking Input Instance • Data Structure (System) Invariant

39. Fairy God MotherLoop Invariants I run infinitely faster than the monster.so my only goal is to get to shore without him there. The only problem is that he runs 4 times faster than I swim. Me: Radius 1, speed 1, time 1.Him: Half circ., speed 4, time = 3.14/4 < 1He wins.

40. Fairy God MotherLoop Invariants He runs back and forth to catch me.And I change my direction to avoid him.Can I get away?

41. Fairy God MotherLoop Invariants My progress is the distance swum from the center. What is your wish? • It is my job to • establish • maintain • and get the post condition • from this loop invariant

42. Fairy God MotherLoop Invariants My goal now is not to excapeor to make progress.My only goal is to extablish the loop invariant by ..? Moving to the center.

43. Fairy God MotherLoop Invariants My goal now is not to excapeor to make progress.My only goal is to maintain the loop invariantby ..? by moving opposite him in my circle. But he moves 4 times faster!!! So the circumference of your circle must be ¼ of his.

44. Fairy God MotherLoop Invariants 1 r rd < d/4 rd r < 1/4 d But he moves 4 times faster!!! So the circumference of your circle must be ¼ of his.

45. Fairy God MotherLoop Invariants If I am less than ¼ from the middle,then I can swim my circle faster than he can run his.(Even though he swims 4 times faster.) Hence, I can maintain the loop invariant,by moving opposite him And can use my extra speed to make progress outward.

46. Fairy God MotherLoop Invariants When I am ¼ from the middle,I can no longer maintain the loop invariant.Hence, I … ? Swim for it. Me: ¾ radius, speed 1, time = ¾ = 0.75Him: Half circ., speed 4, time = 3.14/4 = 0.78 I win!

47. Typical Types of Loop Invariants • Fairy God Mother • More of the input • More of the output • Narowingthe search space • Shrinking Input Instance • Data Structure (System) Invariant

48. Solution Extra More of the Input Loop Invariant The input consists of an array of objects We have read in the first i objects. We will pretend that this prefix is the entire input. We have a solution for this prefix plus some additional information

49. Loop Invariants andDeterministic Finite Automaton L = {a Î {0,1}* | a has length at mostthree and the number of 1'sis odd }.

50. Solution Solution Extra Extra More of the Input Loop Invariant The input consists of an array of objects We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have to one for this larger prefix. (plus some additional information)