Understanding Iteration Abstraction in Software Engineering

1. EECE 310: Software Engineering Iteration Abstraction

2. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Understand the design issues for iterators • Design iterators that support concurrency

3. Problem • Consider the IntSet ADT introduced earlier • Let’s say you want to compute the sum of elements in the set. How would you do it ? • Add a member function computeSum • Copy the rep to another array and return it • Add an indexing operation to the ADT • Add a new iterator abstraction • Which method would you choose and why ?

4. Iteration Abstraction • General way to iterate over the elements of an ADT without tying the iteration to a specific operation (e.g., summation) • Efficient in space and time (as far as possible) For each element e of the ADT: Perform operation o on e

5. Implementation • Implemented using two entities: 1. Generator: Keeps track of the state of the iteration and exposes operations to actually perform the iteration 2. Iterator: Operation of an ADT that returns a generator object to begin the iteration process

6. Iteration Abstraction: Generators Client Code 1 ADT state 1 Generator objects state 2 Client Code 2 Client Code 3

7. Generators : Keeping track of State • Objects that keep track of the state of the iteration in its rep • Implements the Iterator interface in Java • Consists of methods hasNext and next • next method throws NoSuchElementException • Multiple generators can operate on an ADT • Generators can be passed around to procedures

8. Generators: Example public interface Iterator<T> { public boolean hasNext() // EFFECTS: Returns true if there are more elements to // yield, else returns false public T next( ) throws NoSuchElementException // MODIFIES: this // EFFECTS: If there are more results to yield, // returns the next result and records the state of this // Otherwise, throws NoSuchElementException

9. Iterator: Produces a generator • Operation of an ADT or a stand-alone procedure that returns a generator object • Different iterators can produce different generators (e.g., forward and reverse iterator) • Multiple calls to the iterator can produce different generators (for local iteration) • The specification of the iterator defines the behavior of the generator object it returns – add a requires clause at the end constraining its use

10. Iterator: IntSet Example public class IntSet { ... public Iterator<Integer> elements ( ) // EFFECTS: returns a generator that will produce // all the elements of this, each exactly once, // in arbitrary order. // REQUIRES: this must not be modified while // the generator is in use. }

11. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Design iterators that support concurrency

12. How do we use iterators ? • Step 1: Client invokes the iterator operation on ADT and stores the returned generator object • IntSet s = new IntSet(v); • …. • Iterator g = s.elements(); • Step 2: Perform the iteration using the hasNext and next methods of the generator object • while ( g.hasNext() ) { • Integer o = g.next( ); • int i = o.intValue(); • } • Optional Step: May pass the generator object to different procedures for doing the operation • e.g., int m = computeMax(g); ADT (e.g., IntSet) Has iterator methods that return generator objects, e.g., elements • Generator object: implements the Java iterator interface): • hasNext • next

13. Example: Using Iterators public static int max(Iterator<Integer> g) throws EmptyException { // REQUIRES: g!=null && g contains only Integers // MODIFIES: g // EFFECTS: Throws EmptyException if no elements in g, otherwise return // the maximum element of the collection try{ int m = ( g.next() ).intValue() ; while ( g.hasNext() ) { int x = ( g.next() ).intValue(); if (m < x) m = x; } return m; } catch( NoSuchElementException e) { throw new EmptyException(“Comp.max”); } }

14. Some points on using Generators • Using code interacts with a generator via the iterator interface (defined in java.util package) • Using code must obey the constraint imposed on it by the iterator’s requires clause (given after the EFFECTS clause) • Generators can be passed as arguments and returned as results by procedures • Generators may be primed before using them in loops • Generators may loop infinitely, e.g., list of prime numbers

15. In-class Exercise • Write a procedure that iterates over the elements of the IntSet ADT and prints all Integers that exceed a certain threshold, max How would you do this if you had an iterator that allowed you to choose elements that fall within a certain range, say, min and max.

16. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Understand the design issues for iterators • Design iterators that support concurrency

17. Implementing Iterators public class IntSet { private vector<Integer> els; // The rep ... public Iterator<Integer> elements ( ) // EFFECTS: returns a generator object that will // produce all the elements of this, each exactly // once, in arbitrary oder. // REQUIRES: this must not be modified while // the generator is in use. return new IntSetGen(this); }

18. Implementing generators • All generators implement Java’s iterator interface • Using code interacts with it through the interface • You must implement the methods hasNext and next • Generator has access to the rep of the ADT • Must not be exposed to the using code • Generator has its own rep to store the state of the iteration which also should not be exposed

19. IntSetGen: Scope • IntSetGen is declared as a nested class within the IntSet ADT and it’s scope is private • Clients cannot create IntSetGen objects except through the iterator method of the IntSet ADT • Clients cannot directly access any operation of IntSetGen except through the iterator interface • IntSetGen has undeterred access to the private methods of the IntSet including its rep

20. IntSetGen: Declaration public class IntSet { private vector<Integer> els; // The rep public Iterator<Integer> elements( ) { // Iterator // EFFECTS: … return new IntSetGen( this ); } private static class IntSetGen implements Iterator<Integer> { // Generator object returned by elements }

21. IntSetGen: Rep and Constructor private static class IntSetGen implements Iterator<Integer> { private int index; private IntSet s; // Constructor IntSetGen(IntSet is) { // REQUIRES: is!= null // EFFECTS: Initializes the generator for a // new iteration cycle s = is; index = 0; } Passed in by the elements() method of the IntSet ADT

22. IntSetGen: hasNext operation public boolean hasNext( ){ // EFFECTS: If not reached the end // the elements vector, return true // otherwise, return false if ( index == s.els.size( ) ) return false; else return true; } IntSet ADT’s Rep – can access this because it is declared as a nested class

23. IntSetGen: next operation public Integer next( ) throws NoSuchElementException { // EFFECTS: If there is a next object, return it // Otherwise, throw the NoSuchElementException if ( hasNext( ) ) return s.els.get(index++); else throw new NoSuchElementException(“IntSet.elements”); } Use the name of the iterator method that produced the generator object

24. In-class exercise • Implement an Iterator for the IntSet ADT, to return all integers of the IntSet that satisfy a given critereon. The iterator takes as its argument a type Check with a public method: public boolean checker(Integer) - that returns true if and only if the Integer satisfies the criteria Assume that the Check class is given to you. Also, write the implementation of its associated generator object.

25. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Understand the design issues for iterators • Design iterators that support concurrency

26. Design Issues: Multiple Iterators • Types may sometimes have multiple iterators • Examples: • ForwardIterators, ReverseIterators for ordered collections such as lists • Iterators that return elements satisfying a certain critereon (say, within a specified range) • Iterators that perform some operation on their elements prior to returning them (e.g., sorting)

27. Design Issues: Changes during Iteration • Default behavior so far: Disallow changes during iteration (specified in post-REQUIRES clause) • If changes are allowed, what are its semantics ? • State of iteration is what it was when the generator was created (requires creating copy) • Iteration sequence changes when changes occur to ADT (difficult to ensure consistency of state) • How do we reset iterator to go back in the collection ? • What if the current element is removed in a list ?

28. Design Issues: Can the generator itself change the ADT ? • The generator can change the rep of the ADT as long as it does not change the abstraction • Example: An iterator that returns the elements of an un-ordered collection such as set in sorted order may sort the collection during the process of iteration • Iterator changes the abstraction exposed by ADT • This is usually best avoided unless there is a compelling need. For example, iterating over a task list may itself spawn new tasks to be added to the list.

29. Testing Iterators • Iterators are tested like procedures, except that they have paths in their specifications that are similar to loops in procedures • Should test for returning generators that produce 0, 1, 2 results/elements • Generators are tested like any other ADT (Boundary conditions, aliasing etc.)

30. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Understand the design issues for iterators • Design iterators that support concurrency

31. Concurrent Iterators: Read only • Assume that we want to iterate over a list in two separate threads, but without modifying it. Is this allowed ? • Yes, as long as we keep the generator objects separate. i.e., we do not share generators • Each generator keeps track of where it is in the iteration – no changes to the state of the ADT • We need to make the iterator method synchronized (why ?)

32. Concurrent Iterators: Read-Write Thread 1 Thread 2 is.add(1); is.add(2); is.add(3); is.add(4); is.remove(2); No guarantees about what iterator returns Iterator<Integer> g = is.elements(); while ( g.hasNext() ) { Integer i = g.next(); System.out.println(i + “ “); }

33. How to solve the concurrent iterator problem ? • Disallow concurrent writes by requiring clients to take a lock before the iteration operation • Also disallows concurrent reads – performance ! • Need to explicitly release lock when done • Make a copy of the collection before iteration • Throw a concurrent modification exception

34. Making a copy • Create a copy of the list when you call the iterator method, and perform iteration over it • Need to ensure that copy operation is performed in the iterator method in synchronized mode • Can be expensive to copy large lists in memory • Need to perform deep copy of elements to disallow any modifications • Iterator may operate on stale version of collection

35. Concurrent modification exception • Exception thrown when the ADT is modified concurrently with the iteration • Thrown to the thread that performs the iteration • Up to the thread what it wants to do with it • Only one exception per concurrent modification will be thrown • ConcurrentModification is a checked exception

36. CME: Usage public ... search (List list) { try { ListIterator iterator = list.listIterator(); while(iterator.hasNext()) { Object o = iterator.next(); //do something ... } } catch (ConcurrentModificationException concEx) { //handle this case, maybe just iterate the collection again return search(list); } }

37. CME: Implementation - 1 02 * The number of times this list has been structurally modified. 03 * Structural modifications are those that change the size of the 04 * list, or otherwise perturb it in such a fashion that iterations in 05 * progress may yield incorrect results. ….. 15 * Use of this field by subclasses is optional. If a subclass 16 * wishes to provide fail-fast iterators (and list iterators), then it 17 * merely has to increment this field in its add(int, Object) and 18 * remove(int) methods (and any other methods that it overrides 19 * that result in structural modifications to the list). A single call to 20 * add(int, Object) or remove(int) must add no more than 21 * one to this field, or the iterators (and list iterators) will throw 22 * bogus ConcurrentModificationExceptions. If an implementation 23 * does not wish to provide fail-fast iterators, this field may be 24 * ignored. 25 */ 26 protected transient int modCount = 0;

38. 30 private class Itr implements Iterator<E> { 31 int expectedModCount = modCount; 32 33 public boolean hasNext() { 34 // implementation omitted 35 } 36 37 public E next() { 38 checkForComodification(); 39 // implementation omitted 40 41 } 42 43 private void checkForComodification() { 44 if (modCount != expectedModCount) 45 throw new ConcurrentModificationException(); 46 } 47 } CME: Implementation - 2

39. Class Exercise • Add a concurrent iterator to the IntSet ADT that throws a ConcurrentModificationException when another thread attempts to modify/remove from the set during the iteration.

40. Learning Objectives • Define the basic elements used in iterators • Use iterators to iterate over a collection • Implement iterators and the associated generator classes for an ADT • Design iterators that support concurrency