CSE 121/131 Programming Spring 2001 Lecture Notes 5 ã 1999-2001 S.Kannan & V.Tannen

1. CSE 121/131ProgrammingSpring 2001 Lecture Notes 5ã 1999-2001 S.Kannan & V.Tannen

2. Concurrent programming When we have at the same time several ongoing (concurrent) “program executions”! There are several situations where concurrency arises naturally: Dealing with an IO device that is substantially slower than the CPU (eg.,disk, screen, keyboard, mouse, network). An efficient program should do something useful while waiting for the completion of an IO operation. The convenient arrangement is for a separate program, running concurrently, to babysit the IO operation. This problem emerged very early in the history of computing technology.

3. Concurrent programming, continued Systems that support multitasking (multiprocessing). Multiple programs run concurrently, supporting multiple users and multiple tasks launched by each user. Tasks run in separate chunks of memory, sharing one or more processors (CPU). Modern graphical user interfaces (GUIs). Allow a user to perform activities concurrently. One example are windows systems which extend the multiuser paradigm. Some GUIs may support concurrent activities that share memory. Runtime environments of modern programming languages. Some activities, such as garbage-collection are best implemented as a separate computation, running concurrently with the main program.

4. Concurrent programming, continued Parallel processing, (high-performance computing) Taking advantage of multiprocessor systems to solve time-consuming problems requires concurrent programming. It also requires quite clever algorithms, so this area is not just an application of concurrent programming techniques. In concurrent programming we distinguish between programs and “executions” of programs. Why? Because we may deal with several concurrent executions of the same program text. Program executions are called processes or threads.

5. Concurrent programming, continued The term process is used in operating systems. “heavyweight” processes, a.k.a kernel processes or tasks. Each has a separate address space (space of memory locations) “lightweight” processes, also called kernel threads. They execute in the same address space.

6. Concurrent programming, continued The term thread normally refers to a user-level concept, rather than an operating system one. Threads are usually associated with high-level programming languages (such as Java, as we are about to see). User-level threads are also “lightweight”: they share the same address space they can be created terminated synchronized significantly faster than processes

7. Thread operations (in general, and in Java) The basic needs are: thread creation and termination thread communication and synchronization For thread creation we must associate with the new thread a program that the thread will execute. In Java, threads are objects and they execute the program contained in a standard method called run(). The execution of the thread is initiated by the method start() and terminated either “internally” when the run method returns, or “externally” through the method stop().

8. Thread operations, continued Threads communicate information by sharing the same address space. In Java, this is done by sharing objects and accessing and updating their state. When two or more threads access a common resource such as shared data, we face situations known as race conditions. These arise when the global result of the program depends on precisely which thread does what and when to the shared data. At some level, the system “sequentializes” such concurrent operations but in a way that is unpredictable by the programmer. Thus, the programmer must impose explicitly the desired order of operations on shared data. This is called synchronization.

9. Critical regions Synchronization is done in critical regions (mutual exclusion code): code fragment with the property that only one thread can execute them at any given time (the executing thread locks the code fragment) When critical regions are nicely organized (similarly to abstract data types) as a group of shared data and functions that access the data, then they are called monitors.

10. Synchronized methods In Java, the objects of any class can be instrumented to behave like monitors. The user requests this by declaring one or more methods as synchronized. The monitor's data structures are the fields of the object. The monitor's functions are the synchronized methods of the object.

11. Synchronized methods, continued A thread locks an object when executing one of its synchronized methods. Two different threads cannot execute (the same or different) synchronized methods of the same object at the same time. (But they can execute plain methods of the same object!) Terminology: threads try to acquire locks on objects. If the objects are locked, threads block on them until acquisition. When exiting a synchronized method, threads release the lock. class Sync_Obj { synchronized void sync_method() { ... } void plain_method() { ... } }

12. Condition variables Critical regions are not enough. A thread may acquire a lock only to find out that shared data has not been updated. It must then release the lock and “nap” until some condition holds. Another thread may insure the condition holds and then “wake up” the first one with a signal (notification). Such capabilities are usually called condition variables andare part of the monitor features.

13. Wait, notify, etc In Java there is only one(nameless) condition variable for each synchronized object and it is realized through the methods wait(), notify()and notifyAll() These are methods of the synchronized object itself. They are declared in the class Object (not in the class Thread). They must be called from synchronized methods of the object. Thread thr executes wait() of object obj : thr is suspended, its lock on objis released and thr is put in the wait queue of obj . Some thread executes notify() (notifyAll() ) of object obj : one of ( all ) the threads in the wait queue of obj are dequeued and put among the threads that are trying to acquire the lock on obj. When they do, they resume execution after the wait() that caused them to suspend.

14. Other synchronization operations In addition, special mechanisms for suspension, interruption and for and producing certain events are associated with the threads themselves. A thread can wait for another thread to terminate by using the join() method. There are methods (in the class Thread) to put a thread to sleep for a period of time, to suspend it indefinitely until an explicit execution resumption is invoked, and to yield control of execution if other threads are waiting to execute

15. The Sieve of Eratosthenes This program generates all prime numbers between 2 and a value MAX_PRIME. Eratosthenes' algorithm is the following. Generate all numbers starting from 2 and pass them through a sequence of sieves (filters). Each filter tests for divisibility by a prime number and lets through those numbers that are not divisible (and therefore candidate primes). In the beginning, the only filter we have is the one that tests for divisibility by 2. A new filter is created the first time the last filter lets a number p through. Indeed, p must be a prime (why?) and the new filter will be set up to test for divisibility by p. The potential parallelism here comes from the fact the filters could perform their work simultaneously since at any given moment they work on different numbers. A good analogy is that the numbers come down a pipe that is interrupted by successive filters. This is an example of pipelined parallelism. In Java we implement each filter as a thread.

16. The Sieve of Eratosthenes, continued main ...,6,5,4,3 ...,11,9,7,5 Filter divisibleby 2 Filter divisibleby 3 7,11,13,17... 17,19,23,29... 11,13,17,19... 13,17,19,23... Filter divisibleby 11 Filter divisibleby 5 Filter divisibleby 7

17. The Sieve of Eratosthenes, continued class Filter extends Thread { private int prime; private Channel input, output; private boolean lastThread; private Filter next; public Filter (int val, Channel in) { super ("Filter" + Integer.toString(val)); prime = val; input = in; lastThread = true; // When created, this IS last } public void run () { try { my_run(); // Annoying! } catch (InterruptedException e) {} }

18. The Sieve of Eratosthenes, continued private void my_run () throws InterruptedException { while (true) { int p = input.get(); if (p == -1) { // no more to come, must terminate if ( !lastThread ) { output.put(-1); // this thread not last, must // signal termination to next next.join(); // wait for next to terminate } stop(); // terminate too } // otherwise: if ( p % prime != 0 ) { // if p not div by prime if (lastThread) { // then p is a prime! Primes.res[p] = true; output = new Channel (); next = new Filter(p, output); // create and start next.start(); // next filter lastThread = false; } else output.put(p); }}}} // pass along

19. The Sieve of Eratosthenes, continued public class Primes { private static final int MAX_PRIME = 50; static boolean[] res = new boolean[MAX_PRIME + 1]; public static void main (String args []) throws InterruptedException { Channel out = new Channel (); Filter first = new Filter (2, out); first.start(); for (int p = 3; p <= MAX_PRIME; p++) out.put(p); out.put(-1); // signal end of sequence to first filter first.join(); // wait for first filter to terminate System.out.println(); for (int p = 2; p <= MAX_PRIME; p++) if (res[p] == true) System.out.print (p + " "); System.out.println(); } } // Output: // 2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 The primality results are stored in an array of booleans. While this array is globally shared by the filters, each filter accesses a separate entry in the array so we do not need to synchronize it.

20. The Sieve of Eratosthenes, continued class Channel { private int value; private boolean sent; public Channel () { sent = false; } public synchronized void put(int val) throws InterruptedException { while ( sent ) wait(); sent = true; value = val; notify(); } public synchronized int get() throws InterruptedException { while ( !sent ) wait(); sent = false; notify(); return value; } } The passing of numbers between the filters needs to be synchronized. We do it with shared objects in which the numbers are stored and then retrieved (often called shared buffers or channels of communication).

21. The Sieve of Eratosthenes, continued Note the whileloop that keeps testing for sent even after being woken up from wait. In this case we have only two threads that share a channel and we could get away with a simple if. But this is bad practice. If we did this, then to satisfy ourselves that the program is correct we need to examine the program globally. As the code stands now, a local inspection suffices, and the extra check is a small price to pay for this. Moreover, if we modify the program and make more than two threads share the same channel, an if would simply not suffice. Indeed, there is no guarantee that between the moment the signaling thread relinquishes the lock on the shared object and the moment the waiting thread reacquires the lock, a third thread could not have changed the state of the shared object. Note also the pesty InterruptedException that need to be declared in the headers although it cannot happen in this program. Ultimately we need to catch it in run because its header is fixed by the language specification without declaring it.

22. The producers-consumers problem We have seen in the Sieve of Eratosthenes example how two threads communicate using a simple ``channel'', or ``buffer'' that can hold one data item at any given time. More generally, we may wish to resolve the so-called PRODUCERS-CONSUMERS problem, in which several threads (producers) wish to pass information to one consumer) thread, or even several threads (consumers), as long as it doesn't matter which of the consumers gets it. Using a buffer of size 1 for this creates a bottleneck. Thus we show here how to implement buffers of arbitrary fized size. To store the buffers' data we use queues, thus providing the additional guarantee that the data produced by a given thread is consumed in the order in which it was put in the buffer. We assume in what follows an implementation of queues of fixed capacity.

23. Producers-consumers, continued Using queues, we now build buffers. We replace the pesty InterruptException with a runtime exception that is not statically checked, and we do it just once by defining a new ``waiting'’ method: mywait() class MyInterruptXcp extends RuntimeException {} class Buffer { final void mywait() { try { wait(); } catch (InterruptedException e) { throw new MyInterruptXcp(); } } private Queue qQ; Buffer(int capacity) { qQ = new Queue(capacity); }

24. Producers-consumers, continued ... synchronized void put(Object o) { while (qQ.isfull()) mywait(); try { qQ.enqueue(o); } catch (QueueXcp e) {} //shouldn't happen! notify(); } synchronized Object get() { Object tmp = null; while (qQ.isempty()) mywait(); try { tmp = qQ.front(); qQ.dequeue(); } catch (QueueXcp e) {} //shouldn't happen! notify(); return tmp; }

25. Producers-consumers, continued Note the complications introduced by the need to either catch or declare the “normal” exception QueueXcp even though in this code we test separately for emptyness and fullness of the queue. Note also that we had to change the name of the “waiting” method rather than just override it, because it is final in Object (presumably in order to deter the kind of politically incorrect hacking we did here). To test all this, we make up producer threads that generate integers and put them in a common buffer, and consumer threads that get the integers out of the common buffer and print them (see next).

26. Producers-consumers, continued class Producer extends Thread { private int min, step, max; Producer(int m, int s, int M) { super(); min = m; step = s; max = M; } public void run() { for (int i=min; i<=max; i=i+step) BufferTest.buf.put(new Integer(i)); }} class Consumer extends Thread { private int N; Consumer(int n) { super(); N = n; } public void run() { for (int i=1; i<=N; i++) { // (pretty)prints BufferTest.buf.get() details omitted }}}

27. Producers-consumers, continued public class BufferTest { static Buffer buf = new Buffer(10); public static void main (String args []) { Producer p1 = new Producer(0,3,99); Producer p2 = new Producer(1,3,100); Producer p3 = new Producer(2,3,101); Consumer c = new Consumer(102); c.start(); p1.start(); p2.start(); p3.start(); } }

28. Producers-consumers, continued /* Output: 0 3 1 2 6 4 5 9 7 8 12 10 11 15 13 18 21 14 17 16 19 24 27 20 23 22 25 30 33 26 29 28 31 36 39 32 35 34 37 42 45 38 41 40 43 48 51 44 47 46 49 54 57 50 53 52 55 60 63 56 59 58 61 66 69 62 65 64 67 72 75 68 71 70 73 78 81 74 77 76 79 84 87 80 83 82 85 90 93 86 89 88 91 96 99 92 95 98 101 94 97 100 */

29. Producers-consumers, continued ... p1.start(); p2.start(); c.start(); p3.start(); ... /* Output: 0 3 6 9 12 15 18 21 24 27 30 2 5 33 36 1 4 8 11 39 42 7 10 14 17 45 48 13 16 20 23 51 54 19 22 26 29 57 60 25 28 32 35 63 66 31 34 38 41 69 72 37 40 44 47 75 78 43 46 50 53 81 84 49 52 56 59 87 90 55 58 62 65 93 96 61 64 68 71 99 74 67 70 73 76 79 82 85 88 91 94 97 100 77 80 83 86 89 92 95 98 101 */ If we change the order in which the threads are started, the output becomes very different !

30. Producers-consumers, continued We could attempt to figure out the thread scheduling policy by looking at such results, but the real lesson is that in concurrent programs one cannot really assume anything about how threads (of equal priority) interleave! Concurrent programs must use synchronization to make sure that the results that we care about are not dependent on the thread interleaving.