Mastering Concurrent Programming in Java

Concurrent Programming in Java http://flic.kr/p/7HT8qT

What are you goingto learn about today? • Concurrency in Java • What • How • Why • Problems with concurrency • … and some solutions! http://flic.kr/p/8JpkTg

Recall this step in a servlet transaction A different thread handles each request Head First Servlets and JSP (2nd edition), pp. 95–96

One servlet, potentially many threads Spoiler: This can lead to big problems! Head First Servlets and JSP (2nd edition), p. 101

What could go wrong with this servlet? public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } Design intent:if ensuresthat poll()will return a job Hint: Threads proceed nondeterminstically relative to one another http://flic.kr/p/9ksxQa

Imagine there is one job in the queue, andtwo clients send POST requests… public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } 2 1

Thread States Running Ready Blocked/Waiting Terminated

Imagine there is one job in the queue, andtwo clients send POST requests… public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } 2 1 Client 2 executes first

Imagine there is one job in the queue, andtwo clients send POST requests… public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } 2 1 Client 2 completes the call to isEmpty(),and then a context switch occurs

Imagine there is one job in the queue, andtwo clients send POST requests… public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } 2 1 Another context switch occurs

Imagine there is one job in the queue, andtwo clients send POST requests… public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … } 2 1 Error! Client 2 calls poll() on an empty queue

Nondeterministic thread scheduling increases complexity and causes nasty problems public class Foo implements Runnable { private int x = 0; private int y = 0; public void thread1() { x = 10; ///////////// A y = x + 10; ///////// B } public void thread2() { x = y; ////////////// C y = 0; ////////////// D } } Race condition: Some schedules produce errors

Time-of-Check-to-Time-of-Use (TOCTOU) Errors:A common type of race condition public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … }

So why would you want concurrency anyway? Performance. Performance. Performance. Increase responsiveness. Do more work in less time. Increase throughput. http://flic.kr/p/9ksxQa

Why would it be lame to have just one thread? Requests would have to be processed in sequenceQuick requests might have to wait for slow ones http://flic.kr/p/9ksxQa Head First Servlets and JSP (2nd edition), p. 101

How do we preventrace conditions?Synchronization! Java provides two basic mechanisms: • Mutex locks • For enforcing mutually exclusive access to shared data • Condition variables • For enabling threads to wait for application-specific conditions to become true http://flic.kr/p/aGcqRp

Mutex locks • States: • unlocked • locked by exactly 1 thread with 0 or more waiting threads • Operations: • lock() (aka acquire) • unlock() (aka release) • Operations are atomic • Threads that call lock() on a held lock must wait http://flic.kr/p/678N6L

Common Mutex Lock Usage Pattern class X { private final ReentrantLock mutex = new ReentrantLock(); … public void m() { mutex.lock(); try { … method body … } finally { mutex.unlock() } } } Reentrant means that lock() can be called by the holding thread without deadlock Trick to make sure unlock happens on return Threads must lock m() before entering This pattern ensures that only one method executes m() at a time

In Java, every object has an intrinsic lock(inherited from class Object)Java provides a synchronized keyword for doing implicit acquires/releases class X { public synchronized void m() { // ... method body ... } } Can you fix our servlet example now?

What do you think about this solution? public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected synchronized void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … Just these two lines need to execute atomically It’s correct, but generally BAD! It essentially serializesdoPost()—which defeats the purpose of threads! http://flic.kr/p/9ksxQa

Here’s a design that maximizes(?) concurrency public class AssignJobServlet extends HttpServlet { private Queue<Job> jobQueue; … protected void doPost(… request, … response) … { PrintWriter out = response.getWriter(); Job currentJob = null; synchronized(this) { if (!jobQueue.isEmpty()) { currentJob = jobQueue.poll(); } } if (currentJob != null) { printWorkOrder(out, currentJob); } else { printNoJobsAvailable(out); } } … Synchronizedblock Alternative design: Put the synchronization in the queue

Design Issue:Client- vs. Supplier-Side Synchronization Client-side Supplier-side Pro: Synchronization logic localized Con: Client must find a supplier with appropriate synchronization behavior Pro: Proven synchronization logic less risky Con: Client must learn supplier’s interface—especially synch. contract • Con: Synchronization logic spread throughout code • Pro: Client has complete control over synchronization behavior • Con: Novel synchronization designs risky • Pro: Client understands (hopefully) synchronization behavior Supplier-side tends to be preferred in practice (but not always) See java.util.concurrent for thread-safe containers

Monitor Pattern: Most common way to implement supplier-side synchronization • Monitor object: One whose methods are executed in mutual exclusion • All public methods are synchronized http://flic.kr/p/7ZkGEH

Monitor Job Queue Example public class MonitorJobQueue { private Queue<Job> jobQueue; public synchronizedboolean offer(Job job) { return jobQueue.offer(job); } public synchronized Job poll () { // Returns null if empty. return jobQueue.poll(); } } All public methods synchronized, thusonly one thread at a time can be“inside” a MonitorJobQueue object

public class MonitorJobQueue { private Queue<Job> jobQueue; public synchronizedboolean offer(Job job) { return jobQueue.offer(job); } public synchronized Job poll () { // Returns null if empty. return jobQueue.poll(); } } What if a poll() caller needed to get a job once one was available? How to implement? • One way: Keep trying in a loop. Maybe sleep in between tries. • Better way: Sleep while queue is empty. Get signal to wake up when there is a job available. http://flic.kr/p/9ksxQa

Condition variables • Object for enabling waiting for some condition • Always associated with a particular mutex • Mutex must be held while invoking operations • Operations • await() (aka wait) • signal() (aka notify) • signalAll() (aka notifyAll; aka broadcast)

MonitorJobQueue with Conditions class MonitorJobQueue { private Queue<Job> jobQueue; final Lock lock = new ReentrantLock(); final Condition notEmpty = lock.newCondition(); public Job poll() throws … { lock.lock(); try { while (jobQueue.isEmpty()) { notEmpty.await(); } return jobQueue.poll(); } finally { lock.unlock(); } } public boolean offer(Job job) { lock.lock(); try { nonEmpty.signal(); return jobQueue.offer(job); } finally { lock.unlock(); } } }

Object also has one intrinsic condition variable class MonitorJobQueue { private Queue<Job> jobQueue; public synchronized Job poll() { while (isEmpty()) { wait(); } return jobQueue.poll(); } public synchronizedboolean offer(Job job) { notify(); return jobQueue.offer(job); } } If you need multiple conditions (e.g., notEmpty and notFull),you cannot use intrinsic locks/conditions

Summary • Java Threads • Concurrency is for performance • Race conditions • Mutex locks • Condition variables • Monitor pattern http://flic.kr/p/YSY3X

Mastering Concurrent Programming in Java