Concurrent Queues

1. Concurrent Queues שירן חליבה

2. Outline: • Some definitions • 3 queue implementations : • A Bounded Partial Queue • An Unbounded Total Queue • An Unbounded Lock-Free Queue

3. Introduction and some definitions: • Pools show up in many places in concurrent systems. Forexample, in many applications, one or more producer threads produce items to beconsumed by one or more consumer threads. • To allow consumers to keep up, we can place a bufferbetween the producers and the consumers. • Often, pools act as producer–consumer buffers. • A pool allows the same item to appear more than once.

4. Introduction and some definitions cont. • A queue is a special kind of pool with FIFO fairness. • Itprovides an enq(x) method that puts item x at one end of the queue, called thetail, and a deq() method that removes and returns the item at the other end of thequeue, called the head.

5. Bounded vs. Unbounded • A pool can be bounded or unbounded. • Bounded • Fixed capacity • Good when resources an issue • Unbounded • Holds any number of objects

6. Blocking vs. Non-Blocking • Problem cases: • Removing from empty pool • Adding to full (bounded) pool • Blocking • Caller waits until state changes • Non-Blocking • Method throws exception

7. Total vs. Partial • Pool methods may be total or partial. • A method is total if calls do not wait for certain conditions to become true. For example, a get() call that tries to remove an item from an empty pool immediately returns a failure code or throws an exception. A total interface makes sense whenthe producer (or consumer) thread has something better to do than waitfor the method call to take effect.

8. Total vs. Partial • A method is partial if calls may wait for conditions to hold. For example, a partial get() call that tries to remove an item from an empty pool blocks until an item is available to return. A partial interface makes sense when the producer (or consumer) has nothing better to do than to wait for the pool to become nonempty (or non full).

9. Queue: Concurrency y=deq() enq(x) tail head enq() and deq() work at different ends of the object

10. Concurrency y=deq() enq(x) head tail Challenge: what if the queue is empty or full?

11. A Bounded Partial Queue Sentinel First actual item head tail deqLock Lock out other deq() calls enqLock Lock out other enq() calls permits 8 Permission to enqueue 8 items

12. Enqueuer head tail deqLock No need to lock tail? Read permits Lock enqLock enqLock permits 8 OK

13. Enqueuer head tail deqLock enqLock Enqueue Node permits 8 7 getAndDecrement() (Why atomic?)

14. Enqueuer head tail deqLock If queue was empty, notify waiting dequeuers enqLock permits Release lock 8 7

15. Unsuccesful Enqueuer head tail deqLock Read permits enqLock permits 0 Uh-oh

16. Dequeuer head tail deqLock Read sentinel’s next field enqLock Lock deqLock permits 7 OK

17. Dequeuer head tail deqLock Read value enqLock permits 7

18. Make first Node new sentinel Dequeuer head tail deqLock enqLock permits 7

19. Dequeuer head tail deqLock enqLock permits 7 Release deqLock

20. Dequeuer head tail deqLock enqLock Increment permits (no need lock?) permits 8 Answer: we had to hold the lock while enqueuing to prevent lots of enqueuers from proceeding without noticing that the capacity had been exceeded. Dequeuers will notice the queue is empty when they observe that the sentinel’s next field is null

21. Unsuccesful Dequeuer head tail deqLock Read sentinel’s next field enqLock permits 8 uh-oh

22. Bounded Queue public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

23. The ReentrantLock is a monitor (The mechanism that Java uses to support synchronization ). Allows blocking on a condition rather than spinning.How do we use it?(*More on monitors: http://www.artima.com/insidejvm/ed2/threadsynch.html)

24. Lock Conditions public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

25. Await • Releases lock associated with q • Sleeps (gives up processor) • Awakens (resumes running) • Reacquires lock & returns q.await()

26. Signal • Awakens one waiting thread • Which will reacquire lock q.signal();

27. A Monitor Lock Lock() waiting room unLock() Critical Section

28. Unsuccessful Deq Lock() waiting room Deq() await() Critical Section Oh no, Empty!

29. Another One Lock() waiting room Deq() await() Critical Section Oh no, Empty!

30. Enqueur to the Rescue Yawn! Yawn! Lock() waiting room Enq( ) signalAll() Critical Section unLock()

31. Monitor Signalling Yawn! Yawn! waiting room Critical Section Awakend thread might still lose lock to outside contender…

32. Dequeurs Signalled Yawn! waiting room Found it Critical Section

33. Dequeurs Signalled Yawn! waiting room Critical Section Still empty!

34. Dollar Short + Day Late waiting room Critical Section

35. Why not signal()?

36. Lost Wake-Up Yawn! Lock() waiting room Enq( ) signal () Critical Section unLock()

37. Lost Wake-Up Yawn! Lock() waiting room Enq( ) Critical Section unLock()

38. Lost Wake-Up Yawn! waiting room Critical Section

39. Lost Wake-Up waiting room Found it Critical Section

40. What’s Wrong Here? zzzz….! waiting room Critical Section

41. Enq Method public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … }

42. Cont… public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }

43. The Enq() & Deq() Methods • Share no locks • That’s good • But do share an atomic counter • Accessed on every method call • That’s not so good • Can we alleviate this bottleneck? What is the problem?

44. Split the Counter • The enq() method • Decrements only • Cares only if value is zero • The deq() method • Increments only • Cares only if value is capacity

45. Split Counter • Enqueuer decrements enqSidePermits • Dequeuer increments deqSidePermits • When enqueuer runs out • Locks deqLock • Transfers permits • (dequeuer doesn't need permits- check head.next) • Intermittent(תקופתי) synchronization • Not with each method call • Need both locks! (careful …)

46. An Unbounded Total Queue • Queue can hold an unbounded number of items. • The enq() method always enqueues its item. • The deq() throws EmptyException if there is no item to dequeue. • No deadlock- each method acquires only one lock. • Both the enq() and deq() methods are total as they do not wait for the queue to become empty or full.

47. An Unbounded Total Queue

48. A Lock-Free Queue Sentinel head tail • Extension of the unbounded total queue • Quicker threads help the slower threads • Each node’s next field is an: • AtomicReference<Node> • The queue itself consists of two • AtomicReference<Node> fields: • head and tail

49. Compare and Set CAS

50. LockFreeQueue<T> class