Deadlock Resolution via Exceptions for Dependable Java Applications

1. Deadlock Resolution via Exceptions for Dependable Java Applications Fancong Zeng Dept. of Computer Science Rutgers University 06/25/2003

2. Problem and Breakthrough • Problem: the productivity of dependable applications containing deadlock-free code only is unsatisfactory. • A necessary breakthrough is to provide a practical approach for incorporating code that is not deadlock free into dependable applications.

3. Deadlocks and Exceptions • (Fitness in nature) Deadlock occurrences represent not only abnormal states but also fatal errors. • Most modern programming languages, e.g., Java, support exception handling. • This paper proposes an approach of deadlock resolution via exceptions for dependable Java applications.

4. Outline of the Talk • Concepts and Designs • Deadlocks, exceptions, handlers and resolvers • Applications • Resolving deadlocks in existing code • Adjusting homegrown code to avoid deadlock • Dealing with false positives • Implementation • Programming effort • Performance • Conclusions and future work

5. Deadlocked Threads • Deadlocked threads cannot run any more due to an outstanding resource request which is never granted. • “Cyclic waits” are not required as a necessary condition; • More resource types lead to a broader application scope. • Resource Examples • Locks: threads need to acquire necessary locks in order to enter monitors. • Notifications: threads blocked on condition variables need notifications in order to be enabled for scheduling.

6. Deadlock Detection • Blocked group • If all threads in a group defined by the programmer are blocked at the same time, then the programmer believes they are blocked forever. • (Key) specifying groups: Exploiting existing ThreadGroup APIs to specify explicit groups; normal application threads not belonging to any explicit group belong to the default group. • Time out • If a thread has not run for an excessive long time, then it is fairly likely that it won’t run any more. • (Key) timeout value: Exploiting existing thread join() API to specify and dynamically change the timeout value. • Knot detection • If there is a knot in a dependence graph, then there is (possibly) a deadlock. • (Key) runtime edge establishment: • thread X holds the lock requested by thread Y  an L-edge from X to Y; • thread X holds the lock of the monitor providing the cond. var. thread Y is blocked on an N-edge from X to Y.

7. Deadlock Exceptions A DeadLock (DeadEnd, DeadKnot, resp.) instance is created and raised if a deadlock is detected via “blocked group” (“time out”, “knot detection”, resp.) RuntimeException DeadLock DeadEnd DeadKnot A B means that B is a subclass of A, and the condition for B is detected before that of A when both detection procedures are triggered. DeadCycle

8. Fields in DeadLock • The number of threads involved (special value: 0 for no deadlock) • The group id of this deadlock (special value: -1 for not applicable, 0 for the default group) • For each thread involved, • The thread object • Whether this thread has been delegated a deadlock exception • The type of the resource this thread is waiting for, e.g., locks. Moreover, if instances of the resource type can be missed, e.g., notifications, then • Whether the thread has ever missed an instance of the requested resource • Whether the thread has ever received an instance of the requested resource • The owners/producers of the resource this thread is waiting for

9. Local Deadlock Handlers • Local handlers: catch blocks designed for deadlock exceptions and installed for normal application threads. For example, public void run() { boolean successful=false; while (!successful) try { a1.transfer(a2,amount); successful = true ;} catch (DeadCycle e) { //wait for a while }}

10. Global Deadlock Handlers • Global Handlers: catch blocks designed for deadlock exceptions and installed for deadlock resolvers, which are special threads. • Why need deadlock resolvers? • Catching all deadlock exceptions • Realizing flexible recovery actions • Two deadlock resolvers • One for DeadEnd exception • The other for the other deadlock exceptions

11. A Simplified Architectural View Deadlock Resolvers and global handlers Normal Application Threads and local handlers Application Delegate deadlock exceptions Raise deadlock exceptions Exception Management Deadlock Detection JVM Report deadlocks A JVM instance supporting deadlock exceptions

12. Outline of the talk • Concepts and Designs • Deadlocks, exceptions, handlers and resolvers • Applications • Resolving deadlocks in existing code • Adjusting homegrown code to avoid deadlock • Dealing with false positives • Implementation • Programming effort • Performance • Conclusions and future work

13. A money-transfer example Context: There are two simultaneous transactions: one is to transfer money in the amount of s2c from saving account s to checking account c, the other is to transfer money in the amount of c2s from c to s. Problem: programming bugs in existing unchangeable code causes potential deadlocks

14. Code for class Account class Account { … // unchangeable public synchronized void transfer(Account to, int amount) { synchronized (to) { if (value < amount) { try {to.wait();} catch (InterruptedException e){}} if (value>=amount) { to.value = to.value + amount; value=value-amount; this.notifyAll();} }} ...}

15. Possible deadlocks • DeadCycle caused by a lock order bug • The two threads can hold a lock and wait for the lock held by the other thread • DeadKnot caused by a nested monitor call • The thread blocked on a condition variable holds a lock requested by the other thread that holds the lock corresponding to the condition variable • DeadLock caused by an assumption of execution order. • A thread blocked on a condition variable is never awakened up.

16. Installing deadlock resolvers public static void main(String[] args) { … ThreadGroup HG = new ThreadGroup(“DeadlockResolverGroup”); DeadlockResolver DH = new DeadlockResolver(s,c,s2c,c2s); Thread resolver = new Thread(HG,DH, “NoTimerResolver”); resolver.setPriority(Thread.MAX_PRIORITY); resolver.start(); …}

17. Implementing global handlers class DeadlockResolver implements Runnable { … public void run() {boolean cont = true; while (cont) { try{Thread.currentThread.join();} catch (DeadCycle e1) { if (e1.waiters[0].getName.equals(“C2S”)) e1.waiters[0].interrupt(); else e1.waiters[1].interrupt(); } catch (DeadKnot e2) {…} catch (DeadLock e3) {…} … }}}

18. Dealing with DeadCycle class S2CTransfer implements Runnable{ … //unchangeable void run() { a1.transfer(a2,amount);} class C2STransfer implements Runnable {… public void run() { boolean successful=false; while (!successful) try { a1.transfer(a2,amount); successful = true ;} catch (DeadCycle e) { //wait for a while }}}

19. Dealing with DeadKnot class DeadlockResolver implements Runnable { … catch (DeadKnot e2) { //stop a thread waiting for a notification if (e2.waitfors[0][0] == ‘N’) { trans_stopped=e2.waiters[0].getName(); e2.waiters[0].stop();} else { trans_stopped = e2.waiters[1].getName(); e2.waiters[1].stop();} } …}

20. Dealing with DeadLock class DeadlockResolver implements Runnable { … catch (DeadLock e3) { if (e3.number == 0) //no deadlock if (trans_stopped == null) cont = false; else if (trans_stopped.equals(“S2C”){ trans_stopped = null; new Thread(new S2CTransfer(s,c,s2c), “S2C”).start();} else { trans_stopped = null; new Thread(new S2CTransfer(c,s,s2c), “C2S”).start();} else if (e3.number ==1) {e3.waiters[0].stop(); cont = false;}}…}

21. Outline of the talk • Concepts and Designs • Deadlocks, exceptions, handlers and resolvers • Applications • Resolving deadlocks in existing code • Adjusting homegrown code to avoid deadlock • Dealing with false positives • Implementation • Programming effort • Performance • Conclusions and future work

22. Programming efforts • A Latte 0.9.1 JVM is used. • Only 5 files were touched. • Less than 1000 lines of code were added/modified. • Essentially, only the thread package and the exception handling module need to be slightly modified.

23. Cost of deadlock detection • DeadEnd: O(n) • DeadCycle: O(n) • DeadKnot: O(n) (in current implementation) • DeadLock: Constant (n is the number of threads involved.)

24. Frequencies of deadlock detection • DeadEnd: Timer expiration • DeadCycle: A contended lock request • DeadKnot: A contended lock request (in current implementation) • DeadLock: A contended lock request, a wait() invocation, and a thread termination.

25. Outline of the talk • Concepts and Designs • Deadlocks, exceptions, handlers and resolvers • Applications • Resolving deadlocks in existing code • Adjusting homegrown code to avoid deadlock • Dealing with false positives • Implementation • Programming effort • Performance • Conclusions and future work

26. Conclusions and future work • This paper describes an approach of deadlock resolution via exceptions, and shows that it is practical in developing dependable Java applications containing code that may deadlock. • Future work includes: • To implement the ideas in an industry product; • To support more resource types, e.g. RMI messages; • To deal with common safety properties; • To use a JIT compiler to improve detection precision, to reduce detection latencies and to semi-automate recovery code generation. (“self-healing”)

27. Welcome to read the paper • There are some typos in the paper, e.g., in Figure 2, line 23: "successful = false;" should be "successful=true;". But the typos can be healed by the paper itself (“self-healing”). • Welcome to send your comments to me: fzeng@cs.rutgers.edu, especially from industry people.