Semantics of Multithreaded Java Jeremy Manson and William Pugh

1. Semantics of Multithreaded JavaJeremy Manson and William Pugh Background Material Jack Newton University of Alberta newton@cs.ualberta.ca

2. Overview Multithreaded Programming in Java Memory Consistency Models Java Memory Model Problems with the Java Memory Model

3. Multithreaded Programming in Java • Java is the first widely used programming language to provide support for concurrent programming at the language level • Java supports concurrency through the use of threads, and provides mechanisms for synchronization between threads

4. Java Threads: Example • This example will illustrate the basic principles of multithreaded programming in Java sum sumEditor sumPrinter int aint bint c Sum sum Sum sum

5. Java Threads: Example(Continued) public class Sum { int a, b, c; public Sum() { } public String toString() { return this.a + " + " + this.b + " = " + this.c; } }

6. Java Threads: Example(Continued) public class SumEditor extends Thread { Sum sum; public SumEditor(Sum _sum) { this.sum = _sum; } public void run() { for (int i = 1; i < 10; i++) { this.sum.a = i; this.sum.b = i; this.yield(); this.sum.c = this.sum.a + this.sum.b; } } }

7. Java Threads: Example(Continued) public class SumPrinter extends Thread { Sum sum; public SumPrinter(Sum _sum) { this.sum = _sum; } public void run() { for (int i = 1; i < 10; i++) { System.out.println(sum); this.yield(); } } }

8. Java Threads: Example(Continued) public class ThreadingExperiment { public ThreadingExperiment() { } public static void main(String[] args) { Sum sum = new Sum(); SumEditor sumEditor = new SumEditor(sum); SumPrinter sumPrinter = new SumPrinter(sum); System.out.println("main: starting threads"); sumEditor.start(); System.out.println("main: started sumEditor thread"); sumPrinter.start(); System.out.println("main: started sumPrinter thread"); System.out.println("main: exiting"); } }

9. Java Threads: Example(Continued) • Output from this example main: starting threads main: started sumEditor thread main: started sumPrinter thread main: exiting 1 + 1 = 0 2 + 2 = 2 3 + 3 = 4 4 + 4 = 6 5 + 5 = 8 6 + 6 = 10 7 + 7 = 12 8 + 8 = 14 9 + 9 = 16

10. Java Threads: Example(Continued) • What’s going wrong? sumEditor sum sumPrinter for (int i = 1; i < 10; i++) { this.sum.a = i; this.sum.b = i; this.yield(); this.sum.c = this.sum.a + this.sum.b; } 4 for (int i = 1; i < 10; i++) { System.out.println(sum); this.yield(); } 4 a 0 1 4 4 4 b 1 0 4 4 c 0 2 4 1 + 1 = 0

11. Java Threads: Synchronization • Java provides two methods to synchronize threads • Synchronized methods • Synchronized statements • Both of these methods use object locks to enforce mutual exclusion • Only one thread can have a lock on an object at a time • If a thread attempts to obtain a lock on an object that is already blocked, the thread will block

12. Java Threads: Synchronization(Continued) • Synchronized Methods • Object to obtain lock on is implicit • Instance of class the synchronized method appears in for non-static methods • Class method the synchronized method appears in for static methods public synchronized Object getValue() • Synchronized Statements • Object to obtain lock on is explicit synchronized (object) { … }

13. Java Threads: Synchronization(Continued) Thread 1 Lock object object Thread 2 Blocked Blocked Thread 3 Thread 4 Blocked Lock

14. Java Threads: Synchronized Example public class SumEditor extends Thread { Sum sum; public SumEditor(Sum _sum) { this.sum = _sum; } public void run() { for (int i = 1; i < 10; i++) { synchronized (sum) { this.sum.a = i; this.sum.b = i; this.yield(); this.sum.c = this.sum.a + this.sum.b; } } } }

15. Java Threads: Synchronized Example(Continued) public class SumPrinter extends Thread { Sum sum; public SumPrinter(Sum _sum) { this.sum = _sum; } public void run() { for (int i = 1; i < 10; i++) { synchronized (sum) { System.out.println(sum); this.yield(); } } } }

16. Java Threads: Synchronized Example(Continued) • Impact of synchronization sumEditor sum sumPrinter for (int i = 1; i < 10; i++) { synchronized (sum) { this.sum.a = i; this.sum.b = i; this.yield(); this.sum.c = this.sum.a + this.sum.b; } } 4 for (int i = 1; i < 10; i++) { synchronized (sum) { System.out.println(sum); this.yield(); } } 4 a 1 0 4 4 4 b 0 1 4 4 4 4 c 2 0 4 1 + 1 = 2

17. Memory Models • When we discuss memory models, we really mean memory consistency models • A memory consistency model should provide a formal specification of how the memory system behaves • This memory model essentially constrains the possible values that a read of a variable can return

18. Memory Models(Continued) • Memory models allow a programmer to reason about the potential outcomes of both synchronized and unsynchronized programs • Memory models take on special significance with Java, where data races could be used to compromise security

19. Memory Models: Sequential Consistency • Sequential consistency provides an intuitive model of memory consistency • Lamport defines a system to be sequentially consistent if: “the result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by the program”

20. Memory Models: Sequential Consistency(Continued) P1 P2 P3 P4 PN … Main Memory

21. Memory Models: Sequential Consistency(Continued) P1 P2 P3 P4 PN … Main Memory

22. Strong / Relaxed Memory Models • Strong memory models, such as sequential consistency, severely restrict the set of potential optimizations that can be applied • For example, compiler optimizations such as code motion, register allocation, and common sub-expression elimination are disallowed under sequential consistency [Maessen 2000] • Relaxed (or weak) memory models relax memory ordering constraints to allow for certain optimizations

23. Memory Models: Coherence • Coherence is an example of a relaxed memory model • Coherence is similar to sequential consistency, with the following relaxation: • For a given processor/thread, the program order is maintained on a per-location basis

24. Memory Models: Coherence P1 P2 P2 PN R(a) R(b) W(d) W(d) … R(b) R(c) R(a) R(a) W(a) W(a) W(c) W(b) R(b) W(b) R(a) a b c d Main Memory

25. Java Memory Model • The Java Memory Model (JMM) is defined in Chapter 17 of the Java Language Specification • The JMM is hard to understand • Ambiguous in places • Complicated enough that even the authors of the Java Language Specification didn’t completely understand it • All current JVM implementations violate the specification to some degree

26. Java Memory Model: Terminology • A variable refers to • Static variable of a loaded class • Field of an allocated object • Element of an allocated array • All variables are stored in a main memory that is shared by all threads • Every thread has a working memory where it keeps working copies of variables

27. Java Memory Model Main Memory Buffer Buffer Buffer … Working Memory Working Memory Working Memory Execution Engine Execution Engine Execution Engine Thread 1 Thread 2 Thread N

28. Java Memory Model(Continued) Main Memory write read Buffer Buffer Buffer store load … Working Memory Working Memory Working Memory assign use Execution Engine Execution Engine Execution Engine Thread 1 Thread 2 Thread N

29. Java Memory Model(Continued) Main Memory lock/unlock Buffer Buffer Buffer … Working Memory Working Memory Working Memory Execution Engine Execution Engine Execution Engine Thread 1 Thread 2 Thread N

30. Java Memory Model(Continued) Main Memory 3 3 3 1 1 2 unlock lock Buffer Buffer Buffer … Working Memory Working Memory Working Memory 2 3 1 1 3 3 2 6 3 Execution Engine Execution Engine Execution Engine Thread 1 Thread 2 Thread N

31. Java Memory Model: Example 4 x = y = 0 • When Thread 1 and Thread 2 have finished executing, can we have: a = 1, b = 1? • Answer: Yes! Threads Begin Thread 1 Thread 2 4 4 a = x b = y 4 4 y = 1 x = 1 Jan-Willem Maessen, Arvind, Xiaowei Shen. Improving the Java Memory Model using CRF. In Proceedings of the 15th Annual Conference on Object-Oriented Programming Systems, Languages and Applications, Portland, OR, Oct. 2000.

32. Java Memory Model: Prescient Stores • The JMM allows prescient stores: storing the value that will be assigned to a variable by a future assign operation before the assign occurs • The prescient store of a variable V is allowed in thread T if the following conditions are satisfied: • If the store on V occurs, the assign is bound to occur • No lock operation occurs between the store and assign • No load of V occurs between the store and assign • No other store of V occurs between the store and assign

33. Java Memory Model: Example(Continued) Thread 1 read x x = y = 0 Threads Begin Thread 2 read y Thread 1 Thread 2 a = x b = y load y load x use y y = 1 x = 1 use x assign x Main Memory assign y write read store x Buffer store y store load Working Memory write y assign use Execution Engine write x …

34. Java Memory Model: Example(Continued) Thread 1 Thread 1 store y read x Thread 2 Thread 2 read y load y write y load x read y use y read x use x load y assign x load x assign y use y store x use x store y assign x assign y write y store x write x write x

35. Java Memory Model: Volatile • Variables declared volatile are treated specially: • Read from main memory on every use • Written to main memory on every assign • Accesses to volatiles are sequentially consistent (i.e. prescient stores not allowed) • Access to longs/doubles are atomic • The memory semantics of accessing a volatile are similar to accessing an object exclusively through synchronized get() and set() methods

36. Java Memory Model: Final • Fields declared as final can only be assigned to once in the constructor for the class in which they are defined • The Java Memory Model does not discuss final fields at all! • Several optimizations for final fields are obvious, but are not permitted under the current JMM

37. Java Memory Model:Summary • The JMM is difficult to understand – research papers that discuss the JMM interpret it differently • The JMM is at least as strong as Coherence • William Pugh has shown that the JMM is actually stronger than Coherence

38. Problems with the Java Memory Model • Three main problems exist with the current Java Memory Model: • It prohibits many important compiler optimizations • It is incomplete: the semantics for final fields are not defined • It is hard to interpret

39. Problems with the JMM:Compiler Optimizations • The JMM requires Coherence, which prohibits many important compiler optimizations • A compiler would not be permitted to perform fetch elimination Thread 1 Thread 2 Thread 1 Thread 2 v = p.f v = p.f p.f = 2 p.f = 2 w = q.f w = p.f If p == q at runtime (p and q are aliased) x = v x = p.f x = p.f x = v

40. Problems with the JMM:Semantics of Final • The current JMM does not provide any discussion of final fields • Final fields are another potential source of optimizations that cannot be performed under the current JMM • Current semantics may result in immutable objects appearing mutable

41. Problems with the JMM:Complexity • The three stages presented in the JMM make it difficult to reason about the order of memory operations Main Memory write read Buffer store load Working Memory assign use Execution Engine Thread

42. Problems with the JMM:Complexity(Continued) • The current JMM is complex and restrictive enough that all current JVM implementations violate it to some degree • Pugh suggests that the JMM, in its current form, will never be clear. New proposals for semantics, such as those in Semantics of Multithreaded Java, should offer a completely new description

43. References • Sarita V. Adve and Kourosh Gharacholoo. Shared Memory Consistency Models: A Tutorial. Rice University ECE Technical Report 9512. • James Gosling, Bill Joy, Guy Steele, Gilad Bracha. The Java Language Specification, Second Edition. • David Holmes. The Java Memory Model: Defining Concurrency Correctness on Multiprocessors. http://www.csse.monash.edu.au/courseware/cse3420/Lectures/Module11/dholmes.ppt. • Jan-Willem Maessen, Arvind, Xiaowei Shen. Improving the Java Memory Model using CRF. In Proceedings of the 15th Annual Conference on Object-Oriented Programming Systems, Languages and Applications, Portland, OR, Oct. 2000.David Mosberger. Memory Consistency Models. University of Arizona Technical Report 93/11. • William Pugh. Fixing the Java Memory Model. Java Grande Conference, June 12-14, 1999. • William Pugh. The Java Memory Model is Fatally Flawed. Concurrency: Practice and Experience.