Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

1. Clustering the JVM using AOP • Jonas Bonér • Terracotta, Inc. • jonas@terracotta.org • http://terracotta.org • http://jonasboner.com

2. Let’s start with a demo • – since a picture says more than a thousand words

3. Agenda • Overview of Terracotta • Get started • The inner workings of a clustered JVM • Real-world examples • Q&A

4. Agenda • Overview of Terracotta • Get started • The inner workings of a clustered JVM • Real-world examples • Q&A

5. Network-Attached Memory Open Source Scalability and High-Availability for the JVM What is Terracotta?

6. What is Terracotta? • Network-Attached Memory • No API: declarative configuration to selectively share object graphs across the cluster • No serialization: plain POJO clustering • Fine-grained replication: field-level, heap-level replication • Cross-JVM coordination: Java Memory Model maintained across the cluster – e.g. cluster-wide wait/notify and synchronized • Large virtual heaps: that exceeds the capacity of a single JVM • Distributed Method Invocations • Runtime monitoring and control

7. Use-cases • Relieving Database Overload • Distributed Caching • Hibernate Clustering • HTTP Session Clustering • Simplifying Application Architecture and Development • Virtual Heap for Large Datasets • Clustering OSS Frameworks (Spring, Struts, Lucene, Wicket, EHCache etc.) • Master/Worker – Managing Large Workloads • POJO Clustering • Messaging, Event-based Systems and Coordination-related Tasks

8. Terracotta approach App Server App Server App Server Web App Web App Web App Business Logic Business Logic Business Logic Frameworks Frameworks Frameworks Frameworks Frameworks Frameworks JVM JVM JVM • Today's Reality • Scale out is complex • Requires custom Java code • Our different approach • Cluster the JVM • Eliminate need for custom code Scale-out Custom Development JMS RMI Serialization Terracotta Server Clustering the JVM

9. The architecture of Terracotta App Server App Server App Server Web App Web App Web App Sessions Objects Sessions Objects Sessions Objects Any Framework Any Framework Any Framework Any Java Objects Any Java Objects Any Java Objects JVM JVM JVM • Terracotta Server • 100% Pure Java • HA Active / Passive Pair • Local JVM Client • Transparent • Pure Java Libraries • Coordinator “traffic cop” • Coordinates resource access • Runtime optimizations • Central Storage • Maintains state across restarts Scale-out Terracotta Library Terracotta Library Terracotta Library Terracotta Server (ACTIVE) Clustering the JVM Terracotta Server Clustering the JVM Terracotta Server (PASSIVE) Clustering the JVM

10. Network-Attached Memory • Management Console • Runtime visibility • Data introspection • Cluster monitoring TC Management Console • Heap Level Replication • Declarative • No Serialization • Fine Grained / Field Level • GET_FIELD - PUT_FIELD • Only Where Resident • JVM Coordination • Distributed Synchronized Block • Distributed wait()/notify() • Fine Grained Locking MONITOR_ENTRY -MONITOR_EXIT • Large Virtual Heaps • As large as available disk • Dynamic paging Scale-out App Server App Server App Server Web App Web App Web App Shared Objects Shared Objects Shared Objects JVM JVM JVM TC Libraries TC Libraries TC Libraries Terracotta Server Clustering the JVM

11. Agenda • Overview of Terracotta • Get started • The inner workings of a clustered JVM • Real-world examples • Q&A

12. Hello World

13. Hello World - tutorial/HelloWorld.java package tutorial; import java.util.*; publicclassHelloWorld{ privateList<String>hellos=newArrayList<String>(); publicvoid sayHello(){ synchronized(hellos){          hellos.add("Hello, World "+newDate()); for (String hello : hellos){ System.out.println(hello); } } } publicstaticvoid main(String[] args){ newHelloWorld().sayHello(); } }

14. Hello World - tc-config.xml <?xml version="1.0" encoding="UTF-8"?> <tc:tc-configxmlns:tc="http://www.terracotta.org/config"> <application> <dso> <roots> <root><field-name>tutorial.HelloWorld.hellos</field-name></root> </roots> <locks> <autolock> <method-expression>* tutorial.HelloWorld*.*(..)</method-expression> <lock-level>write</lock-level> </autolock> </locks> <instrumented-classes> <include><class-expression>tutorial..*</class-expression></include> </instrumented-classes> </dso> </application> </tc:tc-config>

15. Demo

16. Enhanced Hello WorldUsing java.util.concurrent

17. Coordination - HelloWorldConcurrent.java package tutorial; import java.util.*; import java.util.concurrent.CyclicBarrier; publicclassHelloWorldConcurrent{ privateList<String> hellos =newArrayList<String>(); privateCyclicBarrierbarrier; publicvoid sayHello()throwsException{ barrier=newCyclicBarrier(2); barrier.await(); synchronized(hellos){          hellos.add("Hello, World "+newDate()); for(String hello : hellos){ System.out.println(hello); } } } publicstaticvoid main(String[] args)throwsException{ newHelloWorldConcurrent().sayHello(); } }

18. Coordination - tc-config-concurrent.xml <?xml version="1.0" encoding="UTF-8"?> <tc:tc-configxmlns:tc="http://www.terracotta.org/config"> <application> <dso> <roots> <root><field-name>tutorial.HelloWorldConcurrent.hellos</field-name></root> <root><field-name>tutorial.HelloWorldConcurrent.barrier</field-name></root> </roots> <locks> <autolock> <method-expression>* tutorial.HelloWorldConcurrent*.*(..)</method-expression> <lock-level>write</lock-level> </autolock> </locks> <instrumented-classes> <include><class-expression>tutorial..*</class-expression></include> </instrumented-classes> </dso> </application> </tc:tc-config>

19. Demo

20. Agenda • Overview of Terracotta • Get started • The inner workings of a clustered JVM • Real-world examples • Q&A

21. Inner workings of a clustered JVM • Maintain the semantics of the application (e.g Java Language Specification (JLS) and the Java Memory Model (JMM)) across the cluster • Load-time bytecode instrumentation • Hook in java.lang.ClassLoader and -javaagent • Transparent to the user -- classes are woven on-the-fly • Aspect-Oriented Programming techniques • AspectWerkz aspects • Custom bytecode instrumentation using ASM • Declarative configuration (XML) • Simplified pointcut pattern language (based on AspectWerkz)

22. What do we need to maintain across the cluster? • State sharing • Share data in the Java heap for the instances reachable from a shared top-level “root” reference • Make sure each JVMs local heap are in sync (in regards to the clustered parts of the heap) • Thread coordination • Resource access and guarding • Thread signaling

23. Sample application put top level root in shared space instrument clusterable classes lock and start tx intercept notifyAll commit tx and unlock reconcile pending changes record field change short circuit root creation • @Clustered public class Counter { • @Root public final static Counter instance = new Counter(); • private final Object lock = new Object(); • private volatile int counter = 0; • public void increment() { • synchronized(lock) { this.counter++; lock.notifyAll(); } • } • public void waitFor(int expected) { • synchronized(lock) { • while(this.counter < expected) { • try { lock.wait(); } catch(InterruptedException ex) {} • } • } • }

24. State sharing - field-level replication and management • We need to break down the shared object graph into “literals” • int, long, float, String, boolean etc. • Need to be able to detect changes to the object graph • New object reference is added/removed • Literals changes value • Every reference has a “shadow” that is “managing” the reference • Knows if the reference has been changed • Can lazily page in and out the actual value • Page in: Reconcile changes done in another node • Page out: Null out the reference

25. Field-level replication each instance has a shadow that is managing access and modification Grandparent 500 bytes Parent1 Parent2 500 bytes each Child1 Child2 Child3 Child3 16 bytes each Total object graph size: 1548 bytes Terracotta replicates: 16 bytes

26. Manage field access and modification • Intercept the PUTFIELD and GETFIELD bytecode instructions • Create two advice for managing get and set shared field • before(TransparentAccess o, Object value) : • set(* *.*) && isClusteredTarget(o) && args(value) { • … // register changed field value in shadow • } • before(TransparentAccess o) : • get(* *.*) && isClusteredTarget(o) { • … // reconcile any pending changes • }

27. BCI Example – original byte code • public demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager, demo.sharededitor.ui.Renderer); • Code: • 0: aload_0 • 1: invokespecial #1; //Method javax/swing/event/MouseInputAdapter."<init>":()V • 4: aload_0 • 5: aload_2 • … snip • 33: aload_0 • 34: aload_1 • 35: putfield #6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager; • 38: aload_0 • 39: getfield #6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager; • 42: aload_0 • 43: getfield #2; //Field renderer:Ldemo/sharededitor/ui/Renderer; • 46: invokevirtual #7; //Method demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V • 49: return

28. BCI Example – instrumented byte code • public demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager, demo.sharededitor.ui.Renderer); • Code: • 0: aload_0 • 1: invokespecial #99; //Method javax/swing/event/MouseInputAdapter."<init>":()V • 4: aload_0 • 5: aload_2 • … snip • 33: aload_0 • 34: aload_1 • 35: invokevirtual #117; //Method __tc_setobjmgr:(Ldemo/sharededitor/models/ObjectManager;)V • 38: aload_0 • 39: invokevirtual #119; //Method __tc_getobjmgr:()Ldemo/sharededitor/models/ObjectManager; • 42: aload_0 • 43: getfield #101; //Field renderer:Ldemo/sharededitor/ui/Renderer; • 46: invokevirtual #123; //Method demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V • 49: return

29. Thread coordination • We need to maintain the semantics of the Java Memory Model (JMM) across multiple JVMs • Maintain the semantics of: • synchronized(object) {..} • wait()/notify()/notifyAll() • Flushing of changes • Locking optimizations • User defined • Runtime optimized

30. Manage synchronized blocks • Intercept MONITORENTER and MONITOREXIT bytecode instructions • before(Object o) : • isClustered() && lock() && args(o) { • ClusterManager.lock(o); // lock o in cluster • } • after(Object o) : // after finally • isClustered() && unlock() && args(o) { • ClusterManager.unlock(o); //unlock o in cluster + flush changes • }

31. Manage thread coordination (wait/notify) • Intercept the wait(), notify() and notifyAll() methods in java.lang.Object • void around(Object o) : • isClusteredTarget(o) && call(void Object.wait()) { • ClusterManager.objectWait(o); • } • void around(Object o) : • isClusteredTarget(o) && call(void Object.notify()) { • ClusterManager.objectNotify(o); • } • … // remaining methods omitted

32. Agenda • Overview of Terracotta • Get started • The inner workings of a clustered JVM • Real-world examples • Q&A

33. HTTP Session Clusteringwithout Java Serialization

34. HTTP Session Clustering - Benefits • Terracotta Sessions gives you: • Near-Linear Scale • No java.io.Serializable • Large Sessions - MBs • Higher Throughput • Supported Platforms: • Jetty, • JBoss 4.x, • Tomcat 5.0 & 5.5, • WebLogic 8.1, WebLogic 9.2, • WebSphere CE, Geronimo Alpha, WebSphere 6.1

35. HTTP Session Clustering - DummyCart.java package demo.cart; import java.util.*; publicclassDummyCart{ privateList items =newArrayList(); publicList getItems(){ returnCollections.unmodifiableList(items); } publicvoid addItem(String name){         items.add(name); } publicvoid removeItem(String name){         items.remove(name); } }

36. HTTP Session Clustering - carts.jsp <%@ page import="java.util.Iterator" %> <html> <jsp:useBean id="cart" scope="session" class="demo.cart.DummyCart" /> <% String submit = request.getParameter("submit"); String item = request.getParameter("item"); if (submit != null && item != null) { if (submit.equals("add")) { cart.addItem(item); } else if (submit.equals("remove")) { cart.removeItem(item); } }%> <body> <p>You have the following items in your cart:</p> <ol><% Iterator it = cart.getItems().iterator(); while (it.hasNext()) { %><li><% out.print(it.next()); %></li> <% } %> </ol> <form method=“get" action="<%=request.getContextPath()%>/cart.jsp"> <p>Item to add or remove: <input type="text" name="item" /></p> <input type="submit" name="submit" value="add" /> <input type="submit" name="submit" value="remove" /> </form> </body> </html>

37. HTTP Session Clustering - tc-config.xml <?xml version="1.0" encoding="UTF-8"?> <tc:tc-config xmlns:tc="http://www.terracotta.org/config"> <application> <dso> <web-applications> <web-application>cart</web-application> </web-applications> <instrumented-classes> <include> <class-expression>demo.*</class-expression> </include> </instrumented-classes> </dso> </application> </tc:tc-config>

38. Demo

39. Cluster Spring andother OSS frameworks

40. Glimpse of supported integrations • 'Supported' by Terracotta means: • Just include a ‘Configuration Module’ (OSGi bundle) in your config to cluster your application (a one liner) • Plugs in underneath without any setup • Technically, Terracotta supports all integrations as long as it runs on the JVM (f.e. JRuby, PHP)

41. Example Configuration Module- EHCache clustering <?xml version="1.0" encoding="UTF-8"?> <tc:tc-configxmlns:tc="http://www.terracotta.org/config"> <clients> <modules> <modulename="clustered-ehcache-1.3"version="1.0.0"/> </modules> </clients> <application> <dso> <instrumented-classes> <include> <class-expression>tutorial.*..*</class-expression> </include> </instrumented-classes> </dso> </application> </tc:tc-config>

42. Example - Spring clustering Terracotta config Spring config <spring> <applicationname="tc-jmx"> <application-contexts> <application-context> <paths> <path>*/applicationContext.xml</path> </paths> <beans> <beanname="clusteredCounter"/> <beanname="clusteredHistory"/> </beans> </application-context> </application-contexts> </application> </spring> • Terracotta can declaratively cluster Spring beans (Singleton + Session and Custom scoped) with zero code changes • Can also cluster Spring ApplicationContext events, JMX State and Spring Web Flow <beanid="localCounter" class="demo.jmx.Counter"/> <beanid="clusteredCounter" class="demo.jmx.Counter"/> <beanid="localHistory" class="demo.jmx.HistoryQueue"/> <beanid="clusteredHistory" class="demo.jmx.HistoryQueue"/>

43. DemoCluster Spring Web Flow’s Web Continuations (conversational/workflow state)

44. Agenda • Overview of Terracotta • Get started • Real-world examples • Q&A

45. Wrapping up

46. Wrapping up • Terracotta is Network-Attached Memory for the JVM • Turns Scalability and High-Availability into a deployment artifact • Keep the simplicity of POJO-based development – get Scale-Out with Simplicity • Makes mission-critical applications simpler to: • Write • Understand • Test • Maintain • Endless possibilities for clustering and distributed programming – these were just a few • Be creative, use your imagination and have fun…

47. Questions?http://terracotta.orgjonas@terracotta.org