1 / 43

Clustering Implementation in JBoss

Clustering Implementation in JBoss. Jakša Vučković Università di Bologna. Clustering J2EE. Replicating Resources Communication Concurrency Control Failure Handling Controlling Access to External Resources. Replicating Resources. Stateful Session Bean Replication Entity Bean Replication

colin-rhodes
Download Presentation

Clustering Implementation in JBoss

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clustering Implementation in JBoss Jakša Vučković Università di Bologna

  2. Clustering J2EE • Replicating Resources • Communication • Concurrency Control • Failure Handling • Controlling Access to External Resources Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  3. Replicating Resources • Stateful Session Bean Replication • Entity Bean Replication • HTTP Session Replication • JNDI Tree Replication Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  4. Sticky Sessions and Homogeneous Deployment • JBoss’ staff recommends the use of sticky sessions and homogeneous deployment • Using sticky sessions means that all requests of a client arrive to the same node during a session • Homogeneous deployment means that each application component is replicated an all the nodes, so requests don’t need to span on different nodes • Sticky sessions + homogeneous deployment imply that each request will be entirely processed on one node Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  5. Stateful Session Beans • Each instance of a SFSB is associated to a client • The lifetime of a SFSB is determined by a timeout from the last client access • EJBs cannot generate threads • All access to a SFSB instance should be serialized • SFSBs are non-reentrant Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  6. SFSB Replication • After each invocation of a SFSB, if the bean changed, a message is broadcast to the cluster • The message contains the serialized state of the session bean • Before broadcasting the message a distributed lock is acquired • The broadcast operation is synchronous (blocking) with acknowledgements from all the nodes of the cluster Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  7. RMI Clients • The clients downloads an “intelligent” stub from the server and uses it to access the bean • The stub encapsulates the load balancing policy and the failover mechanism • The stubs maintains a list of server addresses which is updated on each request Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  8. RMI Clients - Failover • If a call to a node fails the client stub fails over to another server or raises an exception to the client application • The stub fails over to another server when: • there is a communication exception • The server does not responds because it crashed or a network partition occurred • The stub reports an exception when: • There are no more servers to fail over to • The server reported a GenericClusteringException with the flag COMPLETED_YES or COMPLETED_MAYBE Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  9. RMI Clients - Failover • If a client makes a request to a node and does not receive the response four scenarios are possible: • The node crashed before receiving the request – failover safe • The node received the request but crashed before broadcasting the state – failover safe • The node received and processed the request, broadcast the state and then crashed before sending the reply – the failover node must recognize a duplicate request • The client indicates in the request message if it is a failover or the original request Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  10. Partial Replication Problem • We have two beans S1 and S2 on two nodes A and B node B S1 S2 S2 Client node A S1 S1 S2 S2 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  11. Partial Replication Problem • The client invokes S1 on node A node B S1 S2 S2 Client node A S1 S1 S2 S2 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  12. Partial Replication Problem • S1 invokes S2 node B S1 S2 S2 Client node A S1 S1 S2 S2 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  13. Partial Replication Problem • Invocation of S2 is completed and its state is replicated on node B node B S1 S2 S2 Client node A S1 S1 S2 S2 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  14. Partial Replication Problem • S1 tries to make another call to S2 but node A crashes node B S1 S2 S2 Client node A S1 S1 S2 S2 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  15. Partial Replication Problem • Now we have an inconsistent state in node B node B S1 S2 Client Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  16. Partial Replication Problem • There are two possible solutions to this problem: • A replication aware Transaction Manager would need to roll back the state of S2 in B also • The updated state is broadcast to other nodes only when committing the whole transaction Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  17. Network Partitions • A network failure could divide the cluster in two or more partitions that cannot communicate Node A Node B No shared data (Entity Beans, DB) is accessed!!! Node C Node D Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  18. Network Partitions • The client stub maintains a list of available servers • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node B No shared data (Entity Beans, DB) is accessed!!! Node C Node D client Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  19. Network Partitions • The client stub maintains a list of available servers • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node B No shared data (Entity Beans, DB) is accessed!!! Node C Node D client Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  20. Network Partitions • The client stub maintains a list of available servers • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition Node A Node B No shared data (Entity Beans, DB) is accessed!!! Node C Node D client Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  21. Network Partitions • Each SFSB is associated to a client, so there will be no other clients accessing the instance on another partition A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! C D s1 s1 Client 2 s2 s2 s3 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  22. Network Partitions • Each clients list of available servers will be updated to contain only the nodes on its partition A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! C D s1 s1 Client 2 s2 s2 s3 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  23. Network Partitions Merging • When the cluster merges the clients will still have the old list of nodes A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! C D s1 s1 Client 2 s2 s2 s3 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  24. Network Partitions Merging • Only when the client makes the next request, the bean gets replicated on the other nodes and the client’s list is updated A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! C D s1 s1 Client 2 s2 s2 s3 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  25. Network Partitions Merging • S3 is not updated so the client must maintain a separate list of nodes for each SFSB A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! C D s1 s1 Client 2 s2 s2 s3 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  26. Network Partitions Merging Problem • Suppose node C crashes now A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! D s1 Client 2 s2 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  27. Network Partitions Merging Problem • Client 2 wants to invoke s2 and fails over to node A A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! D s1 Client 2 s2 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  28. Network Partitions Merging Problem • If s2 invokes s3 we find an “out of date” state on this node A B s1 s1 Client 1 s2 s2 s3 s3 No shared data (Entity Beans, DB) is accessed!!! D s1 Client 2 s2 s3 Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  29. Entity Beans • Entity Beans are persistent • Unlike Session Beans, Entity Beans are not associated to clients • Can be accessed by multiple clients, but not concurrently • If an Entity Bean is declared as reentrant it can be accessed concurrently but only from the same transaction Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  30. Entity Beans • JBoss does not replicate Entity Beans, replication is left to the database tier • JBoss controls concurrent access to the database • All JBoss nodes share the same database • It is transparent to JBoss if the database is clustered or not Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  31. Entity Beans • Entity Bean state is loaded from the DB before each business method invocation and stored after the invocation (Commit option B and C) • To each J2EE transaction is associated a DB transaction • There are to policies to deal with concurrency: • Optimistic • Pessimistic Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  32. Optimistic Policy • Transactions are executed concurrently • Each Transaction has it’s own instance of the Entity Bean • Shared row locking is used in the database • Before committing the container verifies if the Bean changed in the database • If yes the transaction is rolled back and re-executed • If not the new value is stored in the DB Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  33. Pessimistic Policy • Transactions are not executed concurrently • Exclusive Row locking at the DB is used • A transaction blocks if it tries to access a locked row until the row is released • A Distributed Deadlock Detection algorithm is used to detect Deadlocks Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  34. Entity Bean Replication Failures • If the node owner of a transaction crashes the DB rolls back the transaction • Network partitions must be handled by the DB clustering mechanism • With a non replicated database, the approach of a primary partition is used where the primary partition is defined to be the one containing the database Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  35. Entity Bean Replication • Relies intensively on DB • Maximizes DB I/O • If we don’t want a single point of failure, we must use a replicated database • Entity Beans are not used as cache • A distributed cache with distributed locks is under development Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  36. HTTP Session Replication • An HTTP Session is implemented as a serializable object • The HTTP Session state is NOT broadcast to the other nodes but replication is implemented through an Entity Bean • The HTTP Session is a cmp-field of that bean • The CMP engine serializes the HTTP Session object along with its contents • The HTTP Session contains also references (remote objects) of SFSBs • SFSBs do not get serialized with the HTTP Session Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  37. HTTP Session Replication • Failures are handled by the EJB Replication mechanism • Client fail over behavior depends on the HTTP dispatcher • A JBoss HTTP Dispatcher is under development Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  38. JNDI Tree Replication • Each node has a local JNDI Tree • All nodes share a replicated HA-JNDI Tree • When a client looks up an object on node N • The replicated HA-JNDI service is invoked first • If it cannot find it there it looks in the local JNDI Tree • If it’s not there it asks all other nodes if they have it in their local JNDI Tree • If it hasn’t been found a NameNotFoundException is thrown Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  39. JNDI Tree Replication • Unlike clients, EJBs access only the local JNDI Tree • This solution was chosen because of • Compatibility with existing applications • Distinction between local and replicated objects • Low network traffic in homogeneous clusters Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  40. Distributed Cache • Deadline for June (JavaONE) • Based on JavaGroups • Will be a Jboss Service • Highly Configurable • Will be applied to Entity Bean Clustering and Session Bean Clustering • Will enable replication of the Database Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  41. Distributed Cache Semantics • Three semantics: • Asynchronous • broadcast update and return immediately • Synchronous • broadcast update and wait for acknowledgements • Serialized Synchronous • acquires locks before updating a bean • ensures ordering but not consistency • consistency is obtained at another level Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  42. Cache Contents • The cache will implement the XAResource interface • This will enable the use of the cache in a Two Phase Commit protocol • The mapping of the XAResource semantics to the cluster is not a trivial problem • This could enable to do the DB update, cluster replication and sending the response to the client an atomic operation Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

  43. Conclusions • JBoss clustering algorithm does not handle all failure patterns correctly • Simply changing the GC semantics will not solve the problems • To make the existing mechanism work correctly • A replication aware Transaction Manager needs to be implemented • During state merging the “out-of-date” beans must be marked • An alternative would be to re-implement the clustering from scratch • The development of the Distributed Cache by JBoss authors is a good opportunity to redefine the clustering algorithm Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna

More Related