Clustering implementation in jboss
This presentation is the property of its rightful owner.
Sponsored Links
1 / 43

Clustering Implementation in JBoss PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

Clustering Implementation in JBoss. Jakša Vučković Università di Bologna. Clustering J2EE. Replicating Resources Communication Concurrency Control Failure Handling Controlling Access to External Resources. Replicating Resources. Stateful Session Bean Replication Entity Bean Replication

Download Presentation

Clustering Implementation in JBoss

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Clustering implementation in jboss

Clustering Implementation in JBoss

Jakša Vučković

Università di Bologna


Clustering j2ee

Clustering J2EE

  • Replicating Resources

  • Communication

  • Concurrency Control

  • Failure Handling

  • Controlling Access to External Resources

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Replicating resources

Replicating Resources

  • Stateful Session Bean Replication

  • Entity Bean Replication

  • HTTP Session Replication

  • JNDI Tree Replication

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Sticky sessions and homogeneous deployment

Sticky Sessions and Homogeneous Deployment

  • JBoss’ staff recommends the use of sticky sessions and homogeneous deployment

    • Using sticky sessions means that all requests of a client arrive to the same node during a session

    • Homogeneous deployment means that each application component is replicated an all the nodes, so requests don’t need to span on different nodes

  • Sticky sessions + homogeneous deployment imply that each request will be entirely processed on one node

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Stateful session beans

Stateful Session Beans

  • Each instance of a SFSB is associated to a client

  • The lifetime of a SFSB is determined by a timeout from the last client access

  • EJBs cannot generate threads

  • All access to a SFSB instance should be serialized

  • SFSBs are non-reentrant

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Sfsb replication

SFSB Replication

  • After each invocation of a SFSB, if the bean changed, a message is broadcast to the cluster

  • The message contains the serialized state of the session bean

  • Before broadcasting the message a distributed lock is acquired

  • The broadcast operation is synchronous (blocking) with acknowledgements from all the nodes of the cluster

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Rmi clients

RMI Clients

  • The clients downloads an “intelligent” stub from the server and uses it to access the bean

  • The stub encapsulates the load balancing policy and the failover mechanism

  • The stubs maintains a list of server addresses which is updated on each request

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Rmi clients failover

RMI Clients - Failover

  • If a call to a node fails the client stub fails over to another server or raises an exception to the client application

  • The stub fails over to another server when:

    • there is a communication exception

    • The server does not responds because it crashed or a network partition occurred

  • The stub reports an exception when:

    • There are no more servers to fail over to

    • The server reported a GenericClusteringException with the flag COMPLETED_YES or COMPLETED_MAYBE

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Rmi clients failover1

RMI Clients - Failover

  • If a client makes a request to a node and does not receive the response four scenarios are possible:

    • The node crashed before receiving the request – failover safe

    • The node received the request but crashed before broadcasting the state – failover safe

    • The node received and processed the request, broadcast the state and then crashed before sending the reply – the failover node must recognize a duplicate request

  • The client indicates in the request message if it is a failover or the original request

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem

Partial Replication Problem

  • We have two beans S1 and S2 on two nodes A and B

node B

S1

S2

S2

Client

node A

S1

S1

S2

S2

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem1

Partial Replication Problem

  • The client invokes S1 on node A

node B

S1

S2

S2

Client

node A

S1

S1

S2

S2

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem2

Partial Replication Problem

  • S1 invokes S2

node B

S1

S2

S2

Client

node A

S1

S1

S2

S2

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem3

Partial Replication Problem

  • Invocation of S2 is completed and its state is replicated on node B

node B

S1

S2

S2

Client

node A

S1

S1

S2

S2

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem4

Partial Replication Problem

  • S1 tries to make another call to S2 but node A crashes

node B

S1

S2

S2

Client

node A

S1

S1

S2

S2

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem5

Partial Replication Problem

  • Now we have an inconsistent state in node B

node B

S1

S2

Client

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Partial replication problem6

Partial Replication Problem

  • There are two possible solutions to this problem:

    • A replication aware Transaction Manager would need to roll back the state of S2 in B also

    • The updated state is broadcast to other nodes only when committing the whole transaction

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions

Network Partitions

  • A network failure could divide the cluster in two or more partitions that cannot communicate

Node A

Node B

No shared data (Entity Beans, DB) is accessed!!!

Node C

Node D

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions1

Network Partitions

  • The client stub maintains a list of available servers

  • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition

Node A

Node B

No shared data (Entity Beans, DB) is accessed!!!

Node C

Node D

client

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions2

Network Partitions

  • The client stub maintains a list of available servers

  • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition

Node A

Node B

No shared data (Entity Beans, DB) is accessed!!!

Node C

Node D

client

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions3

Network Partitions

  • The client stub maintains a list of available servers

  • If a client is not able to contact the server instance it is bound to, it tries to contact the next from the list until it reaches an instance that is its own partition

Node A

Node B

No shared data (Entity Beans, DB) is accessed!!!

Node C

Node D

client

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions4

Network Partitions

  • Each SFSB is associated to a client, so there will be no other clients accessing the instance on another partition

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

C

D

s1

s1

Client

2

s2

s2

s3

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions5

Network Partitions

  • Each clients list of available servers will be updated to contain only the nodes on its partition

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

C

D

s1

s1

Client

2

s2

s2

s3

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging

Network Partitions Merging

  • When the cluster merges the clients will still have the old list of nodes

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

C

D

s1

s1

Client

2

s2

s2

s3

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging1

Network Partitions Merging

  • Only when the client makes the next request, the bean gets replicated on the other nodes and the client’s list is updated

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

C

D

s1

s1

Client

2

s2

s2

s3

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging2

Network Partitions Merging

  • S3 is not updated so the client must maintain a separate list of nodes for each SFSB

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

C

D

s1

s1

Client

2

s2

s2

s3

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging problem

Network Partitions Merging Problem

  • Suppose node C crashes now

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

D

s1

Client

2

s2

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging problem1

Network Partitions Merging Problem

  • Client 2 wants to invoke s2 and fails over to node A

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

D

s1

Client

2

s2

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Network partitions merging problem2

Network Partitions Merging Problem

  • If s2 invokes s3 we find an “out of date” state on this node

A

B

s1

s1

Client

1

s2

s2

s3

s3

No shared data (Entity Beans, DB) is accessed!!!

D

s1

Client

2

s2

s3

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Entity beans

Entity Beans

  • Entity Beans are persistent

  • Unlike Session Beans, Entity Beans are not associated to clients

  • Can be accessed by multiple clients, but not concurrently

  • If an Entity Bean is declared as reentrant it can be accessed concurrently but only from the same transaction

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Entity beans1

Entity Beans

  • JBoss does not replicate Entity Beans, replication is left to the database tier

  • JBoss controls concurrent access to the database

  • All JBoss nodes share the same database

  • It is transparent to JBoss if the database is clustered or not

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Entity beans2

Entity Beans

  • Entity Bean state is loaded from the DB before each business method invocation and stored after the invocation (Commit option B and C)

  • To each J2EE transaction is associated a DB transaction

  • There are to policies to deal with concurrency:

    • Optimistic

    • Pessimistic

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Optimistic policy

Optimistic Policy

  • Transactions are executed concurrently

  • Each Transaction has it’s own instance of the Entity Bean

  • Shared row locking is used in the database

  • Before committing the container verifies if the Bean changed in the database

    • If yes the transaction is rolled back and re-executed

    • If not the new value is stored in the DB

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Pessimistic policy

Pessimistic Policy

  • Transactions are not executed concurrently

  • Exclusive Row locking at the DB is used

  • A transaction blocks if it tries to access a locked row until the row is released

  • A Distributed Deadlock Detection algorithm is used to detect Deadlocks

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Entity bean replication failures

Entity Bean Replication Failures

  • If the node owner of a transaction crashes the DB rolls back the transaction

  • Network partitions must be handled by the DB clustering mechanism

  • With a non replicated database, the approach of a primary partition is used where the primary partition is defined to be the one containing the database

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Entity bean replication

Entity Bean Replication

  • Relies intensively on DB

    • Maximizes DB I/O

  • If we don’t want a single point of failure, we must use a replicated database

  • Entity Beans are not used as cache

  • A distributed cache with distributed locks is under development

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Http session replication

HTTP Session Replication

  • An HTTP Session is implemented as a serializable object

  • The HTTP Session state is NOT broadcast to the other nodes but replication is implemented through an Entity Bean

  • The HTTP Session is a cmp-field of that bean

  • The CMP engine serializes the HTTP Session object along with its contents

  • The HTTP Session contains also references (remote objects) of SFSBs

  • SFSBs do not get serialized with the HTTP Session

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Http session replication1

HTTP Session Replication

  • Failures are handled by the EJB Replication mechanism

  • Client fail over behavior depends on the HTTP dispatcher

  • A JBoss HTTP Dispatcher is under development

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Jndi tree replication

JNDI Tree Replication

  • Each node has a local JNDI Tree

  • All nodes share a replicated HA-JNDI Tree

  • When a client looks up an object on node N

    • The replicated HA-JNDI service is invoked first

    • If it cannot find it there it looks in the local JNDI Tree

    • If it’s not there it asks all other nodes if they have it in their local JNDI Tree

    • If it hasn’t been found a NameNotFoundException is thrown

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Jndi tree replication1

JNDI Tree Replication

  • Unlike clients, EJBs access only the local JNDI Tree

  • This solution was chosen because of

    • Compatibility with existing applications

    • Distinction between local and replicated objects

    • Low network traffic in homogeneous clusters

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Distributed cache

Distributed Cache

  • Deadline for June (JavaONE)

  • Based on JavaGroups

  • Will be a Jboss Service

  • Highly Configurable

  • Will be applied to Entity Bean Clustering and Session Bean Clustering

  • Will enable replication of the Database

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Distributed cache semantics

Distributed Cache Semantics

  • Three semantics:

    • Asynchronous

      • broadcast update and return immediately

    • Synchronous

      • broadcast update and wait for acknowledgements

    • Serialized Synchronous

      • acquires locks before updating a bean

      • ensures ordering but not consistency

      • consistency is obtained at another level

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Cache contents

Cache Contents

  • The cache will implement the XAResource interface

  • This will enable the use of the cache in a Two Phase Commit protocol

  • The mapping of the XAResource semantics to the cluster is not a trivial problem

  • This could enable to do the DB update, cluster replication and sending the response to the client an atomic operation

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


Conclusions

Conclusions

  • JBoss clustering algorithm does not handle all failure patterns correctly

  • Simply changing the GC semantics will not solve the problems

  • To make the existing mechanism work correctly

    • A replication aware Transaction Manager needs to be implemented

    • During state merging the “out-of-date” beans must be marked

  • An alternative would be to re-implement the clustering from scratch

  • The development of the Distributed Cache by JBoss authors is a good opportunity to redefine the clustering algorithm

Clustering Implementation in JBoss – Jakša Vučković, Università di Bologna


  • Login