Enterprise Java Series: Clustering and Load Balancing J2EE Application

1. Enterprise Java Series: Clustering and Load Balancing J2EE Application By : Rudi Har (Rudi.Har@gmail.com) (http://www.jroller.com/page/rudi_har) This material is slide version of Enterprise Talk I : "Enterprise The Beginning" and Enterprise Talk II : "Clustering and Load Balancing J2EE Application on Web Layer"

2. What is Enterprise? a. The Buzzword  • There is a need for distributed transaction ? • More then 30 or 40 or 50 people to develop in minimal 1 year ? • Need interfacing with legacy application, EIS and such things? • Using those expensive giant mainframes? b. Without the Buzzword • A people who want to give you a lot money (too much money) to get things done. (which will be considered as a buzzword by other people  ) • May just means to make something as good as other existing “enterprise”. • So complicated that if you explain to other people they can never get the point  • Ok, what ever it is, when we talk about enterprise we talk about critical and complicated application. • What is Enterprise and what is not is relative. The definition will always change because the meaning of “enterprise” word is so abstract!

3. Why Web Application? a. Without the Buzzword: • Far Better Performance for WAN environment. • Even if there is firewall/network security policy setting between interconnected LAN. eg: some Branches/offices are connected to 1 data center on separated geographic location: Banking, financial company, typical of multinational company. • Web application is far more suitable for this kind of setup. b. The ‘Buzzword’ that you've heard (they’re true sometimes): • Only http is permitted to pass public network. (Good company doesn't use public network anyway) • Flexible Installation (I won't mind to install my software on 50 or so workstations, when it only happens once every year) • Internet enabled. (Well, many internal business application actually doesn't need it  This is the top 3 reason of why a web based enterprise application could be considered as a buzzword. Something you use even if you don't need it all the time 

4. How much is the Demand for Application with High Availability? a. The Buzzword: • Some service oriented company like telecom and banking always try to improve their services. • In terms of application it means they want 24X7 availability. • This kinds of business always want to increase availability from 95% to 98% to 99% to 99,9% to 99,99% to 99,999% and so on. • The cost to increase availability from 99,99 % to 99,999 % is much higher than from 95% to 98%. b. Without the Buzzword: • The company will want that at least service will be available on working hours. • Nowadays more and more company doing business mostly using computer. • Interruption of service will stop the business and cost money! • I've seen this kind of company, especially in country like Singapore where the only things that they have in office is computer and software. • For this kinds of company they need at least fail over enable application. With no SPOF (Single point of failures) and no data loss when failing over. • It’s rarely exist such a robust application that it will never fail. Thus application failover mechanism is desperately wanted!

5. What is Clustering? = Many computers collaborating to do certain task and look transparently as one system from consumer/ client view. The Goal: • No single SPOF (Single Point Of Failure) either on database layer, application layer, not even network layer (example: using multiple data center on separate geographic location) • System failure doesn't make the services totally malfunction (there is failover mechanism) • When failover happen there is no data loss (replication) • To balance the load between many computer to achieve best performance. • Ability to increase application throughput by adding hardware or else. • This will lead to: high availability, reliability, scalability (all the buzzword things we’ve heard)

6. J2EE Clustering? a. Clustering on web layer (sufficient for most application) b. Clustering on business component layer (EJB or other technology) c. Clustering on resources (JNDI, connection pool, etc) d. Clustering on messaging. e. Clustering and CACHING on database. CACHING is capitalized because normally you will think of it as just a buzzword :-) This time we will only explore Clustering on Web Layer  (It's sufficient for most applications)

7. How it works? a. IP Multicast b. Object serialization. c. Some odd primitive ways: persisting state in shared file or database. (note: There is this thing what so called as peer to peer clustering which is the latest technology just invented)

8. Environment Setup Examples? a. 2 request dispatcher (only 1 run at the same time, 1 backup which will taking over when main dispatcher is down) b. 2 or more load balancer - Hardware load balancer - Software load balancer c. 2 or more application server. d. 2 or more database server. Note: This magic "2" number in every layer is minimal to avoid SPOF (Single Point of Failure).

9. What are the things to concern on Web Clustering Strategy? Clustering strategy: • Application without session. State saved in client cookies. - Don't need replication, failover scenario available. - Impossible nowadays for security reason (eg: the unpleasant original developer can easily break it on production environment) and for performance reason (example: storing and reading state from database are expensive tasks) • Sticky load balancing (load balancer sticks particular client to be served by only 1 server). - Normally load balancer add a cookie to the incoming request or rewrite the URL - Don't need replication, but don't have failover scenario. • Non sticky Load balancing - Need replication, failover is available. - Session replication needed for failover. - Session replication needed for load balancing. - Load balancing for replication :-) This one is interesting, I'll show you. - Both load balancing and session replication must be use at the same time.

10. Web Load Balancing Algorithm • Random • Round Robin • Weight Based • Content Based Routing • Advanced Algorithm

11. Programming and Design Guidelines • Make sure your session object is serializable (implements Serializable) • Don't be tempted to keep application state (not session state) in singleton. • Use cluster wide JNDI instead. (but don't used application object, it's not clustered) • Don't let multi frames page accessing (write-read or write-write) same session object if you are developing for a non sticky clustering system. • Consider acceptable trade off between better load balancing algorithm and performance when choosing sticky or non sticky • Consider also the trade off between when using synchronous or asynchronous replication. It’s good on multi frames write-write: multiple modification on session object only replicated once. What kind of application is yours? • Sticky load balancing using server affinity won't work in SSL communication. And by the way almost all good application are using SSL nowadays :-D • Use web server proxy with SSL accelerator to keep track the affinity.

12. Trouble shooting - Got problem on Failing Over Session? - Browser only send back cookie (http header session id) to the same domain :-) - That is the place where the load balancer will solve. From the client view there is only 1 server/domain which is the load balancer. The cluster members stand behind the load balancer.

13. Clustering with Tomcat 5.0 Here are some steps to configure Tomcat clustering: 1. Enable Tomcat clustering service. By default Tomcat clustering service is off. Edit the server.xml file to enable it. Add some line under the <host> entry as shown below: This is undocumented anywhere in Apache website or else  But luckily you can see some remarks in the server.xml file itself 

14. Clustering with Tomcat 5.0 …(continue) 2. Enable Clustering in your Application This is quiet straight forward. To enable your application for clustering add <distributable/> entry into your web.xml file.

15. Clustering with Tomcat 5.0 …(continue) 3. Setting up Load Balancer using Apache 2.x web server • Without load balancer you don’t have access into your own session  This is due to the nature of all browser who will only send back the cookie into the domain who owns it. • Luckily we don’t have to buy expensive load balancer hardware. Apache2 with mod_jk modules can be used as software load balancer ! • Add mod_jk into your Apache installation. • Edit httpd.conf

16. Clustering with Tomcat 5.0 …(continue) • Create worker.properties file under apache conf/ folder • This file contains the load balancing setting and policies that will be used by Apache+modJk This one is well documented on Apache website under mod_jk user references guide 

17. Demo  • Horizontal Scaling Scenario: - No load balancer needed, just redirect URL. - Use session replication. • Vertical Scaling Scenario: - Without load balancing - With load balancing, sticky, no replication (no failover available) - With load balancing, non sticky, no replication (application malfunction) - With load balancing, sticky, with replication (weak load balancing, failover available) - With load balancing, non sticky, with replication (good load balancing, failover available, with trade off)

18. New Problem  • Performance drop when the system grow with many servers. • Overhead of transferring session data to all machines. • Many machines means many users, thus many replications. • Now all machines have to keep redundant session info from many users from many machines! Solution? - Introduce Sub Cluster - Known as Domain in Weblogic and Partition in JBoss. - Not available in Tomcat 

19. Way to go? • Clustering on Business Component Layer (EJB or other technology) • If we have enough time ;-) If we don’t, we’ll talk about it when we meet again sometime and somewhere else on this planet  See You Again…

20. Bonus : Preview on Enterprise Talk  Enterprise Talk V : The Myths About Spring And Hibernate That Can Hurt You Jade inilah dongeng tentang Sejoli dari 2 musim berbeda. Yg satu dgn semangat berbinar-binar datang dari cerahnya musim semi (Spring) yg satunya telah sekian lama bertapa dari segala kegiatan (Hibernating) selama kelamnya musim dingin  Yg satu sangat bersemangat untuk menyamai prestasi yg satunya lagi, yg konon menurut cerita mitos telah menjelma jadi EJB3! [Sebagaimana mitos umumnya, tentu saja tidak benar] Dia sudah mulai menjalani masa jadi seorang beruban, yg bijaksana dan low profile karena jasa-jasanya sudah diakui dan tidak perlu diperjuangkan lagi dengan jalan memuji diri sendiri: "Hey, look, I'm cool...". Berdiam diri dgn tenang dan rendah hati di dalam database (hibernating) lebih relevant buatnya 