Subject: Load balancing with sticky sessions, using Apache http server (version 2.2.14 in my example) with mod_proxy_balancer as the front end balancer and Apache Tomcat 6.0.20 as the application engine. [[Tip should also work for other recent 2.2.x and 6.0.x versions]]
Background
When you have too many big requests for one Tomcat server to look after the compute for all of them, you can set up an Apache http server ("Apache httpd") to act as a switchboard for all incoming requests, farming them on to a whole series of Tomcats to do the real work. It's rather like having a receptionist at the hotel - we only have one on duty, but he / she has plenty of capacity to check the customers in, and deal with any follow up inquiries during their stay. The receptionist leaves them to get on in their rooms with watching the large TVs, having a relaxing shower, working, sleeping ... and wouldn't dream of actually doing those other things with them, nor would he/she have the time to do so.
When a new customer checks in, a 'random' available room can be allocated from the appropriate pool. But when follow up requests are made by a customer, they have to be applied to the right room. Jo Smith in room 210 will love a knock on his door at 05:00 to tell him that the taxi for Melinda and Robert Brown in room 218 has arrived!
And so it is with httpd - initial requests can be passed to any appropriate system, but then follow up requests which are part of the same sequence must be forwarded to the same machine (((Possible exception - clustering - see
following article))).
Using mod_proxy_balancer without maintaining state
Here's an example where requests to the url
http://www.melkshamhotel.co.uk/manor/
will be passed on to a balancer called "robin", which will share out the traffic between two machines on its local subnet, running the web application there that's called "latmjdemo". If neither of these machines has a running Tomcat on it, then a third machine has been set up as a hot standby to take the traffic.
ProxyPass /manor/ balancer://robin/
# Two servers, sharing traffic (2 to 1 ratio)
# Third server is only used when neither of the others is available
<Proxy balancer://robin>
BalancerMember http://192.168.200.218:8880/latmjdemo loadfactor=10
BalancerMember http://192.168.200.210:8880/latmjdemo loadfactor=5
BalancerMember http://192.168.200.219:8880/latmjdemo status=+H
</Proxy>
That code works well, except that "state is not maintained". In hotel terms, our receptionist will provide it to a random room when asked for a follow up service. The scheme works well for 'single shot' requests, but fails where there's a whole series of pages involved such as in an online ordering system, where cookies will be involve. We need to tell our receptionist that a request isn't a new one -
it's for the folks in room 218!
Maintaining state with mod_proxy_balancer
Your Java Servlet application that's running on Tomcat will almost certainly be using the built in session handling capacities within the code - we have examples
here and
here. Such applications will serve a cookie - in hotel terms, a room card - which the customer presents to the receptionist at each subsequent service request.
Then:
a) Each individual Tomcat needs to have a unique "jvmroute" set - there will be a line in the server.xml file that needs to be configure along the lines of:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm6">
b) The mod_proxy_balancer needs to be told which cookie is the one that's used for the sticky session, and which jvmroute has been used on which server in the balanced group. Here's a configuration sample:
ProxyPass /manor/ balancer://rednose/ stickysession=JSESSIONID|jsessionid
# Ensure that cookies get rewritten on their way back
ProxyPassReverseCookiePath /latmjdemo /manor
<Proxy balancer://rednose>
BalancerMember http://192.168.200.218:8880/latmjdemo/servlet loadfactor=10 route=jvm6
BalancerMember http://192.168.200.210:8880/latmjdemo/servlet loadfactor=10 route=jvm7
BalancerMember http://192.168.200.219:8880/latmjdemo/servlet loadfactor=10 route=jvm8
</Proxy>
These are working, tested examples. The Apache httpd mod_proxy_balancer manual is somewhat unclear with regard to the need for the route to be set (at both Tomcat and in the balancer) and indeed it could be (mis)read to imply that it isn't necessary to set the route - as id it has some large table of all the sessionids so that it knows which to forward to where. If you try to run balanced sticky sessions without the route, you'll fail - you'll be passed to what appears to be a random Tomcat each time, even if you have "stickysession" set in your ProxyPass.
Other Notes
1. With PHP, your session cookie does NOT by default include a "jvmroute" and you'll need to find a mechanism to add one. Mark Round has blogged about this
here - a story that's beyond this Java based post, but we can help delegates work this out on our
LAMP deployment course or on a one on one day if you already know the rest of the LAMP stuff!
2. You should ensure that you have code in your proxy configuration that prevents you being used as an open proxy ...
3. Under normal circumstances, sticky sessions are very hard indeed to test as you're looking to get virtually identical resources from each machine in the group. We have a test servlet
here which reports on the server being hit by name, to help in debugging / tracking your session. (This same example will be used in my
follow up article on clustering)
4. The balancer manager (part of mod_proxy_balancer) is a further useful monitoring tool. Turn it on as follows in the httpd configuration:
<Location /watch>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 192.168.200.
<Location>
5. There is a file containing various proxy configuration examples
here, which we pulled in to our main httpd.conf via an include. You'll need to choose the bits you want - there are various conflicting proxies in there that will default to issuing warnings.
(written 2009-10-29, updated 2009-10-30)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
A690 - Web Application Deployment - Clustering and load balancing [934] Clustering, load balancing, mod_rewrite and mod_proxy - (2006-11-21)
[1121] Sharing the load with Apache httpd and perhaps Tomcat - (2007-03-29)
[1771] More HowTo diagrams - MySQL, Tomcat and Java - (2008-08-24)
[1993] Load Balancing - Hardware or Software? - (2009-01-15)
[1995] Automated server heartbeat and health check - (2009-01-16)
[2059] Sharing the load between servers - httpd and Tomcat - (2009-02-28)
[2483] Clustering on Tomcat - (2009-10-30)
[3293] Distributing the server load - yet ensuring that each user return to the same system (Apache httpd and Tomcat) - (2011-05-18)
[3339] Simplest ever proxy configuration? - (2011-06-28)
[3892] Distributed, Balanced and Clustered Load Sharing - the difference - (2012-10-13)
[4432] Java web application for teaching - now with sessions and clustering / load balancing demonstrations - (2015-02-20)
A655 - Web Application Deployment - Using Tomcat and Apache httpd Together [436] Linking Apache httpd to Apache Tomcat - (2005-09-05)
[576] Why run two different web servers - (2006-01-25)
[631] Apache httpd to Tomcat - jk v proxy - (2006-03-03)
[1376] Choosing between mod_proxy and mod_rewrite - (2007-10-02)
[1383] Monitoring mod_jk and how it is load balancing - (2007-10-07)
[1549] http, https and ajp - comparison and choice - (2008-02-22)
[1552] Extra public classes in deploying Apache httpd and Tomcat - (2008-02-24)
[1938] Predictive Load Balancing - PHP and / or Java - (2008-12-13)
[1940] URL rewriting with front and back servers - (2008-12-14)
[1944] Forwarding session and cookie requests from httpd to Tomcat - (2008-12-14)
[3018] Tuning Apache httpd and Tomcat to work well together - (2010-10-27)
[3999] Handling failures / absences of your backend server nicely - (2013-02-08)
A608 - Web Application Deployment - Apache httpd mod_proxy [1006] Apache httpd and Apache Tomcat together tips - (2006-12-24)
[1566] Strange behaviour of web directory requests without a trailing slash - (2008-03-06)
[1767] mod_proxy and mod_proxy_ajp - httpd - (2008-08-22)
[1939] mod_proxy_ajp and mod_proxy_balancer examples - (2008-12-13)
[2062] Virtual hosting and mod_proxy forwarding of different domains (httpd) - (2009-03-01)
[2273] Three recent questions on Tomcat Convertors - (2009-07-07)
[2325] Apache, Tomcat, mod_proxy - (2009-08-06)
[3680] How can I run multiple web servers behind a single IP address? - (2012-04-02)
Some other Articles
Santa Special - Trowbridge and Melksham to SwindonHow do I set up a constant in Python?Finding text and what surrounds it - contextual grepLoad balancing with sticky sessions (httpd / Tomcat)Sample code with errors in it on our web sitePantomimes around Melksham - 2009/2010 seasonAccidentally typed ci rather than vi?How did I do THAT?By train ...