With the advent of cloud computing enterprises have started reaching out to the masses by deploying server applications in cloud. The age old server application platforms cater to scaling for an enterprise scale, where the number of users using the system were in the 10,000’s. Recent trends show that product companies have started realizing the advantages of cloud and have Saas offerings that attract hundreds of customers. Even if each customer has a few thousands of customers, it puts the required scaling to be supported by the Saas provider at millions of users. With this shift in number of expected users for a system, it is high time we started questioning the way applications are scaled. Is it correct for us to think of scaling as an after thought after the application has been written or should be design for it as a part of the application design?
Anyone who has worked in a enterprise world, has clustered servers at one point or the other and has experienced the pain of clustering. Clustering is typically done for “load balancing”. “Load balancing” balances the user requests across multiple servers hosting the same application. The key questions we have to ask here are
- “Should only requests be load balanced”?
- “Is request processing so monolithic that it has to be processed by a single server or can it be distributed across servers”?
We find that typical requests in a complex server application involves multiple functions being run before the response is sent back. Can the functions that are running to process a request be load balanced? This is the concept of EJB load balancing in J2EE servers. But this involves setting up the correct cluster. Can this be done by programming correctly rather than thinking about it as a an after thought?
Newer technologies such as hadoop, memcache etc have introduced a concept of scale out rather than scale up. While clustering can also be thought of as scale out, does it really implement the actual “scale out” concept as hadoop does? Can something different be done where applications can also be scaled out without the complexity of a cluster setup?
One possible solution for this can be the use of a distributed JVM which attempts to distribute processing and heap across multiple JVM instances. For eg., JVMs such as JESSICA implements such technology. But these JVMs are still in research and are not available for commercial use. Also, typically these JVMs distribute irrespective of the work submitted. A server platform does heterogeneous work, where processing a request is not the only work done. It also does clean up, synchronization etc. It would be good for the program written to specify if the functionality can be distributed or not. We need a JVM that can support both a single JVM operation and a distributed JVM operation.
Can a library be written over the existing JVMs that distribute work based on how it is coded? Can we have a library to which what has to be distributed is specified while how it is distributed is automatically handled by the library?