An excellent article about scalability of Wang Yu based on his years of working in an internal laboratory. In these years, he has helped testing dozens of Java application in variety of different solutions. You can read full article here, the first part of this article will discuss scaling Java application vertically (scale up). Following is my quotes of some interesting points:
The key for the difference lies in the vision of the architect when designing the products. All these scaled-well Java applications were well prepared for the scalability, from the requirement collection phase, system design phase to the implementation phase of the products’ life cycle. Your Java application scalability is really based on your vision.
Scalability is a desirable property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner, or to be readily enlarged. For example, it can refer to the capability of a system to increase total throughput under an increased load when resources (typically hardware) are added.
1. Hot lock is the key enemy of scalability
The synchronized keyword will force the scheduler to serialize operations on the synchronized block. If many threads compete for the contended synchronizations, and only one thread is executing a synchronized block, then any other threads waiting to enter that block are stalled. If no other threads are available for execution, then processors may sit idle. In such situations, more CPUs can help little on performance.
To avoid the hot lock problem, following suggestions may be helpful:
- Make synchronized blocks as short as possible
- Reducing lock granularity
- Avoid lock on static methods
- Using lock free data structure in Java SE 5.0
Fortunately, in Java SE 5.0 and above, you can write wait-free, lock-free algorithms under the help with hardware synchronization primitives without using native code. Almost all modern processors have instructions for updating shared variables in a way that can either detect or prevent concurrent access from other processors. These instructions are called compare-and-swap, or CAS.
One successful story about the lock free algorithms is a financial system tested in our laboratory, after replaced the “Vector” data structure with “ConcurrentHashMap”, the performance in our CMT machine(8 cores) increased more than 3 times.
2. Race condition can also cause the scalability problems
Too many “synchronized” keywords will cause the scalability problems. But in some special cases, lack of “synchronized” can also cause the system fail to scale vertically. The lack of “synchronized” can cause race conditions, allowing more than two threads to modify the shared resources at the same time, and may corrupt some shared data.
3. Non-Blocking IO vs. Blocking IO
The large number of Java threads will cause the JVM and OS busy with handling scheduling and maintenance work of these threads, instead of processing business logic. More over, more threads will consume more JVM heap memory (each thread stack will occupy some memory), and will cause more frequent garbage collection.
4. Single thread task problem
Parallelization is the solution. To parallelize the single thread task, you must find a certain level of independence in the order of operations, then use multiple threads to achieve the parallelization. In this case, the customer had refined their “annual report generation” task to generate monthly reports first, then generate the annual report based on those 12 monthly reports. “Monthly reports” are just transition results, since such reports are useful for the end users. But “monthly reports” can be generated concurrently and will be used to generate the final report quickly. In this way, this scenario was scaled to 4-CPU SPARC servers very well, and exceeded the AMD server more than 80% on performance.
5. Scale Up to More Memory
Java garbage collection relieves programmers from the burden of freeing allocated memory, in doing so making programmers more productive. The disadvantage of a garbage-collected heap is that it will halt almost all working threads when garbage is collecting. In addition, programmers in a garbage-collected environment have less control over the scheduling of CPU time devoted to freeing objects that are no longer needed. For those near-real-time applications, such as Telco systems and stock trade systems, this kind of delay and less controllable behavior are big risks.
In other memory-centric Java applications tested in our laboratory, we found the following characteristics:
1. Every request processing action needed big and complex objects
2. It kept too many objects into HttpSession for every session.
3. The HttpSession timeout was too long, and HttpSession was not explicitly invalidated.
4. The thread pool, EJB pool or other objects pool was set too large.
5. The objects cache was set too large.
Those kinds of applications don’t scale well. When the number of concurrent users increasing, the memory usage of those applications increases largely. If large numbers of live object cannot be recycled in time, the JVM will spend considerable time on garbage collection. On the other hand, if given too much memory (in a 64-bit JVM), the JVM will still spend considerable time on garbage collection after running for a relatively long time.
The conclusion is that Java applications are NOT scalable by given too much memory. In most cases, 3GB memory assigned to Java heap (through “-Xmx” option) is enough (in some operating systems, such as Windows and Linux, you may not be able to use more than 2G memory in a 32-bit JVM). If you have more memory than the JVM can use (memory is cheap these days), please give the memory to the other applications within the same system, or just leave it to the operating system. Most OSs will use spare memory as a data buffer and cache list to improve IO performance.
6. Other Scale Up Problems
Some scalability problems in Java EE applications are not related to themselves. The limitation from external systems sometime will become the bottleneck of scalability. Such bottlenecks may include:
- Database management system: This is the most common bottleneck for most of enterprise and Web 2.0 applications, for the database is normally shared by the JVM threads. So effectiveness of database access, and the isolation levels between database transactions will affect the scalability significantly. We have seen a lot of projects where most of the business logic resides in the database in terms of stored procedures, while keeping the Web tier very lightweight just to perform simple data filtering actions and process the stored procedures in database. This architecture is causing a lot of issues with respect to scalability as the number of requests grow.
- Disk IO and Network IO
- Operating System: Sometimes the scalability bottleneck may lie in the limitation of the operating system. For example, putting too many files under the same directory can cause file systems to slow when creating and finding a file.
- Synchronous logging: This is a common problem about scalability. In some of the cases, the problem was solved by using a logging server such as Apache log4j. Others have used JMS messages to convert synchronous logging to asynchronous one.
Source: http://www.theserverside.com/tt/articles/article.tss?l=ScalingYourJavaEEApplications
