Will an Application Benefit from Caching?
Often the answer is yes, especially if the application is I/O bound. If an application is I/O bound, it depends on the rate at which data can be obtained. If it is CPU bound, then the time taken principally depends on the speed of the CPU and main memory. Caching can improve performance and also reduce the load on a web server.
Speeding up CPU-bound Applications
CPU bound applications are often sped up by:
Improving algorithm performance
Parallelizing the computations across multiple CPUs (SMP) or multiple machines (clusters).
Upgrading the CPU speed.
A cache can temporarily store computations for reuse, including but not limited to:
Large web pages that have a high rendering cost
Authentication status, where authentication requires cryptographic transforms
Speeding up I/O-bound Applications
Many applications are I/O bound, either by disk or network operations. In the case of databases they can be limited by both.
Network operations can be bound by a number of factors:
Time to set up and tear down connections
Latency, or the minimum round trip time
Throughput limits
Overhead for marshalling and unmarshalling
The caching of data can often help significantly with I/O bound applications. For example, you might use BigMemory Maxfor:
Data Access Object caching for Hibernate
Web page caching, for pages generated from databases
Increased Application Scalability
The corollary to increased performance is increased scalability. Suppose you have a database that can perform up to 100 expensive queries per second. Beyond that threshold, the database backs up and if addition connections occur, the database slowly dies.
In this case, caching is likely to reduce the workload. If caching can cause 90% of those 100 queries to be cache hits and not impact the database, the database can scale 10 times higher.