Data Freshness and Expiration
Databases and other systems of record (SORs) that were not built to accommodate caching outside of the database do not normally come with any default mechanism for notifying external processes when data has been updated or modified.
When using BigMemory Go as a caching system, the following strategies can help to keep the data in the cache in sync:
Data Expiration Use the eviction algorithms included with Ehcache, along with the
timeToIdleSeconds and
timetoLiveSeconds settings, to enforce a maximum time for elements to live in the cache (forcing a re-load from the database or SOR).
Message Bus: Use an application to make all updates to the database. When updates are made, post a message onto a message queue with a key to the item that was updated. All application instances can subscribe to the message bus and receive messages about data that is updated, and can synchronize their local copy of the data accordingly (for example by invalidating the cache entry for updated data)
Triggers: Using a database trigger can accomplish a similar task as the message bus approach. Use the database trigger to execute code that can publish a message to a message bus. The advantage to this approach is that updates to the database do not have to be made only through a special application. The downside is that not all database triggers support full execution environments and it is often inadvisable to execute heavy-weight processing such as publishing messages on a queue during a database trigger.
The Data Expiration method is the simplest and most straightforward. It gives you the most control over the data synchronization, and doesn't require cooperation from any external systems. You simply set a data expiration policy and let Ehcache expire data from the cache, thus allowing fresh reads to re-populate and re-synchronize the cache.
If you choose the Data Expiration method, you can read more about the cache configuration settings in "Cache Eviction Algorithms" in the Developer Guide for BigMemory Go and review the timeToIdle and timeToLive configuration settings in "Setting Expiration." The most important consideration when using the expiration method is balancing data freshness with database load. The shorter you make the expiration settings - meaning the more "fresh" you try to make the data - the more load you will place on the database.
Try out some numbers and see what kind of load your application generates. Even modestly short values such as five or ten minutes will produce significant load reductions.