General Steps for Setting Up WAN Replication
The following steps provide an overview of setting up BigMemory WAN Replication:
1. Ensure that the Terracotta clusters that will use WAN replication can run as expected without WAN replication.
See the BigMemory Max Installation Guide and the BigMemory Max Administrator Guide for more information on installing a cluster.
After verifying the Terracotta clusters, shut down all processes in order to set up for WAN replication.
2. Enable WAN replication for all caches that will be replicated across your WAN.
For each cache to be replicated, its ehcache.xml configuration file must include the wanEnabledTSA attribute set to "true" within the <terracottaConfig> element:
<terracottaConfig wanEnabledTSA="true"/>
Note: Caches that will share data must have the same name.
3. Configure at least one Orchestrator for each region.
Each Orchestrator requires a dedicated wan-config.xml configuration file. The wan-config.xml specifies the caches that will be Masters and failover Masters by listing the locations of the Master caches. All of the Masters listed for a cache must be in the same region. (Replica caches do not need to be specified in the Orchestrator configuration file, as they will register with their Master caches upon startup.)
Following is a sample wan-config.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<wan-config xmlns="http://config.wan.terracottatech.com">
<bind host="0.0.0.0" port="9003"/>
<logs>/path/to/mylogs</logs>
<maxConnectionRetryCount>8</maxConnectionRetryCount>
<replicatorIntervalMillis>125</replicatorIntervalMillis>
<replicationMode>unidirectional</replicationMode>
<replicaDisconnectBehavior>reconnectResync</replicaDisconnectBehavior>
<cacheManager ehcacheConfigURI="file:///path/to/ehcache-1.xml">
<cache name="test-cache-1">
<master host="masterhost-A" port="9001"/>
<master host="masterhost-B" port="9002"/>
</cache>
<cache name="test-cache-2">
<master host="masterhost-A" port="9001"/>
</cache>
</cacheManager>
<cacheManager ehcacheConfigURI="file:///path/to/ehcache-2.xml">
<cache name="test-cache-3">
<master host="masterhost-B" port="9002"/>
</cache>
</cacheManager>
// Optional for version 4.1.3 and above--see step 5 below
<userClassDirectory>/temp/user_lib</userClassDirectory>
</wan-config>
Note: The ehcache.xml referenced in an Orchestrator's wan-config.xml must include the location of the tc-config.xml for that Orchestrator's region. The location is specified in the terracottaConfig element of the ehcache.xml. For more information, refer to "Defining a Distributed Configuration" in the BigMemory Max Configuration Guide.
4. Start the Terracotta Server Array in each region.
5. (Optional) If you have any non-JDK value types in your caches, and/or if you are using custom attribute extractors for Search, you will need to add these to the classpath of the Orchestrator process.
a. Place any jars containing custom attribute extractors or custom user classes in a directory.
b. For versions 4.1.0 through 4.1.2 only: Define the environment variable WAN_USER_LIB to the absolute path of the directory. For example:
export WAN_USER_LIB=/temp/user_lib
The WAN shell script will then add all .JAR files under the /temp/user_lib directory to the Orchestrator's classpath.
For version 4.1.3 and above only: Starting with 4.1.3, the WAN_USER_LIB has been deprecated. The functionality previously provided by WAN_USER_LIB is now handled by the userClassDirectory tag in each Orchestrator's wan-config.xml file. Following is a sample userClassDirectory tag in a wan-config.xml file:
<userClassDirectory>/temp/user_lib</userClassDirectory>
The WAN shell script will then add all .JAR files under the /temp/user_lib directory.
6. Use the start-wan script to start the Orchestrators.
To specify the configuration file location, use -f command-line option, for example:
<bigmemory_kit>/server/bin/start-wan.sh -f /path/to/wan-config.xml
7. Start your application. You must start the instances with the Master caches first, and then start those with Replica caches.
Upon starting the app servers, your clusters should begin replicating data across the WAN.