The TerracottaConfiguration File
This document describes the elements of the
Terracotta configuration file (a.k.a.
config file), which serves to define the configuration of a Terracotta cluster including its stripes and nodes. Refer to
Cluster Architecture for details on different TSA topologies.
A sample config file is provided in the kit under server/conf. It can be used as the starting point. Some entries in the file have inline comments describing the configuration elements. Be sure to start with a clean file for your configuration.
Use cases
Unlike previous Terracotta releases, a config file isn't needed to configure or start the servers. However, a config file is handy for the following purposes:
1.
Export and viewing the configuration of a given cluster.
2. Backing up and version-controlling the configuration of a given cluster.
3. Using the current configuration as a foundation to build up a new configuration.
4.
Import the cluster configuration on running nodes of an unconfigured cluster.
The config file is only used as a convenience to feed the configuration settings into unconfigured servers. Once that is performed, the servers will never again utilize or reference the config file, but will instead utilize their internal configuration (which is persisted in the server’s config-dir).
Configuration File Format
To help illustrate the format of the config file, consider a cluster that comprises two stripes and four nodes. Nodes node1 and node2 belong to Stripe-A. Nodes node3 and node4 belong to Stripe-B.
The configuration file format requires that all stripes defined for the cluster are declared with the stripe-names property:
stripe-names:<stripe_name_1>,<stripe_name_2>,...,<stripe_name_n>
The declaration of stripes for our example is:
stripe-names:Stripe-A,Stripe-B
The configuration file format also requires that all nodes belonging to a stripe are declared with the node-names property:
<stripe_name_1>:node-names:<node_name_1-1>,<node_name_1-2>,...,<node_name_1-n>
<stripe_name_2>:node-names:<node_name_2-1>,<node_name_2-2>,...,<node_name_2-n>
...
<stripe_name_m>:node-names:<node_name_m-1>,<node_name_m-2>,...,<node_name_m-n>
The declaration of nodes for our example is:
Stripe-A:node-names:node1,node2
Stripe-B:node-names:node3,node4
With the node and stripe names declared, settings are defined at cluster, stripe or node level by referencing the appropriate stripe name, node name or no name (in the case of cluster level setting):
1. Node-level scope:
<node_name>:<setting> OR node:<node_name>:<setting>
2. Stripe-level scope:
<stripe_name>:<setting> OR stripe:<stripe_name>:<setting>
3. Cluster-level scope:
<setting>
4. When the same setting appears multiple times in the config file and that setting is directly or indirectly ascribed to the same node, lower scoped entries will take precedence over higher scoped entries when determining that setting's value for that particular node, where node-scope < stripe-scope < cluster-scope.
For example, consider this snippet:
backup-dir=pathA
node2:backup-dir=pathB
StripeB:backup-dir=pathC
The above snippet will effectively impart the following configuration to the cluster:
node1:backup-dir=pathA
node2:backup-dir=pathB
node3:backup-dir=pathC
node4:backup-dir=pathC
In the above example, the cluster-scoped entry backup-dir=pathA is applied to every node in the cluster. But the lower scoped entry node2:backup-dir=pathB overrides the cluster-scoped entry for node2 and assigns its backup directory as pathB. Similarly, the stripe-scoped entry StripeB:backup-dir=pathC overrides the backup directory which backup-dir=pathA would have ascribed to nodes node3 and node4, with the value pathA.
Examples
This section describes the various properties supported in a config file.
Minimal configuration
A config file can get pretty large, especially when the cluster contains a large number of nodes. However, most configuration properties have default values, because of which the config file can be reduced in size if the default values are acceptable. The only mandatory property for a single-node cluster is:
<node_name>:hostname=localhost
which specifies the hostname to localhost. The following default values are used:
Property | Default value | Comments |
offheap-resources | main:512MB | |
client-lease-duration | 150 seconds | |
client-reconnect-window | 120 seconds | Defines the client reconnect time window as 120 seconds. |
<node_name>:hostname | %h | Sets the host name of the node to the fully-qualified host name of the machine. |
<node_name>:port | 9410 | Sets the port for this server process to 9410. |
<node_name>:bind-address | 0.0.0.0 | Sets the bind address for the port to the wildcard address 0.0.0.0 |
<node_name>:group-port | 9430 | Sets the intra-stripe communication port for this server process to 9430. |
<node_name>:group-bind-address | 0.0.0.0 | Sets the bind address for the group port to the wildcard address 0.0.0.0 |
<node_name>:metadata-dir | %H/terracotta/metadata | |
<node_name>:data-dirs | main:%H/terracotta/user-data/main | |
<node_name>:log-dir | %H/terracotta/logs | Sets the server logging directory to %H/terracotta/logs |
Note:
%h and
%H in the above default values point to the hostname of the machine and home directory of the current user respectively. See the section
Parameter Substitution for more information.
Several other configuration properties are omitted (i.e. not assumed to have defaults), which are:
Property | Comments |
cluster-name | The name of the cluster |
whitelist | Whether to enable IP whitelist security |
ssl-tls | Whether to enable SSL/TLS based security |
authc | Security authentication setting to be used |
security-dir | The security root directory for this node |
audit-log-dir | Directory containing the node's security audit logs |
backup-dir | Directory to be used to contain the
backup of this node |
public-hostname | |
public-port | |
Security configuration
The following snippet demonstrates how to enable IP whitelisting and SSL/TLS based
security along with security event auditing on a single node cluster (with node named
node1):
)
failover-priority=availability
whitelist=true
ssl-tls=true
authc=certificate
node1:hostname=localhost
node1:audit-log-dir=/path/to/audit/dir
node1:security-dir=/path/to/security/dir
High availability configuration
High-availability can be enabled by configuring more than one node in a stripe. The following snippet defines two nodes in a cluster containing a single stripe:
failover-priority=availability
node1:hostname=localhost
node1:port=9410
node1:group-port=9430
node2:hostname=localhost
node2:port=9510
node2:group-port=9530
Multistripe with HA configuration
Stripes can be added to a Terracotta cluster to scale it out. Additionally, high-availability in a stripe can be enabled by configuring more than one node. The following snippet defines a cluster with two stripes with two nodes each:
failover-priority=availability
node1:hostname=localhost
node1:port=9410
node1:group-port=9430
node2:hostname=localhost
node2:port=9510
node2:group-port=9530
node3:hostname=localhost
node3:port=9610
node3:group-port=9630
node3:hostname=localhost
node3:port=9710
node3:group-port=9730
The preceding snippet can be simplified:
failover-priority=availability
hostname=localhost
node1:port=9410
node1:group-port=9430
node2:port=9510
node2:group-port=9530
node3:port=9610
node3:group-port=9630
node4:port=9710
node4:group-port=9730