Settings
A cluster topology is defined and behaves according to the configuration of the cluster, its stripes, and their constituent nodes. The configuration of each of these entities is established by ascribing values to one or more of their supported settings.
Each setting has the potential to be queried (via the get command), modified (via the set command) and in some cases 'removed' or 'undone' (via the unset command). Importation of settings (via the import command) is another avenue to configure an entity.
Each setting possesses a unique set of rules governing how it can be changed. These rules depend upon the setting's applicable scope (i.e. cluster, stripe, or node) and whether the entity is CONFIGURED (i.e. Activated) or UNCONFIGURED.
Configuration Rules for UNCONFIGURED (not Activated) Nodes and Clusters
An unconfigured cluster offers the greatest flexibility where modifying settings is concerned. This is because the settings of UNCONFIGURED nodes can always be modified and changing them will never require a restart. However, once a node is activated, certain settings can not be modified and in some cases, modifying a node's settings will require the node to be restarted.
The following table shows which Config Tool commands can be executed against each supported setting when the node is UNCONFIGURED (i.e. not Activated).
Setting | Node | Stripe | Cluster | Default |
license-file | | | set unset | - |
cluster-name | | | get set unset import | - |
client-reconnect-window | | | get set unset import | 120 seconds |
client-lease-duration | | | get set unset import | 150 seconds |
failover-priority | | | get set unset import | - |
stripe-name | | get set import | get | <generated> |
name | get set import | get | get | <generated> |
hostname | get import | get | get | %h |
port | get import | get | get | 9410 |
public-hostname | get set unset import | get set unset | get set unset | - |
public-port | get set unset import | get set unset | get set unset | - |
group-port | get set unset import | get set unset | get set unset | 9430 |
bind-address | get set unset import | get set unset | get set unset | 0.0.0.0 |
group-bind-address | get set unset import | get set unset | get set unset | 0.0.0.0 |
data-dirs | get set unset import | get set unset | get set unset | main:%H/terracotta/user-data/main |
metadata-dir | get set unset import | get set unset | get set unset | %H/terracotta/metadata |
log-dir | get set unset import | get set unset | get set unset | %H/terracotta/logs |
backup-dir | get set unset import | get set unset | get set unset | - |
tc-properties | get set unset import | get set unset | get set unset | - |
logger-overrides | get set unset import | get set unset | get set unset | - |
security-dir | get set unset import | get set unset | get set unset | - |
audit-log-dir | get set unset import | get set unset | get set unset | - |
authc | | | get set unset import | - |
ssl-tls | | | get set unset import | FALSE |
whitelist | | | get set unset import | FALSE |
offheap-resources | | | get set unset import | main:512MB |
config-dir | | | | - |
node-uid | get import | get | get | <generated> |
stripe-uid | | get import | get | <generated> |
cluster-uid | | | get import | <generated> |
lock-context | | | import | - |
Configuration Rules for CONFIGURED (Activated) Nodes and Clusters
Once a cluster is activated and its underlying nodes are CONFIGURED, there is less flexibility as to which settings can be modified including whether the requested setting modification can be incorporated at runtime or if one or more Terracotta server restarts will be required.
The following table shows which Config Tool commands can be executed against each supported setting when the node is CONFIGURED (i.e. Activated).
Requirements (see Table below)
CO - Cluster Online - All nodes must be online in order for the
set or
unset command to be accepted.
CR - Cluster Restart - Every node in the cluster must be restarted after executing a
set or
unset command.
NR - Node Restart - Only the node(s) directly impacted by the
set and
unset command must be restarted. The Config Tool will list these impacted nodes following successful execution of the command, asking for them to be restarted.
AoN - All or None - All nodes in the cluster, or zero nodes in the cluster, can possess a configured value for the specified setting.
Note:
When executing set or unset against a setting which does not possess any requirements (CO, CR, NR or AoN) the setting may be automatically incorporated at runtime, without requiring a server restart. However, the system may determine that a restart of one or more nodes is required. Following successful execution of the command, the system will list the node(s), if any, that will require a restart.
Note:
Whenever the set or unset command is executed, the Active server for each stripe comprising the cluster must be online in order for the command to be accepted. That is, no setting can be modified unless every Active server in the cluster is online. This rule applies to all settings.
Setting | Node | Stripe | Cluster | Requirements |
license-file | | | set unset | |
cluster-name | | | get set | |
client-reconnect-window | | | get set unset | |
client-lease-duration | | | get set unset | |
failover-priority | | | get set | CO, CR |
stripe-name | | get | get | |
name | get | get | get | |
hostname | get | get | get | |
port | get | get | get | |
public-hostname | get set unset | get set unset | get set unset | AoN |
public-port | get set unset | get set unset | get set unset | AoN |
group-port | get | get | get | |
bind-address | get | get | get | |
group-bind-address | get | get | get | |
data-dirs | get set | get set | get set | AoN |
metadata-dir | get set | get set | get set | |
log-dir | get set unset | get set unset | get set unset | NR |
backup-dir | get set unset | get set unset | get set unset | AoN |
tc-properties | get set unset | get set unset | get set unset | NR |
logger-overrides | get set unset | get set unset | get set unset | |
security-dir | get set unset | get set unset | get set unset | NR, AoN |
audit-log-dir | get set unset | get set unset | get set unset | NR, AoN |
authc | | | get set unset | CO, CR |
ssl-tls | | | get set unset | CO, CR |
whitelist | | | get set unset | CO, CR |
offheap-resources | | | get set | |
config-dir | | | | |
node-uid* | get | get | get | |
stripe-uid* | | get | get | |
cluster-uid* | | | get | |
lock-context* | | | set unset | |
*Internally used system property
Config Tool Commands
The following sections describe each supported Config Tool
command and provide examples on their usage.
In some cases, we also show examples of
start_tc_server.sh|bat script usage. Refer to
Starting and Stopping the Terracotta Server for complete details on using this script.
Note:
Any command requiring a hostname:port parameter can be substituted with the ipAddress:port of the target node.
Cluster Activation and Topology Changes
Activate
The activate command is used to activate the nodes of a cluster when the nodes are UNCONFIGURED. The activation process will automatically perform the following steps:
1. Validate the configuration consistency
2. Validate the license
3. Write the validated cluster configuration inside the configuration directory of all nodes
4. Restart all activated nodes
Once a cluster is activated, it becomes usable for Terracotta clients.
Syntax:
activate [-connect-to <hostname[:port]> [-config-file <file>]] [-cluster-name <name>] [-license-file <file>] [-restart-wait-time <duration>] [-restart-delay <duration>] [-restrict]
Option | Description |
-connect-to <hostname[:port]> | Terracotta server instance to which the Config Tool will connect and execute the command. |
-cluster-name <name> | The name given to the activated cluster. |
-config-file <file> | Configuration file containing the node definitions, including their stripe topology, to be activated. |
-license-file <file> | File path to the license file. |
-restart-wait-time <duration> | Maximum time to wait for the nodes to restart. Default: 120 seconds. |
-restart-delay <duration> | Delay before the server restarts itself. Default: 2 seconds. |
-restrict | Restricts the activation process to only the -connect-to node. |
Examples:
1. Activating an unconfigured cluster whose topology has already been defined (e.g. through a series of previously executed node/stripe/cluster attach commands):
config-tool.sh activate -connect-to host1:9410 -cluster-name MyCluster -license-file /path/to/license.xml
In this scenario, each of the nodes must be online and running in DIAGNOSTIC mode in order for the activation to proceed.
Note that the connect-to option cannot be used as the node connection details used by the Config Tool are defined inside the configuration file.
This flag shouldn't be used as a general way to activate a cluster. To work, this flag requires that the targeted node is unconfigured and has a topology that is the same as the cluster it needs to join, and other nodes also need to know about this node.
This is an action that can help recover from a broken configuration state, or when an `attach` command has failed. Examples of scenario where this command can be useful:
Repair a node when normal activation has failed
Repair an `attach` command that did not complete (topology was updated in existing nodes, but new nodes failed to be activated)
Re-activate a node after it has been reset
attach new nodes to a cluster where some nodes have already been started with `-auto-activate`
config-tool.sh activate -license-file /path/to/license.xml -config-file cluster.cfg
3. Activating an unactivated node belonging to an already activated cluster.
In this scenario, node7 has entered an UNCONFIGURED state but is still part of the cluster and thus needs to be re-activated. Restricted activation will only run the activation process on the targeted node.
config-tool.sh activate -connect-to host7:9417 -config-file cluster.cfg -restrict
Attach
The attach command is used to add nodes to a cluster by either (1) adding nodes to a stripe or (2) adding whole stripes and their child nodes to the cluster.
Syntax:
attach (-to-cluster <hostname[:port]> -stripe <hostname[:port]> | -to-stripe <hostname[:port]> -node <hostname[:port]>) [-restart-wait-time <duration>] [-restart-delay <duration>]
Option | Description |
-stripe <hostname[:port] | Stripe to add; identified by any node (as hostname:port ) belonging to stripe. |
-to-cluster <hostname[:port] | Destination cluster; identified by any node (as hostname:port ) belonging to any other stripe within the cluster. |
-node <hostname[:port]> | Node to add; identified as hostname:port. |
-to-stripe <hostname[:port]> | Destination stripe; identified by any node (as hostname:port ) belonging to to-stripe. |
-restart-wait-time <duration> | Maximum time to wait for the nodes to restart. Default: 120 seconds. |
-restart-delay <duration> | Delay before the server restarts itself. Default: 2 seconds. |
Note:
In the event that errors are reported by the Config Tool when executing the
attach command, note the error messages and refer to the
Troubleshooting Guide for guidance.
Examples:
1. Attaching two UNCONFIGURED nodes to form a single stripe:
Start node3 as UNCONFIGURED:
start-tc-server.sh -failover-priority=availability -name=node3 -hostname=host3 -port=9413 -config-dir=/path/to/node3/repository
Start node4 as UNCONFIGURED:
start-tc-server.sh -failover-priority=availability -name=node4 -hostname=host4 -port=9414 -config-dir=/path/to/node4/repository
Attach node4 to node3 to form a stripe with two nodes:
config-tool.sh attach -to-stripe host3:9413 -node host4:9414
2. Attaching a node to an existing stripe:
Start node5 as UNCONFIGURED:
start-tc-server.sh -failover-priority=availability -name=node5 -hostname=host5 -port=9415 -config-dir=/path/to/node5/repository
Attach node5 to the stripe containing nodes node3 and node4:
config-tool.sh attach -to-stripe host3:9413 -node host5:9415
3. Attaching a stripe to a cluster:
In the previous step we created a 3-node stripe (
Stripe-B in our
example cluster). Assuming that
Stripe-A has already been constructed, we can attach
Stripe-B to the cluster by specifying any node belonging to
Stripe-B (
node3 in this case) and any other node belonging to another stripe in the cluster (
node1 in
Stripe-A in this case):
config-tool.sh attach -to-cluster host1:9410 -stripe host3:9413
Similarly, Stripe-C can be attached to the cluster. Here we pick node7 from Stripe-C and node3 from Stripe-B:
config-tool.sh attach -to-cluster host3:9413 -stripe host7:9417
Detach
The detach command is used to remove nodes from a cluster by either (1) removing nodes from their parent stripes or (2) removing whole stripes and their child nodes from the cluster.
Syntax:
detach (-from-cluster <hostname[:port]> -stripe <hostname[:port]> | -from-stripe <hostname[:port]> -node <hostname[:port]>) [-stop-wait-time <duration>] [-stop-delay <duration>]
Option | Description |
-stripe <hostname[:port] | Stripe to remove; identified by any node (as hostname:port) belonging to stripe. |
-from-cluster <hostname[:port] | Destination cluster; identified by any node (as hostname:port) belonging to any other stripe within the cluster. |
-node <hostname[:port]> | Node to remove; identified as hostname:port. |
-from-stripe <hostname[:port]> | Destination stripe; identified by any node (as hostname:port) belonging to from-stripe. |
-stop-wait-time <duration> | Maximum time to wait for the nodes to restart. Default: 120 seconds. |
-stop-delay <duration> | Delay before the server restarts itself. Default: 2 seconds. |
Note:
In the event that errors are reported by the Config Tool when executing the
detach command, note the error messages and refer to the
Troubleshooting Guide for guidance.
Examples:
1. Detach a node from a stripe. In our
example cluster we could detach
node4 from
Stripe-B with:
config-tool.sh detach -from-stripe host3:9413 -node host4:9414
2. Detach a stripe from a cluster. Specify any node (active or passive) within the cluster (from-cluster) that does not belong to the stripe we wish to detach, and specify any node (active or passive) belonging to the stripe we wish to detach. Here we detach Stripe-C from Stripe-A:
config-tool.sh detach -from-cluster host2:9412 -stripe host7:9417
Note:
When detaching a stripe, all nodes in the detached stripe are stopped.