Terracotta Ehcache 10.7 | Terracotta Server Administration Guide | SSL / TLS Security Configuration in Terracotta | Cluster Security
 
Cluster Security
Connection encryption
You can enable connection encryption using the ssl-tls property. When <ssl-tls> is specified, you must also supply:
* security-dir with the path to the security root directory.
* authc with the authentication scheme to use. See Authentication for more information.
With SSL/TLS configured, the specified security root directory must contain a valid truststore in the trusted-authority subdirectory and a valid keystore in the identity subdirectory. The identity certificate is required even if file-based or LDAP-based authentication is chosen. The choice of authentication scheme may require presence of additional files in the security root directory.
On the client, the security root directory must contain a trusted authority certificate in the trusted-authority subdirectory. Again, the choice of authentication scheme may mean other files, including an identity certificate, are also required in the security root directory.
Authentication
To configure authentication in the cluster, use authc with the appropriate authentication scheme. Supported authentication schemes are file, ldap, and certificate. When authentication is configured, a security-dir, with the path to the security root directory, must also be specified.
Important:
It is highly recommended that if you configure an authentication scheme, you also configure encrypted connections using the ssl-tls property, otherwise an attacker could acquire credentials by eavesdropping on the unencrypted connection.
Certificate-based authentication
To configure certificate-based authentication, use the certificate authentication scheme in the authc property, and set ssl-tls to true. Also, clients must have an appropriate identity certificate keystore in their security root directory.
File-based authentication
To configure file-based authentication, use the file authentication scheme in the authc property.
The server's security root directory must contain a users.xml file, which is a list of all valid users with a password hash for each user. The only password hashing algorithm currently supported is bcrypt.
Example of a users.xml file for authentication:
<users>
<user>
<username>alex</username>
<password>
<algorithm>bcrypt</algorithm>
<hash>$2a$10$UoM85/5I4SnIbrOQuFZ43ekffuQKSxZmL93bR9VMcdr2URmPyjyX2</hash>
</password>
</user>
<user>
<username>beth</username>
<password>
<algorithm>bcrypt</algorithm>
<hash>$2a$10$6D6c79lE0k/0SxrEtnfhGe2Yr.ygG0rFP1QzeyD9qshIMRrpUMOAS</hash>
</password>
</user>
</users>
Note:
Hashes should start with bcrypt version $2a$.
Generating bcrypt password hashes
There is a command line utility for generating bcrypt password hashes. The bcrypt script is located in tools/security/bin under the product installation directory as bcrypt.bat for Windows platforms, and as bcrypt.sh for Unix/Linux.
When running the bcrypt script, you must specify the number of rounds. The number to choose depends on both the performance of the servers on which authentication takes place and also how you decide to trade off security against speed. A higher number creates hashes that are harder for attackers to crack, but are also harder for your servers to verify. Increasing the number of rounds by 1 doubles the difficulty. You may find that a value between 10 and 13 is suitable.
The bcrypt script can read the password from the console:
bcrypt.sh -n 10
or directly from the command line:
bcrypt.sh -n 10 pa$$w0rd
The console form of the command is preferred because when a password is used on the command line, it can be written into files such as the command history and also readable using commands such as ps.
LDAP-based authentication
To configure LDAP-based authentication, use the ldap authentication scheme in the authc property.
The server's security root directory must contain an ldap.properties file. The properties in the ldap.properties file are the same properties that are used to configure LDAP integration in other Software AG products. See LDAP Properties for a full list of supported properties.
Example of an ldap.properties file for authentication:
url=ldap://ldapserver.example.com:389
userrootdn=ou=People,dc=example,dc=com
uidprop=uid
personobjclass=person
Auditing
To configure security event auditing, use the audit-log-dir property along with security-dir and at least one form of security (i.e. one of ssl-tls, authc, or whitelist). The directory specified in audit-log-dir must exist already, and must be appropriately access controlled to prevent illegitimate access to audit logs.
IP Whitelisting
Introduction
The IP whitelisting feature enables you as the cluster administrator to ensure that only clients from known IP addresses can access the TSA. You can use this feature to prevent malicious clients from establishing connections to the TSA. The term "clients" here refers to clients communicating with the Terracotta servers using the TSA wire protocol.
Note:
It should be understood that usage of this feature in itself does not provide a strong level of security for the TSA. The ideal way to enforce connection restrictions based on IP addresses would be to use host-level firewalls.
The whitelist file
A whitelist file is a plain-text file containing a list of IPs. Only clients running on these IPs are allowed to access the TSA. The server IPs specified in the config file, and the localhost IPs of the server are always whitelisted. An empty whitelist file has the semantics of blacklisting all the IPs, except the IPs fetched from the config file, and those corresponding to localhost.
The following rules need to be followed for a whitelist file to be considered valid, and the entries in it to be parsed properly:
1. The whitelist file must be named whitelist.txt and placed in the security root directory.
2. The entries can be IP addresses, or CIDR notations (to represent IP ranges). Any entry that is not a valid IP address or a valid CIDR is ignored.
3. Each line in the file can contain either a single IP address or a comma-separated list of IP addresses.
4. Lines beginning with # are considered as comments, and are ignored during parsing.
5. Blank lines are ignored.
The following is an example of a valid whitelist file:
# Whitelist for Terracotta cluster

# Caching clients
192.168.5.28, 192.168.5.29, 192.168.5.30
10.60.98.0/28

# Other clients
192.168.10.0/24
Usage
To configure IP whitelisting, use whitelist along with the security-dir property.
If the whitelist.txt file is not found in the security root directory, or there is an error reading the file, the server startup will fail with an appropriate error message.
If hostnames are used in the config file, the server attempts to resolve these hostnames to IPs. If the resolution fails, the server startup fails with an appropriate error message. Note that hostname resolution is done for the config file only, and any hostnames present inside the whitelist.txt file are ignored.
A multi-stripe cluster should be started with the same whitelist.txt file, or identical copies of the file. Similarly, when updates to the file are desired, they should be performed on all the stripes, as described in the following section.
Dynamic updates
After a cluster is started with whitelisting enabled, entries can be dynamically added to or removed from the whitelist without the need for server restarts. To perform a dynamic update, edit the whitelist.txt file contained in the server security root directories, and run the ipwhitelist-reload command to notify the servers in the cluster to reload the whitelist.txt file. Refer to the The "ipwhitelist-reload" Command section for more details.
Errors during whitelist reload, if any, are logged in individual Terracotta Server logs. Thus, after every update operation, server logs should be checked to verify that the updates took effect in all the servers.
If a cluster is not yet activated and the whitelist file needs to be reloaded on the servers, the server-level ipwhitelist-reload command can be used. It may also be helpful when the machine from where the cluster tool is to be used is itself not whitelisted initially. In this scenario, adding this machine's IP to the whitelist, and running the server-level ipwhitelist-reload command ensures that cluster tool can configure the cluster later.
Note:
If any failures happen while reading the whitelist.txt file during a dynamic update, the update is ignored and the server continues with the current whitelist. No partial updates are applied.
Connection Behaviour
When a client connects to a server, the server accepts the socket connection, and verifies the IP of the incoming client connection against the whitelist. If it finds that the client IP is not whitelisted, it closes the socket connection.
If a whitelisted client is removed from the whitelist via a dynamic update, it remains connected to the cluster as long as there is no network disconnection or explicit connection closure from the client. Subsequent connection attempts from the client to cluster will fail.
Full example of secure server configuration
Here is an example of a security configuration section in the config file of a single stripe, two node cluster, that configures auditing, encrypted connections, LDAP authentication and IP whitelisting:
authc=ldap
ssl-tls=true
whitelist=true
stripe.1.node.1.audit-log-dir=/path/to/audit-directory
stripe.1.node.1.security-dir=/path/to/security-root-directory
stripe.1.node.2.audit-log-dir=/path/to/audit-directory
stripe.1.node.2.security-dir=/path/to/security-root-directory
With the configuration in the above example, the server's security root directory would contain a truststore in the trusted-authority subdirectory, a keystore and a credentials.properties in the identity directory, an ldap.properties in the access-control subdirectory and a whitelist.txt file.
The keystore is required so that the server can prove to clients that it is the server they are expecting. The credentials.properties file is required so that the server can connect to other servers in the stripe. If there are no other servers in the stripe, the credentials.properties file should not be present.
Client configuration
To configure a client to connect to a secured cluster, you need to give the client a path to the client's security root directory. This should contain, for example, the credentials that the client needs to connect to the cluster.
Command line tools
To enable command line tools to connect to a secure cluster, a command must be prefixed with -srd (or long option --security-root-directory).
The following example shows the use of the config tool get command with the -srd option specified:
./config-tool.sh -srd /path/to/security-root-directory get -c data-dirs -s localhost
stripe.1.node.1.data-dirs=main:%H/terracotta/user-data/main
Attempting to connect to a secure cluster without the -srd option will fail. Commands without this option retain their behavior.
Ehcache Client
An Ehcache client can define either an XML or a programmatic configuration, both of which support security configuration. The following are the examples of usages of each:
1. API Example
PersistentCacheManager cacheManager = CacheManagerBuilder
.newCacheManagerBuilder()
.with(EnterpriseClusteringServiceConfigurationBuilder
.enterpriseSecureCluster(connectionURI,
securityRootDirectoryPath) // 1
.autoCreate())
.build(true);
1
EnterpriseClusteringServiceConfigurationBuilder enterpriseSecureCluster(URI, Path) lets you create a CacheManager using a secure connection. The first argument is the URI of the Terracotta cluster, appended with the CacheManager name. The second argument is the Path to the client's security root directory. EnterpriseClusteringServiceConfigurationBuilder enterpriseSecureCluster(Iterable<InetSocketAddress>, String, Path) serves the same function, with the added support for IPv6 addresses.
Note:
If the URI or Iterable<InetSocketAddress> contains host names, make sure that they match the host names specified in the server certificates.
The EnterpriseClusteringServiceConfigurationBuilder enterpriseCluster(URI) API continues to provide unsecured connections to unsecured clusters.
2. XML Example
<ehcache:config
xmlns:ehcache="http://www.ehcache.org/v3"
xmlns:tc="http://www.terracottatech.com/v3/terracotta/ehcache">

<ehcache:service>
<tc:cluster>
<tc:connection url="${cluster-uri}/CM" security-root-directory="${security-root-directory}"/>
<tc:server-side-config auto-create="true"/>
</tc:cluster>
</ehcache:service>

</ehcache:config>
1
security-root-directory lets you specify the Path to the client's security root directory. Not passing this option retains the behavior of communicating with an unsecured cluster.
TC Store Client
1. API Example
DatasetManager datasetManager = DatasetManager.secureClustered(
connectionURI, securityRootDirectoryPath) // 1
.build();
1
DatasetManager.secureClustered(URI, Path) lets you create a DatasetManager using a secure connection. The first argument is the URI of the Terracotta cluster. The second argument is the Path to the security root directory which is to be used for the connection. DatasetManager.secureClustered(Iterable<InetSocketAddress>, Path) serves the same function, with the added support for IPv6 addresses.
Note:
If the URI or Iterable<InetSocketAddress> contains host names, make sure that they match the host names specified in the server certificates.
The DatasetManager.clustered(URI) API continues to provide unsecured connections to unsecured clusters.
2. XML Example
<clustered xmlns=
"http://www.terracottatech.com/v3/store/clustered"> <!--1-->
<cluster-connection>
<server host="localhost" port="9410"/>
<security-root-directory>/path/to/security-root-directory
</security-root-directory> <!--2-->
</cluster-connection>
</clustered>
1
Declares a clustered DatasetManager configuration.
2
security-root-directory lets you specify the Path to the client's security root directory. Not specifying this element retains the behavior of communicating with an unsecured cluster.