Setting up Inter-Realm Communication

The communication between realms in a cluster can be secure (encrypted) or non-secure (non-encrypted). The communication is implemented by defining one or more interfaces on each realm.

Since all realms in a cluster are required to have the same configuration (so that for example if the master realm goes offline, one of the other realms can become the new master), you must ensure that any interface definitions on one realm match the interface definitions on all other realms in the cluster.

For non-encrypted inter-realm communication, you can set up the interfaces to use either NSP (Socket Protocol) or NHP (HTTP Protocol). In general, we recommend you to use NSP rather than NHP for non-encrypted inter-realm communication.

For encrypted inter-realm communication, you can set up the interfaces to use either NSPS (Secure Socket Protocol) or NHPS (Secure HTTP Protocol). In general, we recommend you to use NSPS rather than NHPS for encrypted inter-realm communication.

Each realm contains by default one predefined interface, and this interface uses the NSP protocol (i.e. socket protocol without encryption). Also by default, the interface is configured to be usable for inter-realm communication as well as for communication between realm and clients.

If you do not define any additional interfaces on the realm, all communication between the realm and other realms, and between the realm and clients, will use this interface. You can set up a cluster consisting of multiple realms, each of them having just this one default interface defined.

However, in general we recommend you to set up two NSP interfaces on each realm for non-encrypted communication, namely one interface for only inter-realm communication and one interface for only client communication to the realm. The options for setting up this configuration are available in the Enterprise Manager, under the Comms > Interfaces > Basic tab of each realm.

On the interface that you will use for inter-realm communication, use the following settings:

Similarly, on the interface that you will use for client communication with the realm, use the following settings:

After you make these changes, restart all of the realms, to ensure that the new interfaces are activated.

When you form the cluster, communication between realms in the cluster will use the NSP interface that you have configured for inter-realm communication.

The assumed starting point in this scenario is that there is no cluster formed yet. All of the realms that will later form the cluster need to be configured.

The steps required are as follows:

1. If you intend to use self-signed certificates, or if you intend to use a custom truststore (which contains the public certificates associated with each Universal Messaging realm's private certificate), the keystore and the truststore must be added to the Universal Messaging JVM process.

In the file Server_Common.conf on each realm, provide details of the truststore and keystore, according to the following pattern:

wrapper.java.additional.7="-Djavax.net.ssl.trustStore=<TRUSTSTORE>
wrapper.java.additional.8=-Djavax.net.ssl.trustStorePassword=<TRUSTSTORE_PWD>
wrapper.java.additional.9="-Djavax.net.ssl.keyStore=<KEYSTORE>
wrapper.java.additional.10=-Djavax.net.ssl.keyStorePassword=<KEYSTORE_PWD>

wrapper.java.additional.7="-Djavax.net.ssl.trustStore=
/webmethods/truststores/um_truststore.jks"
wrapper.java.additional.8=-Djavax.net.ssl.trustStorePassword=nirvana
wrapper.java.additional.9="-Djavax.net.ssl.keyStore=
/webmethods/keystores/um_keystore.jks"
wrapper.java.additional.10=-Djavax.net.ssl.keyStorePassword=nirvana

a. Add one interface using the NSPS protocol, to be used only for inter-realm communication.

Note: The demo certificates generated by the Universal Messaging Certificate Generator tool (see the section How to generate certificates for use) are only valid for the loopback interface (localhost / 127.0.0.1). Therefore, if you use these demo certificates, ensure that the adapter that you add is bound only on the loopback interface.

For this interface, set the following options (in the Enterprise Manager, they are located under the Basic or Certificates tabs of the interface definition screen):

The reason for disabling client certificate validation is because Universal Messaging does a certificate exchange between realms already when constructing a cluster, so doing another certificate exchange at the SSL layer would be redundant.

Specify Certificates and Truststore on the interface as you would normally.

2. Enforce the SSL level in the realm (using a JVM argument in Server_Common.conf). Example: to enforce TLS1.2 globally on the Universal Messaging server, set:

b. Add one more interface using the NSPS protocol, to be used only by clients for communication with the realm. For this interface, set the following options:

3. Disable the setting for inter-realm communication on the original, non-encrypted, interface.

5. Restart all Universal Messaging realms (to make sure all JVM arguments are activated).

The assumed starting point in this scenario is that there is already a cluster in which the inter-realm communication is not encrypted, i.e. the interface protocol is NHP or NSP, and you want to change this to encrypted communication, i.e. using the interface protocol NHPS or NSPS.

Here are the steps to follow to switch from non-encrypted to encrypted inter-realm communication in a Universal Messaging cluster:

2. In the file Server_Common.conf on each realm, provide details of the truststore and keystore, as described in the previous section (Setting up encrypted inter-realm communication).

4. On each realm, create two NSPS interfaces, as described in the previous section.

5. Under the Certificates tab for each of the NSPS interfaces, add a reference to the custom truststore and the keystores containing the server signed certificates, for example:

Key store path : /webmethods/keystores/um_keystore.jks
Trust store path : /webmethods/truststores/um_truststore.jks

7. Set the environment variables CAKEYSTORE and CAKEYSTOREPWD for each realm to reference the truststore containing the CA root chain, and the truststore's password. You can set up these variables as follows:

a. Open the file Admin_Tools_Common.conf that is located in UniversalMessaging/java/<instanceName>/bin, where <instanceName> is the name of the realm server.

set.default.CAKEYSTORE=/webmethods/keystores/um_keystore.jks
set.default.CAKEYSTOREPASSWD=nirvana

Note that if these variables have already been assigned a value elsewhere in the session, for example in a startup script, the values defined here in Admin_Tools_Common.conf will be ignored.

By restarting the Enterprise Manager after setting values for CAKEYSTORE and CAKEYSTOREPASSWD, the Enterprise Manager will be able to connect over a secured interface.

9. Disable the inter-realm connection option on each realm's non-encrypted interfaces.

When a Universal Messaging realm starts for the first time, it automatically generates a public/private key pair for encryption purposes and stores it in the internal keystore server.jks file in the realm's data/RealmSpecific directory. The public keys of other nodes are also added to this file whenever the realms are added to form a cluster.

These auto-generated keys are used for server identification only; basically whenever two realms establish a connection, they will exchange a single signed message as part of the handshake routine, in order to confirm they know each other.

After this initial handshake has taken place, all encrypted communication between realms in a cluster uses separate keys and keystores, as stated in the section Setting up encrypted inter-realm communication above.