Separating Client and Cluster Communication
When configuring Universal Messaging for clustering, it is not essential but it is often recommended to have a dedicated interface for client communications and a dedicated interface for cluster communications. There are various reasons for this, as described in this section.
Occasionally it is necessary to stop a client connection while some operation is being carried out on the server or cluster. For example, you may want to delete and recreate a channel, so you need to stop your applications from publishing while this is done. However, you do not want to stop the cluster communication during this time, so it is good practice to separate your cluster communication from client communication. Replication of data, state and other cluster negotiation traffic is very high priority so we need to ensure that it is not delayed in any avoidable way. If the cluster cannot form, then no client traffic will be able to flow, so it is vital that the client traffic cannot impact the cluster formation. If you have lots of clients constantly trying to access the server and getting disconnected because the cluster is not formed, they will be competing to connect at the same time as nodes within the cluster are trying to connect to one another.
By adding separate interfaces onto your servers you can separate client communication from cluster communication and also disable client communication if necessary.
There is no server-defined limit on the number of interfaces that you can create, so it might be a good idea to have different interfaces for different applications. This will allow you to easily disable connections from particular applications.
Adding a Dedicated Interface for Cluster Communication
The first thing to do is ensure that you have more than one interface per server. You can add an interface by selecting the server under the Realms node in Enterprise Manager, then opening the Comms panel. Click the Add Interface button to bring up the dialog for adding an interface.
In the dialog, enter the interface details as required. Also ensure that the Auto Start checkbox is enabled, so that if you stop and start the server, your interface is automatically enabled.
Now you will have two interfaces. In this example we have two NHP interfaces, one on port 11000 and one on port 11010. We are going to make the interface running on port 11010 the interface used for cluster communication. The default settings will allow the interface to be used for cluster communication (and the nodes will automatically be notified about this new interface). We recommend you to disable Advertise Interface, which is available as a checkbox under the panel Comms > Interfaces > Basic. This will stop the interface information being sent to clients.
Note:
Do not disable the checkbox Allow Client Connections under Comms > Interfaces > Basic; this is important because Administration API connections, e.g. Enterprise Manager, still count as client connections and we always want to be able to connect an Enterprise Manager. You can optionally disable Allow for InterRealm on the other interface but it is not necessary.
So now we have our interfaces set up correctly and the interface for cluster communication will not be advertised for clients to use. It is now the responsibility of the system administrator to ensure that the cluster interface is not used by clients for normal connections; all applications should use the other interface.
Temporarily Disconnecting the Client Interface
Let us assume that we want to temporarily disable client connections. All we have to do is stop the interface that we are using for out client connections. This is the best way to stop client traffic, but an alternative is to disable the Allow Client Connections checkbox temporarily.
Note:
your Enterprise Manager may be connected to the client interface, so if you stop this interface or disable client communication, you will get disconnected. It is important that you still allow client connections on the cluster interface simply so that you can connect an Enterprise Manager to that interface while your other applications remain disconnected.
Once you are ready to allow clients back in, you can re-enable the client interface.
Further Considerations
Network partitioning
Rather than relying on clients not knowing about the cluster interface, it is often a good idea to bind that interface to a different network or VLAN. This is done by specifying an adapter to bind to rather than using 0.0.0.0. This way you restrict connections on this interface to users within that network. It is in fact best practice to bind to a specific network interface rather than 0.0.0.0, especially if running containers.
Stricter Access Control
In the example above we simply ensure that clients do not know about the cluster interface. To enforce this you could use network partitioning as explained above, or you could restrict access using ACLs or VIA entries. It is not possible to set interface-specific ACL entries, so to restrict connections you would need to temporarily stop them connecting using the server level ACL. To do this you would add a specific ACL entry for your administration-related connection and then disable access for everyone else.
Another alternative is to use VIA entries, which are set on individual interfaces. A VIA entry lets you restrict what interfaces a user and host are allowed to connect to. You would need to add an entry that covers all clients into the VIA list for the client interface and leave the VIA list empty for the cluster interface.
For more information on VIA entries, see
Interface VIA Rules.
One more alternative is to use three interfaces, an administration interface, a cluster interface and a client interface. You could disable client communication on the cluster interface to limit traffic to cluster communication only. You could then use SSL on the administration interface and only provide administration users with the certificate required to connect. This way you can disable the client interface to stop client communication and no client will be able to connect to the cluster interface at all.