Handling Many Connections on the Server Side

To handle many simultaneous inbound connections to Universal Messaging, you might need to do the following:

When handling many inbound connections that do not require the HTTP or HTTPS protocol, we recommend using the Universal Messaging Socket Protocol (nsp) or the Universal Messaging SSL Protocol (nsps).

For more information about what native communication protocols Universal Messaging supports, see Native Communication Protocols.

The Universal Messaging server does not allocate individual threads for each incoming connection. Instead, the server uses a thread pool named Accept Threads to accept the connection and a separate thread pool named Select Threads to monitor data to be read or written to the socket.

When a Universal Messaging server is handling many incoming connections at the same time, you might need to adjust several attributes of the server interface to which the client connects. You configure the attibutes on the Comms > Interfaces tab for a server in the Enterprise Manager as follows:

Accept Threads. Increase the Accept Threads value, otherwise some of the connections will be queued in a backlog queue waiting to be accepted.

Backlog. Increase the capacity of the Backlog queue, otherwise when the capacity of the backlog queue is reached, additional connection requests will be denied.

Note:
On Linux, you might need to set the kernel parameter net.core.somaxconn. For Linux 2.2 and greater, the parameter specifies the maximum queue length for completely established sockets waiting to be accepted.

Auth Time. Increase the time to authenticate incoming connection requests if connection authentication is slow and causes the server to close the connection.

Select Threads. Keep the Select Threads value low even if the interface is handling a lot of connections. The number of select threads should not typically exceed the number of cores available.

Select Threads transfers reading or writing of network data to separate read or write thread pools. You configure the maximum count of threads for the read and write thread pools in the ReadThreadPoolMaxSize and WriteThreadPoolMaxSize properties on the Config > Thread Pool Config panel in the Enterprise Manager. Read pool threads are used for processing incoming client requests, while write pool threads are involved in writing data to client connections.

For more information about the attributes of an interface, see Basic Attributes for an Interface.

Universal Messaging uses in-memory buffers to handle network traffic. By default, these network buffers use the Java direct (off-heap) memory. To configure the Java direct memory for a realm in the Enterprise Manager, you go to Config > Connection Config > UseDirectBuffering.

If the connections handle a lot of traffic or if there are too many connections, the server might return the error "java.lang.OutOfMemoryError Direct buffer memory". In this case, you should increase the amount of direct memory allocated for the Universal Messaging server process by using the -XX:MaxDirectMemorySize=<size> Java command-line argument.

For information about -XX:MaxDirectMemorySize, see the Oracle documentation on Java command-line options.

For more information about how to troubleshoot the "java.lang.OutOfMemoryError Direct buffer memory" error and configure direct memory, see Troubleshooting.

In case of a slower network when clients are across WAN, Software AG recommends that you increase the socket read and write buffer size based on bandwidth-delay product.

Be aware that increasing the read and write buffer size impacts the direct memory allocated by the server. You might need to adjust your direct memory accordingly.