Optimizing requests to Cumulocity IoT with concurrent connections
In order to provide better performance in requests to the Cumulocity IoT platform, you can configure the transport to use multiple client connections to perform requests concurrently. This can provide improved performance, but may also change the ordering in which requests are executed and responses are returned. By default, the Cumulocity IoT transport tries to use multiple connections and restricts ordering to avoid races that may affect your EPL application. However, this may be either insufficiently concurrent or insufficiently ordered for your specific use case. In that case, there are several options on how to control the concurrency used. These are described below.
Default behavior
By default, three concurrent connections are created for handling requests to the Cumulocity IoT platform for a given correlator process (a given tenant, for applications running inside the Cumulocity IoT platform).
To avoid obvious races, the following rules apply for concurrency:
All read (
GET) requests can be performed concurrently with each other.
All update (
PUT/
POST) requests relating to different managed objects (for example, a
ManagedObject update, a new
Measurement for a second
ManagedObject and an
Alarm with a third
ManagedObject as a source) can be performed concurrently with each other.
All updates to a single managed object (based on
id, or
source as appropriate) are performed serially in the order they were sent to the transport.
Measurement updates sent without the
withChannelResponse option that may be batched can be processed concurrently with any other request.
Any read request waits for all outstanding update requests before starting.
Any update request waits for all outstanding read requests before starting.
GenericRequest update (
PUT/
POST/
DELETE) operations cannot be automatically tied to a
ManagedObject, so they are never processed concurrently with any other request.
Changing the default behavior
There are two options to change the default behavior through the CumulocityIoT.properties file. These changes apply to the entire correlator. When running as an EPL application in the Cumulocity IoT platform, this setting can be changed for your whole tenant as a tenant option.
You can set the
CUMULOCITY_NUM_CLIENTS property to control the number of clients used. If it is set to 1, then all requests are sent serially using a single connection (disables concurrency). If it is set to another number, then that number of concurrent connections to the platform is created.
You can add the
CUMULOCITY_CONCURRENCY_MODE property to the properties file. By default, this has the value of
auto which has the behavior described above to limit the concurrency to preserve some ordering. You can set the value to
always instead, in which case requests of all types may be processed concurrently and any ordering consequences must be handled in EPL instead.
Creating a new connection with specific behavior
The following table lists the event types that you can use to either create new connections for sending events independently of the default transport or use the default shared connection. In this way, multiple different configurations can be used at the same time. You can use these event types to separate the requests for different EPL applications, or different types of requests within the system. Each of these event types corresponds to a particular combination of settings that can be configured for the default transport. These events can be created using either the
create or the
createForTenant actions. The
create action is used when an application needs to work only with per-tenant deployment. The
create action may take an argument to configure the number of clients to use for the connection depending on the connection type. The
createForTenant action should be used when an application needs to work with multi-tenant deployment or both multi-tenant and per-tenant deployment. The
createForTenant action takes a
TenantDetails event as its first argument and may also take an argument to configure the number of clients to use for the connection depending on the connection type. The
TenantDetails event provides details of the subscribed tenant to the connection object so that it can provide information that is specific to the provided tenant. The
TenantDetails events can be obtained by using tenant subscription events. See
Working with multi-tenant deployments for more details.
Event type | Description |
FullyConcurrentConnection | A connection with multiple clients and no restriction on ordering. The additional argument to the create() or createForTenant() method on that event type is: integer numClients Configuration equivalents: CUMULOCITY_NUM_CLIENTS=numClients
CUMULOCITY_CONCURRENCY_MODE=always |
SerialConnection | A completely serial connection with no concurrency. There is no argument to the create() or createForTenant() method on that event type. Configuration equivalent: CUMULOCITY_NUM_CLIENTS=1 |
AutoConcurrentConnection | A connection with multiple clients but with the standard ordering restrictions. The additional argument to the create() or createForTenant() method on that event type is: integer numClients Configuration equivalents: CUMULOCITY_NUM_CLIENTS=numClients
CUMULOCITY_CONCURRENCY_MODE=auto |
SharedConnection | A connection to Cumulocity IoT which is configured to handle all requests using a shared default client. |
All of the above event types have the same API to use them, which looks like this:
FullyConcurrentConnection conn := FullyConcurrentConnection.create(5);
monitor.subscribe(conn.getChannel(FindManagedObjectResponse.SUBSCRIBE_CHANNEL));
FindManagedObject fmo := // create a request
send fmo to conn.getChannel(FindManagedObject.SEND_CHANNEL));
on FindManagedObjectResponse(reqId=fmo.reqId) {
// do something
monitor.unsubscribe(conn.getChannel(FindManagedObjectResponse.SUBSCRIBE_CHANNEL));
conn.destroy(); // can't use it after this
}
The connection object provides a getChannel method which can be used to convert the standard channels used into the correct channels for events to go to this connection instead of the default transport.
For more details, see the
com.apama.cumulocity package in the
API Reference for EPL (ApamaDoc) .
Writing your EPL to avoid races
If there are ordering issues due to concurrency, particularly if you want to use a fully-concurrent connection, then you may need to write your EPL to explicitly cater for the possible reordering. This is normally done by listening for the response events that each request has to ensure that it is fully completed before doing any requests which depend on the first one having completed.
For sending a creation or update event to an object that means using the withChannelResponse API and listening for the ObjectCommitted or ObjectCommitFailed response. For example:
monitor.subscribe(Measurement.SUBSCRIBE_CHANNEL);
Measurement m := // create measurement
integer reqId := Util.generateReqId();
send m.withChannelResponse(reqId) to Measurement.SEND_CHANNEL;
on ObjectCommitted(reqId=reqId) and not ObjectCommitFailed(reqId=reqId) {
// do whatever must wait until the Measurement request has been fully completed
}
on ObjectCommitFailed(reqId=reqId) and not ObjectCommitted(reqId=reqId) {
// handle the fact that this failed
}