Receiving messages with APP_CONTROLLED acknowledgements

Apama applications that receive JMS messages can prevent message loss without using correlator persistence by controlling when the application acknowledges the received messages. To do this, use APP_CONTROLLED reliability mode. With APP_CONTROLLED reliability mode, an application can tie the sending of the JMS acknowledgement to application-defined strategies for preserving the effect of the messages. For example, an application might need to ensure JMS messages are not acknowledged to the broker until any output resulting from them has been written to a database, a distributed MemoryStore, a downstream JMS destination, or a connected correlator.

An alternative to using APP_CONTROLLED reliability mode is to use correlator persistence with reliability mode set to AT_LEAST_ONCE. See Using correlator persistence with correlator-integrated messaging for JMS

When reliability mode is set to APP_CONTROLLED, applications are still entirely responsible for handling duplicate messages as well as any message re-reordering that occurs. Applications must be able to cope with any message duplication or reordering caused by the JMS provider implementation or failures in the sender, receiver or broker.

Note: If a license file cannot be found, the correlator is limited to BEST_EFFORT only messaging. See Running Apama without a license file.

In an Apama application, a receiver that is using APP_CONTROLLED reliability mode goes through the following cycle:

1. Receive a batch of messages. Typically, there are several hundred in a batch. The number is controlled by the maxBatchSize and maxBatchIntervalMillis receiver settings. Note that regardless of the value of maxBatchIntervalMillis, the receiver will not be suspended while no events are being received. See Advanced configuration bean properties.

2. Suspend operation at the end of the batch. After suspending, the receiver sends a JMSAppControlledReceivingSuspended event to the context that is handling the messages.

3. Application commits the received messages or commits the results of received messages, such as state changes or output messages to other systems. For example, the received messages might have caused messages to be sent to a database, a distributed MemoryStore, a downstream JMS destination or a connected correlator. These operations may involve a synchronous plug-in call, or sending a request and then listening for an asynchronous event to indicate completion or acknowledgement.

4. Acknowledge receipt of the batch of messages to the JMS broker. After application-specific commit operations for this message batch are complete, the messages no longer need to be retained by the JMS broker. The application calls JMSReceiver.appControlledAcknowledgeAndResume() to acknowledge the message batch and resume receiving. The cycle then starts again.

Following is a simple example of the application logic for responding to JMSAppControlledReceivingSuspended events and allowing the message batch to be acknowledged after the messages have been suitably handed off to another system:

on all JMSAppControlledReceivingSuspended(receiverId="myReceiver")
{
on MyFinishedPersistingReceivedEvents(requestId=persistReceivedEventsSomehow())
{
jms.getReceiver("myReceiver").appControlledAckAndResume();
}
}

The code below shows an example of using APP_CONTROLLED receiving, together with flush acknowledgements from the JMS sender. See Sending messages reliably with application flushing notifications. With this strategy, received JMS messages are acknowledged to the JMS broker only after the context gets an acknowledgement from the JMS sender that all the associated output messages have been sent to the JMS broker.

on all JMSAppControlledReceivingSuspended(receiverId="myReceiver")
{
on JMSSenderFlushed(requestId =
jmsConnection.getSender("mySender").requestFlush()){
jms.getReceiver("myReceiver").appControlledAckAndResume();
}
}

It is important to use the same context to process the messages from a given receiver and to call appControlledAckAndResume().

To improve the throughput of an APP_CONTROLLED receiver, try adjusting the maxBatchSize and maxBatchIntervalMillis receiver settings. The goal is to balance the time spent receiving JMS messages and the time spent committing the results. If the batches are too small then throughput can decrease. If the batches are too large then latency can increase and the JMS broker could use excessive memory to hold the unacknowledged messages.

It is possible to use the APP_CONTROLLED reliability mode for a receiver in a persistence-enabled correlator. In this case, process the messages and call appControlledAckAndResume() from a non-persistent monitor. Acknowledgements cannot be controlled from a persistent monitor because the JMS acknowledgement would get out of sync with the monitor state after recovery. If you try to call appControlledAckAndResume() from a persistent monitor an exception will be thrown.

Note: JMS messages that result in mapping failures cannot be handled by the EPL application so they are usually acknowledged automatically.