Apama 10.15.0 | Deploying and Managing Apama Applications | Tuning Correlator Performance | Correlator pipelining | Configuring pipelining with engine_connect
 
Configuring pipelining with engine_connect
The engine_connect tool allows direct connection of running correlator instances. The executable for this tool is located in the bin directory of the Apama installation. Running the tool in the Apama Command Prompt or using the apama_env wrapper (see Setting up the environment using the Apama Command Prompt) ensures that the environment variables are set correctly.
Note: 
Some of the functions of the engine_connect tool can be performed from within EPL. For more information, see Using the Management interface.
Synopsis
To configure pipelining, run the following command:
engine_connect [ options ]
When you run this command with the –h option, the usage message for this command is shown.
Description
engine_connect connects a source correlator (the sender) to a target correlator (the receiver). The target correlator will receive events from the specified channel(s) of the source correlator. Source and target correlators must already be running.
Alternatively, if you specify the –f option, engine_connect reads connection information from the specified file and sets up each connection found therein (see Configuring pipelining through the client API for details of the file format). The engine_connect tool expects the specified file to be in the local character set. If the configuration file is in UTF-8, specify the -u option in addition to the -f option. If the filename provided to -f is a hyphen (-), then connection information is read from the standard input device (stdin) until end-of-file.
The connection between the source and target correlators is persistent. When one of the correlators stops running, then when that correlator restarts it automatically reconnects with the other correlator.
The tool is silent by default unless an error occurs. For verbose progress information, use the –v option.
Options
The engine_connect tool takes the options listed below.
Note: 
Many of these options can also be specified as elements of a YAML configuration file (with different element names). If an option is specified in both the command line and a YAML configuration file, then the command line takes precedence. For further information, see Configuring the correlator and especially the topic Setting up connections between correlators in a YAML configuration file.
Option
Description
-h | --help
Displays usage information.
-sn host | --sourcehost host
Name of the host on which the source (event sending) correlator is running. The default is localhost. However, you can use the default or specify localhost only when the source correlator and the target correlator are running on the same host. In all other situations, you must specify the public IP address or the name of the host. This ensures that the host of the target correlator can resolve the name/address of the source correlator host. Non-ASCII characters are not allowed in host names.
-sp port | --sourceport port
Port on which the source (event sending) correlator is listening. The default is 15903.
-tn host | --targethost host
Name of the host on which the target (event receiving) correlator is running. The default is localhost. However, you can use the default or specify localhost only when the source correlator and the target correlator are running on the same host. In all other situations, you must specify the public IP address or the name of the host. This ensures that the host of the source correlator can resolve the name/address of the target correlator host. Non-ASCII characters are not allowed in host names.
-tp port | --targetport port
Port on which the target (event receiving) correlator is listening. The default is 15903.
-c channel | --channel channel
Named channel on which to send/receive events. You can specify the -c option multiple times to send/receive events on multiple channels. You must specify the -c option at least once for each sender/target pair. Until you do, no events emitted by the sender correlator are received by the target correlator. An event is discarded if it is sent on a channel for which you did not specify the -c option.
-m mode | --mode mode
Indicates whether there is one connection (-m legacy) between the sender and target correlators or one connection for each specified channel (-m parallel).
The default behavior is that there is one connection between the sender and target correlators. The tool uses the same connection for every channel. Events sent on any channel are delivered to the default channel in the target correlator and all events are delivered in order. You can specify default behavior by specifying -m legacy or --mode legacy.
To create a connection for each specified channel, specify -m parallel or --mode parallel. Events sent on a named channel are delivered to the same named channel in the target correlator. Events sent on the same channel are delivered in order. Events sent on different channels may be re-ordered.
You also specify the -m option when you specify the -x option to disconnect. If you are using a separate connection for each channel, you should specify -m parallel when you specify the -x option. If you are using one connection for all channels, you should specify -m legacy when you specify the -x option.
-x | --disconnect
When you specify the -x option, the behavior depends on whether you also specify the -c option.
If you specify the -x option and you do not also specify the -c channel option, then the source correlator stops sending events to the target correlator. Each connection between the source correlator and the target correlator is terminated.
If you specify the -x option and the -c channel option and the tool is using one connection for each channel, then the source correlator terminates only the connection(s) it was using for the channel(s) you specify. Any other connections being used for other channels continue to be used. You can specify the -x option with one or more instances of the -c channel option. Remember to also specify -m parallel.
If you specify the -x option and the -c channel option and the tool is using one connection for all channels, then the source correlator stops sending events on only the channel(s) you specify. The source correlator continues to send events on any other channels it was already sending events on. If there are no other channels, then the source correlator no longer sends events to the target correlator. However, the connection between the two correlators remains in place. Remember to also specify -m legacy.
-s | --qdisconnect
Disconnect if slow (only takes effect on the first connection).
-f file | --filename file
Read connection information from the named file. If this option is specified, the options -sn, -sp, -tn, -tp and -c are all ignored. This file must be in the local character set or in UTF-8 format. If it is UTF-8, specify the -u option in addition to this option.
-u | --utf8
Indicates that the connection information file is in UTF-8.
-v | --verbose
Requests verbose output during engine_connect execution.
-V | --version
Displays version information for the engine_connect tool.
Exit status
The engine_connect tool returns the following exit values:
Value
Description
0
All connections were established successfully.
1
One or more source correlators could not be contacted.
2
One or more target correlators could not be contacted.
3
A problem occurred establishing the connection; request invalid.
4
Target correlator failed to contact the source.
5
Some other error occurred.
Comparison of legacy and parallel connection modes
Legacy connection mode
Parallel connection mode
0 or 1 connection between two correlators.
Any number of connections between correlators.
Events sent on different channels are delivered in the order in which they are sent.
Events sent on different channels may be delivered in a different order from the order in which they were sent.
Sending an event to a named channel delivers the event to the default channel.
Sending an event to a named channel delivers it to only that channel.
Unlike Universal Messaging for passing events between correlators.
Similar to Universal Messaging for passing events between correlators.
Same behavior as releases earlier than Apama 5.2.
New behavior starting with Apama 5.2.
Universal Messaging has no mechanism for enforcing ordering among events sent on different channels. However, Universal Messaging is the better alternative when you want to use a large number of channels to send events between components. Without Universal Messaging, the use of two TCP connections with threads on both ends of the connection might reach the limit of how many channels can have dedicated connections.
Avoid mixing connection modes
Successive command lines that specify the same source/target hosts/ports build on each other. While this makes it possible to mix the legacy and parallel connection modes, you should avoid doing that. Mixing connection modes can cause an event to be delivered twice to the same channel. For example:
engine_connect -tp 15902 -sp 15903 -c channelA -c channelB
engine_connect -tp 15902 -sp 15903 -c channelA -c channelC -m legacy
The result of the first command is that there is one (legacy) connection for sending/receiving events on channelA and channelB. The result of the second command is that there is a dedicated connection for sending/receiving events on channelA and a dedicated connection for sending/receiving events on channelC. Events sent on channelA would be delivered twice: once on the legacy connection and once on the dedicated connection.
Examples
Because you can specify command lines that build on each other, you could set up a connection and add named channels later. You can also unsubscribe the channels you have added so that no events are sent or received. The connection remains and you can re-add channels at a later time. However, until you specify the -c option for a given connection, no events emitted by the source correlator are received by the target correlator. Consider the following command line:
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904
The correlators on host1 and host2 are connected but no channels have been subscribed and therefore no events are sent/received. To send and receive events, specify the -c option as in the following command line:
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904 -c CHAN1 -c CHAN2
Now the connected correlators can use CHAN1 and CHAN2 to send/receive events. To add another channel, execute this command:
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904 -c CHAN3
The correlators are now using CHAN1, CHAN2, and CHAN3 to send/receive events. To stop using CHAN2, execute the following command. The correlators continue to use CHAN1 and CHAN3.
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904 -x -c CHAN2
To stop sending and receiving events, execute the following command. Note that the correlators remain connected until one of them stops. There is no penalty for this connection.
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904 -x
In this example, the following command is equivalent to the previous command.
engine_connect -sn host1 -sp 15903 -tn host2 -tp 15904 -x -c CHAN1 -c CHAN3