Reproducing correlator behavior from a replay log
To use a replay log to reproduce correlator behavior, you must do the following:
1. Run the extract_replay_log Python utility on your replay log.
2. Start a correlator on the default port (15903).
3. Run the replay_execute script that the extract_replay_log utility generates.
Invoking the extract script
The
extract_replay_log.py script is in the
utilities directory in your Apama installation directory. You must have at least Python 2.4 to run this utility. You can download Python from
http://www.python.org. If you are using Linux or Solaris, you probably already have Python installed.
The format for running the extract_replay_log utility is as follows:
extract_replay_log.py [options] replayLogFile
Replace replayLogFile with the path for the replay log you want to extract. If you specify the first replay log in a series, the subsequent replay logs must be in the same directory as the first replay log.
The options you can specify are as follows:
Option | Description |
-o= dir or --output=dir | Specifies the directory that you want to contain the output from the extract_replay_log utility. The default is the current directory. |
-l= lang or --lang=lang | Specifies the language of the script that the extract_replay_log utility generates. Replace lang with one of the following:  shell generates the replay_execute.sh UNIX shell script. batch generates the replay_execute.bat Windows batch file. This is the default. |
--noReplayMode | Indicates that you do not want to extract information needed to replay non-deterministic behavior in its original order. The script extracts all input events but not enough information to reproduce non-deterministic behavior in the same order as when the replay log captured it. Therefore the results of replaying the non-deterministic behavior might be different than the original results. For details, see About replay logs and parallel processing in the parallel processing section of Developing Apama Applications in EPL. This option has meaning only if you generated the replay log with the --replayLog option rather than the --inputLog option. |
-v or --verbose | Indicates that you want verbose utility output. |
-h or --help | Displays help for the utility. |
The extract_replay_log utility generates the following:
A script whose execution duplicates the correlator activity captured by the replay log.
Event files — each one is prefixed with
replay_.
MonitorScript and possibly JAR and Correlator Deployment Package (CDP) files — each one is prefixed with
replay_.
Starting the correlator
Before you invoke the replay_execute script, start the correlator that you want to use to replay the log. Start this correlator on the default port (15903) and specify the following in the correlator startup command:
Specify the
--replayMode option to indicate that you are going to invoke the
replay_execute script. This option:
Enables replay of non-deterministic correlator behavior.
Implies the
-XignoreEnqueue option, which instructs the correlator to ignore the output of
enqueue statements. The replay log captures enqueued events, as well as when enqueued events appear on the input queues of public contexts. During replay, the correlator uses the captured events, not the newly enqueued events.
As mentioned before, the correlator typically processes events faster during replay than during the original activity. During replay, you want to ensure that the correlator processes enqueued events in the same place in the input stream as during the original correlator activity. Consequently, you want the replay correlator to use the captured enqueued events and to process them according to the captured processing order.
Implies the -
Xclock option, which disables the correlator’s internal clock. The replay log captures the time ticks during the original correlator activity. During replay, you want the correlator to use those captured time ticks. This ensures that the reproduced behavior reflects the original relationship between time ticks and the event stream.
Specify any other options that you specified when you started the correlator whose behavior was captured. For example, if the replay log will feed JAR files into the correlator, specify the
-j option and also specify the
-J option to pass the same options or properties that were previously passed to the embedded JVM.
If the captured behavior includes the use of the
integer.rand() and/or
float.rand() functions, be sure to specify the
-XsetRandomSeed option. This option starts the correlator with the random seed value you specify. To exactly replay the captured behavior, specify the integer that the
extract_replay_log.py script identifies as the random seed.
The correlator uses the random seed to calculate the random numbers returned by the rand() functions. The same random seed returns the same sequence of random numbers.
Invoking the replay script
If you want, you can edit the generated event files, EPL files, or JAR files to slightly modify the behavior you are about to replay. For example, you might add logging for debugging purposes. However, there are restrictions on what you can change:
You cannot insert any of the following:
calls to
integer.getUnique() or
rand()emit,
spawn...to,
enqueue, or
enqueue..to statements
context constructors
You cannot change the number of parseable events sent to the correlator. For example, you cannot attach a dashboard component to the replay log because the dashboard components work by sending events to the correlator.
You cannot change the number of event definitions and monitors injected.
Making any of these changes can potentially alter the behavior of later operations.
If you are using the MemoryStore and the correlator reads or writes to a store on disk then to accurately play back execution you must have a copy of that store as it was before the correlator modified it. Also, if you are using the MemoryStore from multiple contexts it is unlikely to replay correctly because the order of interaction with the MemoryStore is not preserved in the replay log.
When the correlator is running in the correct mode, and you have optionally edited the generated files, you are ready to invoke the replay_execute script. The replay_execute script tries to replay the contents of the replay log into the correlator running on the default port.
While the correlator exactly reproduces the activity captured in the replay log, it can execute the same activity faster during replay than when it was executed originally. This is because the correlator already has all the events it needs to process; it does not have to wait for any events. Replaying a log is typically significantly faster than original correlator activity. It is possible that you will find that the time it takes to replay a log is not much less than the time it took for the original activity. In this case, it is possible you were running too close to capacity during the original run. If that is the case, you risk not being able to keep up with the event flow during regular correlator execution. If you anticipate higher event flow then you should investigate optimizing your application or running it on a faster computer.
Copyright © 2013
Software AG, Darmstadt, Germany and/or Software AG USA Inc., Reston, VA, USA, and/or Terracotta Inc., San Francisco, CA, USA, and/or Software AG (Canada) Inc., Cambridge, Ontario, Canada, and/or, Software AG (UK) Ltd., Derby, United Kingdom, and/or Software A.G. (Israel) Ltd., Or-Yehuda, Israel and/or their licensors.