Working with normalized events

The function of a decoding codec plug-in is to convert incoming messages into a standard normalized event format that can be processed by the Semantic Mapper. Events sent upstream to an encoding codec plug-in are provided to the plug-in in this same format.

Normalized events are essentially dictionaries of name-value pairs, where the names and values are both character strings. Each name-value pair nominally represents the name and content of a single field from an event, but users of the data structure are free to invent custom naming schemes to represent more complex event structures. Names must be unique within a given event. Values may be empty or NULL.

When assigning names to fields in normalized events, keep in mind that the fields and transport attributes for event mapping conditions and event mapping rules both use a list of fields delimited by spaces or commas. This means, for example that <id fields="Exchange EX,foo" test="==" value="LSE"/> will successfully match a field called Exchange, EX or foo, but not a field called Exchange EX,foo. While fields with spaces or commas in their names may be included in a payload dictionary in upstream or downstream directions, they cannot be referenced directly in mapping or id rules.

To construct strings for the normalized event fields representing container types (dictionaries, sequences, or nested events), use the Event Writer utility found in the AP_EventWriter.h header file, which is located in the include directory of the Apama installation. The following examples show how to add a sequence and a dictionary to a normalized event (note the escape character (\) used in order to insert a quotation mark into a string).

#include <AP_EventWriter.h>

AP_EventWriter *map, *list;
AP_NormalisedEvent *event;
AP_EventWriterValue key, value;

list=AP_EventWriter_ctor(AP_SEQUENCE, NULL);
list->addString(list, "abc");
list->addString(list, "de\"f");

map=AP_EventWriter_ctor(AP_DICTIONARY, NULL);
key.stringValue="key1"; value.stringValue="value";
map->addDictValue(map, AP_STRING, key, AP_STRING, value);
key.stringValue="key\"{}2";
value.stringValue="value\"{}2";
map->addDictValue(map, AP_STRING, key, AP_STRING, value);

event=AP_NormalisedEvent_ctor();
event->functions->addQuick(event, "mySequenceField",
event->functions->list->toString(list));
event->functions->event->functions->addQuick(event,
"myDictionaryField", event->functions->map->toString(map));

AP_EventWriter_dtor(list);
AP_EventWriter_dtor(map);

AP_EventWriter* event = AP_EventWriter_ctor(AP_EVENT, "MyEvent");

AP_EventWriterValue val;
val.refValue = NULL;

//add an 'empty' any
event->addAny(event, AP_EMPTY, val, NULL);

//add an 'empty' optional as the second field
event->addOptional(event, AP_EMPTY, val);

val.intValue = 100;
event->addAny(event, AP_INTEGER, val, NULL);
val.intValue = 200;
event->addOptional(event, AP_INTEGER, val);

//Add an 'any' field containing 'optional'
AP_EventWriter *opt = AP_EventWriter_ctor(AP_EVENT, "optional");
opt->addInt(opt, 1);

val.refValue = opt;
event->addAny(writer, AP_EVENT, val, "optional<integer>");

Fields names and values of normalized events are in UTF-8 format. This means that the writer of the codec needs to ensure that downstream events are correctly formed and the codec should expect to handle UTF-8 coming upstream.

The NormalisedEvent.h header file defines objects and functions that make up a special programming interface for constructing and examining normalized events. It contains two main structures:

This structure represents a single normalized event. It has a pointer to a table of client-visible functions exported by the object called AP_NormalisedEvent_Functions. This function table provides access to the operations that may be performed on the event object.

In addition, the AP_NormalisedEvent_ctor constructor function is provided to create a new event instance. AP_NormalisedEvent_dtor destroys a normalized event object, and should be called when the event is no longer required to free up resources.

This structure can be used to step through the contents of a normalized event structure, in forwards or reverse order. It contains a function table defined by AP_NormalisedEventIterator_Functions, which includes all of the functions exported by a normalized event iterator.

AP_NormalisedEventIterator_dtor destroys a normalized event iterator object, and should be called when the iterator is no longer required to free up resources. There is no public constructor function; iterators are created and returned only by AP_NormalisedEvent functions.

In both AP_NormalisedEvent and AP_NormalisedEventIterator functions, there is always a pointer to the corresponding structure. This is analogous to the implicit this pointer passed to a C++ object when a member function is invoked on it.