Using the Payload Extraction plug-in

Apama events are rigidly structured and need to comply with an event type definition. This describes the structure of a particular event, in particular its name, as well as the order, name, type and number of its constituent fields.

However, external events, even when they are of the same ‘type’ or nature (for example, all Trade events or News headlines) might vary in format and structure, even when originating from the same source or feed.

This issue is of particular relevance in the IAF and its semantic mapper module, where rules define how to translate and map from external events to Apama events and vice-versa.

In order to accommodate this, Apama provides for the concept of a payload field within an Apama event. The payload field, typically the last field in an event type definition, can embed any number of additional optional fields in addition to the always-present primary fields.

For example, consider an external message that can appear in several guises, but where each always consists of a particular subset of critical fields together with a variable number of additional optional fields.

If it is desired that these varying guises are mapped to a single Apama event type, then this needs to be defined so that its fields correspond to the subset of critical (and always present) fields, followed by a payload field into which the additional (and optional) fields are embedded.

Note: The Payload Extraction correlator plug-in is deprecated as of Apama 5.0. All Apama adapter products now use dictionary-format payloads rather than string-format payloads. The dictionary-format payload is considerably more efficient in almost all cases. While string-format payloads are still accepted in Apama 5.1, this support, including the Payload Extraction correlator plug-in, is deprecated and will be removed in a future release of the Apama platform. It is recommended that all third-party adapters are migrated to dictionary-format payloads as soon as possible.