Universal Messaging 10.11 | Administration Guide | Command Line Administration Tools | The ExportEventsFromOfflineMemFile Tool
 
The ExportEventsFromOfflineMemFile Tool
Overview
The ExportEventsFromOfflineMemFile tool is a command-line recovery tool that dumps events from mem files of Universal Messaging realm server stores to XML or JSON format. Event filtering can be applied while dumping with the tool. Events can also be dumped to both XML and JSON format in a single run of the tool, resulting in two output files that have a common structure.
The output file will contain a first element describing the export details and then a list of events. The ExportDetails element contains the tool version number and the protocol buffer file descriptor set in base64-encoded format, if the protocol buffer file descriptor set was specified by the user when starting the export tool.
Event structure in the output file
In XML and JSON output files, the Event element contains the following information:
id: event EID (nPublished.getKey());
size: event size (nPublished.getSize());
chanID: event ChannelAttributesId (nPublished.getChannelAttributesId());
ttl: event TTL (nPublished.getTTL());
tag: event tag (nPublished.getTag());
eventData: event data;
persistent: event isPersistent property (nPublished.isPersistant());
transient: event isTransientPropery(nPublished.isTransient());
headerProps: event header attributes;
dictionaryProps: event dictionary properties;
For now, the XML output file event eventData value is base64-encoded.
In the JSON output file, the event element contains an additional property eventDataFormat which can contain the value "Base64" or "GoogleProtobufJson".
In the JSON output file event element eventData, the value can be base64-encoded or can be a JSON node containing protocol buffer data.
XML output file example
<?xml version="1.0" encoding="utf-8"?>
<EventsDetails>
<ExportDetails>
<ToolVersion>1.0</ToolVersion>
<Descriptor>CtwCCgxTY2hvb2wucHJvdG8i7gEKBlBlcnNvbhISCgRuYW1lGAEgAigJUgRuYW1lEg4KAmlkGAIgAigFUgJpZBIUCgVlbWFpbBgDIAEoCVIFZW1haWwSKQoFcGhvbmUYBCADKAsyEy5QZXJzb24uUGhvbmVOdW1iZXJSBXBob25lGlIKC1Bob25lTnVtYmVyEhYKBm51bWJlchgBIAIoCVIGbnVtYmVyEisKBHR5cGUYAiABKA4yES5QZXJzb24uUGhvbmVUeXBlOgRIT01FUgR0eXBlIisKCVBob25lVHlwZRIKCgZNT0JJTEUQABIICgRIT01FEAESCAoEV09SSxACIk4KBlNjaG9vbBIhCgdzdHVkZW50GAEgAygLMgcuUGVyc29uUgdzdHVkZW50EiEKB3RlYWNoZXIYAiACKAsyBy5QZXJzb25SB3RlYWNoZXJCC0IJU2Nob29sU3Vi</Descriptor>
</ExportDetails>
<Events>
<Event>
<id>0</id>
<size>306</size>
<chanID>63565653663178134</chanID>
<ttl>0</ttl>
<tag>tag0</tag>
<eventData>dGVzdGRhdGFib2R5MCBkYXRh</eventData>
<persistent>true</persistent>
<transient>false</transient>
<dictionaryProps>
<item>
<name>string_key</name>
<value>value0</value>
<type>String</type>
</item>
<item>
<name>boolean_key</name>
<value>true</value>
<type>Boolean</type>
</item>
<item>
<name>int_key</name>
<value>0</value>
<type>Integer</type>
</item>
<item>
<name>long_key</name>
<value>0</value>
<type>Long</type>
</item>
<item>
<name>short_key</name>
<value>0</value>
<type>Short</type>
</item>
<item>
<name>byte_key</name>
<value>0</value>
<type>Byte</type>
</item>
<item>
<name>char_key</name>
<value>0</value>
<type>Character</type>
</item>
<item>
<name>byte_arr_key</name>
<value>dGVzdDA=</value>
<type>byte[]</type>
</item>
<item>
<name>float_key</name>
<value>0.0</value>
<type>Float</type>
</item>
<item>
<name>double_key</name>
<value>0.0</value>
<type>Double</type>
</item>
</dictionaryProps>
<headerProps>
<item>
<name>nrvpub.time</name>
<value>1588946429779</value>
<type>Long</type>
</item>
<item>
<name>nrvpub.host</name>
<value>127.0.0.1</value>
<type>String</type>
</item>
<item>
<name>nrvpub.name</name>
<value>ekob</value>
<type>String</type>
</item>
<item>
<name>JMSDeliveryMode</name>
<value>PERSISTENT</value>
<type>String</type>
</item>
<item>
<name>JMSPriority</name>
<value>4</value>
<type>Byte</type>
</item>
</headerProps>
</Event>
</Events>
</EventsDetails>
JSON output file example
[
{
"toolVersion": "1.0",
"descriptor": "CtwCCgxTY2hvb2wucHJvdG8i7gEKBlBlcnNvbhISCgRuYW1lGAEgAigJUgRuYW1lEg4KAmlkGAIgAigFUgJpZBIUCgVlbWFpbBgDIAEoCVIFZW1haWwSKQoFcGhvbmUYBCADKAsyEy5QZXJzb24uUGhvbmVOdW1iZXJSBXBob25lGlIKC1Bob25lTnVtYmVyEhYKBm51bWJlchgBIAIoCVIGbnVtYmVyEisKBHR5cGUYAiABKA4yES5QZXJzb24uUGhvbmVUeXBlOgRIT01FUgR0eXBlIisKCVBob25lVHlwZRIKCgZNT0JJTEUQABIICgRIT01FEAESCAoEV09SSxACIk4KBlNjaG9vbBIhCgdzdHVkZW50GAEgAygLMgcuUGVyc29uUgdzdHVkZW50EiEKB3RlYWNoZXIYAiACKAsyBy5QZXJzb25SB3RlYWNoZXJCC0IJU2Nob29sU3Vi"
},
{
"eventData": {
"encodedData": "dGVzdGRhdGFib2R5MCBkYXRh"
},
"eventDataFormat": "Base64",
"id": 0,
"size": 306,
"chanID": 63565653663178136,
"ttl": 0,
"tag": "tag0",
"isPersistent": true,
"isTransient": false,
"type": 0,
"dictionaryProps": [
{
"name": "string_key",
"value": "value0",
"type": "String"
},
{
"name": "boolean_key",
"value": "true",
"type": "Boolean"
},
{
"name": "int_key",
"value": "0",
"type": "Integer"
},
{
"name": "long_key",
"value": "0",
"type": "Long"
},
{
"name": "short_key",
"value": "0",
"type": "Short"
},
{
"name": "byte_key",
"value": "0",
"type": "Byte"
},
{
"name": "char_key",
"value": "0",
"type": "Character"
},
{
"name": "byte_arr_key",
"value": "dGVzdDA=",
"type": "byte[]"
},
{
"name": "float_key",
"value": "0.0",
"type": "Float"
},
{
"name": "double_key",
"value": "0.0",
"type": "Double"
}
],
"headerProps": [
{
"name": "nrvpub.time",
"value": "1588946429779",
"type": "Long"
},
{
"name": "nrvpub.host",
"value": "127.0.0.1",
"type": "String"
},
{
"name": "nrvpub.name",
"value": "ekob",
"type": "String"
},
{
"name": "JMSDeliveryMode",
"value": "PERSISTENT",
"type": "String"
},
{
"name": "JMSPriority",
"value": "4",
"type": "Byte"
}
]
}
]
Difference between XML and JSON export formats
The JSON output file event element can have eventData as a JSON node and "eventDataFormat" : "GoogleProtobufJson" if events are dumped from the mem file of a protobuf channel and the export tool had the input parameter protobufdescriptor specified.
Here is an example of a JSON event dumped from a protobuf channel with protobufdescription specified :
{
"eventData": {
"student": [
{
"name": "StudentName0",
"id": 1,
"email": "StudentName0@softwareag.com",
"phone": [
{
"number": "19150",
"type": "HOME"
}
]
}
],
"teacher": {
"name": "TeacherName0",
"id": 2,
"email": "TeacherName0@softwareag.com",
"phone": [
{
"number": "77430",
"type": "HOME"
}
]
}
},
"eventDataFormat": "GoogleProtobufJson",
"id": 0,
"size": 215,
"chanID": 39103550381024424,
"ttl": 0,
"tag": "tag0",
"isPersistent": true,
"isTransient": false,
"type": 0,
"headerProps": [
{
"name": "nrvpub.time",
"value": "1588946429899",
"type": "Long"
},
{
"name": "nrvpub.host",
"value": "127.0.0.1",
"type": "String"
},
{
"name": "nrvpub.name",
"value": "ekob",
"type": "String"
},
{
"name": "JMSType",
"value": "School",
"type": "String"
},
{
"name": "JMSDeliveryMode",
"value": "PERSISTENT",
"type": "String"
},
{
"name": "JMSPriority",
"value": "4",
"type": "Byte"
},
{
"name": "JMSMsgType",
"value": "6",
"type": "Integer"
}
]
}
Input parameters
-protobufdescriptor
The -protobufdescriptor input parameter is an optional parameter specifying the path to the protocol buffer file descriptor set for filtering events based on event data.
The protocol buffer file descriptor set can be received as:
protoc.exe <proto_file_name>.proto --descriptor_set_out=<protocol_buffer_file_descriptor_set_name>.fds
If the input parameter protobufdescriptor is specified and the mem file's store is a protobuf channel with the same protobuf descriptor, then event filtering can be done based on the event protocol buffer data. The protocol buffer descriptor will be exported to the output file as a base64-encoded string.
If the input parameter protobufdescriptor was not specified and the mem file's store is a protobuf channel, then event filtering cannot be done based on the event protocol buffer data.
If the input parameter protobufdescriptor is specified but the mem file's store is a protobuf channel with a different protobuf descriptor, then dumping will not be performed.
If the input parameter protobufdescriptor is specified but the mem file's store is not a protobuf channel, then the event will be considered as nPublished events anyway. The protocol buffer descriptor will be exported to the output file as a base64-encoded string.
batchsize
The default value of the batch size is 100. It can be optionally specified when running the tool. It defines the number of events which will be read/loaded to the memory/ from the mem file, filtered and then written to the output file at once.