Universal Messaging - Advanced Filtering with Selectors
Universal Messaging supports standard selector based filtering and some advanced filtering concepts which will be described here .
Content Sensitive Filtering
Each Universal Messaging event can contain an event dictionary and a tag, as well as a byte array of data. Standard filtering, as defined by JMS, allows dictionary entries to be evaluated based on the value of the dictionary keys prior to delivering the data to the consumer.
Universal Messaging also supports a more advanced form of filtering based on the content of the event data (byte array) itself as well as time and location sensitive filtering. Universal Messaging also supports filtering based on arrays and dictionaries contained within event dictionaries. There is no limit to the dept of nested properties that can be filtered.
Filtering based on nested arrays and dictionaries
An event dictionary can contain primitive types as well as dictionaries. They can also contain arrays of primitive types and arrays of dictionaries. Universal Messaging supports the ability to filter based on these nested arrays and dictionaries.
if an nEventProperties object contains a key called NAMES which stores a String[] then it is possible to specify a filter that will only deliver events that match based on values within the array.
NAMES[] = 'myname'
- Returns events where any element in the NAMES array = 'myname'
NAMES[1] = 'myname'
- Returns events where the second element in the array = 'myname'
Similarly, if the array was an nEventProperties[] it would be possible to filter based on the values within the individual nEventProperties objects contained within the array.
For example if the event's nEventProperties contains a key called CONTACTS which stores an nEventProperties[] then the following selectors will be available.
CONTACTS[].name = 'aname'
- Return events where the CONTACTS array contains an nEventProperties which contains a key called name with the value 'aname'
CONTACTS[1].name = 'aname'
- Return events where the second element in the CONTACTS array of nEventProperties contains a key called name with the value 'aname'
CONTACTS[].NAMES[] = 'myname'
- Return events where the CONTACTS array contains a NAMES arrays with a value 'myname' somewhere in the NAMES array.
EventData Byte[] Filtering
Universal Messaging's filtering syntax supports a keyword called 'EVENTDATA' that corresponds to the actual byte array of data within the Universal Messaging event. There are a number of operations that can be performed on the event data using this keyword.
This enables a reduction in the amount of data you wish to send to clients, since rather than querying pre-determined dictionary values, you can now query the actual data portion of the event itself without having to provide dictionary entries. If you have a message structure and part of this structure includes the length of each value within the structure, then you can refer to each portion of data. Alternatively if you know the location of data within you byte array these can be used for filtering quite easily.
Below is a list of the available operations that can be performed on the EVENTDATA.
EVENTDATA.LENGTH()
- Returns the length of the byte[] of the data in the event.
EVENTDATA.AS-BYTE(offset)
- Returns the byte value found within the data at the specified offset.
EVENTDATA.AS-SHORT(offset)
- Returns a short value found within the data at the specified offset. Length of the data is 2 bytes.
EVENTDATA.AS-INT(offset)
- Returns a int value found within the data at the specified offset. Length of the data is 4 bytes.
EVENTDATA.AS-LONG(offset)
- Returns a long value found within the data at the specified offset. Length of the data is 8bytes.
EVENTDATA.AS-FLOAT(offset)
- Returns a float value found within the data at the specified offset. Length of the data is 4 bytes.
EVENTDATA.AS-DOUBLE(offset)
- Returns a double value found within the data at the specified offset. Length of the data is 8 bytes.
EVENTDATA.AS-STRING(offset, len)
- Returns a String value found within the data at the specified offset for the length specified.
EVENTDATA.TAG()
- Returns the String TAG of the event if it has one.
For example, we could then provide a filter string in the form of :
EVENTDATA.AS-STRING(0, 2) = 'UK'
If we knew that at position 0, the first 2 bytes would be a string that represents a value you wish to filter on.
If we had data with 5 string values of varying length, and each length was prepended to each string in 2 bytes, then we could evaluate any portion of the string as follows:
EVENTDATA.AS-STRING(0, EVENTDATA.AS-INT(0)) LIKE 'LON'
and the second string value after that would be calculated as follows:
EVENTDATA.AS-STRING( EVENTDATA.AS-SHORT(0)+4, EVENTDATA-AS-SHORT(EVENTDATA.AS-SHORT(0)+2) )
The offset is calculated based on the length of the first string + the 2 bytes of the first strings size and 2 bytes for the size of the second string (Hence +4). This offset gives you the size of the second string. Then you just need to get size of the second string, which is found by EVENTDATA-AS-SHORT(EVENTDATA.AS-SHORT(0)+2).
This provides a powerful way of embedding functions within functions in order to evaluate the data within an event.
Time Sensitive Filtering
Universal Messaging's filtering syntax also supports a function called 'NOW()' that is evaluated at the server as the current time in milliseconds using the standard Java time epoch. This function enables you to filter events from the server using a time sensitive approach. For example, if your data contained a dictionary key element called 'DATE_SOLD' that contained a millisecond value representing the data when an item was sold, one could provide a filter string on a subscription in the form of:
DATA_SOLD < (NOW() - 86400000)
Which would deliver events corresponding to items sold in the last 24 hours. This is a powerful addition to the filtering engine within Universal Messaging.
Location Sensitive Filtering
Universal Messaging's filtering engine supports a keyword called DISTANCE. This keyword is used to provide geographically sensitive filtering. This allows the calculation of the distance between two points on the earths surface as defined by the latitude and longitude.
For example, if you were designing a system that tracked the location of a tornado, as the tornado moved position, the latitude and longitude would correspond to the geographical location on the earth's surface. As the position of a tornado changed, an event would be published containing the new latitude and longitude values as keys within the dictionary ('latitude' and 'longitude' respectively). Using this premise, you could provide a filter in the form of:
DISTANCE(Lat, Long, Units)
where :
Latitude : the floating point value representing the latitude
Longitude : the floating point value representing the longitude
Units : Optional string indicating the return value to be
K: Kilometers < Default >
M : Miles
N : Nautical Miles
For example :
DISTANCE ( 51.50, 0.16, M ) < 100
Which would deliver events that corresponded to tornadoes that were less than 100 miles away for the latitude and longitude values provided in the filter string.
The DISTANCE keyword provides a valuable and powerful extension to Universal Messaging's filtering capabilities. If you require information that is sensitive to geographical locations
