Matching only the latest event for a given field
A query input can optionally limit the window to only contain the most recent item for each value of a given field or action of the event. This is performed by the with unique operator, which is followed by one or more fields or actions of the input event type.
For example, consider a query looking at sensor data from a number of sensors on the same production line, with events that specify the productionLine and sensorId. The query compares sensor values between different machines and sensors on the same production line, so the query can be keyed on the productionLine field of events, but not on the sensorId field. However, only the latest event for each sensor is required. By specifying a with unique sensorId clause, only the latest value of each sensor is used.
If you add a with unique clause, if there is more than one item in the window that has the same value for all the fields or actions listed in the with unique clause, then only the most recent event is considered to be in the window and can match the pattern. The suppression of duplicates occurs after the within and/or retain clauses apply. For example:
inputs {
Sensor() key productionLine retain 3 with unique sensorId;
}
Given the following events, the window contains only those marked in the third column of the following table (assuming all are for the same productionLine):
Event | sensorId | Window contains | Notes |
1 | A | 1(A) | |
2 | B | 1(A), 2(B) | |
3 | C | 1(A), 2(B), 3(C) | |
4 | B | 3(C), 4(B) | Event 1 is discarded due to retain 3. Event 2 is discarded as event 4 has the same sensorId. |
5 | D | 3(C), 4(B), 5(D) | |
6 | C | 4(B), 5(D), 6(C) | Event 3 is discarded due to retain 3. |
7 | D | 6(C), 7(D) | Event 4 is discarded due to retain 3. Event 5 is discarded as event 7 has the same sensorId. |
Note that the
with unique is applied after the
retain expression. Any
with unique expression does not affect window sharing (see also
Queries can share windows) nor how much data is stored.
The with unique clause comes after the sizing of the window (within, retain) and before, if present, the time from, wait or or clauses used for specifying source time.
with unique can list a number of comma-separated members or calls to actions, where the action name is followed by parentheses. Actions used in a with unique clause must take no parameters and return a value. The ordering is unimportant.
For example, using with unique upperName() for an event definition such as the following would only keep one event for each value of the name field, ignoring case:
event E {
string name;
action upperName() returns string { returns name.toUpper(); }
}
Contact Support
|
Community
|
Feedback