Using heartbeat events with source timestamps
When using source timestamps, if a query's input has no events for a period of time, then the query will wait for the specified wait time for that query before evaluating events. This can cause unacceptable delays in processing events from other inputs. Some data sources may provide heartbeat events with timestamps which signal that communication from the data source to the queries system is working correctly. If these events occur but no input events have been received, then the query can infer that no input events, or only the input events received, have occurred, and thus the query's input is definitive upon receiving a heartbeat, without having to wait any further. If communication is disrupted or delayed, then the heartbeat events will similarly be delayed, and the query will wait, as it has to in order to process delayed events.
Heartbeat events are specified on the input event type's definition or per input of the query. They are only used if a query input is using source timestamps, that is, it has a wait clause specified. The heartbeat can be specified as a @Heartbeat annotation on the event definition, which should name the fully qualified event type to use as heartbeat events.
If a query input contains a time from clause, then the heartbeat must be explicitly named with an or heartbeat-type clause after the wait clause. For example, these two are equivalent:
@TimeFrom("getEcuTime")
@Heartbeat("CarHeartbeat")
event CarEngineTemp { .. }
...
query ... {
inputs {
CarEngineTemp() key carId within 1 hour wait 6 hours;
}
...
or:
query ... {
inputs {
CarEngineTemp() key carId within 1 hour time from getEcuTime wait 6 hours or CarHeartbeat;
}
...
The following rules apply for the heartbeat event:
The heartbeat event cannot be filtered.
The heartbeat event must share the same key fields and the same types as the input event type. In the above example, both
CarEngineTemp and
CarHeartbeat must have a field named
carId which is of the same type in each event type.
The heartbeat event must have a matching action for obtaining the source time. In the above example, both
CarEngineTemp and
CarHeartbeat must have an action of the signature
action getEcuTime() returns float. Typically, these would have the same implementation, as the heartbeat would have source timestamps in the same form as the input events; but the implementation of these methods may be different for heartbeat events (see
Out of order events.)
The heartbeat event cannot be used as an input in the pattern, unless it is also listed as an input event in its own right.
The same heartbeat event type may be used for different inputs of the same query (this is typical, as a query may use a number of different types of events from the same data source, such as a car in the above example).
When a heartbeat event is received and processed, it will step forward the definitive time for all inputs that specify that heartbeat event. Thus, if all inputs use the same heartbeat event, then that heartbeat can step forward the definitive time, allowing the query to evaluate events received on some inputs without having to wait for the input wait time on other inputs where no input events were received.
Typically, heartbeat events will be delivered regularly. The rate at which heartbeat events are sent is dependent on the data source, but the queries system must be able to handle all of the heartbeat events from all data sources as well as the input events. Some devices may only send the heartbeats under certain conditions, for example, a car may only send heartbeats if the engine is running or the car is occupied. If no heartbeat events are received, then queries will use the wait time specified in the input before evaluating any events received, as needed.
Note that queries assume that the heartbeat events are delivered in the same order as input events. If an input event arrives with a timestamp before a previous heartbeat event, it will be discarded.
Typically, heartbeat events will be events that come from the same data source as the input events they are used with. Thus, any communications disruption affecting the input events will affect the heartbeat events in the same way. This is not a requirement; if some other system has knowledge of when a data source is connected or disconnected, the heartbeat events could be sent from that system - but if the system incorrectly sends heartbeat events and input events are delayed, then input events may be discarded.