Filter early to minimize resource usage
To minimize processing and memory overhead it is preferable to filter streams as early as possible in the processing chain or network. Filtering early can reduce the number of items processed or retained in memory and can also reduce the size of the items held. If possible, filter items right at the beginning of the query chain, that is, in the event template.
For example, it is preferable to rewrite this query:
from l in all LargeEvent()
within largeWindowPeriod
where l.key = key
select mean(l.value);
If the key is static, rewrite it this way:
from l in all LargeEvent(key=key)
within largeWindowPeriod
select mean(l.value);
If the key is dynamic, rewrite it this way:
from v in
from l in all LargeEvent()
where l.key = key select l.value
within largeWindowPeriod select mean(v);
In the static case, the correlator filters the large event before the event gets to the window. In the dynamic case, the embedded query filters the event before the event gets to the window in the enclosing query. Because the select statement specifies only l.value, the correlator discards the rest of the event. There is no need to bring the whole event into the window.
Contact Support
|
Community
|
Feedback