Monitoring running queries
To help you monitor queries that are running on a given correlator, Apama provides data about active queries in DataViews. To display the information provided by these DataViews, you can create a dashboard in which an end user can:
Monitor query runtime performance.
Determine whether a query is behaving as intended. For example, you can see how incoming events are distributed across partitions. If you are expecting a particular send and match rate, you can see if you are getting the results you expect.
Ensure that the window size (the number of events in the window) is not too large. The expectation is that your application is designed so that partitioning keeps any given window size as small as possible.
The Queries_Statistics_Sample that is provided with Apama (located in the \samples\queries directory of your Apama installation) contains such a dashboard. It shows you how to build a dashboard that allows you to monitor the performance of running queries.
For information about exposing DataViews in dashboards, see
Building Dashboard Clients.
A running query is either a non-parameterized query instance or a parameterization. For each running query, there is a DataView for each of its input event types. For example, if a query instance has two input event types, then there are two DataViews that provide statistics for that query, one for each input event type.
Each DataView:
Contains data about the activity during the last second of one running query and one of its input event types.
Contains the fields described in the table below. The value contained in each field is an exponentially weighted moving average (EWMA).
Is updated every 10 seconds by default if the information has changed since the last update.
By sending a SetQueryStatisticsPeriod event, you can control the frequency of the statistics gathering or disable query statistics entirely. For example, to update the query statistics every second:
com.apama.queries.SetStatisticsUpdatePeriod(1,1)
To disable query statistics entirely:
com.apama.queries.SetStatisticsUpdatePeriod(0,0)
Statistical Field | Description |
% Threads Active EWMA | Apama uses multiple threads to process a given query. This is the percentage of those threads that were used within the last second to process the input event type that this DataView provides information for. While there is not a linear correlation, as this percentage goes down, the reliability of the rest of the statistics becomes weaker. This is because a smaller proportion of threads are contributing information. |
Avg. Inbound Event Rate/s EWMA | The average rate per second at which events of this type are being processed. |
Avg. % of Successful Matches EWMA | The average percentage of the number of received events that cause a match. |
No. Unique Keys Processed EWMA | The number of unique query partitions that were accessed for this event type within the past second. |
Avg. Window Size/Key EWMA | The average window size (number of events that it contains) of each unique partition that was accessed within the past second. |
The display name of these DataViews is Correlator Query Statistics.
After a non-parameterized query is injected into the correlator, Apama provides a DataView for each input event type and begins writing data to it. After a non-parameterized query is deleted, Apama no longer makes the DataViews for that query instance available.
For a parameterized query, after a parameterization is created, then Apama adds new DataViews and begins populating them. When a parameterization is deleted, then Apama no longer provides the DataViews that correspond to that parameterization. If the definition of a parameterized query is deleted, then Apama no longer provides DataViews for any parameterization of that query.
To help you monitor queries that are running across multiple correlators in a cluster, Apama also provides the same type of performance statistics provided for a given correlator but where the underlying data has been aggregated across all the clustered correlators running those queries.
The display name of these DataViews is Cluster-Wide Query Statistics.
This means that for each query running on a correlator, two types of monitoring data are provided:
Statistics generated from data from only that correlator.
Statistics generated from data aggregated across all correlators in the cluster running that same query.