New Apama queries capability
The new Apama queries capability lets business analysts and developers create scalable applications that process events originating from very large populations of real-world entities. Scaling, both vertically (same machine) and horizontally (across multiple machines), is inherent in Apama query applications. Scaled deployments on multiple machines use distributed cache technology to maintain and share application state. This makes it easy to deploy across multiple servers, and keep the application running even if some servers are taken down for maintenance or fail. Apama queries can be used alongside EPL monitors in the same correlator process, interacting by sending events between them.
Incoming events that queries process are partitioned by, for example, customer account numbers, car license plate numbers, devices or some other entity. In a query application, the correlator processes the events in each partition independently of other partitions.
Advantages of Apama queries over Apama monitors:
When used in conjunction with BigMemory, queries provides active-active availability. That is, queries can be run in a cluster, where every node in the cluster contributes processing resources. The number of nodes can be changed dynamically without losing state.
Queries can be run on correlator clusters.
Disadvantages of Apama queries compared to Apama monitors:
Higher latency than monitors. Latency is of the order of milliseconds to seconds rather than microseconds to milliseconds. Exact values depend on the deployment and the types of events being processed.
Apama monitors allow you to write custom and more powerful EPL applications that do not have the declarative and structural bounds that queries have.
To take advantage of the scalability and availability that the queries platform offers, the problem your application needs to solve should meet one or more of the following requirements:
Different partitions for a given query must be completely independent. However, different queries can use different partition keys for the same event types. For example, one query may partition ATM withdrawals by
cardNumber, and another by
atmId.
The average number of events in each event window in a partition is low. The recommendation is less than 50 events. For example, if ATM withdrawals are partitioned by
cardNumber then a window that retains withdrawals for a three-day period is fine because the typical number of withdrawals per card is likely to be low. While it is possible to have hundreds of withdrawals for a single card number, that would be an exceptional case and probably indicative of suspicious behavior.
Other than the history of events, no state is required. Queries do not provide for state to be stored. However, it is possible to mix monitors and queries in the same deployment.
The time between events destined for the same partition is typically long, that is, more than a few seconds between events.
The exact ordering between events is not critical. A query may treat two events for the same partition that occur close in time as having occurred in an order that is different from the order in which they were sent.
Apama queries are designed to be easy to develop for both the business analyst and the application developer. Graphical tools to specify the application design and full round-trip engineering allow both the business analyst and the developer to work on the same queries. At the developer level, an Apama query is defined using the Apama event processing language, EPL. A new channel, com.apama.queries, is provided for sending events to queries. To implement queries, Apama provides:
The new Query Designer editor in Apama Studio. This graphical user interface lets business analysts define queries without the need to write code.
New EPL constructs for application developers to write code that defines queries. A query is defined in a
.qry file. A query cannot contain a monitor and a monitor cannot contain a query. A query can use many, but not all EPL constructs.
Information and instructions for configuring, deploying and managing queries. To scale query deployment across multiple machines, you use Apama's distributed MemoryStore with a JMS bus such as Universal Messaging.