Polling Coordination
When a polling notification is started, it executes polls on an interval according to its schedule configuration. When polling is coordinated across a cluster, a poll is executed on only one of the active instances at each scheduled interval. Which instance executes depends on configuration and timing.
Integration Server supports a coordination mode configuration setting for each polling notification node. This configuration is normally set from Integration Server Administrator on the same page where the schedule is set. The following coordination mode values are supported:
Coordination Mode Setting | Description |
Disabled | Disables all cluster coordination for this node (both polling and callback coordination). Instances of this node will act entirely independently with no consideration for the cluster. |
Standby | The first instance of this notification to start will execute all polls until that instance either shuts down, or fails. When that occurs, the first active notification instance that detects that the original instance is no longer polling will take its place. |
Distributed | Behaves much like standby, except that at the end of each interval, the instance that first detects that the time to poll has arrived will execute the poll. When the server clocks are properly synchronized, generally the server with the lightest load will execute the poll. |
Starting with Integration Server 8.0, the coordination information for clustered polling notifications is stored in the cluster’s shared cache.
All coordination of clustered polling notifications is done through an entry specific to the node in the cluster's shared cache. When the first instance of a notification (with coordination enabled) is introduced into a cluster, a new shared cache session is created and populated with the name of the server hosting the node, and the state of the node. As that node is copied to other servers in the cluster, each instance is registered in the same shared cache session. The states of these respective instances are used to determine when coordinated callbacks should be executed.
Important:
Always copy polling notification nodes instead of creating new nodes with the same name and configuration. Each polling notification node is created with a GUID that forms part of the message ID of all documents published by the notification. If the instances do not have the same GUID, it can interfere with duplicate-message detection facilities.
When the first instance of a clustered polling notification is started, that instance is marked as "primary", its schedule and coordination settings are recorded, and the time it calculates for the next poll are all recorded in the shared cache session. Being "primary" means that first instance will execute the first poll (barring failures). When it schedules the next poll, it will release the "primary" status if coordination is distributed, or retain it if coordination is configured for standby mode.
When another instance of the notification starts, it first detects that a previous instance was already started. Since it is not first, it overwrites its own cluster and schedule settings with those recorded in the shared cache. It then schedules itself to "wake up" at the next scheduled poll time. When it wakes up, if another instance is marked as primary, the notification instance will verify that the indicated instance is still active, then reschedule itself to wake up periodically until it detects that the poll was completed within the configured time limit. It then reschedules itself to wake up at the next scheduled poll and repeats the process. If the polling time arrives and no instance is marked as primary, or it detects that the primary instance is no longer functioning, then the local instance assumes the primary role and executes the poll as described above. Since all coordination is based on timestamps recorded in the shared cache, it is very important for server clocks to be synchronized.