Concepts of Persistent Messaging

This document provides a brief introduction to the concepts of the persistent store and its role in EntireX for providing persistent messaging within the client/server model. It covers the following topics:

The table Persistent Store Drivers lists the implementation choices available to each operating system for accessing the physical persistent store. See also Using Persistence and Units of Work, Broker UOW Status Transition and Managing the Broker Persistent Store under z/OS | Linux | Windows | BS2000.


Client Server Model: Persistent Messaging

EntireX provides persistent messaging within the client/server model. This is achieved by storing all persistent messages on disk so that if a system failure occurs, messages will automatically be recovered allowing applications to be restarted without any loss of data. The section Using Persistence and Units of Work describes implementation issues and how to use persistence and units of work in EntireX Broker. Units of work can also be used without persistence; units of work which are the vehicle for persistent messaging.

The following figure illustrates the concept of persistent messages.

graphics/persist.png

Persistence in an EntireX Broker's unit of work (a group of logically related messages) has the following four variations:

  • Both the unit of work and its status have persistence.

  • The unit of work does not have persistence, but its status does.

  • The unit of work has persistence, but its status does not.

  • Neither the unit of work nor its status has persistence.

The status of a message is information about the message rather than the actual message data itself. This enables the sender to determine the progress of the message and determine if it has been received by the partner and whether processing was successfully completed. This gives applications the option of having the Broker kernel store only the message status and not the message itself, provided the application has been written to resend data from a known point in the event of system failure. This option can afford significant performance benefits over storing the whole message data.

To support transaction control in a coordinated operation of distributed systems, EntireX can group logically related messages into "units of work" that are committed to the EntireX Broker for further transmission when complete. In case of failure on the server side, the receiving program can backout the whole unit of work; this makes it available for processing later or by another server.

Definitions of Persistent Messaging Terms

UOW

A unit of work (UOW) is a set of one or more messages that are processed as a single unit. The sender of a UOW adds messages to the UOW and then indicates that the UOW is complete (COMMIT). The UOW and its messages are not visible to the receiver until the sender has committed the UOW. Once the UOW is committed, the receiver can receive the messages, and can indicate when the UOW is complete (COMMIT).

Persistent Store

The persistent store is used for storing unit-of-work messages to disk. This means message and status information can be recovered after a hardware or software failure to the previous commit point issued by each application component.

Persistent Store Drivers

A persistent store driver is an executable, or a load module, that implements access to the physical persistent store. There is one persistent store driver for each persistent store type. The following table shows the persistent store options:

Persistent Store Type Description Operating System Notes
Adabas Uses Adabas database. z/OS, Linux, Windows, BS2000 Adabas, Software AG's ADAptable dataBASe, is a high-performance, multithreaded, database management system.
DIV Uses IBM Data In Virtual facility on z/OS. z/OS This persistent store option is implemented as a VSAM linear data set.
CTREE c-tree© is an embedded local database that can be used as your persistent store. Linux and Windows c-tree© is the fast and reliable embedded database of FairCom Corporation®.

See also Managing the Broker Persistent Store under z/OS | Linux | Windows | BS2000 and also PSTORE-TYPE under Broker-specific Broker Attributes.

UOW Lifetime

Each UOW has a lifetime value associated with it. This is the period of time that the UOW is allowed to exist without being completed. This time starts when the UOW is initially created and runs until the UOW is completed. A UOW is completed when it is successfully:

  • cancelled or backed out by its sender, or

  • cancelled or committed by its receiver.

If the UOW is in ACCEPTED status when this lifetime expires, the UOW is placed into a TIMEOUT status. Lifetime timeouts will not occur when the UOW is in either RECEIVED or DELIVERED status.

A special "pseudo-clock" is maintained for UOW lifetimes. This clock is implemented in such a way that it only runs when the Broker is active. This prevents a UOW lifetime from expiring while the Broker is down or otherwise unavailable.

Persistent UOW

Persistence is an attribute of a UOW (unit of work). If a UOW is persistent, its messages are saved in the persistent store when the sender COMMITs the UOW and they are retained until the receiver COMMITs or CANCELs the UOW, or until its lifetime expires. If the Broker or system should fail after the UOW is committed by the sender, the UOW (and its conversation) will be restored to their last, stable status when the Broker restarts.

Persistent Status

Persistent status is an attribute of a UOW (unit of work). If a UOW has persistent status, the status of the UOW is maintained in the persistent store, and is updated whenever the status changes. The persistent status remains in the persistent store after the UOW is completed, until its status lifetime has expired.

A persistent status value represents a multiple of the UOW lifetime value. Thus if a UOW has a lifetime of 5M (whereby M stands for minutes) and a persistent status value of 4, the status of the UOW would be deleted 20M (5M*4) after the UOW was completed.

Availability of Persistent Store

Warning:
The persistent store must be available before you attempt to start or restart the Broker; otherwise your Broker will not initialize.

Introduction

The PSTORE must be available for the Broker to start. Subsequently, Broker will continue to function even if the PSTORE becomes unavailable and applications issuing non-persistent commands will continue without interruption. However, Broker will not be able to process commands relating to persistence until the PSTORE becomes available again.

Users issuing commands involving persistence - for example units of work with persistence - are notified of the unavailability of the PSTORE through an ACI return code. In addition, persistent commands being processed at the point of unavailability are backed out, and details of the PSTORE problem are written to the Broker log file.

There are several reasons for the PSTORE becoming unavailable. Examples:

  • unavailability of the PSTORE file(s)

  • capacity of PSTORE file being exceeded

  • in the case of Adabas, termination of the database

Disconnect the Persistent Store

You can remove the state "No new Units of Work" - that is, no new persistent data - using CIS. If the PSTORE capacity is exceeded, an error message is written to the Broker log file. You must use Command and Information Services (CIS) to ensure that users cannot issue further commands creating new units of work.

During the time the PSTORE is unavailable, there is no timeout processing for unit-of-work records kept in the PSTORE. In addition, some in-memory resources relating to persistent items, such as conversation control blocks, are also retained until the PSTORE becomes available again and normal processing is resumed for all commands.

See executable command request DISCONNECT-PSTORE under ETBCMD: Executable Command Requests.

Connect the Persistent Store

Subsequently, you can use CIS to make the PSTORE available again, allowing users only to issue commands consuming records from the PSTORE rather than producing new ones. After a period of operation in this state, the contents of the PSTORE will be reduced sufficiently, and you can remove the state "No new Units of Work" through CIS.

See executable command request CONNECT-PSTORE under ETBCMD: Executable Command Requests.

Persistent Store Report

You can create an optional report file that provides details about all records added to or deleted from the persistent store. This section details how to create the report and provides a sample report.

Configuration

To create a persistent store report, use Broker's global attribute PSTORE-REPORT with the value YES.

When the attribute value YES is supplied, all created or deleted persistent records will be reported if the output mechanism is available.

If the value NO is specified, no report will be created.

The report file is created using the following rules:

BS2000

LINK-NAME ETBPREP assigns the report file. Format REC-FORM=V, REC-SIZE=0, FILE-TYPE ISAM is used by default.

Linux

Broker creates a file with the name PSTORE.REPORT in the current working directory.

If the environment variable ETB_PSTORE_REPORT is supplied, the file name specified in the environment variable will be used.

If Broker receives the command-line argument -p, the token following argument -p will be used as the file name.

Windows

Same as Linux.

z/OS

DDNAME ETBPREP assigns the report file. Format RECFM=FB, LRECL=121 is used.

Sample Report

The following is an excerpt from a sample PSTORE report.

EntireX 10.9     PSTORE Report      2016-10-18 10:46:18    Page     1

Identifier        Elements  Total length  Record Type   Date       Time          Action     Remaining Postpone Attempts
0000000000000000        1           760   Master        2016-10-18 10:46:18.529  Created
0010000000000001        1          5022   Conversation  2016-10-18 10:46:36.216  Created
0010000000000002        1          5022   Conversation  2016-10-18 10:46:36.216  Created
0010000000000003        1          5022   Conversation  2016-10-18 10:46:36.217  Created
0010000000000001                          Conversation  2016-10-18 10:46:48.036  Postponed            6
0010000000000001                          Conversation  2016-10-18 10:47:18.470  Accepted             6
0010000000000002                          Conversation  2016-10-18 10:47:57.856  Postponed            6
0010000000000002                          Conversation  2016-10-18 10:48:27.476  Accepted             6
0010000000000003                          Conversation  2016-10-18 10:48:27.477  Postponed            6
0010000000000003                          Conversation  2016-10-18 10:48:57.480  Accepted             6
0010000000000003                          Conversation  2016-10-18 10:48:57.481  Postponed            5
0010000000000003                          Conversation  2016-10-18 10:49:27.484  Accepted             5
0010000000000001                          Conversation  2016-10-18 10:52:25.505  Deleted
0010000000000002                          Conversation  2016-10-18 10:52:25.506  Deleted
0010000000000003                          Conversation  2016-10-18 10:53:26.508  Deleted

The following fields are provided in the report:

  • Identifier provides the UOWID (record ID).

  • Elements gives the number of messages per UOW when creating or loading records.

  • Total Length gives the size of the raw record when creating or loading records.

  • Record Type describes the type of the data. Following types are currently supported:

    • Cluster: a special record for synchronization purposes

    • Conversation: a unit of work as part of a conversation

    • Master: a special record to manage the persistent store

  • Date and time of the action

  • Action describes the action of Broker. The following actions are currently supported:

    • Accepted: UOW status was changed from POSTPONED to ACCEPTED

    • Created: record is created

    • Deleted: record is deleted

    • Postponed: UOW status was changed from DELIVERED to POSTPONED

  • Remaining postpone attempts.