The persistent store is used for storing unit-of-work messages to disk. This means message and status information can be recovered after a hardware or software failure to the previous commit point issued by each application component.
Under z/OS, the broker persistent store can be implemented with:
the Adabas database of Software AG
a VSAM linear data set (LDS) accessed using Data In Virtual (DIV)
This document covers the following topics:
See also Concepts of Persistent Messaging.
EntireX provides an Adabas persistent driver. This enables Broker unit of work (UOW) messages and their status to be stored in an Adabas file. It is designed to work with Adabas databases under z/OS, Linux, Windows and BS2000, and can be used where the database resides on a different machine to the Broker kernel. For performance reasons, we recommend using EntireX Broker on the same machine as the Adabas database.
The Adabas persistent store driver module is contained within
the regular Broker load library or binaries directory. Module ADAPSI is
activated by specifying the PSTORE-TYPE
parameter. Use the supplied job EXBJ015 from data set
EXX109.JOBS to define and install the persistent
store file in your Adabas database. This job creates and loads the Adabas file into the database.
If a HOT start is performed, the Broker kernel must be executed on the same platform on which also the previous Broker executed. This is because some portions of the persistent data are stored in the native character set and format of the Broker kernel. It is also necessary to start Broker with the same Broker ID as the previous Broker executed.
If a COLD start is executed, a check is made to ensure the Broker ID and
platform information found in the persistent store file is consistent with the
Broker being started (provided the persistent store file is not empty). This is
done to prevent accidental deletion of data in the persistent store by a
different Broker ID. If you intend to COLD start Broker and to utilize a
persistent store file which has been used previously by a different Broker ID,
you must supply the additional PSTORE-TYPE parameter FORCE-COLD=Y.
Perform regular backup operations on your Adabas database. The persistent store driver writes C1 checkpoint records at each start up and shut down of Broker.
Significant performance improvements can be achieved using Adabas/Fastpath where available. See Adabas/Fastpath documentation for details of installation and configuration of Adabas/Fastpath.
For performance reasons, execute Broker on the same machine as Adabas.
During startup, Broker writes the following C1 checkpoint records to
the Adabas database. The time, date and job name are recorded in the Adabas
checkpoint log. This enables Adabas protection logs to be coordinated with
Broker executions. This information can be read from Adabas, using the ADAREP utility with option
CPLIST:
| Broker Execution Name | Broker Execution Type | Adabas |
|---|---|---|
| ETBC | Broker Cold Start | Normal Cold Start |
| ETBH | Broker Hot Start | Normal Hot Start |
| ETBT | Broker Termination | Normal Termination |
Caution should be exercised when defining the block size
(BLKSIZE) parameter for the Adabas persistent store.
This determines how much UOW message data can be stored within a single Adabas
record. Therefore, do not define a much larger block size than the size of the
maximum unit of work being processed by Broker. (Remember to add 41 bytes for
each message in the unit of work.) The advantage of having a good fit between
the unit of work and the block size is that fewer records are required for each I/O operation.
It is necessary to consider the following Adabas parameters and settings when using Adabas for the persistent store file:
| Topic | Description | ||
|---|---|---|---|
| Allowing Sufficient Adabas UQ Elements | Allow sufficient Adabas user queue (UQ) elements each time you start Broker.
The Broker utilizes a number of user queue elements equal to the
number of worker tasks (NUM-WORKER), plus two.
Adabas timeout parameter (TNAE) determines how long the
user queue elements will remain. This can be important if Broker is restarted
after an abnormal termination, and provision must be made for sufficient user
queue elements in the event of restarting Broker.
Sample JCL SAGJ014 is provided in data set EXX109.JOBS for z/OS to enforce clean-up of any user queue element belonging to the previous Broker job. This JCL can be inserted into the job step before starting up Broker. |
||
| Setting Size of Hold Queue Parameters | Consideration must be given to the Adabas hold queue parameters
NISNHQ and NH. These must be
sufficiently large to allow Adabas to add/update/delete the actual number of
records within a single unit of work.
Example: where there are 100 message within a unit of work and the
average message size is 10,000 bytes, the total unit of work size is 1 MB. If,
for example, a 2 KB block size (default |
||
| Setting Adabas TT Parameter | Consideration must be given to the Adabas transaction time
(TT) parameter for cases where a large number of records
is being updated within a single unit of work.
|
||
| Defining LWP Size | Sufficient logical work pool (LWP) size
must be defined so that the Adabas persistent store can update and commit the
units of work. Adabas must be able to accommodate this in addition to any other
processing for which it is used.
|
||
| Executing Broker Kernel and Adabas Nucleus on Separate Machines | If Broker kernel is executed on a separate machine to the
Adabas nucleus, with a different architecture and codepage, then we recommend
running the Adabas nucleus with the UEC (universal
conversion) option in order to ensure that Adabas C1 checkpoints are legible
within the Adabas checkpoint log.
|
||
Setting INDEXCOMPRESSION=YES |
This Adabas option can be applied to the Adabas file to reduce by approximately 50% the amount of space consumed in the indexes. This is the default setting in job EXBJ015, which is supplied in data set EXX109.JOBS to define the Adabas persistent store file. | ||
| 4-byte ISNs | If you anticipate having more than 16 million records within the persistent store file, you must use 4-byte ISNs when defining the Adabas file for EntireX. | ||
| Specification of Adabas LP Parameter |
If this is not large enough, Broker will detect an error (response 9; subresponse - 4 bytes - X'0003',C'LP') and Broker will not be able to write any further UOWs. See the description of the |
To calculate the Adabas file size it is necessary to estimate the number of records being stored. As an approximate guide, there will be one Adabas record (500 bytes) for each unprocessed unit of work, plus also n records containing the actual message data, which depends on the logical block size and the size of the unit of work. In addition, there will be one single record (500 bytes) for each unit of work having a persisted status.
Always allow ample space for the Adabas persistent store file since the continuous operation of Broker relies of the availability of this file to store and retrieve information.
Note:
If the Adabas file space is exceeded, no new units of work will be accepted.
In this example there are 100,000 Active UOW records at any one time. Each of these is associated with two message records containing the message data. UOW records are 500 bytes in length. Each message record contains 2,000 bytes. In addition, there are 500,000 UOW status records residing in the persistent store, for which the UOW has already been completely processed. These are 500 bytes long.
Note:
The actual size of the data stored within the UOW message records is
the sum of all the messages within the UOW, plus a 41-byte header for each
message. Therefore, if the average message length is 59 bytes, the two 2,000
bytes, messages records, could contain n = 4,000 / (59+41), or 40 messages.
Adabas is assumed to compress the message data by 50% in the example (this can
vary according to the nature of the message data).
3-byte ISNs and RABNs are assumed in this example. A device type of 8393 is used; therefore, the ASSO block size is 4,096, and DATA block size is 27,644. Padding factor of 10% is specified.
The following example calculates the space needed for Normal Index (NI), Upper Index (UI), Address Converter (AC) and Data Storage (DS).
| Calculation Factors | Required Space |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Entity Requiring Space | Total Required Space |
| Normal Index (NI) | = 23,868 blocks |
| Upper Index (UI) | = 512 blocks |
| Data Storage (DS) | = 14,068 blocks |
| Address Converter (AC) | = 587 blocks |
Entire Net-Work
If you intend to use Adabas persistent store through Entire
Net-Work, see the Entire Net-Work documentation for installation and configuration details.
Adabas Versions
Adabas persistent store can be used on all Adabas versions currently
released and supported by Software AG.
Prerequisite Versions of Entire Net-Work with Adabas
See the Adabas and Entire Net-Work documentation to determine
prerequisite versions of Entire Net-Work to use with Adabas at your site.
Note:
From EntireX version 9.7, broker attribute PSTORE-VERSION
must be set to 4 for a persistent store
of type DIV. If you were using a lower version, you will need to perform a cold start.
See PSTORE=COLD under Broker-specific Broker Attributes.
This section covers the following topics:
The persistent store is implemented as a VSAM linear data set (LDS) accessed using Data In Virtual (DIV). This DIV persistent store is a container for units of work.
DIV is an access method that utilizes the system paging facilities for fast I/O to and from an LDS. Performance is best if the LDS is placed on the fastest storage device available such as those used for paging. An LDS may span multiple volumes.
The DIV persistent store has an internal structure that is formatted by
EntireX Broker during a COLD start (see broker attribute PSTORE). This
format is controlled by format parameter statements that the Broker reads from
the attribute file. See DIV-specific Attributes.
Persistent store data sets are maintained using the IBM z/OS utility
IDCAMS. See Operations using IDCAMS for more information.
Note:
This method of specifying persistent store format parameters is deprecated, but still supported for compatibility reasons.
We recommend you use the attributes introduced with EntireX version 9.12 instead.
See DIV-specific Attributes.
The persistent store format parameters define how a persistent store is
formatted during a cold start operation and how it is accessed during all
operations. These parameters are supplied in the attribute file section
DEFAULTS=DIV.
Knowledge of the application usage of units of work (UOWs) will be very helpful
in selecting appropriate values for the parameters used to define the
persistent store.
The persistent store format parameters file must begin with the word
DEFINE, followed by "parameter name parameter value"
specifications. Each parameter name must be separated from the parameter value
by whitespace (blanks, tabs, or new lines). Comments may be added to the file.
A comment begins with /* and ends with */, just as in the C language. The
parameters must be entered in uppercase. In the following parameter
descriptions, lowercase is used to denote variables:
DEFINE STORE name
DDNAME ddname
DATASPACE NAME name-of-dataspace
DATASPACE PAGES count-of-pages
| Parameter | Value | Opt/ Req |
Description |
|---|---|---|---|
STORE |
A8 | R | Defines an internal name that is used to identify the persistent store. |
DDNAME |
A8 | R | Defines the JCL DDNAME that will be used to access the persistent store.
|
DATASPACE NAME |
A8 | O | Defines the name of the data space that will be used to map the persistent store.
Default value= |
DATASPACE PAGES |
126-524284 | O | Defines the size of the dataspace used to map the persistent store (size=DATASPACE-PAGES * 4 KB).
We recommend using the maximum value.
Default value is 2048. |
DEFAULTS = DIV DIV = "DEFINE STORE PSD01 DDNAME STORE01"
This section covers the following topics:
The following IDCAMS statement can be used to allocate the persistent store. It assumes that the local environment is using z/OS SMS for data management. SMS allows for simple definition, but it may not be used at your site or it may not provide optimal performance. You may therefore need to modify the following statement under the direction of your local system administrator:
DEFINE CLUSTER (NAME(dsn_pstore) - MEGABYTES(15,5) - SHAREOPTIONS(1,3) - /*this is required*/ LINEAR) /*this is required*/
where dsn_pstore is the DSNAME
you chose for the PSTORE linear data set (LDS).
Note:
The size of the linear data set (LDS) should be 16K times the value
specified for DATASPACE PAGES. For example, 16K * 2048 =
32768K = 32M would be the LDS size needed to contain the default number of
pages. Less than the required amount will cause initialization to fail; more will be unused space.
The following statement lists the catalog information for the linear data set:
LISTCAT ENTRIES(dsn_pstore) ALL
where dsn_pstore is the DSNAME
you chose for the PSTORE linear data set (LDS).
The following statement prints the contents of a persistent store in dump format:
PRINT IDS(dsn_pstore) DUMP
REPRO IDS(dsn_pstore) ODS(your_backup_name)
Sample JCL is provided as member IDCAMS in the installation source
library. Each operation is contained in a separate DD. To select an operation,
modify the SYSIN DD DDNAME= to the name of the DD enclosing the statements to be selected.