Installing Entire Transaction Propagator on BS2000/OSD

This document describes the steps for installing the Entire Transaction Propagator (product code ETP) on BS2000/OSD.

Related Topic:

For information on the features and functions provided by the Entire Transaction Propagator, see the Entire Transaction Propagator documentation.

Notation vrs or vr:

When used in this document, the notation vrs or vr represents the relevant product version (see also Version in the Glossary).


Entire Transaction Propagator Version 1.5.2 Compatibility with Earlier Releases

It is essential that the ETPNUC module and the Natural programs for Entire Transaction Propagator in the Natural system library SYSETP have the same version. When using a Version 1.5.2 ETPNUC module with Entire Transaction Propagator Version 1.4.1 Natural programs - or vice versa - can cause unpredictable results.

The Natural profile parameter ETPSIZE (see the Parameter Reference documentation) is still accepted by Natural, but it is no longer necessary to specify this parameter. The required storage (approx. 10 KB) will be automatically allocated when the first call to an Entire Transaction Propagator database is issued. Any value specified for the ETPSIZE parameter will be ignored. The size of the ETPSIZE parameter should be left at its default (ETPSIZE=0).

As the Natural profile parameter DATSIZE (see the Parameter Reference documentation) is automatically adjusted to the required length, it is no longer necessary to adjust the value of the Natural DATSIZE parameter for a Natural environment running a replication task. However, the DATSIZE in such an environment will be approximately 170 KB.

After Entire Transaction Propagator Version 1.5.2 is installed and when the MENU command is invoked for the first time in the Entire Transaction Propagator maintenance utility, the Entire Transaction Propagator administration file is automatically migrated to the Entire Transaction Propagator Version 1.5.2 format and an appropriate message is displayed.

Afterwards, any attempt to access the migrated administration file from an earlier version of the Entire Transaction Propagator maintenance utility is denied.

Before enabling the new Entire Transaction Propagator maintenance utility, all log files must be empty and all log and confirmation files must be accessible for the migration process. Migration of the administration file from the Entire Transaction Propagator Version 1.4 format to the Entire Transaction Propagator Version 1.5 format is performed automatically when the Entire Transaction Propagator Version 1.5 maintenance utility is invoked for the first time. If a previous Entire Transaction Propagator version is installed, you should use the Natural SYSMAIN utility (see the Utilities documentation) to delete the contents of the library SYSETP before installing Entire Transaction Propagator Version 1.5.2. Following installation of Entire Transaction Propagator Version 1.5.2, the maintenance utility of any earlier Entire Transaction Propagator version will be denied access to the administration file.

Otherwise, Entire Transaction Propagator Version 1.5.2 is compatible with Entire Transaction Propagator Version 1.4.1. The FDT (Field Definition Table) of the administration, log and confirmation files remains unchanged.

Prerequisites

See General Prerequisites and System Support in the section Overview of the Installation Process.

Installation Medium

The installation medium contains the following files required for product installation:

File Name Contents
ETPvrs.SYSF System file definition for multiple use:

administration, logging and confirmation

ETPvrs.SYS1 System file definition for administration
ETPvrs.SYS2 System file definition for logging
ETPvrs.SYS3 System file definition for confirmation
ETPvrs.SRCE Example program for using Entire Transaction Propagator services from 3GL programs
ETPvrs.PAMS Source modules
ETPvrs.JOBS Sample installation jobs
ETPvrs.INPL Natural objects
ETPvrs.ERRN Natural error messages
ETPvrs.FDTA System file for containing FDT definitions for all Entire Transaction Propagator files

Copy the files into your environment as described in Copying Files to a BS2000/OSD Disk in the section Installing Natural.

Installation Procedure

Be sure to read Installation Process and Major Natural Features before you start the installation procedure.

Step 1: Load the Entire Transaction Propagator System Files

(Job I050, Step 5300)

The ETPvrs.SYSF file has a Field Definition Table (FDT) that is suitable for containing the administration, confirmation and log files within one physical Adabas file.

Load the ETPvrs.SYSF file from the installation medium by using Job I050.

The following files are suitable for installing individual Entire Transaction Propagator files:

  • ETPvrs.FDTA

    This file contains the same FDTs as the ETPvrs.SYSF file, but in a format suitable for loading with the Adabas ADACMP utility.

  • ETPvrs.SYSn

    This file contains separate sample Adabas files that can be loaded using the Adabas ADALOD utility. These files have FDTs suitable for defining individual administration, log and confirmation files.

The Adabas utility parameters ISNREUSE=YES (for mainframes) and/or REUSE=ISN (for UNIX, Windows or OpenVMS systems) can be set to reuse freed ISNs as they become available for Entire Transaction Propagator master, replicate, confirmation, administration and/or log files.

Step 2: Modify the Natural Parameter Module

Before linking the assembler modules as described in the next step, refer to Step 4: Specify Master File Databases and Step 8: Define the Administration File.

Reassemble and relink the Natural parameter module after you have modified it.

Note:
You must relink the modified Natural parameter module to all Natural nuclei that update a master file or start a replication task.

Step 3: Link the Assembler Modules

You must link the ETPNUC module to all Natural nuclei that update a master file or start a replication task. We recommend that you relink Natural with the ETPNUC module.

To avoid the need to relink Natural with ETPNUC, specify the Natural profile parameter RCA=ON or RCA=NATGWETP in the Natural parameter module to allow dynamic loading of the Entire Transaction Propagator during Natural startup. In this case, perform the following steps:

  1. Rename the module ETPNUC in the Entire Transaction Propagator module library to NATGWETP.

  2. Place the NATGWETP module in a library from which your TP monitor or batch system can perform dynamic loads. In Com-plete systems, NATGWETP can be loaded as a resident program.

Step 4: Specify Master File Databases

Use the NTDB macro in the Natural parameter module to specify the databases containing the master files:

NTDB ADABAS,dbid,ETP

Or:

NTDB ADABAS,(dbid,dbid,...),ETP

where dbid specifies one or more databases separated by commas, each containing one or more master files.

You can define the same database as an ENTIRE database and also as an Entire Transaction Propagator database if you specify both options for the NTDB macro:

NTDB ADABAS,dbid,ETP,ENTIRE

It is also possible to specify the databases containing the master files dynamically at Natural startup using the DB parameter:

DB=(ADABAS,(dbid,dbid,...),ETP)

The first time that a database which is defined as an Entire Transaction Propagator database is accessed, Entire Transaction Propagator automatically obtains a buffer (ETPSIZE) of the required size (approx. 10 KB). Any value specified for the Natural profile parameter ETPSIZE is ignored. The size of the ETPSIZE parameter should be left at its default (ETPSIZE=0).

Step 5: Install the Maintenance Utility

The Entire Transaction Propagator maintenance utility is a menu-based control facility for defining and managing master and replicate files. Although not required, we recommend that you install the maintenance utility on every node containing a master file.

Step 6: Load the Natural Objects and Natural Error Messages

(Job I061, Steps 5300, 5301)

Load the Natural objects and Natural error messages specific to the Entire Transaction Propagator by performing the following steps:

  1. Step 5300: Load the ETPvrs.INPL file (contains the Natural objects) into your Natural FNAT system file by using the Natural INPL utility.

  2. Step 5301: Load the ETPvrs.ERRN file (contains the Natural error messages) into your Natural FNAT system file by using the ERRLODUS program of the Natural SYSERR utility (described in the Utilities documentation).

Warning:
The SYSETP library should be protected against general access with Natural Security or an equivalent security facility to prevent uncoordinated changes to the administration file. Such changes can destroy the consistency and integrity of the master and replicate files.

Step 7: Define Natural Security Options

The Entire Transaction Propagator and the maintenance utility support the concept of functional security, meaning that selected functions can be allowed or disallowed under control of Natural Security. When a user is restricted by Natural Security from performing a specific Entire Transaction Propagator function, the function is not displayed on the user's corresponding menu.

Note:
Disallowing the general dialog functions - for example, EXIT, CANCEL - can cause unpredictable results.

To install the Entire Transaction Propagator if Natural Security is already installed, perform the following steps:

  1. Log on to the library SYSSEC.

  2. Issue the command ADD LIBRARY SYSETP.

  3. In the Add Library menu, enter the appropriate information and select Additional Options.

  4. In the ADDITIONAL OPTIONS window, select Functional Security.

  5. In the FUNCTIONAL SECURITY window:

    • Define the command processor WADNCP1 for the library SYSETP.

    • Enable or disable keywords (for example, DELETE, MASTER ...) or functions (for example, REPLICATE TRANSACTIONS) as required for your site. Note that any restrictions you define here apply to all users.

    In a similar way, you can restrict the availability of keywords or functions for every user that has access to SYSETP.

Step 8: Define the Administration File

Each database containing an Entire Transaction Propagator master file should also contain an administration file to hold all master and replicate file definitions. The logical file ID of the administration file must be 200. To define the administration file for your Natural applications, specify the NTLFILE macro in the Natural parameter module:

NTLFILE 200,dbid,fnr

where fnr is the physical file number of the administration file and dbid is its physical database ID.

The administration file setting can also be changed dynamically at the start of the Natural session, using the Natural profile parameter LFILE:

LFILE=(200,dbid,fnr)

The administration file must always be defined before using the Entire Transaction Propagator; if the Entire Transaction Propagator maintenance utility is used, the utility prompts the user for an administration file if no valid LFILE definition is found.

The Entire Transaction Propagator also works correctly, even if the Natural macro NTTF or the profile parameter TF is used. However, the Entire Transaction Propagator may not work properly if you have an Adabas user exit installed which modifies the database ID or file number in the Adabas control block.

Warning:
Do not change any of the information in the administration file while it is being used by a replication task.

To install an administration file in your database, use either Job I050 or the Adabas ADALOD utility to load ETPvrs.SYS1. You can keep the administration file quite small since it contains only a single record for every:

  • master file definition

  • replicate file definition

  • log file

  • replicate database

  • user profile

Step 9: Define a Master File and its Log File

A master file is normally an existing Adabas file. If the master file is new, you must first create the file as a normal Adabas file. This description assumes that the file to be defined as a master already exists.

Before a master file can be defined, an administration file must first be defined. See Step 8: Define the Administration File.

After defining the administration file, perform the following steps to define a master and log file:

  1. Stop all updating on the file that is to be defined as a master file.

  2. Copy the file to be defined as a master file. We recommend that you use the Adabas ADAULD utility for this purpose; the Adabas ADASAV utility may also be used, but only where the file will be reloaded on the same device type as before. Note the exact date and time of the copy.

  3. Using the Entire Transaction Propagator maintenance utility's Master File Definition Maintenance menu (see Master File Task Screens in the Entire Transaction Propagator documentation), specify the master file and log file. If desired, all master files on a database can share the same log file.

  4. Restart the database operation to make the master file available again.

From this time on, all changes applied to the master file will be recorded in the log file. Any replicate files of the master file can now be defined without interfering with the master file operation, providing the log file contains all changes applied to the master file since the copy in Step 2 above was created. For more information, see Updating the Administration File in the Entire Transaction Propagator documentation.

Transaction logging will start as soon as a master file is defined and a new Natural session with the appropriate administration file is started or the master file is updated from within the Natural session which was used to define the master file. Therefore, the procedure described above might not be applicable, especially when a new master file is to be defined in a running environment. If it cannot be guaranteed that no updates are applied to the to-be-defined master file that is to be defined while the above steps are executed, perform the following steps (note that it is required that the log file specified for a master file is empty when the master file is defined):

  1. Stop the transaction replication processes.

  2. Define the new master file.

  3. Shut down the database containing the new master and log files.

  4. Copy the master file as described above.

  5. Use the Adabas ADADBM utility with the REFRESH function to refresh the log file.

  6. Restart the database to make the master file available again; transaction logging will start as soon as a user updates the master file;

  7. Copy the master file's contents into the replicate file;

  8. Restart transaction replication.

Install a log file in your database by using either Job I050 or the Adabas ADALOD utility to load ETPvrs.SYS2.

The number of records in the log file depends on the number of transactions that update master files between two successive invocations of the Clean up log file maintenance function (provided that all transactions are replicated), which is described in the Entire Transaction Propagator documentation. The approximate number of log file records equals:

(transaction count)*(updates per transaction, +1)

When loading the log file, the ADALOD utility parameter PGMREFRESH=YES is required if you want the Clean up log file function to refresh the log file for improved performance.

Note:
It is impossible to refresh a log file that is also used as an administration or confirmation file.

Step 10: Define and Initialize the Replicate and Confirmation Files

After the master file is defined, the replicate files can be defined. A confirmation file must also be defined on each database where a replicate file is defined. If desired, multiple replicate files on a database can share the same confirmation file. To define a replicate file, perform the following steps:

  1. Using the Add replicate file definition screen (described in the Entire Transaction Propagator documentation) of the online maintenance utility, specify the database IDs and file numbers for the replicate, master and confirmation files:

  2. Load the unloaded copy of the master file into the replicate file using the Adabas ADALOD utility (if the file was unloaded with ADAULD) or ADASAV (if the file was unloaded with ADASAV). Specify the parameter USERISN=YES for the related Adabas mainframe utilities. For the replicate files on UNIX, Windows or OpenVMS systems, the option USERISN must be specified when loading a non-empty file unless the distribution key is used as the replication criterion. If an empty replicate file is created, the option USERISN or parameter setting USERISN=YES is not required.

    When specifying the MAXISN parameter while defining the replicate file you should remember that, when using the records' ISNs as the replication criterion, the Address Converter (AC) is not automatically extended. This can occur if the Entire Transaction Propagator issues Adabas N2 calls to add new replicate file records and specify an ISN value that exceeds the file's MAXISN value; in such a case, a response code 113 is returned. Specify a MAXISN value large enough for future extensions of the replicate file.

  3. Using the Display transactions function of the maintenance utility (described in the Entire Transaction Propagator documentation), check for any log file entries that have been made since the master file was copied.

  4. If a replicate file contains a subset of the master file records, you should either delete all unneeded records as defined by the specified distribution key ranges or copy only the selected subset of records from the master file. This can be done using one of the following methods:

    • Use a Natural program to delete the unneeded records from the replicate file.

    • Use a Natural program to copy only the selected records from the master file to an intermediate file which is then subsequently copied to the replicate file.

    • Use the SELCRIT and SELVAL parameters of the Adabas utility function ADAULD UNLOAD to select only the subset for unloading. This is the recommended method.

      Warning:
      After the replicate file has been initialized, it should not be manually changed. Otherwise, the consistency with the master file could be destroyed.
  5. To install a confirmation file in your mainframe database, use either Job I050 or the Adabas ADALOD utility to load ETPvrs.SYS3 from the installation medium. For confirmation files on UNIX, Windows or OpenVMS systems, use the confirmation file field definitions in file ETPvrs.FDTA to supply field definitions for the Adabas ADAFDU utility. For every replicate file that uses the confirmation file, the latter file contains a single record.

  6. If the log file contains any entries for the master file, start the replication task for the replicate file using the Replicate transactions maintenance function described in the Entire Transaction Propagator documentation. The task checks the appropriate administration file for master or replicate files in the file range to be processed. The task then synchronizes the master and replicate file by applying all updates to the replicate file that are not already applied. The replicate file is now available for use.

To add other replicate files of an existing master file, perform the steps above after creating an up-to-date copy of the master file. Note that when creating a replicate file, no logged changes to the master file should be removed from the log file. If this rule is followed, a replicate file can be added without affecting the normal mode of operation.

Step 11: Set Related Parameters

  • Natural WH Parameter

    When running multiple Entire Transaction Propagator replication tasks in parallel, specify WH=ON to avoid NAT3145 errors (record already in hold status for another user) when two tasks attempt to access the same record simultaneously.

  • Reusing ISNs in Entire Transaction Propagator Files

    The Adabas utility parameters ISNREUSE=YES (for mainframes) and/or REUSE=ISN (for UNIX, Windows or OpenVMS systems) can be set to reuse freed ISNs as they become available for Entire Transaction Propagator master, replicate, confirmation, administration and/or log files.

  • Transactions with Many Updates

    If transactions that include a lot of updates are to be logged, increase the value of the Adabas ADARUN parameter LDEUQP. The required size for transaction logging can be computed as:

    LDEUQP = (updates per transaction * 29)

Step 12: Modify the WADUSER2 and/or WADUSER3 User Exits

The two user exits WADUSER2 and WADUSER3 are delivered in source form in the library SYSETP.

  • Optional User Exit WADUSER2

    The optional user exit WADUSER2 is a Natural subprogram for controlling file replication. WADUSER2 is called after the Entire Transaction Propagator decides whether the record in question is to be replicated or not.

    The WADUSER2 user exit, defined as User Exit 2, is called only if the user exit option is specified when a master file is defined (see Master File Task Screens in the Entire Transaction Propagator documentation). An example of WADUSER2 is included in the SYSETP library.

  • Message Handler WADUSER3

    The subprogram WADUSER3 is used to display all messages issued by the replication task. WADUSER3 can be modified to filter the task messages and, if desired, send them directly to the operator console. The WADUSER3 subprogram receives the error number, severity level and the message text from the replication task. This allows the user to select the messages to be displayed. By means of the Natural CMWTO entry (for an example, see the program WTO in the library SYSEXTP), the messages can be sent to the operator console.

    Warning:
    The user exits WADUSER2 and WADUSER3 should neither issue Adabas calls that update a database file nor should they issue any End Transaction (ET) or Back Out Transaction (BT) commands; otherwise, the results are unpredictable.

Step 13: Run the Entire Transaction Propagator in Batch Mode

In batch mode, command execution is possible only by means of direct commands. For a list of direct commands and their minimum abbreviations, see Entering Direct Commands in the Entire Transaction Propagator documentation.

The following example is for a batch file for starting a replication task:

LOGON SYSETP                                      (1)*
MENU                                              (2)*
REPLICATE TRANSACTIONS                            (3)
1,1,65535,65535,1,65535,00:30:00,200,1000,N,EXIT  (4)
EXIT                                              (5)
FIN                                               (6)

* If Natural Security is installed, lines (1) and (2) may have to be changed (see Starting the ETP Maintenance Utility in the Entire Transaction Propagator documentation).

Line (4) contains the parameters for the corresponding Replicate transactions Entire Transaction Propagator maintenance utility screen. Parameters are entered from top to bottom, left to right. Line (5) exits from the Entire Transaction Propagator maintenance utility.

Warning:
We recommend that you start replication tasks in batch mode.

The following example is for a batch file for deleting successfully replicated transactions:

LOGON SYSETP                                      (1)*
MENU                                              (2)*
CLEANUP LOGFILE                                   (3)
1,1,65535,65535,,,N,00:30:00,200,1000,EXIT        (4)
EXIT                                              (5)
FIN

* If Natural Security is installed, lines (1) and (2) may have to be changed (see Starting the ETP Maintenance Utility in the Entire Transaction Propagator documentation). Line (4) contains the parameters for the corresponding Cleanup logfile Entire Transaction Propagator maintenance utility screen. Parameters are entered from top to bottom, left to right. Line (5) exits from the Entire Transaction Propagator maintenance utility.

Warning:
We recommend you to start tasks that successfully delete replicated transactions in batch mode.

If a window is displayed in batch mode, all fields are protected; the reason for this is that in most cases it is not possible to determine the number of selectable items. Therefore, the only meaningful command is PROCESS. The following is an example to reset the in-use flag for all replicate files:

MENU
RESET IN-USE * *
PROCESS
EXIT
EXIT
FIN

To run the above examples without problems, the following parameters in the Natural parameter module must be specified:

ID=','   (default setting)
IM=D
PC=OFF   (default setting)

Either the Natural statement SET CONTROL '+' or the terminal command %= cancel the effect of PC=OFF.