Using Utilities

This document covers the following topics:

Where appropriate, specific sections are used when there are substantial handling differences between platforms (Linux and Windows).


Assigning Input and Output Devices

All commands to utilities are read from stdin and the output is directed to stdout. The standard input and output may be directed to files that are normally supported by the operating system. By default, the standard output does not include the utility parameters specified, but if you set the environment variable ADAPARLOG to YES, the parameter specifications are copied to stdout, provided the parameters are not specified interactively.

If the utility produces one or more output files, the environment variable corresponding to each output file must be set to a legal file name, before the utility is started. For detailed information concerning the assignment of environment variables, refer to the utilities in question. The output files produced by each utility and the environment variable assignments for these files are described near the beginning of each utility chapter. If the environment variable for an output file is not set before the utility is started, the output file is created in the current directory, and the name of the output file is the same as the environment variable. If the environment variable for a command log file or a protection log file is not set before the utility is started, the output file is created in the current directory, and the name of the output file is the same as the environment variable with a sequence number appended.

Several utilities also require environment settings for user exits and hyperexits. For more details, refer to the section User Exits and Hyperexits.

Executing a Utility (Linux)

The utilities may be executed interactively or in the background.

Prerequisites

You must have the necessary permissions in order to execute a utility. The installation sets the permissions as follows: utilities may be executed by the Software AG products administrator user, e.g. sag, and by all users that belong to the corresponding group, e.g. sag. If this group is not your current group, it is recommended that you change the group with the newgrp command before you execute a utility (including ADANUC); failing to do so may lead to other users encountering permission problems because the group for IPC resources created by the user is the current group of the user who executes the utility, and some IPC resources allow access for only the group.

Before you execute a utility, you must set the required environment variables if the names of the files to be used differ from the default values (see the section "Environment Variables" in this chapter).

The program dbgen generates a file called assign.bsh (for Bourne shell and Korn shell) or assign.csh (for C shell) when they create a new database. This file can be sourced in the shell to set the environment variables for the container files, command log and protection log. To source this file, change your working directory to your database directory and enter:

.  ./assign.bsh

If you do not set the environment variables for container files externally, the utilities extract the settings themselves via the configuration files ADABAS.INI and DBxxx.INI, as described in the Adabas Extended Operation section.

Utility input lines contain control statements that consist of strings defining the settings of parameters, a function to be executed or both.

All functions and parameters are described in the documentation.

The utilities are located in the directory $ADAPROGDIR. If this directory is defined in the PATH environment variable, you can execute the utility directly by specifying its name at the operating system prompt, otherwise you have to precede the utility name by "$ADAPROGDIR". The following examples assume that the directory name $ADAPROGDIR is defined in the PATH environment variable.

Executing a Utility Interactively

You execute a utility interactively by specifying its name followed by carriage return at the operating system prompt.

When a utility starts, it displays an informational message consisting of date, time and version number. It then prompts with its name and a colon for input of control statements:

%ADAREP-I-STARTED, 30-MAY-2024 11:38:16, Version 7.3.0.0 (Linux 64Bit)
adarep:

In interactive mode, the input line starts directly after the utility prompt.

Utilities support the autocompletion of control statements by using the <TAB> key in interactive mode. They also support the autocompletion of values for control parameters when the possible values are already defined. This applies for example to the ENCRYPTION parameter of ADABCK. After entering ENCRYPTION= on the prompt, you can use the <TAB> key to complete to the possible values (NO, AES_128_XTS, AES_256_XTS).

Utilities save entries that are made in a temporary history at runtime. You can use the up and down arrow keys to go back and forth in history to reuse already made input. After terminating the utility, the history is lost.

To make the history persistent, create a directory .adabas in your $HOME directory. Each utility stores their command history in their own history file. For example, ADABCK will write its history to $HOME/.adabas/adabck.history.

Ensure that the $HOME and .adabas directories have restrictive read and write permissions when entering sensitive data at the utility command prompt and saving it to the history file. Saving can be prevented by starting the command line with a space character.

Removing the .adabas directory disables the persistency of the command history.

You can extend the input over several physical input lines by terminating each line with a backslash ("\") followed by carriage return. The utility outputs the prompt ">" at the start of each continuation line.

Executing a Utility at Call Level

The control statements also be specified at the call level, e.g. for the utility adarep:

adarep dbid=20 summary

The control statements can also be redirected from an input file. If, for example, the file rep.in contains the lines:

dbid=20
summary

then you can run adarep with these two control statements by specifying:

adarep <rep.in

The output produced by the utility usually goes to stdout, which means that it can be redirected. So, for example, to redirect the output of the sample adarep call to the file rep.out, specify:

adarep <rep.in >rep.out

Notes:

  1. Please be aware that the Linux shells have a special handling for some characters such as parentheses, quotes and double quotes. This means that you must change the specification of the parameters accordingly if the parameter values contain one or more of these characters.
  2. Blanks are equivalent to a line feed in interactive input - if a blank should appear in a parameter value, it must be preceded by a backslash or occur between quotes or double quotes.
  3. Setting the environment variable ADAPARLOG to YES may help you to find errors in the parameter specification - it displays the parameters as specified interactively.

Examples

adadcu fields 'NEW RECORD',AA,AB end_of_fields

Invalid, because it is equivalent to

adadcu
fields
NEW RECORD,AA,AB
end_of_fields

The quotes are missing.

adadcu fields \'NEW RECORD\',AA,AB end_of_fields

Invalid, because it is equivalent to

adadcu
fields
'NEW
RECORD',AA,AB
end_of_fields

The literal must be specified on one line.

adadcu fields \'NEW\ RECORD\',AA,AB end_of_fields

Valid, this is equivalent to

adadcu
fields
'NEW RECORD',AA,AB
end_of_fields
adadcu "fields 'NEW RECORD',AA,AB end_of_fields"

Invalid, because it is equivalent to

adadcu
fields 'NEW RECORD',AA,AB end_of_fields

fields and end_of_fields must be specified on separate lines.

adadcu fields "'NEW RECORD'",AA,AB end_of_fields

or

adadcu fields "'NEW RECORD',AA,AB" end_of_fields

Valid, both are equivalent to

adadcu
fields
'NEW RECORD',AA,AB
end_of_fields

Switching Parameter Input from Command Line to Standard Input

It is possible to switch the parameter input for a utility from the command line to standard input by entering a '+' character (plus sign) after the last parameter specified in the command line.

Example:

adafdu dbid=35 file=36 + <employee.fdu

Executing a Utility (Windows)

The utilities may be executed interactively or in the background.

In order to be able to execute Adabas utilities of an installed Adabas version, some environment variables must be set. This is done when you select Adabas Server Command Prompt.

If you want to execute Adabas commands from the standard command prompt, you must configure a ‘local’ Adabas Server Environment as described in Completing the Installation.

  • You can set the environment variables required for Adabas as system environment variables by selecting Set Adabas System Environment. You can delete these system environment variables again by selecting Unset Adabas System Environment.

  • If you don’t want to set the environment variables as system environment variables, you can execute the command startenv.cmd in the bin folder of the current Adabas version. You can find out the name of the folder by right-clicking on the Adabas Command Prompt and selecting Properties.

Before you execute a utility offline, you must set the required environment variables if the names of the files to be used differ from the default values (see the section "Environment Variables" in this chapter).

If you do not set the environment variables for container files externally, the utilities extract the settings themselves via the configuration files ADABAS.INI and DBxxx.INI, as described in the Adabas Extended Operation documentation.

Utility input lines contain control statements that consist of strings defining the settings of parameters, a function to be executed or both.

All functions and parameters are described in this manual.

The utilities are located in the subdirectory "Adabas" of the installation directory. If this directory is defined in the PATH environment variable, you can execute the utility directly by specifying its name at the operating system prompt; otherwise you have to precede the utility name by "%ADAPROGDIR%". The following examples assume that the directory name %ADAPROGDIR% is defined in the PATH environment variable.

Executing a Utility Interactively

You execute a utility interactively by specifying its name followed by carriage return at the operating system prompt.

When a utility starts, it displays an informational message consisting of date, time and version number. It then prompts with its name and a colon for input of control statements:

%ADAREP-I-STARTED, 30-MAY-2024 11:38:16, Version 7.3.0.0 (Windows 64Bit)
adarep:

In interactive mode, the input line starts directly after the utility prompt.

You can extend the input over several physical input lines by terminating each line with a backslash ("\") followed by carriage return. The utility outputs the prompt ">" at the start of each continuation line.

Executing a Utility at Call Level

The control statements also be specified at the call level, e.g. for the utility adarep:

adarep dbid=20 summary

The control statements can also be redirected from an input file. If, for example, the file rep.in contains the lines:

dbid=20
summary

then you can run adarep with these two control statements by specifying:

adarep <rep.in

The output produced by the utility usually goes to stdout, which means that it can be redirected. So, for example, to redirect the output of the sample adarep call to the file rep.out, specify:

adarep <rep.in >rep.out

Note:
Blanks are equivalent to a line feed in interactive input - if a blank should appear in a parameter value, it must occur between double quotes.

Examples

adadcu fields 'NEW RECORD',AA,AB end_of_fields

Invalid, because it is equivalent to

adadcu
fields
'NEW
RECORD',AA,AB
end_of_fields

The literal must be specified in one line.

adadcu "fields 'NEW RECORD',AA,AB end_of_fields"

Invalid, because it is equivalent to

adadcu
fields 'NEW RECORD',AA,AB end_of_fields

fields and end_of_fields must be specified in separate lines.

adadcu fields "'NEW RECORD'",AA,AB end_of_fields

or

adadcu fields "'NEW RECORD',AA,AB" end_of_fields

valid, both are equivalent to

adadcu
fields
'NEW RECORD',AA,AB
end_of_fields

Switching Parameter Input from Command Line to Standard Input

It is possible to switch the parameter input for a utility from the command line to standard input by entering a '+' character (plus sign) after the last parameter specified in the command line.

Example:

adafdu dbid=35 file=36 + <employee.fdu

Executing a Utility Remotely

If the option LOCAL_UTILITIES is not set for an active nucleus of a given Adabas database, the database can be accessed remotely by certain Adabas utilities, using SOFTWARE AG's product ENTIRE NET-WORK or NET-WORK ACCESS, provided that the architecture of the local and remote machines is identical (for example, that both machines use byte-swapping). The utilities that can access a database remotely are:

  • ADACMP

  • ADADBM

  • ADAFDU

  • ADAREC

  • ADAREP

  • ADATST

  • ADAULD

For ADADBM and ADAULD, only the functions which are allowed when the nucleus is running can be executed remotely.

Note:
ADAFDU cannot be executed remotely if you define an ADAM file or if you specify the parameter FORMAT.

Utility Syntax

Functions and Arguments

The arguments of a function are called parameters. Parameters may have various values. A parameter that has only two possible values, which are logically complementary, is called a switch.

Functions are represented by reserved keywords. Three types of functions are used:

  • Functions requiring no arguments;

  • Functions requiring one argument;

  • Functions requiring more than one argument.

A function that does not require an argument is a switch, and can be enabled or disabled by keywords such as LOCK and UNLOCK.

Arguments are of the following type:

  • Strings
    A string is a sequence of valid ASCII characters with the exception of comma `,', carriage return <CR>, semicolon `;' opening `(' and closing brackets `)', which are all regarded as string delimiters. Embedded blanks and tabs are removed. The backslash `\' can be used as an escape character.

    Most strings are converted to upper case. Instead of the equals sign, it is also possible to specify a colon ‘:’. The difference is that strings following a colon are not converted to upper case. Although a colon may also be specified for other parameters where the utility syntax description does not allow a colon, it is recommended to use a colon only in these cases; otherwise the results of the parameter specification may be undefined.

    Example:

    adatst: rb=test
    adatst: rb
      <<<<<< RECORD BUFFER >>>>>>
    
    00000000    54455354 ........ ........ ........    TEST............
    adatst: rb:test
    adatst: rb
      <<<<<< RECORD BUFFER >>>>>>
    
    00000000    74657374 ........ ........ ........    test............
    

    If you specify “rb=test”, the string “test” is converted to upper case, while it is not converted if you specify “rb:test”.

  • Keywords
    A keyword is a predefined string.

  • Numbers
    A number is a string of digits representing an unsigned integer. Numbers can be entered in hexadecimal format by preceding the digits with the two characters "0x".

  • Number ranges
    A number range is a number followed by a hyphen (-) and another number.

  • Lists of keywords, strings, numbers or number ranges
    A list is a sequence of keywords, strings or numbers, separated by commas and enclosed in brackets.

Arguments are specified according to the following rules:

  • The first argument is specified after the equal sign `=' following the function keyword;

  • Subsequent arguments are specified by using keywords followed by an equal sign and a value.

Examples:

adadbm: recover
adadbm: renumber = (3,14)
adadbm: reuse = (isn,ds), file = 6
adarec: regenerate = *
adarec: plog = 654
adarec: checkpoint = synp
adarec: block = 0x1A

The list of arguments may be continued in the following lines. In the case of a multi-function utility, the function is executed if this list is complete and correct.

Symbols used in Syntax Diagrams

If an argument is mandatory or has a default value, this is indicated by the characters `M' or `D' in the list of arguments at the beginning of the utility description.

Checking Current Parameter Settings

The current values of all parameters that were set or preset can be displayed by entering an asterisk `*':

adarep:*<cr>
%ADAREP-I-PARSET setting of COUNT=<set>
%ADAREP-I-PARSET setting of DBID=5
%ADAREP-I-PARSET setting of FDT=<set>
%ADAREP-I-PARSET setting of FILES=(10)

If a switch is set, it is displayed as "<set>".

Absolute Time

Some utilities (ADACLP, ADADBM, ADAREP) require arguments in the form of an absolute date or date and time string. The string used must correspond to the following absolute format:

dd-mmm-yyyy[:hh:mm:ss]

For more detailed information about how to use absolute dates and times, please refer to the individual utilities concerned.

Single- and Multi-function Utilities

Depending on the number of functions that can be performed during one run, a utility is called a single-function or multi-function utility.

In a single-function utility, the correctness of each keyword value is checked after it is entered. The completeness of the set of arguments is checked when the utility is executed by pressing CTRL/D (Linux) on a line by itself or by pressing CTRL/Z followed by a carriage return (Windows). A single-function utility is terminated after the execution of a function.

A multi-function utility accepts sequences of function specifications and/or parameter settings. The sequence must correspond to a certain logical order. A function is executed when this sequence is complete and correct. The execution of a function in a multi-function utility is terminated with a message indicating successful or non-successful completion. The utility itself does not terminate after the execution of a function.

Terminating a Utility

The QUIT control statement or its abbreviation "Q" is used to terminate a utility. Also EXIT can be used as a synonym for QUIT.

Note:
Utility control parameters are always converted to upper case. It is, therefore, also possible to specify QUIT and EXIT in lower case. QUIT and EXIT can also be specified in the FDT parsing mode, for example, after specifying the FIELDS parameter. However, no conversion to upper case is done in the FDT parsing mode after specifying the LOWER_CASE_FIELD_NAMES parameter; in this case, you must specify QUIT or EXIT in upper case.

In a single-function utility, QUIT aborts the function and terminates the utility. An EOF (CTRL/D on a line by itself [Linux] or CTRL/Z followed by a carriage return [Windows]) first executes the function and then terminates the utility.

In a multi-function utility, QUIT terminates the utility. If entered while setting the function parameters, QUIT aborts the function and terminates the utility. However, an EOF tries to execute the function before termination. If some of the parameters required have not been specified, the utility requests their entry before the function is executed.

In all cases of termination, a statistical summary of the IOs made on the input and output files is given and an informational message is issued with utility name, date, time and elapsed time e.g.:

%ADAFRM-I-IOCNT,      500  IOs on dataset WORK
%ADAFRM-I-IOCNT,      800  IOs on dataset DATA
%ADAFRM-I-IOCNT,      600  IOs on dataset ASSO
%ADAFRM-I-TERMINATED, 30-MAY-2024 11:38:16, elapsed time: 00:00:51

If a utility terminates successfully, the final TERMINATED message is displayed and the program exit status is set to 0, otherwise an ABORTED message is issued and the program exit status is set to a non-zero value. Multi-function utilities always terminate successfully if they are used interactively without input redirection.

In some cases, utilities display the TERMINATED message and deliver a non-zero exit status to indicate an exception, for example the utility ADAVFY if an error has been found, or the utility ADACMP if records were rejected.

Error Handling

Incorrect sections of utility commands are indicated by up arrows in the following line and an error message in a subsequent line. Errors are processed from left to right.

If an error occurs in a sequence of parameter settings, all settings before the error are executed.

adadbm: refresh = 30
                   ^      
%ADADBM-E-NOTLOAD,  file not loaded

The position of the error in the input line is indicated by an up arrow in the following line.

Adabas Sequential Files

Overview

The Adabas nucleus or utilities usually work with sequential files for input and output. In addition to these sequential files, there are also internal files with a uniform internal Adabas format (Protection Log, for example), and external files with a user-defined format (decompressed data, for example).

Some Adabas sequential files can be directed to a named pipe. The following table lists all of the internal Adabas sequential file types and the corresponding environment variables that can be set to a file in the file system. If the environment variable is not defined, Adabas uses a file in the current working directory with the name of the environment variable.

Sequential File Type Environment Variable
Protection Log DEVPLGx, NUCPLGx(*), PLPLEX, PLPPLGx, RECPLGx
Command Log CLPCLGx, DEVCLGx, NUCCLGx
Backup File BCK00nx, DEV00nx, ULD00nx
Backup File Copy BCKOUTx, DEVOUTx
Compressed Data CMPDTAx, DCUDTAx, DEVDTAx, MUPDTAx, ULDDTAx
Descriptor Value Table CMPDVTx, DEVDVTx, MUPDVTx, ULDDVTx
ADAMUP Temporary Data MUPTMPx
ADAMUP Log
(DTA/DVT)
DEVLOGx, MUPLOGx
Reorder File DEVEXPx, ORDEXPx
Utility Error File ERRINx, CMPERRx, DCUERRx,
DEVERRx, INVERRx, MUPERRx,
RECERRx,
ADAMUP ISN List MUPISN
ADAMUP Temporary Working Space for LOB Processing MUPLOB, MUPLBI
Field Definition Data FDUFDT, CMPFDT, DCUFDT
Decompressed Raw Data CMPIN, DCUOUT
Nucleus Audit Trail log NUCADTx

where: n = 1 - 9; x is omitted or is greater than 1.

Platform Dependencies

Adabas sequential files have different internal formats on high-order-first and low-order-first platforms, but they are compatible between all platforms with the same byte order.

Examples

Platform 1 Byte Order Platform 2 Byte Order Compatible
LINUX (Intel) Low-order-first Windows Low-order-first Yes

Only the following files are compatible between all platforms:

  • Decompressed data files

  • Pure text files, for example FDT files

If you want to exchange date between high-order-first and low-order-first platforms, you should proceed as follows:

  1. Unload with ADAULD

  2. Decompress with ADADCU

  3. Compress with ADACMP

  4. Load with ADAMUP

If this is not possible because the decompressed records are too large, you must write a program that reads data from the source database. The program can either write data to an intermediate file in the file system, from where it is read by a second program which writes data to the target database, or it can store the data directly in the target database via Entire Net-Work.

Using Named Pipes

Named pipes can be used to save storage space when the output of a utility is to be processed by another utility or application, or if the input of a utility is output by another utility or application. Named pipes can be used for all Adabas sequential files except MUPDTA, MUPDVT, MUPISN, MUPLBI, MUPLOB, MUPLOG, MUPTMP, NUCPLG, ORDEXP, PLPLEX, PLPPLG, ULD00n.

Named pipes under Windows are transient objects. If an environment variable contains the name of a named pipe (for example \\.\pipe\anyname), the utility that attempts to write to the named pipe will create it implicitly. Then this utility waits for a predefined amount of time until the reading utility comes up and begins to read from the named pipe. If the time elapses without the reading utility coming up, the writing utility aborts with an I/O error.

Multiple Extents For Adabas Sequential Files

If the space allocated to a sequential file becomes exhausted while it is being created by a utility, Adabas attempts to continue the file in another free slot. This applies to all sequential files if the file system where the sequential file is located becomes full. Multiple environment variables may be used with this technique, with a counter being appended to the name of the environment variable.

Example (C shell)

setenv  NUCPLG  /dev/rdsk/c4d0s2
setenv  NUCPLG2  /usr/adabas/PLOG_2
setenv  NUCPLG3  /dev/rdsk/c7d0s2

These three devices are used for the Protection Log during a nucleus session.

An alternative method is to set all of the file/device names within one environment variable:

Note:
For Linux platforms the delimiter is a blank, and on Windows platforms it is the semicolon character ";".

Example (C shell)

setenv NUCPLG "/dev/rdsk/c4d0s2  /usr/adabas/PLOG_2  /dev/rdsk/c7d0s2"

Adabas starts with the first environment variable (in this case NUCPLG), or with its default if the environment variable is not set (the default is a file in the current directory with the same name as the environment variable). When the end of the file is reached, or when the file system becomes full (in cases where a file system is used), the file will be closed, and the next environment variable (NUCLPG2) will be translated and the associated file or disk section opened. Then it allocates free space from that disk section to be used for the next extent. If a subsequent environment variable does not exist (e.g. NUCPLG4), Adabas will wrap around to the first disk section (NUCPLG).

If only one environment variable is defined and assigned to a disk section, the subsequent file extents will all be created in the same disk section. If only one environment variable is defined and assigned to a file in a file system, Adabas cannot continue if the file system becomes full.

When a switch occurs from one file extent to another, the closed extent usually gets renamed by appending the extent number to its name. For example, a file extent called PLG.3 in a disk section will be renamed to PLG.3(1), and a file extent called /tmp/BCK_DB in a file system will be renamed to /tmp/BCK_DB(1). The name of the new file extent contains the new file extent number. Thus, in the example just mentioned, the new file extent in the disk section is named PLG.3(2) and the new file extent in the file system is named /tmp/BCK_DB(2).

Note:
The size of the value of an environment variable is limited to 512 bytes or more, depending on the operating system. If you specify a large number of extents in one environment variable, it may happen that the value of the environment variable is truncated, and this will result in an error. In this case, you must specify the extents in separate environment variables.

Calling Utilities

There is one important restriction that applies to using the utilities with multiple extent files: input redirection is not allowed if more than one file extent is required and only one environment variable is set to only one value. In this case, the utility aborts with a message of the following form:

%ADABCK-E-SFNOTPO, subsequent file extents not possible with input redirection

The following modes will work successfully:

Interactive (Linux)

adabck <RETURN>
dbid=5, dump=4 <RETURN>
<CTRL/D>

Interactive (Windows)

adabck <RETURN>
dbid=5, dump=4 <RETURN>
<CTRL/Z><RETURN>

Interactive, command line

adabck dbid=5 dump=4

Batch job

adabck dbid=5 dump=4 &

Reading Multiple-Extent Files

When reading sequential files that are split into multiple extents, Adabas checks whether an extent is finished and prompts for the file/device path name of the next extent at which reading is to continue, if just the normal environment variable is given.

Example:

ADAREC regenerates a Protection Log that is spread over 2 sections. The environment variable RECPLG must point to the first extent of protection log 5 (PLG.5(1)). Once it has been processed, ADAREC prompts for the file/device path name of the next extent:

 Enter next file/device path name to continue PLG.5(2)

After opening the subsequent extent, Adabas checks to see whether the given extent is the one it expects. If it is not the correct one, an error message is displayed and the prompt for the correct file/device path name is displayed again. If such a subsequent prompt appears, you can abort further processing by entering "quit". Situations can arise where no subsequent protection log extent exists although Adabas expects one. In this case, you can enter "close" to terminate the processing of the current protection log file without error.

It is also possible to process the files in one go, without prompting, when using `counted' environment variables. In this case, the same technique is used as when creating the files.

Example

C Shell:

setenv RECPLG /dev/rmt/0m
setenv RECPLG2 /dev/rdsk/c10d0s2
setenv RECPLG3 /dev/rdsk/c7d0s2

Bourne Shell, Korn Shell:

RECPLG = /dev/rmt/0m
export RECPLG
RECPLG2 = /dev/rdsk/c10d0s2
export RECPLG2
RECPLG3 = /dev/rdsk/c7d0s2
export RECPLG3

Windows:

set RECPLG=\\.\TAPE0
set RECPLG2=C:\BCK_2
set RECPLG3=D:\BCK_3

Example

An alternative method is to set all of the file/device names within one environment variable.

If the mass update utility ADAMUP uses a data file (DTA) that is split into 2 extents, then the following can be set:

C Shell:

setenv  MUPDTA  "/dev/rdsk/c5d0s2  /usr/adabas/DTA_2"

Bourne Shell, Korn Shell:

MUPDTA  = "/dev/rdsk/c5d0s2  /usr/adabas/DTA_2"export MUPDTA

Windows:

set MUPDTA=\\.\TAPE0;C:\adabas\DTA_2

Optimization of ADAMUP and ADAINV Execution

In order to create the index, the utilities ADAINV and ADAMUP must sort the descriptor values. The efficiency of the sort execution can be impacted by the size of utility parameter LWP, which specifies the amount of memory that can be used additionally to the minimum memory allocated for the utilities. In the following, the impact of the size of the LWP parameter for the execution of the sort operations is described.

  • Depending on the size of the LWP parameter, an area in memory is provided for an in-memory sort. If LWP is large enough, all descriptor values can be sorted in memory, and the index can be created, and no SORT containers are required. The required value for an in-memory sort is displayed by the ADAMUP and ADAINV SUMMARY functions.

  • If the area is too small for an in-memory sort, the data to be sorted are divided in subsets that can be sorted in memory. The number of subsets depends on the size of the LWP parameter: the number of subsets can be reduced by increasing the size of LWP. After the in-memory sort of these subsets, the sorted subsets are written to the SORT container.

  • Once all of the subsets have been sorted, the sorted subsets must be merged. However, it is only possible to merge a limited number of subsets in a single pass. If the number of subsets does not exceed the limit, all descriptor values are sorted after the merge, and the index can be created. The required LWP value for a single- pass merge and the required size of the SORT container are displayed by the ADAMUP and ADAINV SUMMARY functions.

  • If the number of subsets is higher, the merged subsets are written to disk again, and another merge pass is required; this also means that the SORT container must have be double the size compared to the size required if a single merge pass is sufficient. The required size of the SORT container is displayed by the ADAMUP and ADAINV SUMMARY functions.

  • The required size of the SORT container depends only on the number of merge passes, but not on the exact value of the LWP parameter: If the LWP parameter is large enough for a single-pass merge, you need half the amount of disk space that is required for a merge in two or more passes.

  • An in-memory sort is not necessarily faster than a sort with one or more additional merge passes. The reason for this is that the sort algorithm randomly accesses the provided memory. With LWP=0, you usually don’t have many memory-cache misses, but with a larger LWP parameter, you have a lot. This can increase the sort time more than the time required for merging the sorted subsets. Merging the sorted subsets doesn’t cause many memory cache misses, and also the negative impact of the I/Os for the sort container is limited, since the I/Os are performed asynchronously, and it may happen that data are still in the file system cache for the sort container.

  • The performance of the different system components may vary greatly between different computers; therefore, the recommendation is to try the different alternatives if you perform ADAMUP or ADAINV regularly. If you don’t perform these utilities often, it may not be worth the effort.

  • Performing ADAMUP SUMMARY is very fast, because the descriptor summary is already created when the MUPDTA and MUPDVT files are created, and is stored in the MUPDTA file. ADAINV SUMMARY, on the oither hand, must read all DATA blocks of the file to create the descriptor summary, and, therefore, is relatively slow.

  • The SUMMARY functions also calculate the recommended TEMP sizes: the utilities can process only one descriptor at a time, but they always see the values of all descriptors. In order to prevent ADAINV having to reread the DATA blocks, the values for the other descriptors are stored on TEMP, grouped by descriptor. It should be possible to store at least the largest descriptor; ideally TEMP should be large enough to process all descriptors in one pass. Because ADAINV does not know the descriptor sizes in advance, the TEMP size required for one pass are relatively large; therefore ADAINV SUMMARY also displays the TEMP size required for processing the descriptors in two passes.

  • The values computed by the SUMMARY functions are not the exact minimum values; the values are rounded up, i.e. these values are sufficient, but smaller values may be also sufficient.

Synchronization Between Nucleus and Utilities

The Utility Communication Block (UCB)

Most utilities can write to the database containers and can be performed both in online mode, i.e. when the nucleus is active, and in offline mode, i.e. when the nucleus is not active. This means that a synchronization mechanism is required for the nucleus and the utilities, independent of an active nucleus, in order to guarantee the integrity of the database.

For this purpose, each Adabas database contains a Utility Communication Block (UCB). The UCB is stored in RABN 3 of the ASSO container. Each time that utilities need to have read or write access for one or more files, corresponding UCB entries must first be created that contains the following information:

  • The file number(s) of the Adabas file(s) to be processed;

  • The access mode (read or write).

In addition, the access to some global data structures in the database containers must be synchronized between the nucleus and the utilities.

In online mode, the synchronization between utilities and nucleus is always done via the nucleus: The utilities issue special Adabas commands (U calls), and the nucleus updates the UCB and other global database blocks.

In offline mode, the utility itself updates the UCB and the other global database blocks; in order to avoid a situation in which two utility processes concurrently update these blocks, the access is protected via a semaphore. The semaphore protects only the access to the global data structures; the access to the file-specific data is protected via the UCB.

When a utility terminates, it normally removes its UCB entry again, but if a utility terminates abnormally, it can happen that the UCB entry for the utility is not removed. In this case, it is necessary to remove the UCB manually using ADADBM RESET=UCB.

Note:
There are some read-only utilities that do not synchronize with the nucleus and other utilities (ADAREP and ADAPRI). This has the advantage that these utilities can always be executed, with the risk, that in some rare cases, it may happen that the utility terminates abnormally.

Databases in Read-Only Mode

A database can also be in read-only mode, either if the containers are write-protected, or via specifying the nucleus parameter OPTION=READONLY. For a database in read-only mode, it is not possible to write the UCB; nevertheless a synchronization is necessary because the read-only mode may be caused only by the nucleus parameter OPTION=READONLY, and the containers are writable. The nucleus and utilities access a shared memory where a General Control Block (GCB) is stored; this is a shared memory that is created by the first nucleus or utility process started, and removed again when the last nucleus and utility process using it, is terminated. This shared memory is used to mark the database as read-only. This is done when the shared memory is created, and either the containers are read-only, or the nucleus is started with OPTION=READONLY.

The following rules apply:

  • If the shared memory is marked as read-only, it is only possible to start the nucleus in read-only mode, or to start utility functions that only read the database containers. A writing nucleus or utility can be started again only after the shared memory has been removed again;

  • If the shared memory is not marked as read-only, it is not possible to start the nucleus in read-only mode, because it is possible that the utility that created the shared memory modifies the database, and a read-only nucleus assumes that this does not happen.