This document contains information related to database performance monitoring and tuning.
The following topics are covered:
It is the responsibility of the DBA to monitor the database environment on a continuing basis in order to ensure that an efficient level of service is provided while maintaining database integrity. This responsibility for monitoring takes the form of a variety of activities and procedures, some of which are covered in this section.
The responsibility of the DBA to maintain database performance at an acceptable level is critical and difficult. The sources of degradation of service are numerous and often difficult to trace, while the process of making adjustments can be complex.
The DBA must implement a set of procedures which are designed to foresee degradation before the event and to adjust the operation or design of the database in an orderly and controlled way. This set of procedures will include the following activities:
Identifying potential sources of degradation;
Establishing tools for monitoring database performance;
Controlling the implementation of adjustments;
Controlling the implementation or redesign of the database to meet the new requirements.
The DBA should make regular reports on database usage and performance to both data processing and user management. These reports should be as factual as possible, but also include recommendations for the tuning of the database environment as he develops them. It should be remembered that tuning, while benefiting the organization as a whole, may adversely affect the service received by one or more users. Any decision on tuning should, therefore, be taken by all affected users.
The DBA should establish appropriate controls and monitor them to assist him in ensuring the integrity of the database.
Computer-generated control totals can be checked and cross-footed between computer processing runs or generated reports. Batch responses (or inquiries) may include such information as the exact run time, search parameters, time of last update of data, and the primary parameter controls. This increases the confidence level and helps to ensure the integrity of the database.
The problem of control totals is one which will take on many different guises at different installations. It is not possible to lay down hard and fast rules in this area. However, it is felt appropriate to give some general guidelines.
The DBA should ensure that proper consideration is given to the following areas in the design of each application system which will use the database:
What controls can be checked on every batch update run? For example, record counts, additions, deletions, updates;
What controls require a full file pass to check them? For example, value field hash totals;
What input transactions, Adabas logs, etc. should be retained in order to be able to recover when control totals are found to be wrong at the end of a given period;
Are 'localized' control totals (e.g., by branch, product group) of any use in identifying the areas affected by a file control total error?
The following table illustrates some of the monitoring statistics that may be used and what adjustments to (or tuning of) the database environment may result.
Changes in | May require tuning of ... | ||||
---|---|---|---|---|---|
Database Structure | Access Method Used | Hardware or Software Configuration | Processing Priority | Disk Storage Allocation | |
Terminal and line traffic | X | X | X | X | |
Response times (application performance) | X | X | X | X | X |
Access totals by user and descriptor | X | X | X | ||
Database size | X | X | X | X | |
Database growth rate | X | X | X | X |
When any alteration is made to a production database, great care must be taken to ensure a continued high level of reliability and integrity. Whatever the change, the DBA must make sure that the decision is the right one and that it is properly and accurately implemented. He should retain absolute control over the tuning process and ensure that it follows the formal acceptance procedures.
The DBA must be very careful not to over-react to changes in the items listed in the table. A sudden change in line traffic, response times, etc., may only be a temporary affair. It is far better to wait for a while to see whether this is a permanent trend or a temporary disturbance to the normal way of operating. Another way of viewing the table is that the tuning required may be necessary when a new project will cause a significant change in terminal and line traffic, response times, etc. The DBA can then act in advance to minimize these effects before the new application system is implemented.
The Adabas command log option of the Adabas nucleus may be used to generate information on all the commands issued by users to Adabas.
Some of the information provided is:
User identification
Time of day
What command was used
Which file was accessed
Which record was accessed
What Adabas response code was received
How long the command took to perform
This information provides a further level of knowledge of the activity against the database. It can be displayed using the utility ADACLP.
showcls is a tool that analyses the client queue semaphore of a database. It is called with the following syntax:
showcls <dbid>
showcls displays the status of each client queue element of the database:
Status | Meaning |
---|---|
Free | The client queue entry can be used for a new client connection. |
Obsolete | The client queue entry can be reused after the next client queue cleanup. |
In use | The client queue entry is currently in use. |
showipc is a tool that the DBA can use in order to look at the UNIX kernel ipc configuration, and to look at the Adabas/NET-WORK ipc structures and clean them up. For each database, showipc displays the shared memory, the message queues, and the semaphores associated with the database in question.
The following options are available:
showipc [-adfhiksv] [-e<string>] [-g<g_name>] [-r<sec>] [-u<u_name>] [<dbid>...]
One or more <dbid> arguments may be supplied. If <dbid> is omitted, showipc processes the ipc structures for all active databases. In addition to the numeric values that represent valid DBIDs, the strings CSCI, ACS and U can also be supplied. CSCI (client server communication interface) returns information for all ipc elements of CSCI users. ACS returns information about the ipc elements used by NETACS (NET-WORK access server). U indicates all elements that cannot be associated with a database (database unknown).
showipc also supports multiple logical NET-WORK nodes. The NET-WORK ipc structures are identified by the <dbid> zero and a NET-WORK ID. Unless the -a or -e options are used (see below), information is only displayed for the default NET-WORK ID. If the environment variable NET_WORK_ID is set, the corresponding NET-WORK ID will also be processed.
Note:
The information provided above concerning NET-WORK is only relevant
for NET-WORK Version 2, which is no longer supported with Adabas Version 6.
However, showipc of Adabas Version 6 also supports Adabas Version 5 databases,
which can still be used with NET-WORK Version 2.
The following table shows the meanings of the various options that are supported:
Option | Meaning |
---|---|
-a | displays the structures of all of the logical NET-WORK nodes that are running simultaneously. |
-d | used with the -k option, this forces a cleanup of IPC driver
resources, even if there are no ipc elements remaining for the database. The -d option should always be accompanied by a database ID. Note: |
-e<string> | specifies additional NET-WORK IDs for which information is to be displayed. Because the first character of a NET-WORK ID must be unique, each character of <string> uniquely identifies one NET-WORK ID. |
-f | used with the -k option, this forces the removal of the NETACS and CSCI elements (which would not be deleted otherwise). |
-g<g_name> | searches for the ipc elements that are owned by the group <g_name>, instead of the default users ('sag', 'adabas', 'natural', 'esq', $SIPGROUP). |
-h | calls up a help screen that explains how to use showipc. |
-i | used with the -k option, this requests confirmation before deleting any ipc element. |
-k | removes ipc elements that are no longer in use. Before it removes anything, showipc checks to see whether the GCB shared memory of the database is still in use. If it is still in use, any removal requests are rejected. Software AG strongly recommends that the -i option is used with the -k option. showipc is run with the effective user ID of root, otherwise you would not be able to remove the IPC resources if they were created by an Adabas nucleus process that was started by another user. |
-r<sec> | the specified command is repeated every <sec> seconds. The repetition can be stopped by entering CRTL-C. |
-s | displays the ipc configuration values of the running UNIX kernel. This can be useful to check whether the UNIX kernel is configured correctly. |
-u<u_name> | searches for the ipc elements that are owned by the user <u_name>, instead of the default users ('sag', 'adabas', 'natural', 'esq', $SIPUSER). |
-v | displays the database information in verbose format. |
The first column of the default output format contains either "NET" for NET-WORK (followed by the NET-WORK ID if it is not the default NET-WORK ID), or "NACS" for NETACS, or "CSCI" for CSCI, or the numeric DBID of the database in question. This is followed by a list of the ipc elements for the database. The entry for each element consists of an abbreviation to indicate its type, and, separated by a colon, the ipc id of the element. This id can then be used for the "ipcrm" command ("showipc -k <dbid>" is, however, much more convenient). The following abbreviations are used:
ATB | attached buffer shared memory |
CSA | common shared area |
CSM | CSCI shared memory |
DRV | Adabas ipc driver shared memory |
GCB | general control block |
GDT | global database table (NET-WORK) |
OPR | shared memory created with ADAOPR (called with CSA=) |
PHx | protocol handler with ID x (NET-WORK) |
ESi | Adabas SQL server shared memory ID i |
CLM | communication client message queue |
CSQ | CSCI message queue |
RSQ | user response queue |
SRV | communication server message queue |
SVQ | nucleus thread queue |
USQ | user request queue |
CLS | communication client semaphore |
CSS | CSCI semaphore |
PSE | private semaphore |
SEM | other semaphore |
ESQ | Adabas SQL server semaphore |
The verbose output format (-v option) displays additional information for each element. Three tables are displayed for each database: one for shared memory, one for message queues, and one for semaphores. The columns in these tables have the following meanings:
Column | Meaning |
---|---|
TYPE | type of ipc element (see abbreviations) |
NI | NET-WORK ID (if not default); may be truncated to one character |
ID | ipc ID |
SIZE | size of the shared memory in bytes |
Column | Meaning |
---|---|
LOPPID | process ID of the process that performed the last shmop() call |
CRPID | process ID of the creator process |
ATTPROC | number of attached processes (not always supported by the operating system) |
Column | Meaning |
---|---|
TYPE | type of ipc element (see abbreviations) |
NI | NET-WORK ID (if not default); may be truncated to one character |
ID | ipc ID |
NBYTES | number of bytes in the message queue |
NMSGS | number of messages in the message queue |
LSND | process ID of the last sending process |
LRCV | process ID of the last receiving process |
Column | Meaning |
---|---|
TYPE | type of ipc element (see abbreviations) |
NI | NET-WORK ID (if not default); may be truncated to one character |
ID | ipc ID |
For NETACS elements, the column TYPE is replaced by DOMN, which represents the NETACS domain.
Note:
Because showipc reads the kernel memory structures, it must be run
with the effective user id of root.
If the running kernel does not have the default name (for example
"/hp-ux" for HP-UX,, ...), showipc will abort with an appropriate
error message. This can be prevented by setting the environment variable
SIP_KERNAM to the correct name, in csh for example "setenv SIP_KERNAM
'/mykernel'".
The following environment variables are used together with showipc:
Environment Variable | Meaning |
---|---|
NET_WORK_ID | This specifies a NET-WORK ID to be processed in addition to the default NET-WORK ID (see the showipc options -e and -a for further information). |
SIP_KERNAM | This sets the name of the kernel (see the note above for further information). |
SIPGROUP | If the environment variable SIPGROUP is set, its contents will be appended to the list of default groups (see also option -g). |
SIPUSER | If the environment variable SIPUSER is set, its contents will be appended to the list of default users (see also option -u). For example, if SIPUSER is set to "harry", all of the ipc elements that belong either to the default users (currently "sag", "adabas", "esq" and "natural") or belong to "harry" will be displayed. |
The following messages may be received when using showipc:
Cleared up driver resources for DB xx | |
Explanation |
showipc not only removes ipc structures if the -k option is used, but for Adabas Version 2.1 and above also releases the ipc driver resources for the corresponding database. |
Action |
None. |
Could not attach to global data area of database xxx | |
Explanation |
The CSA of the database xxx is corrupted or not in the format expected by showipc. |
Action |
If this problem occurs with an active database, contact your nearest Adabas support centre. |
Database yy of Adabas Vz.z might be incorrectly displayed by showipc V x.x | |
Explanation |
The database in question is of a more recent version than the current version of showipc, which means that it cannot be guaranteed that the output of the ipc structures for this database is correct. |
Action |
Use an appropriate, more recent version of showipc. |
Db xx is of version ww - please use version ww of showipc to remove | |
Explanation |
The nucleus of database xx is of a higher version than the showipc called. showipc will not remove any ipc structure for this database. |
Action |
Use an appropriate, more recent version of showipc. |
Found inconsistent data structures for DB xxx | |
Explanation |
The internal data structures of database xxx are incompatible with showipc. |
Action |
Check the version of the database. If it is more recent than the version of showipc that you are using, use an appropriate version of showipc. If the problem still occurs, contact your nearest Adabas support centre. |
GCB of DB xx still in use, I do not remove anything | |
Explanation |
showipc -k was called for a database that is still active (running nucleus or being accessed by Adabas utilities). No ipc structures will be removed. |
Action |
If you really want to remove the ipc structures for the database in question, terminate the process that is accessing the database (use showipc -v <xx> to determine the process) and try the command again. |
GDT is still in use by DB xx; not removed | |
Explanation |
There is still at least one database using the GDT: it will not be removed by showipc. |
Action |
None. |
I do not remove private semaphores | |
Explanation |
showipc does not remove Adabas semaphores that are not associated with a specific database. |
Action |
None. Removing the semaphores with ipcrm could cause Adabas utilities to fail/abort. |
NET-WORK xx still running, I do not remove anything | |
Explanation |
The NET-WORK with the specified ID is still active. showipc does not remove any ipc structures that are associated with this NET-WORK ID. |
Action |
None. |
Sorry, but this program has to be run with supervisor privileges | |
Explanation |
The showipc executable does not have the correct permissions set. It must have a set user ID permission, and have the owner root. |
Action |
Check and, if necessary, correct the permissions. |
Sorry, I will remove CSCI elements only with -f option | |
Explanation |
CSCI elements can only be removed if the -f option is used. |
Action |
Add the -f option to the -k option. |
Sorry, I will remove CSCI elements only with -f option | |
Explanation |
NET-ACS elements can only be removed if the -f option is used. |
Action |
Add the -f option to the -k option. |
Structure mismatch for DB xx - Cannot identify version | |
Explanation |
The internal data structures of the database in question cannot be correctly interpreted. |
Action |
This message may occur during the startup of a database: in this case it can be ignored. |
Warning: could not find adanod for clean up of Adabas driver | |
Explanation |
In some cases, the utility adanod is required when cleaning up the Adabas ipc driver resources. |
Action |
Set the environment variable ADANOD to the full path name of the adanod executable. |
Warning: Found <structure> with invalid database ID xx contained | |
Explanation |
A structure that belongs to one of the relevant users has an invalid key. The structure can be either shared memory, a message queue, or a semaphore. |
Action |
Check which process created the structure in question (showipc displays the creator's process ID). If it is not a SOFTWARE AG product, it should be modified so that it does not interfere with Adabas. If it is a SOFTWARE AG product, contact your nearest Adabas support centre. |
Warning: Some NET-WORK IDs may be displayed truncated to one character | |
Explanation |
When the showipc options -e or -a are used, showipc might truncate some NET-WORK IDs to one character. NET-WORK IDs are, however, identified uniquely by their first character. |
Action |
You can force one NET-WORK ID to be expanded by setting the environment variable NET_WORK_ID accordingly. |
showipc
This displays the ipc structures of all Adabas databases and of the default NET-WORK ID.
showipc -av
This displays the ipc structures of all Adabas databases and of all logical NET-WORK IDs in the verbose format.
showipc -ki 127
This removes all of the ipc structures of database 127 if the database nucleus is not running. showipc will ask for confirmation before an ipc structure is removed.
Beginning with Adabas Version 6.1 on OpenVMS Itanium, Adabas uses a set of techniques for exchanging data data between client and server processes. These IPC (Interprocess Communication) techniques are divided into methods for synchronization, message passing and shared memory data exchange.
The IPC types used by Adabas for OpenVMS are implemented in an independent IPC layer, which uses the existing interprocess facilities available on OpenVMS:
Adabas IPC Type | Used OpenVMS Facility |
---|---|
Semaphores | Lock manager, global sections |
Message queues | Mail boxes |
Shared memory | Global sections |
Each IPC resource is represented in the system by a key name of 8 bytes length with the following structure:
Byte | Usage |
---|---|
1-2 | Base identification (always ‘AD’) |
3-4 | Internal identification for resources |
5-6 | Client identification |
7-8 | Database ID |
It is possible to have the same key name for different facilities. For example, Adabas uses the same key name for the ‘common client message queue’ and the ‘common shared area’.
Message Queues | ||
---|---|---|
Key | Identifier | Description |
AD00000C | CLM | Common Client Message Queue |
ADA0000C | SRV | Adanuc Receiver 0 Message Queue |
ADA0010C | SRV | Adanuc Receiver 1 Message Queue |
AD00010C | CLM | Client 1 Message Queue |
AD00020C | CLM | Client 1 Message Queue |
Shared Memory | ||
---|---|---|
Key | Identifier | Description |
ADA1000C | GCB | General Control Block |
ADA0000C | CSA | Common Shared Area |
ADAB000C | ATB | Attached Buffer |
ADAD000C | ATX | Attached Buffer Extended |
Semaphore Array | ||
---|---|---|
Key | Identifier | Description |
ADA3000C | ENQ | Adanuc and Utility Synchronization |
ADA1000C | CLS | Client Communication Semaphore |
ADAC000C | SDA | Server Dead or Alive Semaphore |
The Adabas DBA on the OpenVMS HP Integrity server must be able use IPC methods if emergency situations arise. ADAIPCS and ADAIPCRM are tools that the DBA can use in order to display and maintain IPC resources for each database running on the local node. These tools are necessary in case a database server or a utility are terminated abnormally.
The procedure adaipcs.com provides basic information about the IPC facilities of Adabas. It shows the key of the resource and the database ID to which this key belongs.
The following options are available:
Option | Resource |
---|---|
-M | Shared memory segment |
-Q | Message queues |
-S | Semaphore arrays |
-A | Print all information (default) |
-H | Print help file |
ADAIPCS returns a value of 1 if it completes successfully.
The ICP resources of DBID 41 are displayed.
$ @adaipcs ------ Shared Memory Segments -------- key dbid ADAB0029 41 ADA00029 41 ADA10029 41 ------ Semaphore Arrays -------- key dbid ADAC0029 41 ADA10029 41 ADA30029 41 ------ Message Queues -------- key dbid AD000029 41 ADA00029 41 ADA00129 41 $
The procedure adaipcrm.com removes one or more specified message queues, semaphore sets, or shared memory identifiers of an Adabas database. The database must not be running when IPC resources are removed. If the resource is being held by another process, it will be marked for deletion and remains active until the process in question is terminated.
The following options are available:
Option | Resource |
---|---|
-M shmkey | Remove the shared memory identifier, created with key shmkey, from the system. The shared memory segment and data structure associated with it are destroyed after the last detach. |
-Q msgkey | Remove the message queue identifier, created with key msgkey, from the system and destroy the message queue and data structure associated with it. |
-S semkey | Remove the semaphore identifier, created with key semkey, from the system and destroy the set of semaphores and data structure associated with it. |
-H | Print help file |
ADAIPCRM returns a value of 1 if it completes successfully. It returns a value >1 if any of the operations fail.
The value of the shared memory key is determined using ADAIPCS
$ adaipcrm -M ADAB0029 INFO: Global Section ADAB0029 deleted ... $
The DBA uses ADASHOWIPC to look at the Adabas IPC structures and clean them up. For each database, ADASHOWIPC displays the shared memory, the message queues, and the semaphores associated with that database. If the resource is being held by another process, it will be marked for deletion and remains active until the process in question is terminated.
The following options are available:
Option | Resource |
---|---|
-k | Remove all IPC resources of a database. |
-i | Ask before removing anything. |
-h | Print help file |
<dbid>... | DBID(s) of the database(s) to use. The default is all databases. |
The following abbreviations are used in the output:
Resource Type | Abbreviation | Meaning |
---|---|---|
Shared Memory | CSA | Common shared area |
GDT | Global database table | |
ATB | Attached buffer shared memory | |
GCB | General control block | |
SMP | SMP shared memory | |
Semaphore Arrays | CLS | Common client semaphore |
SDA | Server dead/alive semaphore | |
ENQ | Global semaphore array | |
Message Queues | USQ | User request queue |
SVQ | Nucleus thread queue | |
CLM | Common client message queue | |
RSQ | User response queue | |
SRV | Common server message queue |
$ adashowipc ADASHOWIPC (1.2.5) for Adabas OpenVMS DBID Shared Memory Message Queues Semaphores ----------------------------------------------------- 112 GCB CSA ATB SRV SRV CLM ENQ CLS SDA ATX 113 GCB CSA ATX SRV SRV CLM SDA CLS ENQ ATB CLM CLM CLM CLM CLM CLM CLM $
$ adashowipc -k 112 ADASHOWIPC (1.2.5) for Adabas OpenVMS Nucleus of DB 112 still running, I do not remove anything.
$ adashowipc -k 112 ADASHOWIPC (1.2.5) for Adabas OpenVMS DB 112: removing GCB: shared memory ADA10070 ... successful. DB 112: removing CSA: shared memory ADA00070 ... successful. DB 112: removing ATB: shared memory ADAB0070 ... successful. DB 112: removing ATX: shared memory ADAD0070 ... successful. DB 112: removing SRV: message queue ADA00170 ... successful. DB 112: removing SRV: message queue ADA00070 ... successful. DB 112: removing CLM: message queue AD000070 ... successful. DB 112: removing ENQ: semaphore ADA30070 ... successful. DB 112: removing CLS: semaphore ADA10070 ... successful. DB 112: removing SDA: semaphore ADAC0070 ... successful.
If a problem should occur at a customer site, Software AG's support team wants to supply solutions as fast as possible. To minimize the time required to analyze a problem and deliver a solution, the support team requires detailed information about the problem.
The standard problem information should contain:
Problem description:
Adabas component or function used
error messages (message ID)
short description of the problem
Environment description:
Adabas version
corrections used
operating system version
hardware environment (CPU type, disk device types, etc.)
Can the problem be reproduced ?
What are the steps to reproduce it
Which data is needed to reproduce it
If the problem is not reproducible, the problem information should contain additionally:
Where does the problem occur ?
on all databases or on a single databases only
on databases within a specific environment only
environments where the problem does not occur
When does the problem occur ?
always or only sporadically
in parallel with other events
periods of time when the problem does not occur
When did the problem first occur ?
changes of the environment around this date
last changes of the environment before this date
In the case of data corruption problems:
date of last successful database verification run
utilities used on the corrupted file (checkpoint list)
disk problems encountered
Depending on the nature of the problem, information supplied on tape is necessary for problem analysis. A description of the tape's content must be added, so that Software AG support staff can build an optimal environment into which the tape's contents can be copied. This is particularly important when the tape contains large amounts of data, since sufficient disk space has to be allocated for unloading the data. The problem data should be provided and documented in a way that allows files to be copied selectively.
The tape documentation should contain the label(s) of the tape(s) and a list of all files including their sizes. If backups are contained on the tape, the corresponding information should be supplied for each backup.
The problem data you should supply depends on the kind of problem. The problem should be assigned to one of the following problem categories:
nucleus abort
nucleus autorestart abort
utility abort
data corruption problem
other problem
The following problem data should be supplied on tape:
ADABAS.xxx global section file
ADANUC log file and process dump
ADANUC and ADASHR image
database layout and file information (ADAREP)
protection log and command log file (if available)
The following problem data should be supplied on tape:
files as listed for "Nucleus abort"
a copy of the WORK containing the autorestart data
The following problem data should be supplied on tape:
utility output (log file) and process dump
utility and ADASHR image
database layout and file information (ADAREP)
input and output files
The following problem data should be supplied on tape:
protection logs
utility usage information (checkpoint list)
information about the kind of corruption (ADAVFY output or information about an abort due to the corruption)
The following problem data should be supplied on tape:
information useful to reproduce the problem
any other information that might help, please ask your nearest Software AG Support centre for assistance