This document describes the utility "ADAREC".
The following topics are covered:
The ADAREC utility consists of the following database recovery functions:
The LIST function lists information about a Protection Log.
The REGENERATE function re-applies all of the updates made between two specified checkpoints. The checkpoints used are normally the result of a checkpoint command (C1) but may also be internal checkpoints taken by OP commands from EXU users or utility actions. If the whole database is to be regenerated, certain files may be excluded by using the EXCLUDE_FILES option. The files specified with this option are not regenerated, and the updates that are excluded are reported.
If REGENERATE terminates at a SYNP checkpoint, ADAREC "looks ahead" on the current PLOG to find an alternative restart point for the next run of this PLOG. The utility then displays a list of other utility functions that have to be executed before ADAREC can be restarted. If one or more SYNP checkpoints were found, ADAREC terminates
with exit code 14, if the PLOG contains further transactions to be applied via a restart of ADAREC,
otherwise with exit code 12.
The calculated restart point can be reset or overridden by entering BLOCK = or CHECKPOINT =. Refer to the database report utility ADAREP in this manual for a description of the possible system checkpoint types.
Normally, REGENERATE completes all fully-logged and confirmed transactions. This function is most frequently used when the database (or one or more files) has been restored to a previous status with the RESTORE function of the ADABCK utility.
If the utility writes records to the error file, it will exit with a non-zero status.
Notes:
This utility is a single-function utility. For more information about single- and multi-function utilities, see Adabas Basics, Using Utilities.
| Data Set | Environment Variable/ Logical Name |
Storage Medium |
Additional Information |
|---|---|---|---|
| Control statements | stdin | Utilities Manual | |
| ADAREC messages | stdout | Messages and Codes | |
| Rejected data | RECERR | Disk (* see note) | Output of ADAREC |
| Protection log | RECPLG | Disk |
Note:
(*) A named pipe can be used for this sequential
file.
The sequential file RECPLG can have multiple extents. For detailed information about sequential files with multiple extents, see Adabas Basics, Using Utilities
The following table shows the nucleus requirements for each function and the checkpoints written:
| Function | Nucleus must be active | Nucleus must NOT be active | Nucleus is NOT required | Checkpoint written |
|---|---|---|---|---|
| LIST | X | - | ||
| REGENERATE | X | SYNX |
Data protection information, in the form of `before' and `after' images of all updated records, is written to the Protection Log during each Adabas session. This information is needed to regenerate the updates.
The following control parameters are available:
M DBID = number LIST = keyword REGENERATE = {* [,EXCLUDE_FILES = (number[-number] [,number[-number] ] ... ) } | (number[-number] [,number[-number] ] ... ) PLOG = number D [,[NO]BI_CHECK] [,BLOCK = ([number][,number]) ,CHECKPOINT = ([string][,string])] D [,[NO]ERROR_LOG] D [,ON_ERROR = keyword]
DBID = number
This parameter selects the database to be used.
Note:
Program functions which do not require the nucleus to
be running need the environment variables/logical names set for the container
files.
[PLOG=number,] LIST = keyword
Valid keywords are BRIEF, FULL and RESTART. BRIEF lists the Protection Log number and its creation date. FULL lists additional information about the records on the Protection Log, e.g. the checkpoints, the number of modifications for each file, etc. RESTART displays the restart points that ADAREC writes when it encounters checkpoints while processing.
Note:
The timestamps displayed for checkpoints are the timestamps that were made when the
checkpoints were included in the PLOG. When offline checkpoints are created, the
checkpoints are first written to the checkpoint block in the ASSO. The next time the
nucleus is started, they are written to the checkpoint file with the actual checkpoint
creation date, and to the PLOG with the current date. This implies that the timestamps
for offline checkpoints displayed with ADAREP CHECKPOINT and by ADAREC LIST are
different.
The LIST=FULL function also checks the structure of the Protection Log to ensure that it is internally consistent. If a structural error is detected, a message is output indicating the error type as well as the record and block numbers.
If the Protection Log is within a disk section, the PLOG parameter must be set before LIST can be specified.
adarec: list=brief Protection log 1 - 26-OCT-2006 11:39:03
The creation date of PLOG 1 is displayed.
This function is used to regenerate a whole database or files within a database.
REGENERATE = *, PLOG = number
[,EXCLUDE_FILES = (number[-number][,number[-number]]...)]
[,[NO]BI_CHECK]
[,BLOCK = ([number][,number]),
CHECKPOINT = ([string][,string])]
[,[NO]ERROR_LOG]
[,ON_ERROR = keyword]
This option of the REGENERATE function regenerates a database. A file exclusion list can be used to exclude certain files from the regenerate. ET logic is supported.
During REGENERATE processing, ADAREC sets the database to utility-only mode. Processing terminates if a SYNP checkpoint is encountered. In this case, ADAREC inspects the Protection Log in order to calculate an alternative restart point. This restart point is then displayed together with a list of utility functions that must be executed before processing can be continued. The next call to REGENERATE automatically sets up at this point. The use of the calculated restart point can be overridden by specifying "BLOCK=" or "CHECKPOINT=" (that is, supplying empty values for these keywords). This procedure is repeated until the end of the PLOG is reached. After ADAREC has terminated, the database remains in utility-only mode, because more calls to REGENERATE may follow. After the database regeneration has finished, you can enable the database for normal processing with the ADAOPR command OPTIONS=NOUTILITIES_ONLY.
If this option is set to BI_CHECK, ADAREC checks the consistency of the before images in the Protection Log against the data in the database (is the ISN in use; does the record exist; is there a before image mismatch?). If a mismatch is encountered, ADAREC issues messages containing the relevant information and does not perform the update.
If this option is set to NOBI_CHECK, the consistency check is still made and the ERROR_LOG is implicitly enabled; however, on finding a BI inconsistency, the update is made and the mismatch is reported to the ERROR_LOG (see below). If errors are encountered, only the first error for each file will be displayed, all subsequent errors are logged to the ERROR_LOG. Note that the index might become inconsistent in this case.
However, if the PLOG was written with the NOBI option of the nucleus, it will not contain any before images and the BI_CHECK option cannot be set.
The default is BI_CHECK.
This parameter specifies the numbers of the blocks in the Protection Log files that contain the corresponding checkpoint names. The block numbers can be taken from ADAREC LIST=FULL.
This parameter specifies the starting and ending checkpoint names. The checkpoint names can be taken from the ADAREP database status report or ADAREC LIST=FULL.
If processing is to start at the beginning of the Protection Log file, the first parameter must be omitted.
Setting this option to ERROR_LOG enables the automatic logging of any BI inconsistencies that may be detected when using the NOBI_CHECK option. The contents of the error file produced can be examined using the ADAERR utility. Do not print this error file using the standard operating system print utilities, since the records contain nonprintable characters. See ADAERR for further information.
The default is NOERROR_LOG.
This parameter specifies the files to be excluded when regenerating a complete database. The updates that are excluded are written to a report.
Valid keywords are ABORT and EXCLUDE. The keyword used determines what action to take if ADAREC detects non-fatal errors during processing (e.g. response code 17, file not loaded). ABORT abnormally terminates regenerate processing, and EXCLUDE excludes the file in question from the regenerate if Data Storage errors occur (nucleus response codes 17, 49, 75, 77 and 113).
If, however, an error occurs while updating a file's index (nucleus response codes 75, 76, 77, 98, 165, 166, 167 and 176), only the regeneration of the Data Storage for this file will continue. When the regeneration process is complete, the index of this file is marked as invalid. The ADAINV REINVERT function with the ALL_FIELDS option then has to be run for this file (please refer to the ADAINV utility in this manual for more detailed information). If index errors occur and if the regenerate includes several Protection Logs, all of the Protection Logs should be processed before reinverting the index. Reinverting the index each time a Protection Log results in index errors would waste considerable amounts of time and computer resources.
The default is ON_ERROR=EXCLUDE.
This parameter specifies the log number of the Adabas Protection Log to be used as input for the REGENERATE function. This number can be found with ADAREC using the LIST = BRIEF function.
REGENERATE = (number[-number][,number[-number]]...), PLOG = number
[,[NO]BI_CHECK]
[,BLOCK = ([number][,number]),
CHECKPOINT = ([string][,string])]
[,[NO]ERROR_LOG]
[,ON_ERROR = keyword]
This option of the REGENERATE function re-applies all updates in a Protection Log for the specified files or ranges of files. LOB files specified are ignored, but the LOB files assigned to all base files specified are dumped too.
During regenerate processing, ADAREC locks the files for exclusive use. The regenerate terminates if a SYNP checkpoint is found while processing a protection log. In this case, ADAREC inspects the Protection Log in order to calculate an alternative restart point. This restart point is then displayed with a list of utility functions that must be executed before processing can be continued. The next call to REGENERATE automatically sets up at this point. The use of the calculated restart point can be overridden by specifying "BLOCK=" or "CHECKPOINT=" (that is, supplying empty values for these keywords). This procedure is repeated until the end of the Protection Log is reached.
The files remain locked, because more calls to REGENERATE may follow. After the files regeneration is finished, you must unlock the files with the ADAOPR command UNLOCK.
The following functions are not allowed while ADAREC is active:
ADAOPR ET_SYNC FEOF = PLOG
ADABCK DUMP
ADAOPR STOP to a sub-user while the associated ADAREC user exists
If this option is set to BI_CHECK, ADAREC checks the consistency of the before images in the Protection Log against the data in the database (is the ISN in use; does the record exist; is there a before image mismatch?). If a mismatch is encountered, ADAREC issues messages containing the relevant information and does not perform the update.
If this option is set to NOBI_CHECK, the consistency check is still made and the ERROR_LOG is implicitly enabled; however, on finding a BI inconsistency, the update is made and the mismatch is reported to the ERROR_LOG (see below). If errors are encountered, only the first error for each file will be displayed, all subsequent errors are logged to the ERROR_LOG. Note that the index might become inconsistent in this case.
NOBI_CHECK improves performance at the expense of possible loss of data consistency. We advise you therefore not to use NOBI_CHECK for mission critical databases.
The default is BI_CHECK.
This parameter specifies the blocks in the Protection Log files that contain the corresponding checkpoint names. The block numbers can be taken from ADAREC LIST=FULL.
This parameter specifies the starting and ending checkpoint names. The checkpoint names can be taken from the ADAREP database status report.
If processing is to start at the beginning of the Protection Log file, the first parameter must be omitted. However, if the first checkpoint name is supplied, it must be found in the first Protection Log file.
If processing is to stop at the end of the last Protection Log file, the second checkpoint name must be omitted.
Setting this option to ERROR_LOG enables the automatic logging of any BI inconsistencies that may be detected when using the NOBI_CHECK option. The contents of the error file produced can be examined using the ADAERR utility. . Please refer to the ADAERR utility in this manual for more detailed information.
The default is NOERROR_LOG.
Valid keywords are ABORT and EXCLUDE. The keyword used determines what action to take if ADAREC detects non-fatal errors during processing (e.g. response code 17, file not loaded). ABORT abnormally terminates regenerate processing, and EXCLUDE excludes the file in question from the regenerate if Data Storage errors occur (nucleus response codes 17, 49, 75, 77 and 113).
If, however, an error occurs while updating a file's index (nucleus response codes 75, 76, 77, 98, 165, 166, 167 and 176), only the regeneration of the Data Storage for this file will continue. When the regeneration process is complete, the index of this file is marked as invalid. The ADAINV REINVERT function with the ALL_FIELDS option then has to be run for this file (please refer to the ADAINV utility in this manual for more detailed information). If index errors occur and if the regenerate includes several Protection Logs, all of the Protection Logs should be processed before reinverting the index. Reinverting the index each time a Protection Log results in index errors would waste considerable amounts of time and computer resources.
The default is ON_ERROR=EXCLUDE.
This parameter specifies the log number of the Adabas Protection Log to be used as input for the REGENERATE function. This number can be found with ADAREC using the LIST = BRIEF function.
In this example, database 2 is to be regenerated using the Protection Log 2. File 12 is to be excluded from the regenerate.
adarec: regenerate=*,plog=2 adarec: exclude_files=12 adarec: Protection log 2 - 26-OCT-2006 11:48:59 Block 3 - checkpoint SYNC - 11:49:00 - USERID ADANUC <version> %ADAREC-I-CHKIGN, Checkpoint ignored The following utility functions were executed in the original session: Block 4 - checkpoint SYNP - 11:50:02 - USERID ADADBM REFRESH=13 Block 5 - checkpoint SYNX - 11:50:03 - USERID ADADBM RESET=UCB,IDENT=7 Block 6 - checkpoint SYNP - 11:50:03 - USERID ADADBM RECOVER Re-execute all SYNP utility functions starting from block 4. REGENERATE summary Calculated RESTART point - BLOCK=6,CHECKPOINT=SYNP
Processing of the Protection Log terminated at the SYNP checkpoint in block 4. However, no updates were found on looking ahead and processing can be continued from the calculated restart point in block 6. ADAREC displays a list of the utility functions that must be executed before processing continues. The next call to REGENERATE=* will automatically continue at this calculated restart point.
adarec: regenerate=*,plog=2
%adarec-I-restartp, calculated restart point - block=6,checkpoint=synp
adarec: exclude_files=12
adarec:
Protection log 2 - 26-OCT-2006 11:48:59.86
Block 6 - checkpoint SYNP - 11:50:03.86 - USERID ADADBM RECOVER
%ADAREC-I-CHKSTP, starting checkpoint
1 modifications in file 11
1 modifications EXCLUDED from file 12
4 ET commands issued
Block 7 - checkpoint SYNC - 11:52:38.98 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
Protection log 2 processed
Processing of the Protection Log continues at the calculated restart point. The regenerate terminates successfully.
In this example, database 2 is to be regenerated using the Protection Log 2. Processing is to start at the checkpoint SYNP in block 6 of the Protection Log. If Data Storage errors occur, the file in question will be excluded from the regenerate. If index errors occur, the file's index will be excluded from the regenerate and marked as invalid.
adarec: regenerate=*,plog=2,block=6,checkpoint=synp
adarec: on_error=exclude
adarec:
Protection log 2 - 26-OCT-2006 11:48:59.86
%ADAREC-W-UTIENA, OPTIONS=UTILITIES enabled in nucleus by ADAREC
%ADAREC-W-RECUPD, Updates performed between Nucleus and REGENERATE'S startup
%ADAREC-W-RECCMD, 1 N1 command(s)
Block 6 - checkpoint SYNP - 11:50:03.86 - USERID ADADBM RECOVER
%ADAREC-I-CHKSTP, starting checkpoint
1 modifications in file 11
3 ET commands issued
%ADAREC-E-ISNINUSE, ISN 774 in use in file 12
%ADAREC-I-PLOGRB, from record 14 in block 7 in PLOG 2
%ADAREC-I-UPDEXC, ALL following updates in file 12 will be EXCLUDED
1 modifications EXCLUDED from file 12
1 ET command issued
Block 7 - checkpoint SYNC - 11:52:38.98 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
Protection log 2 processed
An ISN conflict occurred in file 12 and all subsequent updates to this file were excluded. The cause of the error has to be investigated. However, the nucleus was started without `OPTIONS=UTILITIES_ONLY' and an N1 command was issued before the regenerate was started.
The Protection Log was processed to its end, the abort system message is used only to indicate a fatal error.
This example is similar to the previous one, with the exception that processing will abort if Data Storage or index errors are encountered.
adarec: regenerate=*,plog=2,block=6,checkpoint=synp
adarec: on_error=abort
adarec:
Protection log 2 - 26-OCT-2006 11:48:59.86
%ADAREC-W-UTIENA, OPTIONS=UTILITIES enabled in nucleus by ADAREC
%ADAREC-W-RECUPD, Updates performed between Nucleus and REGENERATE'S startup
%ADAREC-W-RECCMD, 1 N1 command(s)
Block 6 - checkpoint SYNP - 11:50:03.86 - USERID ADADBM RECOVER
%ADAREC-I-CHKSTP, starting checkpoint
1 modifications in file 11
3 ET commands issued
%ADAREC-E-ISNINUSE, ISN 774 in use in file 12
%ADAREC-I-PLOGRB, from record 14 in block 7 in PLOG 2
An ISN conflict occurred in file 12 and further processing was aborted.
In this example, database 2 is to be regenerated using the Protection Log 3. The before images in the Protection Log will be checked against the data in the database and mismatches will be displayed on the terminal.
adarec: regenerate=*,plog=3
adarec:
Protection log 3 - 26-OCT-2006 12:10:25.12
%ADAREC-W-UTIENA, OPTIONS=UTILITIES enabled in nucleus by ADAREC
Block 1 - checkpoint SYNC - 12:10:25.12 - USERID ADANUC 3.2/0 PL 0
%ADAREC-I-CHKIGN, Checkpoint ignored
1 ET command issued
%ADAREC-E-RECMIS, Before image mismatch for ISN 3 in file 11
%ADAREC-I-PLOGRB, from record 7 in block 2 in PLOG 3
%ADAREC-I-UPDEXC, ALL following updates in file 11 will be EXCLUDED
1 modifications EXCLUDED from file 11
1 modifications in file 12
3 ET commands issued
Block 2 - checkpoint SYNC - 12:11:44.30 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
Protection log 3 processed
One before image mismatch occurred during processing. As a result, one update was excluded from file 11.
Having restored the files, the same example can be rerun with no consistency check of the before images and with BI error logging enabled.
The Protection Log was processed to its end, the abort system message is used only to indicate a fatal error.
adarec: regenerate=*,plog=3,nobi_check
adarec:
Protection log 3 - 26-OCT-2006 12:10:25.12
%ADAREC-W-UTIENA, OPTIONS=UTILITIES enabled in nucleus by ADAREC
Block 1 - checkpoint SYNC - 12:10:25.12 - USERID ADANUC 3.2/0 PL 0
%ADAREC-I-CHKIGN, Checkpoint ignored
%ADAREC-W-RECMIS, Before image mismatch for ISN 3 in file 11
%ADAREC-I-PLOGRB, from record 7 in block 2 in PLOG 3
1 modifications in file 11
1 modifications in file 12
4 ET commands issued
1 BI_CHECK error in file 11
Block 2 - checkpoint SYNC - 12:11:44.30 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
1 BI_CHECK error in file 11
Protection log 3 processed
One BI_CHECK error occurred during processing.
The Protection Log was processed to its end, the abort system message is used only to indicate a fatal error.
The source of the errors is written to an error file which can be displayed using the ADAERR utility. The first error is logged and also written to the error file. All subsequent errors are written to ERROR_LOG.
The following error file was produced:
%ADAERR-E-RECMIS, Before image mismatch for ISN 3 in file 11 %ADAERR-I-PLOGRB, from record 7 in block 2 in PLOG 3
In this example, database 2 is to be regenerated using the Protection Log 3.
adarec: regenerate=*,plog=3
adarec:
Protection log 3 - 26-OCT-2006 12:10:25.12
%ADAREC-W-UTIENA, OPTIONS=UTILITIES enabled in nucleus by ADAREC
Block 1 - checkpoint SYNC - 12:10:25.12 - USERID ADANUC 3.2/0 PL 0
%ADAREC-I-CHKIGN, Checkpoint ignored
%ADAREC-E-ERRIUP, Error response 165 during index update
%ADAREC-E-Adabas_165, * Invalid descriptor name in DVT
%ADAREC-I-DESNAM, Descriptor name XA
%ADAREC-I-ISNFILE, from ISN 3 in file 11
%ADAREC-I-PLOGRB, from record 7 in block 2 in PLOG 3
%ADAREC-I-REINVERT, REINVERT all descriptors to re-establish INDEX
%ADAREC-I-REGDAT, Regenerating ONLY data-storage for file 11
1 modifications in file 11
1 modifications in file 12
4 ET commands issued
Block 2 - checkpoint SYNC - 12:11:44.30 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
Protection log 3 processed
An invalid descriptor name was encountered during processing. As a result, only the data storage of file 11 was regenerated. All of the descriptors will have to be reinverted in order to reestablish the index.
The Protection Log was processed to its end, the abort system message is used only to indicate a fatal error.
If index errors occur and if the regenerate includes several Protection Logs, all of the Protection Logs should be processed before reinverting the index. Reinverting the index each time a Protection Log results in index errors would waste considerable amounts of time and computer resources.
In this example, database 2 is to be regenerated using the Protection Log 4 after the regenerate processing of Protection Log 3 resulted in an index error.
adarec: regenerate=*,plog=4
adarec:
Protection log 4 - 26-OCT-2006 12:12:00.15
Block 1 - checkpoint SYNC - 12:12:00.15 - USERID ADANUC <version>
%ADAREC-I-CHKIGN, Checkpoint ignored
%ADAREC-E-FCBNAC, file 11's index not accessible
%ADAREC-I-REGDAT, Regenerating ONLY data-storage for file 11
1 modifications in file 11
1 modifications in file 12
4 ET commands issued
Block 2 - checkpoint SYNC - 12:12:19.35 - USERID ADANUC SHUTDOWN
%ADAREC-I-CHKIGN, Checkpoint ignored
REGENERATE summary
Protection log 4 processed
The index error that occurred while processing Protection Log 3 (see example 5) means that file 11's index is no longer accessible. Only the Data Storage of file 11 is regenerated, whereas both the Data Storage and the index of file 12 are regenerated.
The Protection Log was processed to its end, the abort system message is used only to indicate a fatal error.
An interrupted ADAREC run which leaves a UCB entry has to be re-started from the beginning. Because modifications have already been made, a RESTORE database or RESTORE file has to be executed before re-starting ADAREC. However, if there is no UCB entry, the database has not been modified and ADAREC can be re-started.
An abnormally terminated ADAREC (RESTORE/RECOVER) leaves the database in a consistent state, although it is not possible to tell exactly in which state. ADAREC cannot determine which transactions have already been recovered, so it is necessary to repeat the RESTORE operation and restart the ADAREC from the beginning in order to ensure that everything is recovered.
Having performed the first update, ADAREC writes a `started' checkpoint to the checkpoint file, e.g.
SYNX 22-MAR-2007 16:49:46 192 ADAREC REG STARTED