DataSync settings are accessed from the Gear menu
Sort Buffer
In the Sort buffer text box, enter the maximum amount of memory in megabytes used per thread to sort files. The number should
reflect the amount of memory available on the machine used as the sorting buffer and the number of threads used. The actual
amount used may be much smaller if the tables being synchronized are small. The default sort buffer of 80 megabytes should
be compatible with most systems that follow our recommended guidelines for minimal machine configuration.
Transaction Batch Size
Transaction Batch Size refers to how many records are inserted into the target database in a single transaction. Some database
systems have problems if too many records are involved in a transaction, and DataSync offers the ability to set this field
manually. 1000 is the default.
Create indexes before table population
To reorder the synchronization process so that indexes are created first, select Create indexes before table population. This
can have either a negative or positive effect on performance and space requirements of a full synchronization, depending upon
the database system. For instance, if the target is Oracle, it will be faster to create the indexes after the load of the
data. Inserting rows with an existing index updates the indexes on every insert, whereas creation of an index at the end of
the synchronization is somewhat more efficient.
On the other hand, with RMS as the target system, if indexes are created as the synchronization completes, then two entire
copies of the file will exist on the target system, as a sequential file is converted into an indexed file with a CONVERT/FDL
command. The best choice strongly depends on the type of database system you are using and on the amount of resources available
for performing the synchronizations.
Store transaction log on target
To create a transaction log which is stored in the target database, select the Store Transaction Log on Target option. If
this option is not selected, the transaction log is not created. A transaction log is a table that contains the unique index
information for each row that was inserted, updated, or deleted along with a flag that tells what sort of action was performed
on each row. The transaction log table will be created on the target database and will have the name of the source table
with the characters _ACD appended to the end. By using a transaction log, systems that need an audit process can tell exactly
what was changed and what sort of operation was performed on the data. It should be noted, however, that the creation of a
transaction log may slow synchronizations.
Logical update verification analysis
Logical Update Verification Analysis is used in conjunction with the Store Transaction Log on Target option.
For each change verified, the target table transaction log will change the 'C' (for changed) in the ACD column of the transaction
log to 'V' (for verified). If Logical Update Verification Analysis is selected, for each row that has had a change detected
via CRC differences, a column by column search is performed to confirm a difference.
Logical Update Verification Analysis should only be used in special cases, where a careful analysis has shown that more data
is being processed than is necessary. Selecting this option makes the synchronizations run more slowly, since for every change
detected, every single column in the record is compared from source to target - one a column at a time. If a post-process
step that is very slow must be performed on the changes (for example some expensive ETL operations) then logical update verification
analysis might be worthwhile. Some examples might include the following:
Multiple physical values with different internal representations but the same physical meaning, such as:
Positive and negative zero on one's complement machines
Positive signed and unsigned decimal numbers (e.g. signed + 1 and unsigned + 1 have the same meaning but different internal
representation
Insignificant changes. If, for example, the very last bit of a floating point number changed in the mantissa, the actual meaning
of what is stored may not really have changed. For instance, someone might have edited a field that contained the value "2.1"
as a floating point number. Though no changes were made, the value was saved again into the database. It is possible that
the number will differ slightly due to the nature of floating point. A comparison shows that both the source and target numbers
are intended as 2.1, but a
CRC comparison shows that some bit has changed and consider them as completely different.
Masked values. If, for example, a null-terminated string is stored in the source database, anything after the null string
terminator is a physical difference that causes a different CRC, but the data stored may be the same. For instance, the string
of bytes 'D', 'o', 'g', 0, 'f', 'o', 'o', 'd',0 will be interpreted simply as "Dog" since a null terminator (zero) follows
the 'g'. The string of bytes 'D', 'o', 'g', 0, 'f', 'i', 'g', 'h', 't', 0 will also be interpreted simply as "Dog" since a
null terminator (zero) follows the first 'g'. Anything beyond the null terminator has no meaning as far as the database is
concerned. But CRC values will indicate that there has been a change.
Recreate all indexes on target table
By default, only the unique index used to perform incremental updates is created on the target table after synchronization.
Select Recreate all indexes on target table to recreate all indexes from the source table on the target table.
Create targets in separate schemas (SQL Server/Oracle only)
If you are using SQL Server or Oracle is the target database, select Create target tables in separate schemas to create target
tables under different owners. DataSync creates a user for the purpose of creating tables with identical table names from
different source catalogs. For example, if you want to synchronize both the test and production version of a table into a
single SQL Server or Oracle database, CONNX fully qualifies the target name (product.table or test.table) so that there is
no name collision.
Add date/time column cnxSyncDate to target table
This option will append a date and time column named cnxSyncDate to the end of each row in the synchronized target tables.
The date and time of the last synchronization operation performed against the target row appears in the new column.
Disable logging
This option will prevent log information from being written to the datasync.log file. The datasync.log file is stored in the
datasync folder.
Days to Keep History
Synchronization records that are older than the specified value will be purged from the database. This helps keeps the database
size smaller. The default is 365 days.