Processing of Store Clock Values

This document covers various aspects concerning the handling of store clock values in Natural applications.

The following topics are covered:


Original Store Clock and Year 2042 Issue

The Natural system variable *TIMESTMP provides the machine-internal store clock value as provided by the mainframe machine instruction STCK (Store Clock format). The store clock is an 8-byte binary field. It counts the microseconds since 1900-01-01 00:00:00 (UTC) whereby bit position 51 represents exactly one microsecond (bit position 0 is the leftmost bit). The remaining 12 bits can contain higher precision values such as nanoseconds or a processor ID, depending on the machine implementation. Store clock values are unique in a Natural session even if multiple store clock values are taken in one and the same microsecond.

STCK Layout

The store clock will reach its highest possible value on 2042-09-17 23:53:47.370495 (UTC) at which time all 52 bits will be set to 1. Afterwards, the store clock will restart counting from 0. The date on which the clock is reset is referred to below as wrap-date.

After the wrap-date, the store clock will use the same values as in the range from 1900 to 2042. Therefore, it cannot be determined whether a store clock has been taken before or after the wrap-date.

If the store clock covers the years from 1900 to 2042, we call it the original store clock.

The year 2042 issue leads to the following impacts:

  • Compare and sort
    Comparing time-related values should reflect the chronological sequence so that previous values are smaller than later values. Unfortunately, a store clock value taken just before the wrap-date is greater than a store clock value after that date when the store clock was reset to 0. Thus, comparing such store clock values will lead to incorrect results. For example, the store clock of 2040-01-01 (before the wrap-date) is greater than the store clock of 2043-01-01 (after the wrap-date). If both the store clock values taken before and after the wrap-date are used for sorting, they will not correspond to the chronological sequence. A READ which uses the mentioned values as start and end values, will not return any value because the end value is smaller than the start value.

  • Conversion
    If a store clock value is converted into a date, it is assumed that it was taken in the years from 1900 to 2042. If the store clock has actually been taken after the wrap-date, the conversion will therefore lead to an incorrect result. For example, a store clock value was taken on 2043-12-07 (after wrap-date). If this store clock is converted into a date, it will result in 1901-03-21.

  • Calculation of a time difference
    If store clock values are converted into microseconds, they can be used to calculate a time difference. This works fine if the wrap-date is not involved. Subtracting a (big) store clock value taken before the wrap-date from a (small) store clock value taken after the wrap-date will result in an invalid (big negative) result. For example, the calculation of the store clock time difference between the years 2039 and 2043, will result in “-138 years”.

We denote the time while we process only store clock values smaller than the wrap-date (2042-09-17) as the transition period. During this time, the original store clock values should be replaced by one of the approaches described in the following sections.

How to Overcome the Year 2042 Issue

To overcome the year 2042 issue, there are three approaches:

For these approaches, Software AG provides application programming interfaces (APIs) in the system library SYSEXT as described in the section Natural APIs for Store Clock Processing.

The Store Clock with Sliding Window interprets the store clock value in a sliding window. It supports the time range from 1971 to 2114. The binary values cannot be used for sorting or comparing.

The Extended Store Clock is 16 bytes long and supports the time range from 1900 to 38434. The binary values can be used for sorting and comparing. Extended store clock values are not compatible to store clock values (neither original nor with sliding window). Data and programs referring store clock values must be converted in one big step to extended store clock values.

The Smart Store Clock clock is 9 bytes long and supports the time range from 1900 to 38434. The binary values can be used for sorting and comparing. During the transition period, smart store clock values are compatible to original store clock values. Data and programs referring store clock values can be converted step by step to smart store clock values. This is the recommended approach to overcome the year 2042 issue.

Store Clock with Sliding Window

If a store clock value is converted into a date, the original store clock conversion assumes that it was taken in the range from 1900 to 2042. To extend the range, we use a sliding window which supports the range from 1971 to 2114. This implies that the years from 1900 to 1971 are no longer supported.

In the year 1971 the first bit in the store clock value switches to 1. Therefore, this bit is used for the realization of the sliding window:

  • From 1971 to 2042 the first bit is 1. Store clock values of this range are interpreted as before.

  • From 1900 to 1971 and from 2042 to 2114, the first bit is 0. With the sliding window these store clock values are interpreted as 2042 – 2114.

When a store clock with sliding window is converted into microseconds and the first bit is 0, the binary store clock value is extended at the left with x’01’ before the conversion, so that the result always reflects the number of microseconds since 1900-01-01.

The range of the store clock with sliding window

  • starts at 1971-05-11 11:56:53.685248 (UTC) and

  • ends at 2114-01-26 11:50:41.055743 (UTC).

Note:

  • If you sort 8-byte store clock values, they will not be in the chronological sequence even if you use the sliding window. For this task you must use smart or extended store clock values.

  • If you compare 8-byte store clock values, the result will not reflect the chronological sequence even if you use the sliding window. You can use the copycode USR9201D for this task.

  • If a store clock is taken after the end date of the sliding window, it will be wrongly interpreted. In this case you must also use smart or extended store clock values.

  • A store clock value is saved in an 8-byte field in which the last 12 bits are never zero. This fact can be used to determine whether the field for the store clock is still unused. If all 8 bytes in the field are zero (“NULL value”), the field is still unused, and a store clock value was not yet saved. You must ensure that the store clock field does not contain a NULL value before using it in a conversion function.

Compared with the original store clock, the store clock with sliding window provides the following improvements:

  • Conversion
    Natural APIs can be used to convert store clock values with sliding window correctly if store clock values have been taken from 1971 to 2114.

  • Calculation of a time difference
    Natural APIs can be used to convert store clock values with sliding window into microseconds (since 1900-01-01). A difference calculation with these values provides the correct time duration (if the store clock values are from 1971 to 2114).

  • Outlook
    Year 2042 issue is postponed until 2114.

Compared with the smart or extended store clock, the store clock with sliding window provides the following temporary advantages:

  • Size
    The size of the store clock value remains at 8 bytes.

  • Data conversion (compared with extended store clock)
    There is no need to migrate the store clock data to 16-byte values. Saved 8-byte store clock data will be interpreted correctly if it is in the range 1971-2114.

  • Program modifications
    The conversion implies minor changes. The system variable *TIMESTMP can be used further on.

Before migrating original store clock to store clock with sliding window, consider the following:

  • Further code changes will be required before the end of the sliding window in 2114.

  • If your application calls a Natural API which supports the original store clock value, replace it by an API which supports the store clock value with sliding window.

  • If you compare or sort binary store clock values, they will not be in the chronological sequence. See the note above.

Extended Store Clock

The Natural system variable *TIMESTMPX provides the machine-internal extended store clock value as provided by the mainframe machine instruction STCKE (Store Clock Extended format). The extended store clock is a 16-byte binary field. It counts the microseconds since 1900-01-01 00:00:00 (UTC) whereby bit position 59 represents exactly one microsecond (bit position 0 is the very left bit). The original store clock time representation is retained at byte 2-9. An extra overflow byte is added to the left, the so-called epoch index.

STCKE Layout

The extended store clock covers the years from 1900 to 38434.

Compared with the original store clock, the extended store clock provides the following improvements:

  • Compare and sort
    The extended store clock always delivers the expected chronological sequence when you compare or sort values.

  • Conversion
    Natural APIs can be used to convert extended store clock values. Because every extended store clock value in the extended store clock range is unique, the interpretation is always correct.

  • Calculation of a time difference
    Natural APIs can be used to convert extended store clock values into microseconds (since 1900-01-01). A difference calculation with these values provides the correct time duration.

  • Outlook
    No further code change is required in the future.

Before migrating from original store clock to extended store clock, consider the following:

  • The size of the binary field increases from 8 bytes to 16 bytes.

  • Extended store clock values are not compatible with (B8) store clock values. You cannot move a store clock value to an extended store clock value nor vice versa.

    Example:

    #STORECLOCK: DD943485BC302002
    
    MOVE #STORECLOCK TO #EXTENDED
    
    Received: 0000000000000000DD943485BC302002 
    Expected: 00DD943485BC30200200000000000000 
    
  • It is not possible to compare an extended store clock value directly against an original store clock value (nor against a Store Clock with Sliding Window). The extended store clock contains seven additional bytes added to the right and a comparison will not return a correct result.

  • Saved store clock values must be converted into extended store clock values. You may use the copycode USR9201R for this task.

  • If store clock values (B8) are saved in Adabas, you must edit the data so that the new (B16) field contains correct extended store clock values.

  • Every program which processes store clock values must be adjusted to extended store clock values. Because store clock values and extended store clock values are not compatible, the changes must be done in one step.

  • If your application calls a Natural API which supports the original store clock value, replace it by an API which supports the extended store clock value.

  • This complete conversion implies major changes.

  • It is recommended to use the smart store clock instead of the extended store clock. This is described in the following section.

Smart Store Clock

The smart store clock is a 9-byte binary field. It counts the microseconds since 1900-01-01 00:00:00 (UTC) whereby bit position 59 represents exactly one microsecond (bit position 0 is the very left bit). The original store clock time representation is retained at byte 2-9. An extra overflow byte is added to the left, the so-called epoch index. In other words, the smart store clock comprises the first 9 bytes of the extended store clock.

smart

The smart store clock covers the years from 1900 to 38434.

Compared with the original store clock, the smart store clock provides the following improvements:

  • Compare and sort
    The smart store clock always delivers the expected chronological sequence when you compare or sort values.

  • Conversion
    Natural APIs can be used to convert smart store clock values. Because every smart store clock value in the smart store clock range is unique, the interpretation is always correct.

  • Calculation of a time difference
    Natural APIs can be used to convert smart store clock values into microseconds (since 1900-01-01). A difference calculation with these values provides the correct time duration.

  • Outlook
    No further code change is required in the future.

Before migrating from original store clock to smart store clock, consider the following:

  • The size of the binary field increases from 8 bytes to 9 bytes

  • During the transition period:

    • Increasing the length of an original store clock value from (B8) to (B9) gives the correct smart store clock value. This is because in Natural and Adabas, binary fields are right aligned. Increasing the length will fill the overflow byte with H’00’.

    • Smart store clock values are compatible with original store clock values. You can move an original store clock value to a smart store clock value and vice versa.

      Example:

      #STORECLOCK: DD943485BC302002
      
      MOVE #STORECLOCK TO #SMART
      
      Received: 00DD943485BC302002
      Expected: 00DD943485BC302002
      
    • Smart store clock values can be compared with original store clock values.

  • Every program which processes store clock values must be adjusted to smart store clock values. Because store clock values and smart store clock values are compatible, the changes can be done step by step.

  • If your application calls a Natural API which supports the original store clock value, replace it by an API which supports the smart store clock value.

  • This complete conversion implies minor changes.

Transition to Smart Store Clock

Data Definition in Natural

A field containing store clock values is defined as

1 #TIMESTMP (B8)   /* original store clock value

Replace the definition by

1 #TIMESTMP (B9)   /* smart store clock value

Receive Current Store Clock Value

To receive the current store clock value, the following statement was executed

MOVE *TIMESTMP TO #TIMESTMP   /* get current timestamp

During the transition period, the statement can still be used even if the length of #TIMESTMP has increased to (B9). On the long term, the statement should be replaced by

INCLUDE USR9201F '*TIMESTMPX' '#TIMESTMP' /* get current timestamp

Alternatively, you can use a redefinition:

1 #TIMESTMPX (B16)   /* extended store clock value
1 REDEFINE #TIMESTMPX
  2 #TIMESTMP (B9)   /* smart store clock value

And use the following statement:

MOVE *TIMESTMPX TO #TIMESTMPX /* get current timestamp

Move Store Clock Values

During the transition period, store clock values can be moved from one to the other even if the length of one is (B9) and the other is still (B8).

MOVE #TIMESTMP-A TO #TIMESTMP-B   /* move store clock value

Calling a Subprogram

A subprogram uses a store clock value as parameter

1 #TIMESTMP (B8)   /* store clock value

If you increase the length of the field to (B9) and there are programs which call it with (B8), the program will fail. Use in this case the following definition

1 #TIMESTMP (B9) BY VALUE RESULT  /* store clock value

As soon as all callers use (B9) store clock values, the BY VALUE RESULT can be removed.

Compare Store Clock Values

During the transition period, store clock values can be compared with each other even if the length of one is (B9) and the other is still (B8).

If #TIMESTMP-A > #TIMESTMP-B   /* compare store clock values

Field Definition in Adabas

A field containing store clock values is defined as

1,AA,8,B,...

Replace the definition by

1,AA,9,B,...

On the mainframe, you can achieve it with the Adabas Online Service or the ADADBS (Database Services) utility; on Linux and Windows, with the ADAFDU (File Definition) utility. Note that on Linux and Windows, a field containing store clock values must be defined with the HF option (high order first).

With the change of the length, the database contains valid smart store clock values. There is no need to edit the data. If the field is a descriptor, it will provide the correct chronological sequence.

On the Natural side, you must update the DDM accordingly, and if a length is specified, also the related views in the data definitions.

Superdescriptor Definition in Adabas

A field AA containing store clock values is used as parent field of a superdescriptor:

SA=AC(1,20),AA(1,8)

During the transition period, the superdescriptor can still be used even if the length of the parent field has increased to 9 bytes. This is because the bytes in a binary field are counted from right to left.

On the long term, the definition should be replaced by

SA=AC(1,20),AA(1,9)

Every variable in Natural programs (like start and end values) referring the superdescriptor, must be adjusted together with the database changes. Alternatively, you can keep the original superdescriptor and create additionally a new superdescriptor with the new length. Then you can switch one program after the other to the new superdescriptor. Once all programs access the new superdescriptor, the old one can be released (deleted).

Local Store Clock

The Natural system variable *TIMX provides the local time in tenth of seconds. If the local time is required in a higher precision, the Natural API USR9178N can be used. It provides the local time and the corresponding UTC time in store clock format (B8), in microseconds since 1900-01-01 and in the Natural time (T) format. Additionally, it provides the time differential, a standard DATETIME value and the time zone. Functions are available to convert the values into other formats. The copycodes USR9178X, USR9178Y and USR9178Z provide mainly the same functionality as the subprogram USR9178N with an essentially better performance.

Layout of a Local Store Clock Value

The first 7 bytes in the local store clock value have the same content as the original store clock whereby bit positions 0 – 51 contain the microseconds of the local time. Therefore, standard APIs (like USR1023* or USR9201*) can be used for the interpretation of a local store clock value.

The last byte of a local store clock value generated by USR9178*, contains the time differential. With this information it can later be determined in which time zone the local store clock was taken, especially whether it has been taken in the DST (daylight saving time).

Time Differential and Time Zone

The time differential reflects the following difference:

Time differential = Local time - UTC time

The API USR9178N and the corresponding copycodes provide the time differential in units of 1/10 seconds.

The last byte of a local store clock value contains the time differential in units of 15 min as signed integer (format I1).

The API USR9178N also returns the time zone related to the time differential:

  • -hh:ii for a negative time differential, or

  • +hh:ii otherwise.

Standard DATETIME

The API USR9178N returns the standard DATETIME value:

yyyy-mm-ddThh:ii:ss.fff

where fff is the time fraction in milliseconds.

Natural APIs for Store Clock Processing

The following Natural Application Programming Interfaces (APIs) for processing of store clock values are provided in the Natural library SYSEXT:

Natural API
Function
Store Clock
Previous Version
Original
Sliding Window
Smart / Extended
USR1009N Convert store clock with sliding window into microseconds Yes Yes No  
USR1023N Convert time-related variables Yes No No  
USR9178N Maintain local store clock value No Yes No  
USR9201N Convert time-related variables No Yes Yes USR1023N

USR1009N

The API converts a store clock value into microseconds since 1900-01-01. The store clock value is interpreted by default without sliding window (range 1900-2042). An optional parameter is available to interpret the store clock value with the sliding window (range 1971-2114).

It is recommended to use the copycode USR9201Y (with sliding window) or USR1023Y (without sliding window) instead of the API USR1009N for better performance.

USR1023N

The API converts a time-related variable into other formats. The API uses the original store clock (1900 – 2042).

It is strongly recommended to use the API USR9201N instead of USR1023N. Otherwise, store clock values will be incorrectly interpreted after the year 2042. If you take, for example, a store clock on 2043-12-07, the API USR1023N will interpret it as 1901-03-21.

The API USR1023N expects one of the following formats as input:

  • Natural date and time,

  • Natural date,

  • microseconds (since 1900-01-01), or

  • original store clock,

and converts it into all other formats.

Additional to the API, the following copycodes are provided:

  • USR1023W - Convert microseconds into original store clock

  • USR1023X - Convert microseconds into Natural time

  • USR1023Y - Convert original store clock into microseconds

  • USR1023Z - Convert Natural time into microseconds

The copycodes are essentially faster than the subprogram USR1023N.

USR9178N

The API maintains local store clock values. It supports store clock with sliding window (1971 - 2114).

For function “C”, it returns the current values of:

  • Local store clock

  • UTC store clock

  • Time differential in 1/10 seconds

  • Local time in microseconds

  • UTC time in microseconds

  • Local time in Natural (T) format

  • UTC time in Natural (T) format

  • Standard DATETIME

  • Time zone

For function “L”, it converts a local store clock value into the other formats.

For function “U”, it converts a UTC store clock value and a time differential into the other formats.

Additional to the API, the following copycodes are provided:

  • USR9178X - Get current local store clock, UTC store clock and time differential.

  • USR9178Y - Convert local store clock into UTC store clock and time differential.

  • USR9178Z - Convert UTC store clock and time differential into local store clock.

USR9201N

The API is the successor of USR1023N. It supports store clock with sliding window (1971 - 2114), smart and extended store clock (1900 - 38434).

The API expects one of the following formats as input:

  • Natural date and time,

  • Natural time,

  • Natural date,

  • microseconds (since 1900-01-01),

  • store clock with sliding window,

  • extended store clock, or

  • smart store clock,

and converts it into all other formats.

Additional to the API, the following copycodes are provided:

  • USR9201D - Calculate time difference of two store clock values with sliding window

  • USR9201F - Convert extended store clock into smart store clock

  • USR9201G - Convert smart store clock into extended store clock

  • USR9201H - Convert store clock with sliding window into smart store clock

  • USR9201I - Convert smart store clock into store clock with sliding window

  • USR9201J - Convert microseconds into smart store clock

  • USR9201K - Convert smart store clock into microseconds

  • USR9201R - Convert store clock with sliding window into extended store clock

  • USR9201S - Convert extended store clock into store clock with sliding window

  • USR9201U - Convert microseconds into extended store clock

  • USR9201V - Convert extended store clock into microseconds

  • USR9201W - Convert microseconds into store clock with sliding window

  • USR9201X - Convert microseconds into Natural time

  • USR9201Y - Convert store clock with sliding window into microseconds

  • USR9201Z - Convert Natural time into microseconds

Note:
The copycode USR9201D can also be used to compare two store clock values which have been taken in the sliding window range (1971-2114).

Overview of the copycodes USR9201*

The copycodes are essentially faster than the subprogram USR9201N.

Example - Convert a Store Clock with Sliding Window into Microseconds

A test for 10,000 conversions shows that:

  • the subprogram USR9201N requires about 35 ms, and

  • the copycode USR9201Y requires about 5 ms.