Version 4.2.6
 —  Unicode and Code Page Support  —

Frequently Asked Questions

This document covers the following topics:


Why do I get the startup error "Invalid code page specified"?

The code page you have defined with the profile parameter CP does either not exist (see http://demo.icu-project.org/icu-bin/convexp for valid ICU code pages and http://www.iana.org/assignments/character-sets for the appropriate IANA names) or is an invalid default code page for the platform (for example, an EBCDIC code page cannot be used on a Windows, UNIX or OpenVMS platform).

On mainframe platforms, the code page needs to be specified in the Natural configuration file via NTCPAGE (see also NTCPAGE Macro). Code pages that are not entered here are rejected as invalid, although they are available in the ICU implementation. Check whether the same IANA name, CCSID/CCSN or alias name as specified in NATCONFG is used.

Top of page

What is the "default code page"?

The default code page is the code page which is the result of the evaluation of the profile parameter CP. If CP is not filled (Windows, UNIX and OpenVMS), it is the current operating system code page.

Top of page

What default code page is used?

The default code page which is used by Natural for conversions between code page and Unicode and vice versa can be detected by displaying the content of the system variable *CODEPAGE.

Top of page

Should I save all Natural sources in UTF-8 format?

It depends on the characters you want to use and on the platforms on which your sources are located. If you want to use Unicode constants, UTF-8 is the only possibility to store all combinations of characters. However, you can define hexadecimal UH constants which can also be stored in code page sources. The disadvantage of hexadecimal constants is that you have to know the UTF-16 encoding for every character of the constant. On mainframes, UTF-8 format for sources is not possible at all. On UNIX and OpenVMS, UTF-8 sources can only be handled via SPoD; they cannot be handled locally on UNIX or OpenVMS.

Top of page

How can I handle UTF-8 encoding with Natural code?

Use the MOVE ENCODED statement for conversion from UTF-8 to UTF-16: the code page "UTF-8" has to be used for the A format variable.

Top of page

Why are some characters not displayed correctly?

Check if you are using the correct code page. If the code page is correct, check if the selected font supports the characters you want to display.

Top of page

Why do I get an error when I want to edit a Natural source?

The code page which is defined for the source is not correct. When converting the contents of the source to Unicode, a conversion error occurs. Change the encoding of the source so that the conversion to Unicode is successful.

On mainframe platforms, the source is saved with the code page at creation time. You get a conversion error when the source could not be converted from the code page of the saved source into the code page of the current Natural session. You can start Natural with the code page of the source to avoid conversion or you can adjust non-convertible characters in the window which appears when the editor is started.

Top of page

Why do I get an error when I want to save a Natural source?

You have entered characters in the source which cannot be converted to the code page which was used to read the source. Check if you have entered these characters by mistake or if you really want to save the characters in the source. In the first case, remove the faulty characters and save the source. In the second case, save the source in UTF-8 format or, if the characters are contained in U constants, use UH constants instead.

If you have not entered any characters which are not contained in the code page of the source, check whether the profile parameter SRETAIN has been set to OFF. In this case, the source will be saved with the default code page. If the concerned source was previously saved with a different code page, a conversion error may occur.

If you are connected to a mainframe environment via SPoD, the source from the mainframe is converted and edited in Unicode in the SPoD environment. If it is saved, it has to be converted into the code page of the Natural server. A conversion error may occur if a Unicode character is not mapped in the code page of the Natural server session.

If you are in a native Natural for Mainframes environment (without SPoD) you do not get errors when saving a source since a conversion is not performed. The source is saved with the code page information of the current Natural session.

Top of page

How can I find out the encoding of a Natural source?

In Natural Studio, invoke the Properties dialog box for the source node. The General page shows the encoding of the source. If the Encoding text box is empty, no specific encoding is stored for the source. This means that the default encoding is used when reading the source.

The list view windows of Natural Studio also show the encodings of all listed objects.

On mainframe platforms, code page information is part of the Natural source directory. Use the LIST DIR command on Natural for Mainframes to display the directory.

Top of page

How can I change the encoding of a Natural source?

In Natural Studio, invoke the Properties dialog box for the source node. The General page shows the encoding of the source. If this is not the correct encoding, you can change it by choosing the Change button: a list of available code pages is shown and you can select the correct encoding for the source.

On mainframe platforms, you should start your Natural session with the desired code page using the CP parameter. Set the parameter SRETAIN to OFF, edit the source and save it. Now the source has the modified code page information. Or, you can use the SYSCP utility to check or change the code page assignment of a source.

Top of page

How can I convert an existing Natural source into UTF-8 format (Windows, UNIX and OpenVMS only)?

Open the source in the Natural editor with the correct code page. Save the source with Save As and in the Save As dialog box, select UTF-8 as the encoding.

Top of page

Which substitution character is used if a character cannot be converted?

This depends on the direction of the conversion: if a code page character cannot be converted to Unicode, the Unicode substitution character "U+FFFD" is used. If a Unicode character cannot be converted to a code page, the substitution character which is defined by ICU for this code page is used.

On Natural for Mainframes, the substitution character of the code page or, if specified in the configuration file, the place holder character is used.

For the conversion from Unicode to the default code page, the substitution character can be changed on Windows, UNIX and OpenVMS platforms by setting the profile parameter SUBCHAR.

Top of page

Can I use Natural 4.2 sources with previous Natural versions?

On Natural for Mainframes, you can use Natural 4.2 sources with previous Natural versions. The layout of the source has not been changed and the additional code page information of Natural 4.2 sources will simply be ignored if the source is accessed with a previous version.

Top of page

Can I use UTF-8 sources with previous Natural versions?

No. Previous Natural versions do not know any code page information; a UTF-8 source will be interpreted as the current system code page.

Top of page

Why do I get a conversion error when cataloging a source which has UTF-8 format?

A Natural source with UTF-8 format cannot be cataloged because a code point cannot be converted (Windows, UNIX and OpenVMS only).

All A constants in a source with UTF-8 format are converted to the default code page when storing them in the generated program. Either remove the characters which are not contained in the default code page from the A constants or use U constants instead of A constants.

Top of page

Why do I get garbage On UNIX or OpenVMS when displaying U format via a terminal emulation?

All characters which are not contained in the default code page will be replaced with the substitution character of the code page before displaying the output on a terminal emulation. For an ASCII code page, the substitution character defined by the ICU conversion table is often "0x1A", which could be a control character on UNIX or OpenVMS terminals. It is strongly recommended to use the Natural Web I/O Interface when using U format in I/O statements. If using a terminal emulation is essential, the substitution character (SUBCHAR) can be changed to a printable character (for example, "?").

On mainframe platforms, you can still use your terminal emulation since it is possible to replace the substitution character by a displayable place holder character via the NTCPAGE macro. The place holder character avoids garbage in case of non-convertible characters.

Top of page

Can I work with a current SPoD client and an older SPoD server?

Yes, but you should set the code page of the SPoD client to the code page of the server sources.

See also Prerequisites for Natural Single Point of Development at http://documentation.softwareag.com/natural/spod_prereq/prereq.htm.

Top of page

Can I work with a current SPoD server and an older SPoD client?

Yes, but this is not recommended if you have defined encodings for sources.

See also Prerequisites for Natural Single Point of Development at http://documentation.softwareag.com/natural/spod_prereq/prereq.htm.

Top of page