REQUEST DOCUMENT

#### Search Results

Search

# REQUEST DOCUMENT

`REQUEST` `DOCUMENT FROM` url
| `WITH` `[`with-clause`]` |
| `RETURN` `[`return-clause`]` |
`RESPONSE` http-response-code
`[``GIVING` natural-error-number`]`

This document covers the following topics:

- Function

- Syntax Description

- Automatically Generated Headers

- URL Encoding for Special Characters

- HTTP/HTTPS Responses Redirected and Denied

- Examples

For an explanation of the symbols used in the syntax diagram, see Syntax Symbols.

Related Statement: `PARSE XML`

Belongs to Function Group: Internet and XML

## Function

The `REQUEST DOCUMENT` statement is used to retrieve and upload documents on the internet as described in REQUEST DOCUMENT in Statements for Internet and XML Access in the Programming Guide.

For information on Unicode support, see REQUEST DOCUMENT in the Unicode and Code Page Support documentation.

### Restrictions for Protocol Types

For technical reasons, HTTPS is supported under z/OS only.

### No Support for Cookies

Cookies are not supported and are ignored.

## Syntax Description

Operand Definition Table:

Operand | Possible Structure | Possible Formats | Referencing Permitted | Dynamic Definition
--- | --- | --- | --- | ---
`url` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`http-response-code` |  | S |  |  |  |  |  |  |  | I4 |  |  |  |  |  |  |  | yes | yes
`natural-error-number` |  | S |  |  |  |  |  |  |  | I4 |  |  |  |  |  |  |  | yes | no

Syntax Element Description:

Syntax Element | Description
--- | ---
`url` | Location of Document: `url` is the URL to access a document. If `url` is an Internet Protocol Version 6 (IPv6) address in hexadecimal notation, you need to enclose the address in square brackets ([ ]). See also Example 7 - GET Request for an IPv6 Address on Port Number 8080. You cannot embed an Internet Protocol Version 4 (IPv4) address into an IPv6 address.
`with-clause` | WITH Clause: See `with-clause`.
`return-clause` | RETURN Clause: See `return-clause`.
`http-response-code` | RESPONSE: `http-response-code` is the HTTP/HTTPS response status code returned for the request, for example: `200` (request completed). See also HTTP/HTTPS Responses Redirected and Denied. For a list of possible HTTP/HTTPS status codes, refer to the RFC 2616 memorandum published by the World Wide Web Consortium (W3C).
`natural-error-number` | GIVING Option: `natural-error-number` contains the 4-digit Natural error number if the request could not be performed.

### with-clause

[`USER` user-id]
[`PASSWORD` user-password]
[`HEADER` {[`NAME`] header-name-out [`VALUE`] header-value-out} ...]
[`DATA` {`ALL` outbound-document [`ENCODED` [[`IN`] `CODEPAGE` code-page-out]]
|{[`NAME`] variable-name-out [`VALUE`] variable-value-out} ...}]

The `with-clause` is used to specify optional user/password, header and data details for the request.

An empty `with-clause` (that is, no value specified after `WITH`) is ignored.

Operand Definition Table:

Operand | Possible Structure | Possible Formats | Referencing Permitted | Dynamic Definition
--- | --- | --- | --- | ---
`user-id` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`user-password` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`header-name-out` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`header-value-out` | C | S |  |  |  | A |  | N | P | I | F |  | D | T | L |  |  | yes | no
`outbound-document` | C | S |  |  |  | A | U | N | P | I | F | B | D | T | L |  |  | yes | no
`code-page-out` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`variable-name-out` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`variable-value-out` | C | S |  |  |  | A |  | N | P | I | F |  | D | T | L |  |  | yes | no

Syntax Element Description:

Syntax Element | Description
--- | ---
`user-id` | USER: `user-id` is the ID of the user that will be used for the request.
`user-password` | PASSWORD: `user-password` is the password of the user that will be used for the request.
`header-name-out` `header-value-out` | HEADER NAME/VALUE Option: `header-name-out` and `header-value-out` can only be used in conjunction with each other: `header-name-out` is the name of a header variable sent with this request. `header-value-out` is the value of a header variable sent with this request. header-name-out: Header names must not contain a carriage return (CR), a line feed (LF) or a colon (:). This will not be checked by the `REQUEST DOCUMENT` statement. For valid header names, see the HTTP specifications. For compatibility with the web interface, header names can be written with underscore (_) instead of a dash (-). (Internally, the underscore is replaced by a dash). header-value-out: Header values are not allowed to contain CR/LF. This will not be checked by the `REQUEST DOCUMENT` statement. For valid header values and formats, see the HTTP specifications. See also Automatically Generated Headers.
`outbound-document` | DATA ALL Option: `outbound-document` is a complete document that is to be sent. This value is needed for the HTTP REQUEST-METHOD `PUT` (see Automatically Generated Headers).
`code-page-out` | DATA ALL ENCODED Option: Data transfer with the `REQUEST DOCUMENT` statement normally does not involve any code page conversion. If you want to encode outgoing data in a specific code page, use the `CODEPAGE` option: `outbound-document` will be encoded from the default code page (value of the system variable `*CODEPAGE`) to the code page given in `code-page-out`. Encoding and Charset Attributes: If the outbound document contains an encoding (XML) or a charset (HTML) attribute, we recommend that the value of the `ENCODED` option maps the attribute value of the document. Example: If an outbound XML document contains the attribute encoding `="UTF-8"`, code the `REQUEST DOCUMENT` statement with the option `DATA ALL #DOCUMENT ENCODED CODEPAGE 'UTF-8'`.
`variable-name-out` `variable-value-out` | DATA NAME/VALUE Option: `variable-name-out` and `variable-value-out` request only specific `DATA` variable information instead of the complete document. They can only be used in conjunction with each other: `variable-name-out` is the name of a `DATA` variable to be sent with this request. `variable-value-out` is the value of a `DATA` variable to be sent with this request. This value is needed for the HTTP `REQUEST-METHOD` `POST` (URL encoding necessary, especially ampersand (&), equal sign (=), percent sign (%) characters). Restriction: If `variable-name-out` and `variable-value-out` are given and the communication is `http://` or `https://`, by default, the `REQUEST-METHOD` `POST` (see Automatically Generated Headers) with content type `application/x-www-form-urlencoded` is used. During the request, `variable-name-out` and `variable-value-out` will be separated by equal sign (=) and ampersand (&) characters. Therefore, the operands are not allowed to contain equal sign (=), ampersand (&) and, because of URL encoding, percent sign (%) characters. These characters are considered reserved and need to be encoded as indicated in Reserved Characters.

### return-clause

[`HEADER` [`ALL` header-all-in] [[`NAME`] header-name-in [`VALUE`] header-value-in ...]]
[`PAGE` inbound-document [`ENCODED` [[`FOR TYPES` mime-type ...] [`IN`] `CODEPAGE` code-page-in]]]

Operand Definition Table:

Operand | Possible Structure | Possible Formats | Referencing Permitted | Dynamic Definition
--- | --- | --- | --- | ---
`header-all-in` |  | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | yes
`header-name-in` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`header-value-in` |  | S | A * |  |  | A |  | N | P | I | F | B | D | T | L |  |  | yes | yes
`inbound-document` |  | S |  |  |  | A | U |  |  |  |  | B |  |  |  |  |  | yes | yes
`mime-type` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no
`code-page-in` | C | S |  |  |  | A |  |  |  |  |  |  |  |  |  |  |  | yes | no

This clause can be used to specify return information for the headers and/or the document.

Syntax Element Description:

Syntax Element | Description
--- | ---
`header-all-in` | HEADER ALL Option: `header-all-in` contains all header data delivered with the HTTP response. The first line contains the status information and all following lines contain the headers as pairs of name and value. The names always end in a colon (:) and the values end in a line feed (LF). Internally, all carriage returns/line feeds (CR/LF) are transformed into line feeds (LF).
- `header-name-in` `header-value-in` | HEADER NAME/VALUE Option: `header-name-in` and `header-value-in` return only specific header information. They can only be used in conjunction with each other: `header-name-in` is the name for the header information returned by the HTTP request. For compatibility with the web interface, header names can be written with underscore (_) instead of dash (-) characters. Internally, the underscore is replaced by a dash. If `header-name-in` is a blank string, the status information is returned, for example:
      HTTP/1.0 200 OK
`header-value-in` is the scalar or array value required to receive the header data returned by the HTTP request. An array definition is required if more than one occurrence of the same header is expected, for example, multiple Set-Cookie headers. Only one dimension of a multi-dimensional array may contain an index range (see Example 9). An X-array must be materialized before you can use it. If the number of array occurrences exceeds the number of headers, the unused occurrences are reset. If the number of headers exceeds the number of array occurrences, the remaining header values are ignored. For an example of an array definition, see Example 9 - RETURN HEADER NAME VALUE with Array Definition.
`inbound-document` | PAGE Option: `inbound-document` is the document returned for this request. No encoding at all of the returned page will be done; that is, the page will remain encoded as delivered from the HTTP server.
`code-page-in` | PAGE ENCODED Option: Data transfer with the `REQUEST DOCUMENT` statement normally does not involve any code page conversion. If you want to encode incoming data in a specific code page, use the `ENCODED` option: If necessary, `inbound-document` will be encoded in the default code page (value of system variable `*CODEPAGE`) of the Natural session. If the value of `code-page-in` is blank, then `inbound-document` will be encoded from the code page defined with the keyword subparameter `RDCP` to the default code page (A/B) or (U). The keyword subparameter `RDCP` of the profile parameter `XML` is used to specify the name of the default HTML/XML code page. Note "Returned MIME type contains an encoding" means that the HTTP server returns a content-type header with a `charset=` clause, for example: `charset=ISO-8859-1`. For examples of using the `RETURN PAGE ENCODED` clause, see Example 8 - RETURN PAGE ENCODED Clause.
`mime-type` | PAGE ENCODED FOR TYPES Option: As a response of an HTTP/HTTPS request, incoming data may contain binary data (for example, image/gif) or character data (for example, text/html). Together with the response, the `REQUEST DOCUMENT` statement receives a parameter which specifies the type of content of the requested document (MIME type, also known as internet media type). This parameter may contain information about the code page in which the document is encoded. `mime-type` is the list of MIME types for which an encoding of the returned document in `inbound-document` will be performed. If the returned MIME type contains an encoding, `inbound-document` will be encoded from this code page to the default code page (A/B) or (U). If the returned MIME type does not contain an encoding, then `inbound-document` will be encoded from the code page defined with `code-page-in` to the default code page (value of the system variable `*CODEPAGE`) (A/B) or (U). If the returned MIME type does not contain an encoding, then an additional check is performed if the returned MIME type matches one of the types given with `mime-type`. If a match is found, `inbound-document` will be encoded from the code page defined with `code-page-in` to the default code page (A/B) or (U).

## Automatically Generated Headers

For an HTTP request, some headers are required, for example: `REQUEST-METHOD` or content type. These headers will be automatically generated depending on the parameters given with the `REQUEST DOCUMENT` statement.

Note
It is possible to overwrite the automatically generated headers. Natural will not check them for errors. Unexpected errors may occur.

- HTTP REQUEST-METHOD

- Content Type

### HTTP REQUEST-METHOD

The `REQUEST DOCUMENT` statement supports the following HTTP `REQUEST-METHOD`s: `HEAD`, `POST`, `GET` and `PUT`.

The following table shows the HTTP `REQUEST-METHOD` generated depending on the given operands:

| `WITH HEADER` | `WITH DATA` | `RETURN HEADER` | `RETURN PAGE`
HEAD | o | - | x | -
POST | o | x | o | x
GET | o | - | o | x
PUT | o | `DATA ALL` * | o | o

In addition to the standard `REQUEST-METHOD`s mentioned above, the methods `DELETE`, `PATCH`, `OPTIONS` and `TRACE` can be specified in a `REQUEST-METHOD` header.

#### Explanation:

o | Optional. Operand can be optionally specified.
- | Operand cannot be specified.
x | Operand is always specified.
* | Only applies to `DATA ALL` and not `DATA NAME VALUE`.

### Content Type

The `REQUEST-METHOD` `POST` requires a content-type header for the HTTP request. If no content type is explicitly specified, Natural inserts the following default content-type header into the request:

    application/x-www-form-urlencoded

## URL Encoding for Special Characters

When sending `POST` data with the content type `application/x-www-form-urlencoded`, certain characters must be represented by means of URL encoding, which means substituting the character with `%hexadecimal-character-code`. Some basic details are given here:

- Non-ASCII Characters

- Unsafe Characters

- Reserved Characters

For full details of when and why URL encoding is necessary, refer to the memorandums RFC 1630, RFC 1738 and RFC 1808 published by the World Wide Web Consortium (W3C).

### Non-ASCII Characters

All non-ASCII characters (that is, valid ISO 8859/1 characters that are not also ASCII characters) must be URL encoded, for example, the file `köln.html` would appear in an URL as `k%F6ln.html`.

### Unsafe Characters

URL encode the following unsafe characters when you request web pages to avoid server failures:

Unsafe Character | URL Encoding
--- | ---
the tab character | %09
the space character | %20
[ | %5B
\ | %5C
] | %5D
^ | %5E
` | %60
{ | %7B
| | %7C
} | %7D
~ | %7E

### Reserved Characters

Some characters have special meanings in URLs, such as the colon (:) that separates the URL scheme from the rest of the URL, the double slash (//) that indicates that the URL conforms to the Common Internet Scheme syntax and the percent sign (%). Generally, when these characters appear as parts of file names, they must be URL encoded to distinguish them from their special meaning in URLs (this is a simplification, refer to the RFCs mentioned earlier for full details).

Reserved characters are:

Reserved Character | URL Encoding
--- | ---
" | %22
# | %23
% | %25
& | %26
+ | %2B
, | %2C
/ | %2F
: | %3A
< | %3C
= | %3D
> | %3E
? | %3F
@ | %40

## HTTP/HTTPS Responses Redirected and Denied

For a list of HTTP/HTTPS status codes, refer to the RFC 2616 memorandum published by the World Wide Web Consortium (W3C).

The following special considerations apply to the HTTP responses for redirected and denied requests:

- Response 301 - 303 (Redirected)

- Response 401 (Denied/Unauthorized)

### Response 301 - 303 (Redirected)

The HTTP response codes 301, 302 and 303 mean that the URL where the requested document resides has changed and that the request was therefore redirected to another URL. As a response, the return header with the name `LOCATION` will be displayed. This header contains the URL where the requested page has moved to. A new `REQUEST DOCUMENT` request can be used to retrieve the page moved.

HTTP browsers redirect automatically to the new URL, but the `REQUEST DOCUMENT` statement does not handle redirection automatically.

### Response 401 (Denied/Unauthorized)

The HTTP response code 401 means that the requested page can only be accessed if a valid user ID and password are provided with the request. As a response, the return header with the name `WWW-AUTHENTICATE` will be delivered with the REALM needed for this request.

HTTP browsers normally display a dialog with user ID and password, but with the `REQUEST DOCUMENT` statement, no dialog is displayed.

## Examples

- Example 1 - General Request

- Example 2 - Simple GET Request (no data)

- Example 3 - Simple HEAD Request (no return page)

- Example 4 - Simple POST Request (default REQUEST-METHOD)

- Example 5 - Simple PUT Request (with DATA ALL)

- Example 6 - REQUEST DOCUMENT with Proxy Authorization

- Example 7 - GET Request for an IPv6 Address on Port Number 8080

- Example 8 - RETURN PAGE ENCODED Clause

- Example 9 - RETURN HEADER NAME VALUE with Array Definition

Note
There is an example dialog `V5-RDOC` for this statement in the example library `SYSEXV`.

### Example 1 - General Request

    REQUEST DOCUMENT FROM "http://bolsap1:5555/invoke/sap.demo/handle_RFC_XML_POST"
      WITH
        USER #User PASSWORD #Password
        DATA
        NAME 'XMLData'       VALUE #Queryxml
        NAME 'repServerName' VALUE 'NT2'
      RETURN
        PAGE #Resultxml
    RESPONSE #rc

### Example 2 - Simple GET Request (no data)

    REQUEST DOCUMENT FROM "http://pcnatweb:8080"
      RETURN
        PAGE #Resultxml
    RESPONSE #rc

### Example 3 - Simple HEAD Request (no return page)

    REQUEST DOCUMENT FROM "http://pcnatweb"
    RESPONSE #rc

### Example 4 - Simple POST Request (default REQUEST-METHOD)

    REQUEST DOCUMENT FROM "http://pcnatweb/cgi-bin/nwwcgi.exe/sysweb/nat-env"
      WITH
        DATA
        NAME 'XMLData'       VALUE #Queryxml
        NAME 'repServerName' VALUE 'NT2'
      RETURN
        PAGE #Resultxml
    RESPONSE #rc

### Example 5 - Simple PUT Request (with DATA ALL)

    REQUEST DOCUMENT FROM "http://pcnatweb/test.txt"
      WITH
        DATA ALL     #document
      RETURN
        PAGE #Resultxml
    RESPONSE #rc

### Example 6 - REQUEST DOCUMENT with Proxy Authorization

    DEFINE DATA
    LOCAL
    1 #URL              (A) DYNAMIC
    1 #PAGE             (A) DYNAMIC
    1 #HTTPSTAT         (I4)
    1 #PATH             (A) DYNAMIC
    1 #NAME             (A) DYNAMIC
    1 #VALUE            (A) DYNAMIC
    1 #USERID           (A8)
    1 #PASSWORD         (A8)
    1 #AUTHHDR          (A) DYNAMIC
    1 #CREDENTIALS      (A) DYNAMIC
    1 #PADDCHAR         (A2/0:2) INIT  <' ','==','='>
    1 #LENGTH           (I4)
    1 #FACTOR           (I4)
    1 #REMAINDER        (I4)
    /*
    /* #USR4210
    1 PARM-FUNCTION  (A2) INIT <'BA'>
    1 PARM-RC        (I4)
    1 PARM-ERRTXT    (A72)
    1 PARM-A         (A) DYNAMIC
    1 PARM-B         (B) DYNAMIC
    1 PARM-RFC       (B1)
    END-DEFINE
    **
    ASSIGN #URL = 'http://www.softwareag.com/corporate/default.asp'
    REQUEST DOCUMENT FROM #URL
      RETURN PAGE #PAGE ENCODED CODEPAGE ' '
      RESPONSE #HTTPSTAT
    *************************************************************
    ** HTTP response 407 means that the proxy server requires  **
    ** the client to identify itself!                          **
    *************************************************************
    IF #HTTPSTAT = 407
      INPUT           'Please enter userid and password' (AD=I) /
      'Userid  :' #USERID / 'Password:' #PASSWORD
      ASSIGN #AUTHHDR = 'Proxy-Authorization'
      COMPRESS #USERID ':' #PASSWORD INTO PARM-B  LEAVING NO
    *************************************************************
    ** Convert credentials to ASCII                            **
    *************************************************************
      MOVE ENCODED PARM-B TO PARM-B
       CODEPAGE 'ASCII'
    *************************************************************
    ** ENCODE CREDENTIALS WITH BASE64                          **
    *************************************************************
    ** BASE64 demands that the length of the encoded string is **
    ** a multiple of 3. If encoding length does not match this **
    ** the encoded string has to be padded with '=' characters.**
    ** Since USR4210 does not support padding of the base64    **
    ** string with trailing '=' characters, we have to do this **
    ** ourselves.                                              **
    *************************************************************
      CALLNAT 'USR4210N' PARM-FUNCTION PARM-RC PARM-ERRTXT PARM-B PARM-A
        PARM-RFC
    *
      #LENGTH := *LENGTH(PARM-A)
      WRITE PARM-A (AL=40) #LENGTH EJECT
      DIVIDE 3 INTO #LENGTH GIVING #FACTOR REMAINDER #REMAINDER
      COMPRESS  PARM-A #PADDCHAR(#REMAINDER)
        INTO #CREDENTIALS LEAVING NO
    *************************************************************
    ** Build Proxy-Authorization HTTP header                   **
    *************************************************************
      COMPRESS 'Basic '  #CREDENTIALS
        INTO #CREDENTIALS
      ASSIGN #AUTHHDR = 'Proxy-Authorization'
    *************************************************************
    ** Repeat HTTP request with valid client identification    **
    *************************************************************
      REQUEST DOCUMENT FROM #URL
       WITH HEADER NAME #AUTHHDR VALUE #CREDENTIALS
        RETURN PAGE #PAGE ENCODED CODEPAGE ' '
        RESPONSE #HTTPSTAT
    END-IF
    END

### Example 7 - GET Request for an IPv6 Address on Port Number 8080

    REQUEST DOCUMENT FROM "http://[2BFC:5022:4081:0000::C:41]:8080"
      RETURN
        PAGE #Resultxml
    RESPONSE #rc

### Example 8 - RETURN PAGE ENCODED Clause

1. Server returns a header `'Content-type: text/html;charset=UTF-8'`

   Program Code Example 1:

       ...
       RETURN PAGE inbound-document

   Resulting Processing:

   `inbound-document` remains UTF-8 encoded.

   Program Code Example 2:

       ...
       RETURN PAGE inbound-document ENCODED [...]

   Resulting Processing:

   `inbound-document` is converted from UTF-8 to the default code page regardless of whether `code-page-in` and `mime-type` are specified: When a valid encoding is returned in the content-type header, `mime-type` and `code-page-in` are ignored.

2. Server returns a header `'Content-type: text/xml'`

   Program Code Example 1:

       ...
       RETURN PAGE inbound-document ENCODED

   Resulting Processing:

   `inbound-document` remains unconverted since the content-type header does not contain a valid encoding.

   Program Code Example 2:

       ...
       RETURN PAGE inbound-document ENCODED FOR TYPES 'text/xml' IN CODEPAGE 'USASCII'

   Resulting Processing:

   `inbound-document` is converted from USASCII code page to the default code page. In this case, conversion is done according to the programmer's assumption about the encoding of the received page.

   Program Code Example 3:

       ...
       RETURN PAGE inbound-document ENCODED FOR TYPES 'text/html'
           IN CODEPAGE ' '

   Resulting Processing:

   `inbound-document` remains unconverted since the MIME type `'text/html'`, specified in `mime-type`, does not match the MIME type `'text/xml'`, returned in the content-type header.

   Program Code Example 4:

       ...
       RETURN PAGE inbound-document ENCODED IN CODEPAGE ' '

   Resulting Processing:

   `inbound-document` is converted from the default code page, specified with subparameter `RDCP` of profile parameter `XML`, to the default code page.

   Note:
   The default value for the `RDCP` subparameter, which applies if nothing has been explicitly specified, is `ISO-8859-1`. See also XML - Activate PARSE XML and REQUEST DOCUMENT Statements in the Parameter Reference.

### Example 9 - RETURN HEADER NAME VALUE with Array Definition

    DEFINE DATA
    LOCAL
    1 #FROM      (A) DYNAMIC
    1 #HEADER    (A) DYNAMIC
    1 #PAGE      (A) DYNAMIC
    1 #COOKIES (A20/1:3,1:4,2:5)
    1 #RC      (I4)
    END-DEFINE
    ASSIGN #FROM = 'http://www.myserver.com'
    REQUEST DOCUMENT FROM #FROM
       RETURN
            HEADER NAME 'Set-Cookie' VALUE #COOKIES(1,2:3,3)
            PAGE   #PAGE
            RESPONSE #RC
    PRINT #COOKIES(*,*,*)
    END

In the example program above, invalid array definitions (with multiple dimensions) would be:

    RETURN HEADER NAME 'Set-Cookie' VALUE #COOKIES(1:3,2:3,3)
    RETURN HEADER NAME 'Set-Cookie' VALUE #COOKIES(*,2,*)

### On this page

- Function
- Syntax Description
- Automatically Generated Headers
- URL Encoding for Special Characters
- HTTP/HTTPS Responses Redirected and Denied
- Examples