Tracing webMethods EntireX

This document describes the various techniques available for troubleshooting, tracing and logging with EntireX components. It covers the following topics:

Note:
Trace files can contain sensitive personal data (user ID, IP address, SSL certificates and payload data). This is particularly relevant if you have activated EntireX Security. EntireX uses trace files for accounting, diagnostics and error analysis. We recommend you check the different trace opportunities provided by EntireX and delete trace files if they are no longer needed. The various EntireX components will not delete these trace files automatically; this is your responsibility as user. Use the appropriate tools of the respective operating system.


Table Summarizing Tracing for webMethods EntireX Components

EntireX Component Use Tracing Technique for Tracing Technique
Broker ActiveX Control Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
EntireX Broker ACI under Windows Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
EntireX Broker Agent Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Agent
EntireX Broker under Linux Processing within the Broker
Requests to, replies from clients/server
Tracing EntireX Broker
DCOM Wrapper Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
RPC-related problems on the client side
Requests to, replies from RPC Servers
Requests to, replies from the Broker
Tracing the RPC Runtime
EntireX Java ACI Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
Java Wrapper Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
EntireX RPC Server for Java Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing RPC Server for Java
EntireX IDL Tester    
.NET Wrapper Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
RPC-related problems on the client side
Requests to, replies from RPC servers
Requests to, replies from the Broker
Tracing the RPC Runtime
C Wrapper Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
RPC-related problems on the client side
Requests to, replies from RPC servers
Requests to, replies from the Broker
Tracing the RPC Runtime
RPC Server RPC-related problems on the server side
Requests to, replies from RPC clients
Requests to, replies from the Broker
Activating Tracing for the RPC Server for C | .NET
Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing Broker Stubs
EntireX Broker HTTP(S) Agent Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
EntireX RPC Server for XML/SOAP For RPC Server for XML/SOAP-related problems. Tracing the XML/SOAP Runtime
Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
EntireX XML Tester    
EntireX Listener for XML/SOAP For Listener for XML/SOAP-related problems. Tracing the XML/SOAP Runtime
Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
XML/SOAP Wrapper For XML/SOAP Wrapper-related problems. Tracing the XML/SOAP Runtime
Transport-related problems
Requests to, replies from the Broker or Broker Agent
Tracing EntireX Java ACI
EntireX RPC-ACI Bridge   Tracing the EntireX RPC-ACI Bridge

Tracing EntireX Broker

See also EntireX Broker Return Codes.

Switching on Tracing

Start of instruction setTo switch on tracing

  • Set the attribute TRACE-LEVEL in the broker attribute file

    • for minimal trace output to "1"

    • for detailed trace output to "2"

    • for full trace output to "3"

    Example:

    TRACE-LEVEL=2

Switching off Tracing

Start of instruction setTo switch off tracing

  • Set the attribute TRACE-LEVEL in the broker attribute file to 0:

    TRACE-LEVEL=0

    Or:
    Omit the TRACE-LEVEL attribute.

Viewing the Trace Log

The trace file, BrokerID.LOG, is written to the Broker Directory.

Start of instruction setTo view the contents of a log

  • Using Command Central, select an environment in the Environments pane, select the Instances tab, click the name of a product instance, select the Logs tab, click the log alias for a log in the Alias column.

    Or:
    Enter the following command in Command Central:

    sagcc get diagnostics logs

    This retrieves log entries from a log file. Log information includes the date, time, and description of events that occurred with a specified runtime component.
    See Administering EntireX Components with Command Central or the separate Command Central documentation and online help for details.

Deferred Tracing

It is not always convenient to run with TRACE-LEVEL defined, especially when higher trace levels are involved. Deferred tracing is triggered when a specific condition occurs, such as an ACI response code or a broker subtask abend. Such conditions cause the contents of the trace buffer to be written, showing trace information leading up the specified event. If the specified event does not occur, the Broker trace will contain only startup and shutdown information (equivalent to TRACE-LEVEL=0). Operating the trace in this mode requires the following additional attributes in the broker section of the attribute file. Values for TRBUFNUM and TRAP-ERROR are only examples.

Attribute Value Description
TRBUFNUM 3 Specifies the deferred trace buffer size = 3 * 64 K.
TRMODE WRAP Indicates trace is not written until an event occurs.
TRAP-ERROR 322 Assigns the event ACI response code 00780322 "PSI: UPDATE failed".

Dynamically Switching On or Off the EntireX Broker Trace

The following methods are available to switch on or off the EntireX Broker trace dynamically. You do not need to restart the broker for the changes to take effect.

  • etbcmd
    Run command utility etbcmd with option -c TRACE-ON or - c TRACE-OFF. See etbcmd.

  • Command Central
    Use Command Central. See Updating the Trace Level under Administering the EntireX Broker using the Command Central GUI | Command Line.

Tracing Broker Agent

Start of instruction setTo switch on tracing

Start of instruction setTo switch off tracing

  • Set the parameter Trace Option to OFF.

    Or:
    Omit the parameter Trace Option.

Trace Output

The trace output is written to STDOUT.

Tracing Broker Stubs

The broker stubs provide an option for writing trace files.

Start of instruction setTo switch on tracing for the broker stub

  • Before starting the client application, set the environment variable ETB_STUBLOG:

    Trace Level Description
    0 NONE No tracing. Switch tracing off.
    1 STANDARD Traces initialization, errors, and all ACI request/reply strings.
    2 ADVANCED Used primarily by system engineers, traces everything from level 1 and provides additional information, for example the Broker ACI control block, as well as information from the transports.
    3 SUPPORT This is full tracing through the stub, including detailed traces of control blocks, message information, etc.

    Example:

    ETB_STUBLOG=2

If the trace level is greater than 1, unencrypted contents of the send/receive buffers may be exposed in the trace.

The trace file is created in the current directory. The name is pid.etb where pid is the process ID. If you want to write the trace file to a different location, set environment variable ETB_STUBLOGPATH to the desired path.

See also Linux Commands to Set the Environment Variables.

Remember to switch off tracing to prevent trace files from filling up your disk.

Start of instruction setTo switch off tracing for the broker stub

  • Set the environment variable ETB_STUBLOG to NONE or delete it.

Tracing EntireX Java ACI

The EntireX Java ACI provides a system property for tracing.

Start of instruction setTo switch on tracing

  1. When starting the Java virtual machine, set the Java system property entirex.trace

    • for minimal trace output to "1"

    • for detailed trace output to "2"

    • for full trace output to "3".

  2. The programming interface of the EntireX Java ACI allows you to set the trace value by the Java application using the EntireX Java ACI, see Tracing under Writing Advanced Applications in the EntireX Java ACI documentation. There may also be other methods to provide the trace value. See your application documentation.

Start of instruction setTo switch off tracing

  • Set the Java system property entirex.trace to 0 when starting the Java virtual machine

    Or:
    Omit the Java system property entirex.trace when starting the Java virtual machine.

Trace Output

The trace output will be written to STDOUT.

Tracing RPC Server for Java

Start of instruction setTo switch on tracing

  • When starting the Java virtual machine, set the Java system property entirex.trace

    • for minimal trace output to "1"

    • for detailed trace output to "2"

    • for full trace output to "3".

    See Customizing the RPC Server.

Start of instruction setTo switch off tracing

  • Set the Java system property entirex.trace to "0" when starting the Java virtual machine.

    Or:
    Omit the Java system property entirex.trace when starting the Java virtual machine.

Trace Output

The trace output will be written to STDOUT.

Tracing the RPC Runtime

Start of instruction setTo switch on tracing

  • Before starting the client application, set the environment variable ERX_TRACELEVEL to

    • STANDARD for minimal trace output

    • ADVANCED for detailed trace output

    • SUPPORT for full trace output.

Start of instruction setTo switch off tracing

  • Set the environment variable to NONE or delete it.

Trace Output

By default the trace file, ERXTrace.nnn.log, will be written to the trace directory.

The value nnn can be in the range from 001 to 005.

Start of instruction setTo change the trace destination

  • Set the environment variable ERX_TRACEFILE to the desired destination, which can consist of

    file names, folder names and variables for file names, folder names, process ID, thread ID, range.

    The variables are:

    Variable Operating System Description
    %...% Windows environment variable
    $(...) Linux environment variable
    @PID Linux, Win process ID
    @TID Linux, Win thread ID
    @RANGE[ n,m ] Linux, Win m must be greater than n, range is from 0 - 999
    @CSIDL_PERSONAL Windows The user's home directory. The variable will be resolved by Windows shell functions.
    @CSIDL_APPDATA Windows The Application Data Directory. The variable will be resolved by Windows shell functions.
    @CSIDL_LOCAL_APPDATA Windows The Application Data Directory. The variable will be resolved by Windows shell functions.

Related Information

Environment Variables in EntireX

Tracing the XML/SOAP Runtime

This section provides information on tracing the following components:

  • EntireX RPC Server for XML/SOAP

  • EntireX Listener for XML/SOAP

  • EntireX XML/SOAP Wrapper

The following topics are covered:

Note:
A trace of the XML/SOAP Runtime will trace the XML/SOAP communication. If you need to log the communication of the XML Runtime with EntireX Broker, see Tracing EntireX Java ACI.

Enabling Tracing

There are two ways to switch on tracing mode:

Using a Property File

Start of instruction setTo switch on tracing mode using a property file

  1. Copy the trace property file entirex.trace.standard to one of the following locations:

    • the working directory of your client application;

    • the user's home directory;

    • any other location.

  2. Rename the copied file entirex.trace.properties.

  3. Customize entirex.trace.properties as described in Trace Parameters.

  4. If entirex.trace.properties is within the current directory of your client application or your user home directory, it will be located automatically.

    Otherwise, specify the fully qualified or relative file name when starting the Java virtual machine for your client application using property entirex.sdk.default.trace.propertiesfile, example:

    java -Dentirex.sdk.default.trace.propertiesfile ="/MyDirName/entirex.trace.properties" MyClient

Using Trace Parameters of the Java Virtual Machine

Start of instruction setTo switch on tracing mode by specifying the trace parameters of the Java virtual machine

  • Submit the trace parameters when you start the Java virtual machine for the application to be traced. See Trace Parameters. Note that parameter specifications submitted overwrite settings in the property file.

Disabling Tracing

Start of instruction setTo switch off tracing

  • Delete or rename the trace property file if it is located in the working directory or in the user's home directory.

    Or:
    Specify level=NONE when invoking the Java virtual machine:

    java -Dentirex.sdk.default.trace.level = NONE MyClient

Configuring a Trace File for the Listener for XML/SOAP

We recommend to add the following parameter in file conf/axis2.xml located in the Software AG Common Web Services Stack installation:

<parameter name="exx-trace-propertiesfile">file:////path of trace.properties file</parameter>

Example:

<parameter name="exx-trace-propertiesfile">MyDirName/entirex.trace.properties</parameter>

Notes:

  1. If a relative path is specified, the file is located in directory WEB-INF/conf/ in the Web Services Stack web application file that contains the property.
  2. In the parameter section of the file axis2.xml, the value of the parameter exx-trace-propertiesfile can contain definitions of operating system variables, for example location="$EXXDIR/config/entirex.trace.properties".

Configuring a Trace File for the XML/SOAP Wrapper or the RPC Server for XML/SOAP

See Enabling Tracing.

Note:
If the RPC Server for XML/SOAP is running as a daemon, enable tracing by adding a Java system property to the start script or by copying file entirex.trace.properties to the same directory as the start script.

Trace Parameters

The following table provides an overview of trace parameters, their respective values, and how to submit them as arguments when invoking the Java virtual machine for the component to be traced.

Parameter Syntax Description
propertiesfile entirex.sdk.component name.trace.propertiesfile= absolute or relative path including the properties file Provide the location of the entirex.trace.properties file. Only used when the component is started.

Note:
A sample trace property file named entirex.trace.standard with predefined trace settings is contained in the directory ../EntireX/config. This file is a model and must be renamed to the valid name when used.

level entirex.sdk.component name.trace.level = tracelevel You can specify the following trace levels:
Keyword Level Description
NONE No tracing Tracing is switch off
STANDARD User Trace invocation of a component.
ADVANCED Expert For support and diagnostics. Expert knowledge of the component is required.
SUPPORT Expert Full trace output. Otherwise, as above.
directory entirex.sdk.component name.trace.directory = absolute or relative path Default is the working directory.
filename entirex.sdk.component name.trace.filename = FILE|STDOUT|STDERR Specify where tracing data is written to:
Keyword Destination
STDOUT (Default) Console
STDERR Console
FILE File name is generated internally: exx.sdk.component name.threadName. backupNo.log , where backupNo is in the range from ".000" to ".009". Note that the number of files created depends on maximumsize. If more than 10 files are required, the oldest backup file is overwritten.
threadoriented entirex.sdk.component name.trace.threadoriented = true | false
Keyword Description
YES Thread-oriented: trace data is distributed over multiple files (one file per thread)
NO (Default) Trace data is written to one file.
rowlength entirex.sdk.component name.trace.rowlength = maximum_characters_per_row Maximum number of characters per row. If this limit is exceeded, the remaining letters are written to a new line.
maximumsize entirex.sdk.component name.trace.maximumsize = max_file_size Maximum size of the log file. If this limit is exceeded, the log file is renamed and the remaining data is written to a new log file, see filename. Note that this specification has an effect only if filename is set to "FILE".
timeframe entirex.sdk.component name.trace.timeframe = number of day Time period after which the log file is closed. If this time limit has exceeded, the log file is renamed and the remaining data (if any) is written to a new log file. Note that this specification has an effect only if filename is set to "FILE". You can specify the following timeframes:
Keyword Description
1-9+H Number of hours
1-9+D Number of days

If no time frame is defined, only one log file is used until tracing is stopped.

Example: If timeframe has been set to 30D, the current log file is closed and renamed at midnight every thirty days, and tracing is continued with a new log file.

Component Names

Trace properties given in the trace property file might have to be restricted by componentname. The following components are available:

EntireX Component componentname Description
  default The trace property is not restricted to a specific EntireX component.
XML/SOAP Runtime xml.runtime The trace property belongs to the EntireX XML/SOAP Runtime only.

Tracing the EntireX RPC-ACI Bridge

Start of instruction setTo trace Broker calls

  1. Use the system property entirex.trace=[0|1|2|3].

    This trace does not separate the calls to the Broker for RPC from those to the Broker for ACI. The trace levels are:

    • 0 to switch off tracing.

    • 1 to trace Broker calls.

    • 2 to trace Broker calls and the payload.

    • 3 to trace Broker calls and all buffers including the payload.

  2. Redirect the trace to a file with the property entirex.server.logfile. Set this to the file name of the log file, the default is standard output.

Enabling Java Trace of SPM Plug-ins

In some cases a Java trace of the SPM plug-ins is needed to analyze an issue.

Start of instruction setTo enable Java trace of SPM plug-ins

  1. Stop the Platform Manager. On Linux it runs as a daemon.

  2. Edit the file custom_wrapper.conf in <Installation Dir>\profiles\SPM\configuration\custom_wrapper.conf. Add the following line:

    wrapper.java.additional.<n>=-Dentirex.trace=2

    Example:

    #encoding=UTF-8
    # Configuration files must begin with a line specifying the encoding
    # of the file.
    # Put here your custom properties.
    wrapper.successful_invocation_time=10
    wrapper.java.initmemory=32
    wrapper.restart.reload_configuration=TRUE
    wrapper.java.additional.10=-Djava.util.Arrays.useLegacyMergeSort=true
    wrapper.java.additional.20=-Dentirex.trace=2

    In case of issues with SSL, add the line:

    wrapper.java.additional.<n>=-Djavax.net.debug=ssl

    Example with Java trace and SSL trace:

    #encoding=UTF-8
    # Configuration files must begin with a line specifying the encoding
    # of the file.
    # Put here your custom properties.
    wrapper.successful_invocation_time=10
    wrapper.java.initmemory=32
    wrapper.restart.reload_configuration=TRUE
    wrapper.java.additional.10=-Djava.util.Arrays.useLegacyMergeSort=true
    wrapper.java.additional.20=-Dentirex.trace=2
    wrapper.java.additional.30=-Djavax.net.debug=ssl
  3. Restart the Platform Manager.

    The Java trace is written to <installation dir>\profiles\SPM\logs\wrapper.log.

    Tip:
    Search for string "EntireX Java Runtime" for the start of the trace.

Start of instruction setTo stop the Java trace of SPM plug-ins

  1. Remove the additional lines in <Installation Dir>\profiles\SPM\configuration\custom_wrapper.conf.

  2. Restart the Platform Manager.