pysys.writer.console¶

Writers that record test outcomes on the console (stdout) when running PySys.

If no progress writers are explicitly configured in the PySys project XML file, an instance of ConsoleProgressResultsWriter is used. If no summary writer is explicitly configured in the PySys project XML file, an instance of ConsoleSummaryResultsWriter is used.

ConsoleSummaryResultsWriter¶

class pysys.writer.console.ConsoleSummaryResultsWriter(logfile=None, **kwargs)[source]¶

Bases: pysys.writer.api.BaseSummaryResultsWriter, pysys.writer.api.TestOutcomeSummaryGenerator

Default summary writer that is used to list a summary of the test results at the end of execution.

Support the same configuration options as pysys.writer.api.TestOutcomeSummaryGenerator.

cleanup(**kwargs)[source]¶

Called after all tests have finished executing (or been cancelled).

This is where file headers can be written, and open handles should be closed.

Parameters: kwargs – Additional keyword arguments may be added in a future release.

ConsoleProgressResultsWriter¶

class pysys.writer.console.ConsoleProgressResultsWriter(**kwargs)[source]¶

Bases: pysys.writer.api.BaseProgressResultsWriter

Default progress writer that logs a summary of progress so far to the console, after each test completes.

processResult(testObj, cycle=- 1, **kwargs)[source]¶

Called when each test has completed.

This method is always invoked under a lock that prevents multiple concurrent invocations so additional locking is not usually necessary.

Parameters

testObj (pysys.basetest.BaseTest) – Reference to an instance of a pysys.basetest.BaseTest class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran.
cycle (int) – The cycle number. These start from 0, so please add 1 to this value before using.
testTime (float) – Duration of the test in seconds as a floating point number.
testStart (float) – The time when the test started.
runLogOutput (str) – The logging output written to the console/run.log, as a unicode character string. This string will include ANSI escape codes if colored output is enabled; if desired these can be removed using pysys.utils.logutils.stripANSIEscapeCodes().
kwargs – Additional keyword arguments may be added in future releases.

processTestStarting(testObj, cycle=- 1, **kwargs)[source]¶

Called when a test is just about to begin executing.

Note on thread-safety: unlike the other methods on this interface, this is usually executed on a worker thread, so any data structures accessed in this method and others on this class must be synchronized if performing non-atomic operations.

Parameters

testObj – Reference to an instance of a pysys.basetest.BaseTest class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran.
cycle – The cycle number. These start from 0, so please add 1 to this value before using.
kwargs – Additional keyword arguments may be added in a future release.

setup(cycles=- 1, numTests=- 1, threads=- 1, **kwargs)[source]¶

Called before any tests begin.

Before this method is called, for each property “PROP” specified for this writer in the project configuration file, the configured value will be assigned to self.PROP.

Parameters

numTests – The total number of tests (cycles*testids) to be executed
cycles – The number of cycles.
xargs – The runner’s xargs
threads – The number of threads used for running tests.
testoutdir – The output directory used for this test run (equal to runner.outsubdir), an identifying string which often contains the platform, or when there are multiple test runs on the same machine may be used to distinguish between them. This is usually a relative path but may be an absolute path.
runner – The runner instance that owns this writer. The default implementation of this methods sets the self.runner attribute to this value.
kwargs – Additional keyword arguments may be added in a future release.

ConsoleFailureAnnotationsWriter¶

class pysys.writer.console.ConsoleFailureAnnotationsWriter(logfile=None, **kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer that prints a single annotation line to stdout for each test failure, for IDEs and CI providers that can highlight failures found by regular expression stdout parsing.

An instance of this writer is automatically added to every project, and enables itself only if the PYSYS_CONSOLE_FAILURE_ANNOTATIONS environment variable is set.

This class is designed for simple cases. If you need to output in a format that requires escaping of special characters it is best to create a custom writer class.

DEFAULT_FORMAT = '@testFile@:@testFileLine@: @category@: @outcome@ - @outcomeReason@ (@testIdAndCycle@)'¶

This is the default format if the environment variable is empty and format is not provided.

The output looks like this:

c:\myproject\tests\MyTest_001\run.py:4: error: TIMED OUT - This test timed out (MyTest_001 [CYCLE 03])

which is similar to output from “make” and so should be parseable by many tools and IDEs.

enableIfEnvironmentVariable = 'PYSYS_CONSOLE_FAILURE_ANNOTATIONS'¶

The environment variable used to control whether it is enabled.

This writer will be enabled if the specified environment variable is set (either to any empty string or to any value other than “false”).

Set enableIfEnvironmentVariable to “” to ignore the environment and instead enable when running with --record.

format = ''¶

The format that will be written to stdout. If not specified as a writer property in pysysproject.xml, the environment variable PYSYS_CONSOLE_FAILURE_ANNOTATIONS will be used as the format.

The format can include the following placeholders:

@testFile@: the absolute path to the test file (e.g. run.py), using platform-specific slashes.

@testFile/@: the absolute path to the test file, using forward slashes on all OSes.

@testFileLine@: the line number in the test file (if available, else 0).

@runLogFile@: the absolute path to the run log (e.g. run.log), using platform-specific slashes.

@runLogFile/@: the absolute path to the run log (e.g. run.log), using forward slashes on all OSes.

@category@: either error or if it’s a non-failure outcome, warning.

@outcome@: the outcome e.g. FAILED.

@outcomeReason@: the string containing the reason for the failure; this string can contain any characters (other than newline).

@testIdAndCycle@: the test identifier, with a cycle suffix if this is a multi-cycle test.

The default format if the environment variable is empty and format is not provided is DEFAULT_FORMAT.

includeNonFailureOutcomes = 'NOT VERIFIED'¶: In addition to failure outcomes, any outcomes listed here (as comma-separated display names) will be reported (with a @category@ of warning rather than error).

isEnabled(record=False, **kwargs)[source]¶

Determines whether this writer can be used in the current environment.

If set to False then after construction none of the other methods (including setup)) will be called.

Parameters: record – True if the user ran PySys with the --record flag, indicating that test results should be recorded.
Returns: For record writers, the default to enable only if record==True, but individual writers can use different criteria if desired, e.g. writers for logging output to a CI system may enable themselves based on environment variables indicating that system is present, even if record is not specified explicitly.

processResult(testObj, cycle=- 1, **kwargs)[source]¶

Called when each test has completed.

This method is always invoked under a lock that prevents multiple concurrent invocations so additional locking is not usually necessary.

Parameters

testObj (pysys.basetest.BaseTest) – Reference to an instance of a pysys.basetest.BaseTest class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran.
cycle (int) – The cycle number. These start from 0, so please add 1 to this value before using.
testTime (float) – Duration of the test in seconds as a floating point number.
testStart (float) – The time when the test started.
runLogOutput (str) – The logging output written to the console/run.log, as a unicode character string. This string will include ANSI escape codes if colored output is enabled; if desired these can be removed using pysys.utils.logutils.stripANSIEscapeCodes().
kwargs – Additional keyword arguments may be added in future releases.

setup(cycles=- 1, **kwargs)[source]¶

Called before any tests begin.

Before this method is called, for each property “PROP” specified for this writer in the project configuration file, the configured value will be assigned to self.PROP.

Parameters

numTests – The total number of tests (cycles*testids) to be executed
cycles – The number of cycles.
xargs – The runner’s xargs
threads – The number of threads used for running tests.
testoutdir – The output directory used for this test run (equal to runner.outsubdir), an identifying string which often contains the platform, or when there are multiple test runs on the same machine may be used to distinguish between them. This is usually a relative path but may be an absolute path.
runner – The runner instance that owns this writer. The default implementation of this methods sets the self.runner attribute to this value.
kwargs – Additional keyword arguments may be added in a future release.