pysys.writer.outcomes¶

Writers that record test outcomes to a variety of file formats.

JSONResultsWriter¶

class pysys.writer.outcomes.JSONResultsWriter(logfile, **kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter, pysys.writer.api.ArtifactPublisher

Writer to log a summary of the test results to a single JSON file, along with runDetails from the runner, and a list of published artifacts (e.g. code coverage etc).

The following fields are always included for each test result:

testId: the test id, including ~mode suffix if it has modes defined.

outcome: the outcome string e.g. “NOT VERIFIED”.

outcomeReason: the string explaining the reason for the outcome, or empty if not available.

startTime: the time the test started, in %Y-%m-%d %H:%M:%S format and in the local timezone.

durationSecs: how long the test executed for (or None/null if unknown).

testDir: the path to this test under the testRootDir, using forward slashes.

testFile: the path (typically relative to testDir, using forward slashes) of the main file containing the test’s logic, e.g. pysystest.py, run.py etc. This is usually, but not always, a Python file.

If applicable, some tests/runs may have additional fields such as cycle, title and outputDir.

New in version 2.1.

Changed in version 2.2: Added artifacts dictionary recording artifact paths published during execution of the tests, for example code coverage and performance reports.

includeTitle = True¶

By default the title of each test is included in the output.

To save disk space you can set this to False if it is not needed.

includeNonFailureOutcomes = '*'¶

In addition to failure outcomes, any outcomes listed here (as comma-separated display names, e.g. "NOT VERIFIED, INSPECT") will be included. To include all non-failure outcomes, set this to the special value "*".

To save disk space and record only failure outcomes, you may set this to an empty string.

outputDir = None¶

The directory to write the logfile, if an absolute path is not specified. The default is the working directory.

Project ${...} properties can be used in the path.

createTestResultDict(testObj, **kwargs)[source]¶

Creates the dict that will be output for each test result.

Returns: The dict, or None if this result should not be included/

TextResultsWriter¶

class pysys.writer.outcomes.TextResultsWriter(logfile, **kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer to log a summary of the results to a log file in text format.

Only enabled when the record flag is specified.

Can be a useful way to view the current or final status of the current/latest test run. The file is written incrementally as each test completes, but can be sorted (using standard command line tools) to get a nice summary of all the failures.

outputDir = None¶

The directory to write the logfile, if an absolute path is not specified. The default is the working directory.

Project ${...} properties can be used in the path, for example ${eval:os.path.expanduser('~')} for the current user’s home directory.

verbose = False¶: Display more details in the output including the outcome reason and the test title.

New in version 2.2.

XMLResultsWriter¶

class pysys.writer.outcomes.XMLResultsWriter(logfile, **kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer to log results to logfile in a single XML file.

The class creates a DOM document to represent the test output results and writes the DOM to the logfile using toprettyxml(). The outputDir, stylesheet, useFileURL attributes of the class can be overridden in the PySys project file using the nested <property> tag on the <writer> tag.

Variables

~.outputDir (str) – Path to output directory to write the test summary files
~.stylesheet (str) – Path to the XSL stylesheet
~.useFileURL (str) – Indicates if full file URLs are to be used for local resource references

JUnitXMLResultsWriter¶

class pysys.writer.outcomes.JUnitXMLResultsWriter(**kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer to log test results in the widely-used Apache Ant JUnit XML format (one output file per test per cycle).

If you need to integrate with any CI provider that doesn’t have built-in support (e.g. Jenkins) this standard output format will usually be the easiest way to do it.

The output directory is published as with category name “JUnitXMLResultsDir”.

outputDir = None¶

The directory to write the XML files to, as an absolute path, or relative to the testRootDir.

Project ${...} properties can be used in the path.

testsuiteName = None¶: Overrides the content written to the <testsuite name=...> attribute (defaults to @TESTID@), A recommended alternate value is @TESTID_PACKAGE@ which represents the test id up to the final ..

New in version 2.2.

testcaseClassname = None¶: Overrides the content written to the <testcase classname=...> attribute (defaults to @CLASSNAME@), A recommended alternate value is empty string, since the classname (typically “PySys”) is not very useful for most PySys tests.

New in version 2.2.

testcaseName = None¶: Overrides the content written to the <testcase name=...> attribute (defaults to @TESTID@), A recommended alternate value is @TESTID_NO_PACKAGE_OR_MODE@~@MODE@ when using testsuiteName to hold the package part of the test id.

New in version 2.2.

CSVResultsWriter¶

class pysys.writer.outcomes.CSVResultsWriter(logfile, **kwargs)[source]¶

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer to log results to logfile in CSV format.

Writing of the test summary file defaults to the working directory. This can be be over-ridden in the PySys project file using the nested <property> tag on the <writer> tag. The CSV column output is in the form:

id, title, cycle, startTime, duration, outcome