pysys.writer.ci

Writers for recording test results to Continuous Integration providers.

These writers only generate output if the environment variables associated with their CI system are set.

If you wish to run PySys from a CI provider that’s not listed here, for basic functionality all you need to do is run the tests with the --ci option. Many CI providers (including Jenkins) will be able to read the results of PySys tests if you configure them to look in the results directory generated by pysys.writer.outcomes.JUnitXMLResultsWriter. It may also be possible to configure some providers to automatically pick up archived test output from artifacts from the pysys.writer.testoutput.TestOutputArchiveWriter directory.

If you want to implement additional functionality for your CI provider, subclass pysys.writer.api.BaseRecordResultsWriter (using the classes in this module as an example), and consider contributing your implementation back to the PySys project with a pull request.

GitHubActionsCIWriter

class pysys.writer.ci.GitHubActionsCIWriter(logfile=None, **kwargs)

Bases: pysys.writer.api.BaseRecordResultsWriter, pysys.writer.api.TestOutcomeSummaryGenerator, pysys.writer.api.ArtifactPublisher

Writer for GitHub(R) Actions.

Produces annotations summarizing failures, adds grouping/folding of detailed test output, and sets step output variables for any published artifacts (e.g. performance .csv files, archived test output etc) which can be used to upload the artifacts when present. Step output variables for published artifacts are named ‘artifact_CATEGORY’ (for more details on artifact categories see pysys.writer.api.ArtifactPublisher.publishArtifact).

For example, if the step is given the id pysys then the directory containing archived test output for failures can be uploaded using {{ steps.pysys.outputs.artifact_TestOutputArchiveDir }}.

Be sure to include a unique run id (e.g. outdir) for each OS/job in the name of any uploaded artifacts so that they do not overwrite each other when uploaded.

See the PySys sample projects on GitHub(R) for a workflow file you can copy into your own project to enable running your PySys tests with GitHub(R) Actions.

Only enabled when running under GitHub Actions (specifically, if the GITHUB_ACTIONS=true environment variable is set).

failureSummaryAnnotations = True

Configures whether a single annotation is added with a summary of the number of failures and the outcome and reason for all failures.

failureTestLogAnnotations = True

Configures whether annotations are added with the (run.log) log output for each of the first few test failures.

maxAnnotations = 10

Configures the maximum number of annotations generated by this invocation of PySys, to cope with GitHub limits.

This ensure we don’t use up our entire allocation of annotations leaving no space for annotations from other tools. Being aware of this limit also allows to us add a warning to the end of the last one to make clear no more annotations will be shown even if there are more warnings.

TravisCIWriter

class pysys.writer.ci.TravisCIWriter(logfile=None, **kwargs)

Bases: pysys.writer.api.BaseRecordResultsWriter

Writer for Travis CI(R).

Provides folding of test log details, and correct coloring of console output for Travis.

Only enabled when running under Travis (specifically, if the TRAVIS=true environment variable is set).