Project ConfigurationΒΆ
Each PySys project has a configuration file called pysysproject.xml at the top level. This file contains
property elements for any user-defined properties that will be used by your tests such as credentials, server
names, which are available to your tests using self.project.
The project file also contains allows customization of how PySys executes tests and reports on the results.
When starting a new PySys test project you should create a minimal project file from scratch using
pysys makeproject, then add any additional elements you require.
To illustrate the range of available elements, here is a large project file that demonstrates all the main functionality:
<?xml version="1.0" encoding="utf-8"?>
<pysysproject>
<!--
Specify a minimum required pysys version to run these tests
-->
<requires-pysys>1.6.1</requires-pysys>
<!--
Specify a minimum required python version to run these tests
-->
<requires-python>2.7.0</requires-python>
<!--
The following standard project properties are always defined and can be accessed through ${prop} syntax:
${testRootDir} - Path of the root directory containing the pysysproject.xml file
${outDirName} - The basename (with parent dirs removed) from the outdir for this test run.
This may be the name of the current OS or a unique user-specified name for the test run.
${os} - The operating system name e.g. 'windows', 'linux', 'darwin'.
${osfamily} - The operating system family - 'windows' or `unix`.
${startDate} - The date that this test run was started, in a form that can be used in filenames.
${startTime} - The (local) time that this test run was started, in a form that can be used in filenames.
${hostname} - The (non-qualified) name of the host this is running on, suitable for including in filenames.
In addition, within this file ${env.VARNAME} syntax can be used to access environment variables.
-->
<!-- Property identifying the home directory of the application build being tested.
Binaries and configuration files can be specified relative to this directory
to avoid having to hardcode locations inside each individual testcase.
A different directory can be used temporarily by setting environment variable PYSYS_APP_HOME.
Project properties can be used as substitution variables within the project file, and
are set as attributes on the Project class for use by tests. In the example below, "appHome"
would be available to tests as "self.project.appHome".
If a property value contains any properties or environment variables that do not exist,
the "default" is used instead (or "" if a default is not explicitly specified). If the default
also contains undefined properties the project will fail to load.
-->
<property name="appHome" value="${env.PYSYS_APP_HOME}" default="${testRootDir}/.." pathMustExist="true"/>
<!-- If provided, the specified command line will be executed (in testRootDir) to populate the vcsCommit field
in runner.runDetails with the current commit number from your version control system. -->
<property name="versionControlGetCommitCommand" value="git show -s --format=%h"/>
<!--
Set default LANG for child processes on Unix.
Useful for making all machines behave the same. Ensure that the specified
locale is installed on all machines.
-->
<!-- <property name="defaultEnvironsDefaultLang" value="en_US.UTF-8"/> -->
<!--
The defaultEnvirons. prefix can be used to provide a default environment variable for use by
all processes that PySys starts. For example, to set Java(R)'s JAVA_TOOL_OPTIONS environment variable:
-->
<!--
<property name="defaultEnvirons.JAVA_TOOL_OPTIONS" value="-Xmx512M"/>
-->
<!--
Import project properties from a file, using the specified prefix for namespacing.
If pathMustExist=true, the project will fail to load if the specified file does not
exist. If pathMustExist=false the project will silently ignore a missing properties file,
which can be useful when using this feature to provide optional user-specific overrides.
The properties file should be of the format name=value (one pair specified per line).
Each imported property line is handled just the same as a standard "property" element,
so for example other project properties can be referenced withint the values
using ${...} syntax.
The set of property keys that are imported can be filtered with a regular expression using
the includes="regex" and excludes="regex" optional attributes.
-->
<property file="pysys-${osfamily}.properties" pathMustExist="false" prefix="os_"/>
<!--
Add a path to the python path to allow importing custom modules, e.g. extensions to the
BaseTest or BaseRunner classes, or customer test output writers.
-->
<!--
<pythonpath value="${testRootDir}/pysys-extensions" />
-->
<!-- Test plugins are additional classes instantiated when each test's BaseTest is instantiated, and accessible
to testcases (using the specified alias) as a field of the BaseTest. They provide a way to expose extra
functionality for use by your testcases, for example support for additional languages and technologies.
Each plugin class must provide a constructor __init__(self, testObj, pluginProperties) that accepts the owner
testObj (BaseTest instance) as a parameter and a dictionary of plugin properties provided as attributes
and/or <property> elements.
Each plugin class must provide a constructor that accepts the parent testObj (BaseTest instance) as a parameter.
The plugin instance is assigned as a field of the test object using the specified "alias" so that its
methods and fields are available for use. The alias can be any valid Python identifier but must not conflict
with other plugins or fields that PySys sets on the BaseTest; usually a brief lowercase name
identifying your organization or the purpose of the plugin is best. -->
<!--
<test-plugin classname="myorg.plugins.MyTestPlugin" alias="myorg">
<property name="myProp" value="..."/>
</test-plugin>
-->
<!-- Runner plugins are classes that are instantiated when the BaseRunner performs its setup() at
the beginning of a test run. They can provide extra functionality both at the beginning of a test
run, and also (by calling addCleanupFunction from setup()) at the end after testing has finished. For example,
a runner plugin could be used to add support for starting a database server or virtual machine to be shared
by all tests, or to collect.
Each plugin class must provide a constructor __init__(self, runner, pluginProperties) that accepts the owner
runner (BaseRunner instance) as a parameter and a dictionary of plugin properties provided as attributes
and/or <property> elements.
The plugin instance can optionally be assigned as a field of the runner using the specified "alias" so that its
methods and fields are available for use by tests. The alias can be any valid Python identifier but must not
conflict with other plugins or fields that PySys sets on the BaseRunner; usually a brief lowercase name
identifying your organization or the purpose of the plugin is best. -->
<!--
<runner-plugin classname="myorg.plugins.MyRunnerPlugin" alias="myorg">
<property name="myProp" value="..."/>
</runner-plugin>
-->
<!--
For advanced cases it is possible to provide a custom BaseRunner subclass. However, consider whether the
composition "runner-plugin" approach would do the job before using runner inheritance.
For example, to use the MyRunner class in the myorg module use:
-->
<!--
<runner classname="myorg.MyRunner"/>
-->
<!--
Use a custom maker class for constructing new testcases. Custom maker classes can extend from the
ConsoleMakeTestHelper class in order to create templated run.py scripts e.g. when a test module has
a repeatable pattern for running or validating etc.
-->
<!--
<maker classname="myorg.MyMaker"/>
-->
<!--
Configures the writers that implement reporting of test outcomes, typically to disk, to the console, or
to a CI system.
For full details of the configuration properties of each writer, and the API for creating custom writers,
see `pysys.writer` in the API reference.
The writer element specifies the module (which should be available on the pythonpath) and writer classname.
-->
<writers>
<!-- This writer is useful for creating zip archives of failed test output directories when
running in record mode on a machine where it is not otherwise easy to access the output
directories. The destDir could then be uploaded to a CI system (some CI writers
implement this automatically) or manually copied to a file share.
-->
<writer classname="pysys.writer.testoutput.TestOutputArchiveWriter">
<property name="destDir" value="__pysys_output_archives.${outDirName}/"/>
<property name="maxTotalSizeMB" value="1024.0"/>
<property name="maxArchiveSizeMB" value="200.0"/>
<property name="maxArchives" value="50"/>
</writer>
<!-- CI writers automatically enable themselves only if running under
the associated CI environment.
-->
<writer classname="pysys.writer.ci.GitHubActionsCIWriter"></writer>
<writer classname="pysys.writer.ci.TravisCIWriter"></writer>
<!--
Add in the JUnit results writer for PySys test output to be written in the widely-used Apache Ant JUnit XML format.
Use the outputDir property to define the output directory for the JUnit test summary files (the writer will
produce one file per test into this output directory).
-->
<writer classname="pysys.writer.outcomes.JUnitXMLResultsWriter">
<property name="outputDir" value="__pysys_junit_xml"/>
</writer>
<writer classname="pysys.writer.outcomes.XMLResultsWriter" file="__pysys_testsummary_${outDirName}_%Y-%m-%d_%H.%M.%S.xml">
<!--
Set properties on the XML test output writer class. The available properties that
can be set are the stylesheet location, whether to use file URLs in all references
to resources on the local disk, and the directory to write the output file (defaults
to the current working directory). Note that Mozilla Firefox requires the stylesheet
to be located next to the XML file when loading the file, and all references to local
resources must be as file URLs. Internet Explorer and Chrome can load the stylesheet
from any location on the local disk, but cannot load resources when referenced by a
file URL.
The filename template is processed through time.strftime so that time information can
be set into the filename, e.g. a filename template of 'testsummary.%Y-%m-%d_%H.%M.%S' will result
in a file created with a name of testsummary_2008-10-25_21.33.08.xml etc.
<property name="outputDir" value="${testRootDir}"/>
<property name="stylesheet" value="./pysys-log.xsl"/>
<property name="useFileURL" value="true"/>
-->
</writer>
<!--
Add in the test results writer if output to a text file is required
<writer classname="pysys.writer.outcomes.TextResultsWriter" file="__pysys_testsummary_%Y-%m-%d_%H.%M.%S.log">
<property name="outputDir" value="${testRootDir}"/>
</writer>
-->
<!--
Add in the CSV results writer if CSV text output is required. This outputs the test results
in a column separated list, with headings id, title, cycle, startTime, duration, outcome
<writer classname="pysys.writer.outcomes.CSVResultsWriter" file="__pysys_testsummary_%Y-%m-%d_%H.%M.%S.csv">
<property name="outputDir" value="${testRootDir}"/>
</writer>
-->
<!-- Code coverage for .py files -->
<writer classname="pysys.writer.testoutput.PythonCoverageWriter">
<property name="destDir" value="__coverage_python.${outDirName}"/>
<property name="pythonCoverageArgs" value="--rcfile=${testRootDir}/internal/utilities/python_coveragerc"/>
</writer>
<!-- The ConsoleSummaryResultsWriter displays a summary of non-passed outcomes at the end of the test run,
optionally including outcome reason. The ConsoleSummaryResultsWriter is automatically added to the writers
list if no other "summary" writer is explicitly configured.
-->
<writer classname="pysys.writer.console.ConsoleSummaryResultsWriter">
<property name="showTestTitle" value="false"/>
</writer>
</writers>
<default-file-encodings>
<!--
Specify the file encoding to be used for reading/writing text files.
The first pattern that matches is used to determine the encoding. The pattern is a glob-style expression to be
matched case-insensitively against either the full path or the basename using Python's fnmatch.fnmatch method.
The defaults specified here can be overridden or added to by the runner or basetest getDefaultFileEncoding()
method. See pysys.process.user.ProcessUser.getDefaultFileEncoding for more details.
-->
<default-file-encoding pattern="run.log" encoding="utf-8"/>
<default-file-encoding pattern="*.xml" encoding="utf-8"/>
<default-file-encoding pattern="*.json" encoding="utf-8"/>
<default-file-encoding pattern="*.yaml" encoding="utf-8"/>
</default-file-encodings>
<execution-order secondaryModesHintDelta="+100.0">
<!--
The execution-order elements provide a way to globally change the ordering hints specified on individual
tests by adding or subtracting a value from the hints specified on test descriptors in a specified group
and/or mode.
Groups and modes can be identified with a full name or a regular expression.
The secondaryModesHintDelta value is used to order tests so that all tests execute in their primary
mode before any secondary modes are executed. The 2nd mode (the one following the primary mode) has its
execution order hint incremented by secondaryModesHintDelta, the third by 2*secondaryModesHintDelta etc,
which ensures the modes are spaced out. To disable this behaviour and execute all modes of each test
before moving on to the next test set it to 0.0. If not specified, the default value is +100.0.
-->
<!--
<execution-order hint="+20.0" forMode="MyMode_.*"/>
<execution-order hint="-10.5" forGroup="performance" forMode="MyMode"/>
-->
</execution-order>
<!--
In most projects there is no reason to provide a <formatters> element, but it can be used if you wish to customize
the log line format (incl date/time format) or console coloring.
Specify custom formatters for logging to the console or run.log, and/or configure the formatter
appropriately through custom properties. Custom formatters should be defined using the classname
and module attributes and should extend the pysys.utils.logutils.BaseLogFormatter class. If no
classname and module is given the default pysys.utils.logutils.ColorLogFormatter is assumed.
The ColorLogFormatter allows specification of the message and date strings using the messagefmt and
datafmt attributes. Enabling color to the console (stdout) formatter can be done using the color
property, and the colors for supported message types can be specified via the color:<category> property.
See below for more details for the default color types and categories. Supported colors are BLUE,
GREEN, YELLOW, RED, MAGENTA, CYAN, WHITE and BLACK.
-->
<formatters>
<formatter classname="pysys.utils.logutils.ColorLogFormatter"
name="stdout" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt="%H:%M:%S">
<!--
<property name="color" value="true"/>
-->
<property name="color:warn" value="MAGENTA"/>
<property name="color:error" value="RED"/>
<property name="color:traceback" value="RED"/>
<property name="color:debug" value="BLUE"/>
<property name="color:filecontents" value="BLUE"/>
<property name="color:details" value="CYAN"/>
<property name="color:outcomereason" value="CYAN"/>
<property name="color:progress" value="CYAN"/>
<property name="color:performance" value="CYAN"/>
<property name="color:timed out" value="MAGENTA"/>
<property name="color:failed" value="RED"/>
<property name="color:passed" value="GREEN"/>
<property name="color:skipped" value="YELLOW"/>
</formatter>
<formatter classname="pysys.utils.logutils.BaseLogFormatter"
name="runlog" messagefmt="%(asctime)s %(levelname)-5s %(message)s" datefmt=""/>
</formatters>
<project-help>
<!-- Add project-specific text to be appended to the "pysys run -h".
You can use ${...} project properties, or ${$} to escape the $ character.
-->
</project-help>
</pysysproject>