pysys.process.user¶

Contains pysys.process.user.ProcessUser which supports using processes from PySys, and provides the shared functionality of subclasses pysys.basetest.BaseTest and pysys.baserunner.BaseRunner.

STDOUTERR_TUPLE¶

pysys.process.user.STDOUTERR_TUPLE = <class 'pysys.process.user.stdouterr'>¶: Returned by ProcessUser.allocateUniqueStdOutErr to hold a pair of (stdout,stderr) names.

ProcessUser¶

class pysys.process.user.ProcessUser[source]¶

Bases: object

ProcessUser provides support for safely using processes in PySys, including starting processes (with an appropriate output directory), waiting for them to do what they’re supposed to do, and finally ensuring all processes and ports are cleaned up when each test (or the overall runner) terminates.

As the common base class of both pysys.basetest.BaseTest and pysys.baserunner.BaseRunner, ProcessUser also holds shared functionality such as dynamic port allocation, copying files, creating directories etc, and keeps track of the “outcome” generated by some methods (although the outcome value is only used when it is subclassed by BaseTest).

Each ProcessUser instance is responsible for managing the lifetime of the processes started using its startProcess method, and ensuring they are all terminated when the cleanup method is invoked. Additional functions to be executed during cleanup can be registered using addCleanupFunction, for example to delete large output directories, or terminate non-process resources such as docker containers or remote servers.

Apart from the addOutcome method this class is not thread-safe, so if you need to access it from multiple threads be sure to add your own locking around use of its fields and methods, including any cleanup functions.

Variables

input (str) – Full path to the directory containing input files (e.g. testdir/Input)
output (str) – Full path to the directory that output files should be written to (e.g. testdir/Output/<platformname>)
log (logging.Logger) – The Python Logger instance that should be used to record progress and status information.
project (pysys.xml.project.Project) – A reference to the singleton project instance containing the configuration of this PySys test project as defined by pysysproject.xml. The project can be used to access information such as the project properties which are shared across all tests (e.g. for hosts and credentials).
disableCoverage (bool) –
Set to True to disable all code coverage collection for processes started from this instance. For example, to disable coverage in tests tagged with the ‘performance’ group you could use a line like this in your BaseTest:
```
if 'performance' in self.descriptor.groups: self.disableCoverage = True
```
The built-in Python code coverage functionality in startPython checks this flag. It is recommended that any other languages supporting code coverage also check the self.disableCoverage flag.

abort(outcome, outcomeReason, callRecord=None)[source]¶

Raise an AbortException with the specified outcome and reason.

See also skipTest.

Parameters

outcome – The outcome, which will override any existing outcomes previously recorded.
outcomeReason – A string summarizing the reason for the outcome.

addCleanupFunction(fn)[source]¶

Registers a function that will be called as part of the cleanup of this object.

Cleanup functions should have no arguments, and are invoked in reverse order with the most recently added first (LIFO), and before the automatic termination of any remaining processes associated with this object.

e.g. self.addCleanupFunction(lambda: self.cleanlyShutdownProcessX(params))

addOutcome(outcome, outcomeReason='', printReason=True, abortOnError=False, callRecord=None, override=False)[source]¶

Add a validation outcome (and optionally a reason string) to the validation list.

The method provides the ability to add a validation outcome to the internal data structure storing the list of validation outcomes. Multiple validations may be performed, the current supported validation outcomes of which are described in Assertions and outcomes.

The outcomes are considered to have a precedence order, as defined by the order of the outcomes listed above. Thus a pysys.constants.BLOCKED outcome has a higher precedence than a pysys.constants.PASSED outcome. The outcomes are defined in pysys.constants.

This method is thread-safe.

Although this method exists on all subclasses of pysys.process.user.ProcessUser, in practice only pysys.basetest.BaseTest subclasses actually do anything with the resulting outcome.

Parameters

outcome – The outcome to add, e.g. pysys.constants.FAILED.
outcomeReason – A string summarizing the reason for the outcome, for example “Grep on x.log contains ‘ERROR: server failed’”.
printReason – If True the specified outcomeReason will be printed
abortOnError – If true abort the test on any error outcome. This should usually be set to False for assertions, or the configured self.defaultAbortOnError setting (typically True) for operations that involve waiting.
callRecord – An array of strings indicating the call stack that lead to this outcome. This will be appended to the log output for better test triage.
override – Remove any existing test outcomes when adding this one, ensuring that this outcome is the one and only one reported even if an existing outcome has higher precedence.

allocateUniqueStdOutErr(processKey)[source]¶

Allocate unique filenames of the form processKey[.n].out/.err which can be used for the startProcess stdouterr parameter.

The first time this is called it will return names like ('myprocess.out', 'myprocess.err'), the second time it will return ('myprocess.1.out', 'myprocess.1.err'), then ('myprocess.2.out', 'myprocess.2.err') etc.

Parameters: processKey (str) – A user-defined identifier that will form the prefix onto which [.n].out is appended
Returns: A STDOUTERR_TUPLE named tuple of (stdout, stderr)
Return type: STDOUTERR_TUPLE

cleanup()[source]¶

Tear down function that frees resources managed by this object.

Should be called exactly once by the owner of this object when is no longer needed.

Do not override this method, instead use addCleanupFunction.

static compareVersions(v1, v2)[source]¶

Compares two alphanumeric dotted version strings to see which is more recent.

Example usage:

if self.compareVersions(thisversion, '1.2.alpha-3') > 0:
        ... # thisversion is newer than 1.2.alpha-3 

The comparison algorithm ignores case, and normalizes separators ./-/_ so that '1.alpha2'=='1Alpha2'. Any string components are compared lexicographically with other strings, and compared to numbers strings are always considered greater.

>>> ProcessUser.compareVersions('10-alpha5.dev10', '10alpha-5-dEv_10') == 0 # normalization of case and separators
True

>>> ProcessUser.compareVersions(b'1....alpha.2', u'1Alpha2') == 0 # ascii byte and unicode strings both supported
True

>>> ProcessUser.compareVersions('1.2.0', '1.2')
0

>>> ProcessUser.compareVersions('1.02', '1.2')
0

>>> ProcessUser().compareVersions('1.2.3', '1.2') > 0
True

>>> ProcessUser.compareVersions('1.2', '1.2.3')
-1

>>> ProcessUser.compareVersions('10.2', '1.2')
1

>>> ProcessUser.compareVersions('1.2.text', '1.2.0') # letters are > numbers
1

>>> ProcessUser.compareVersions('1.2.text', '1.2') # letters are > numbers 
1

>>> ProcessUser.compareVersions('10.2alpha1', '10.2alpha')
1

>>> ProcessUser.compareVersions('10.2dev', '10.2alpha') # letters are compared lexicographically
1

>>> ProcessUser.compareVersions('', '')
0

>>> ProcessUser.compareVersions('1', '')
1

Parameters

v1 – A string containing a version number, with any number of components.
v2 – A string containing a version number, with any number of components.

Returns

an integer > 0 if v1>v2, an integer < 0 if v1<v2, or 0 if they are semantically the same.

copy(src, dest, mappers=[], encoding=None)[source]¶

Copy a single text or binary file, optionally tranforming the contents by filtering each line through a list of mapping functions.

If any mappers are provided, the file is copied in text mode and each mapper is given the chance to modify or omit each line. If no mappers are provided, the file is copied in binary mode.

In addition to the file contents the mode is also copied, for example the executable permission will be retained.

This function is useful both for creating a modified version of an output file that’s more suitable for later validation steps such as diff-ing, and also for copying required files from the input to the output directory.

For example:

self.copy('output-raw.txt', 'output-processed.txt', encoding='utf-8', 
        mappers=[
                lambda line: None if ('Timestamp: ' in line) else line, 
                lambda line: line.replace('foo', 'bar'), 
        ])

Parameters

src – The source filename, which can be an absolute path, or a path relative to the self.output directory. Use src=self.input+'/myfile' if you wish to copy a file from the test input directory.
dest – The source filename, which can be an absolute path, or a path relative to the self.output directory. If this is a directory name, the file is copied to this directory with the same basename as src.
mappers – A list of filter functions that will be applied, in order, to each line read from the file. Each function accepts a string for the current line as input and returns either a string to write or None if the line is to be omitted.
encoding – The encoding to use to open the file. The default value is None which indicates that the decision will be delegated to the getDefaultFileEncoding() method.

Returns

the absolute path of the destination file.

createEnvirons(overrides=None, addToLibPath=[], addToExePath=[], command=None, **kwargs)[source]¶

Create a new customized dictionary of environment variables suitable for passing to startProcess()’s environs= argument.

As a starting point, this method uses the value returned by getDefaultEnvirons() for this command. See the documentation on that method for more details. If you don’t care about minimizing the risk of your local environment affecting the test processes you start, just use environs=os.environ to allow child processes to inherit the entire parent environment instead of using this method.

Parameters

overrides – A dictionary of environment variables whose values will be used instead of any existing values. You can use os.getenv('VARNAME','') if you need to pass selected variables from the current process as part of the overrides list. If the value is set to None then any variable of this name will be deleted. Use unicode strings if possible (byte strings will be converted depending on the platform). A list of dictionaries can be specified, in which case the latest will override the earlier if there are any conflicts.
addToLibPath – A path or list of paths to be prepended to the default value for the environment variable used to load libraries (or the value specified in overrides, if any), i.e. [DY]LD_LIBRARY_PATH on Unix or PATH on Windows. This is usually more convenient than adding it directly to overrides.
addToExePath – A path or list of paths to be prepended to the default value for the environment variable used to locate executables (or the value specified in overrides, if any), i.e. PATH on both Unix and Windows. This is usually more convenient than adding it directly to overrides.
command – If known, the full path of the executable for which a default environment is being created (passed to getDefaultEnvirons).
kwargs – Overrides of this method should pass any additional kwargs down to the super implementation, to allow for future extensions.

Returns

A new dictionary containing the environment variables.

deleteDir(path, **kwargs)[source]¶

Recursively delete the specified directory.

Does nothing if it does not exist. Raises an exception if the deletion fails.

Parameters

path – The path to be deleted. This can be an absolute path or relative to the testcase output directory.
kwargs – Any additional arguments are passed to pysys.utils.fileutils.deletedir().

getBoolProperty(propertyName, default=False)[source]¶

Get a True/False indicating whether the specified property is set on this object (typically as a result of specifying -X on the command line), or else from the project configuration.

Parameters: propertyName – The name of a property set on the command line or project configuration.

getDefaultEnvirons(command=None, **kwargs)[source]¶

Create a new dictionary of environment variables, suitable for passing to startProcess(), with a minimal clean set of environment variables for this platform, unaffected (as much as possible) by the environment that the tests are being run under.

This environment contains a minimal PATH/LD_LIBRARY_PATH but does not attempt to replicate the full set of default environment variables on each OS, and in particular it does not include any that identify the the current username or home area. Additional environment variables can be added as needed with createEnvirons overrides. If you don’t care about minimizing the risk of your local environment affecting the test processes you start, just use environs=os.environ to allow child processes to inherit the entire parent environment.

The createEnvirons() and startProcess() methods use this as the basis for creating a new set of default environment variables.

If needed this method can be overridden in subclasses to add common environment variables for every process invoked by startProcess, for example to enable options such as code coverage for Java/Python/etc. This is also a good place to customize behaviour for different operating systems.

Some features of this method can be configured by setting project properties:

defaultEnvironsDefaultLang: if set to a value such as en_US.UTF-8 the specified value is set for the LANG= variable on Unix; otherwise, the LANG variable is not set (which might result in use of the legacy POSIX/C encoding).

defaultEnvironsTempDir: if set the expression will be passed to Python eval() and used to set the OS-specific temp directory environment variables. A typical value is self.output.

defaultEnvironsLegacyMode: set to true to enable compatibility mode which keeps the behaviour the same as PySys v1.1, 1.2 and 1.3, namely using a completely empty default environment on Unix, and a copy of the entire parent environment on Windows. This is not recommended unless you have a lot of legacy tests that cannot easily be changed to only set minimal required environment variables using createEnvirons().

Parameters

command –
If known, the full path of the executable for which a default environment is being created (when called from startProcess this is always set). This allows default environment variables to be customized for different process types e.g. Java, Python, etc.

When using command=sys.executable to launch another copy of the current Python executable, extra items from this process’s path environment variables are added to the returned dictionary so that it can start correctly. On Unix-based systems this includes copying all of the load library path environment variable from the parent process.
kwargs – Overrides of this method should pass any additional kwargs down to the super implementation, to allow for future extensions.

Returns

A new dictionary containing the environment variables.

getDefaultFileEncoding(file, **xargs)[source]¶

Specifies what encoding should be used to read or write the specified text file.

This method is used to select the appropriate encoding whenever PySys needs to open a file, for example to wait for a signal, for a file-based assertion, or to write a file with replacements. Many methods allow the encoding to be overridden for just that call, but getDefaultFileEncoding exists to allow global defaults to be specified based on the filename.

For example, this method could be overridden to specify that utf-8 encoding is to be used for opening filenames ending in .xml, .json and .yaml.

The default implementation of this method uses pysysproject.xml configuration rules such as:

<default-file-encoding pattern="*.xml" encoding="utf-8"/>

A return value of None indicates default behaviour, which on Python 3 is to use the default encoding, as specified by python’s locale.getpreferredencoding(), and on Python 2 is to use binary str objects with no character encoding or decoding applied.

Parameters

file – The filename to be read or written. This may be an absolute path or a relative path.
xargs – Ensure that an **xargs argument is specified so that additional information can be passed to this method in future releases.

Returns

The encoding to use for this file, or None if default behaviour is to be used.

getExprFromFile(path, expr, groups=[1], returnAll=False, returnNoneIfMissing=False, encoding=None, reFlags=0)[source]¶

Searches for a regular expression in the specified file, and returns it.

If the regex contains unnamed groups, the specified group is returned. If the expression is not found, an exception is raised, unless returnAll=True or returnNoneIfMissing=True. For example:

self.getExprFromFile('test.txt', 'myKey="(.*)"') # on a file containing 'myKey="foobar"' would return "foobar"
self.getExprFromFile('test.txt', 'foo') # on a file containing 'myKey=foobar' would return "foo"

See also pysys.basetest.BaseTest.assertGrep which should be used when instead of just finding out what’s in the file you want to assert that a specific expression is matched. assertGrep also provides some additional functionality such as returning named groups which this method does not currently support.

Parameters

path (str) – file to search (located in the output dir unless an absolute path is specified)
expr (str) – the regular expression, optionally containing the regex group operator (...)
groups (List[int]) – which regex group numbers (as indicated by brackets in the regex) should be returned; default is [1] meaning the first group. If more than one group is specified, the result will be a tuple of group values, otherwise the result will be the value of the group at the specified index as a str.
returnAll (bool) – returns a list containing all matching lines if True, the first matching line otherwise.
returnNoneIfMissing (bool) – set this to return None instead of throwing an exception if the regex is not found in the file
encoding (str) – The encoding to use to open the file. The default value is None which indicates that the decision will be delegated to the getDefaultFileEncoding() method.
reFlags (int) –
Zero or more flags controlling how the behaviour of regular expression matching, combined together using the | operator, for example reFlags=re.VERBOSE | re.IGNORECASE.

For details see the re module in the Python standard library. Note that re.MULTILINE cannot be used because expressions are matched against one line at a time. Added in PySys 1.5.1.

Returns

A List[List[str]] if returnAll=True and groups contains multiple groups, a List[str] if only one of those conditions is true, or else a simple str containing just the first match found.

getInstanceCount(displayName)[source]¶

(Deprecated) Return the number of processes started within the testcase matching the supplied displayName.

Deprecated: The recommended way to allocate unique names is now allocateUniqueStdOutErr

The ProcessUser class maintains a reference count of processes started within the class instance via the startProcess() method. The reference count is maintained against a logical name for the process, which is the displayName used in the method call to startProcess(), or the basename of the command if no displayName was supplied. The method returns the number of processes started with the supplied logical name, or 0 if no processes have been started.

Parameters: displayName – The process display name
Returns: The number of processes started matching the command basename
Return type: int

getNextAvailableTCPPort(hosts=['', 'localhost'], socketAddressFamily=<AddressFamily.AF_INET: 2>)[source]¶

Allocate a free TCP port which can be used for starting a server on this machine.

The port is taken from the pool of available server (non-ephemeral) ports on this machine, and will not be available for use by any other code in the current PySys process until this object’s cleanup method is called to return it to the pool of available ports.

To allocate an IPv4 port for use only on this host:

port = self.getNextAvailableTCPPort(hosts=['localhost'])

Changed in version 1.5.1: Added hosts and socketAddressFamily parameters.

Parameters

hosts (list(Str)) –
A list of the host names or IP addresses to check when establishing that a potential allocated port isn’t already in use by a process outside the PySys framework. By default we check "" (which corresponds to INADDR_ANY and depending on the OS means either one or all non-localhost IPv4 addresses) and also localhost.

Many machines have multiple network cards each with its own host IP address, and typically you’ll only be using one of them in your test, most commonly localhost. If you do know which host/IP you’ll actually be using, just specify that directly to save time, and avoid needlessly opening remote ports on hosts your’re not using. A list of available host addresses can be found from socket.getaddrinfo('', None).
socketAddressFamily – The socket address family e.g. IPv4 vs IPv6. See Python’s socket module for details.

getOutcome()[source]¶

Get the overall outcome based on the precedence order.

The method returns the overall outcome of the test based on the outcomes stored in the internal data structure. The pysys.constants.PRECEDENT order of the possible outcomes is used to determined the overall outcome of the test, e.g. if pysys.constants.PASSED, pysys.constants.BLOCKED and pysys.constants.FAILED were recorded during the execution of the test, the overall outcome would be pysys.constants.BLOCKED.

The method returns the integer value of the outcome as defined in pysys.constants. To convert this to a string representation use the pysys.constants.LOOKUP dictionary i.e. LOOKUP[test.getOutcome()].

Returns: The overall outcome

getOutcomeReason()[source]¶

Get the reason string for the current overall outcome (if specified).

Returns: The overall test outcome reason or ‘’ if not specified
Return type: string

lock = None¶: A recursive lock that can be used for protecting the fields of this instance from access by background threads, as needed.

log = None¶: The logger instance that should be used to log from this class.

logFileContents(path, includes=None, excludes=None, maxLines=20, tail=False, encoding=None, logFunction=None, reFlags=0)[source]¶

Logs some or all of the lines in the specified file.

If the file does not exist or cannot be opened, does nothing. The method is useful for providing key diagnostic information (e.g. error messages from tools executed by the test) directly in run.log, or to make test failures easier to triage quickly.

Changed in version 1.5.1: Added logFunction parameter.

Parameters

path (str) – May be an absolute, or relative to the test output directory
includes (list[str]) – Optional list of regex strings. If specified, only matches of these regexes will be logged
excludes (list[str]) – Optional list of regex strings. If specified, no line containing these will be logged
maxLines (int) – Upper limit on the number of lines from the file that will be logged. Set to zero for unlimited
tail (bool) – Prints the _last_ ‘maxLines’ in the file rather than the first ‘maxLines’
encoding (str) – The encoding to use to open the file. The default value is None which indicates that the decision will be delegated to the getDefaultFileEncoding() method.
logFunction (Callable[[line],None]) – The function that will be used to log individual lines from the file. Usually this is self.log.info(u' %s', line, extra=BaseLogFormatter.tag(LOG_FILE_CONTENTS)) but a custom implementation can be provided, for example to provide a different color using pysys.utils.logutils.BaseLogFormatter.tag.
reFlags (int) –
Zero or more flags controlling how the behaviour of regular expression matching, combined together using the | operator, for example reFlags=re.VERBOSE | re.IGNORECASE.

For details see the re module in the Python standard library. Note that re.MULTILINE cannot be used because expressions are matched against one line at a time. Added in PySys 1.5.1.

Returns

True if anything was logged, False if not.

mkdir(path)[source]¶

Create a directory, with recursive creation of any parent directories.

This function does nothing (does not raise an except) if the directory already exists.

Parameters: path – The path to be created. This can be an absolute path or relative to the testcase output directory.
Returns: the absolute path of the new directory, to facilitate fluent-style method calling.

project = None¶: The pysys.xml.project.Project instance containing settings for this PySys project.

signalProcess(process, signal, abortOnError=None)[source]¶

Send a signal to a running process.

This method uses the pysys.process.commonwrapper.CommonProcessWrapper.signal to send a signal to a running process.

Should the request to send the signal to the running process fail, a BLOCKED outcome will be added to the outcome list.

Parameters

process – The process handle returned from the startProcess method
signal – The integer value of the signal to send
abortOnError – If True aborts the test with an exception on any error, if False just log it as a warning. (defaults to the defaultAbortOnError project setting)

skipTest(outcomeReason, callRecord=None)[source]¶

Raise an AbortException that will set the test outcome to SKIPPED and ensure that the rest of the execute() and validate() methods do not execute.

This is useful when a test should not be executed in the current mode or platform.

Parameters: outcomeReason – A string summarizing the reason the test is being skipped, for example “Feature X is not supported on Windows”.

startProcess(command, arguments, environs=None, workingDir=None, state=None, timeout=600, stdout=None, stderr=None, displayName=None, abortOnError=None, expectedExitStatus='==0', ignoreExitStatus=None, quiet=False, stdouterr=None, background=False)[source]¶

Start a process running in the foreground or background, and return the pysys.process.commonwrapper.CommonProcessWrapper object.

Typical use is:

myexecutable = self.startProcess('path_to_my_executable', 
        arguments=['myoperation', 'arg1','arg2'],
        environs=self.createEnvirons(addToLibPath=['my_ld_lib_path']), # if a customized environment is needed
        stdouterr=self.allocateUniqueStdOutErr('myoperation'), # for stdout/err files, pick a suitable logical name for what it's doing
        background=True # or remove for default behaviour of executing in foreground
        )

The method allows spawning of new processes in a platform independent way. The command, arguments, environment and working directory to run the process in can all be specified in the arguments to the method, along with the filenames used for capturing the stdout and stderr of the process. Processes may be started in the foreground, in which case the method does not return until the process has completed or a time out occurs, or in the background in which case the method returns immediately to the caller returning a handle to the process to allow manipulation at a later stage, typically with waitProcess.

All processes started in the background and not explicitly killed using the returned process object are automatically killed on completion of the test via the cleanup() destructor.

When starting a process that will listen on a server socket, use getNextAvailableTCPPort to allocate a free port before calling this method.

Parameters

command (str) – The path to the executable to be launched (should include the full path)
arguments (list[str]) – A list of arguments to pass to the command
environs (dict(str,str)) – A dictionary specifying the environment to run the process in. If a None or empty dictionary is passed, getDefaultEnvirons will be invoked to produce a suitable clean default environment for this command, containing a minimal set of variables. If you wish to specify a customized environment, createEnvirons() is a great way to create it.
workingDir (str) – The working directory for the process to run in (defaults to the testcase output subdirectory)
background (bool) – Set to True to start the process in the background. By default processes are started in the foreground, meaning execution of the test will continue only once the process has terminated.
state – Alternative way to set background=True. Run the process either in the FOREGROUND or BACKGROUND (defaults to FOREGROUND). Setting state=BACKGROUND is equivalent to setting background=True; in new tests using background=True is the preferred way to do this.
timeout (int) – The number of seconds after which to terminate processes running in the foreground. For processes that complete in a few seconds or less, it is best to avoid overriding this and stick with the default. However for long-running foreground processes it will be necessary to set a larger number, for example if running a soak test where the process needs to run for up to 2 hours you could set timeout=2*60*60.
stdouterr (str) – The filename prefix to use for the stdout and stderr of the process (out/err will be appended), or a tuple of (stdout,stderr) as returned from allocateUniqueStdOutErr. The stdouterr prefix is also used to form a default display name for the process if none is explicitly provided. The files are created relative to the test output directory. The filenames can be accessed from the returned process object using pysys.process.commonwrapper.CommonProcessWrapper.stdout and pysys.process.commonwrapper.CommonProcessWrapper.stderr.
stdout (str) – The filename used to capture the stdout of the process. It is usually simpler to use stdouterr instead of this.
stderr (str) – The filename used to capture the stderr of the process. It is usually simpler to use stdouterr instead of this.
displayName (str) – Logical name of the process used for display in log messages, and the str(…) representation of the returned process object (defaults to a string generated from the stdouterr and/or the command).
abortOnError (bool) – If true abort the test on any error outcome (defaults to the defaultAbortOnError project setting)
expectedExitStatus (str) – The condition string used to determine whether the exit status/code returned by the process is correct. The default is ‘==0’, as an exit code of zero usually indicates success, but if you are expecting a non-zero exit status (for example because you are testing correct handling of a failure condition) this could be set to ‘!=0’ or a specific value such as ‘==5’.
ignoreExitStatus (bool) –
If False, a BLOCKED outcome is added if the process terminates with an exit code that doesn’t match expectedExitStatus (or if the command cannot be run at all). This can be set to True in cases where you do not care whether the command succeeds or fails, or wish to handle the exit status separately with more complicated logic.

The default value of ignoreExitStatus=None means the value will be taken from the project property defaultIgnoreExitStatus, which can be configured in the project XML (the recommended default property value is defaultIgnoreExitStatus=False), or is set to True for compatibility with older PySys releases if no project property is set.
quiet (bool) – If True, this method will not do any INFO or WARN level logging (only DEBUG level), unless a failure outcome is appended. This parameter can be useful to avoid filling up the log where it is necessary to repeatedly execute a command check for completion of some operation until it succeeds; in such cases you should usually set ignoreExitStatus=True as well since both success and failure exit statuses are valid.

Returns

The process wrapper object.

Return type

pysys.process.commonwrapper.CommonProcessWrapper

startPython(arguments, disableCoverage=False, **kwargs)[source]¶

Start a Python process with the specified arguments.

Uses the same Python process the tests are running under.

If PySys was run with the arguments -X pythonCoverage=true then startPython will add the necessary arguments to enable generation of code coverage. Note that this requried the coverage.py library to be installed. If a project property called pythonCoverageArgs exists then its value will be added as (space-delimited) arguments to the coverage tool.

Parameters

arguments – The arguments to pass to the Python executable. Typically the first one be either the name of a Python script to execute, or -m followed by a module name.
kwargs – See startProcess for detail on available arguments.
disableCoverage – Disables code coverage for this specific process. Coverage can also be disabled by setting self.disableCoverage==True on this test instance.

Returns

The process handle of the process.

Return type

pysys.process.commonwrapper.CommonProcessWrapper

stopProcess(process, abortOnError=None)[source]¶

Stops the specified process, if it is currently running.

Does nothing if the process is not running.

This is equivalent to calling pysys.process.commonwrapper.CommonProcessWrapper.stop(), except it also logs an info message when the process is stopped.

Parameters

process – The process handle returned from the startProcess method
abortOnError – If True abort the test on any error outcome (defaults to the defaultAbortOnError project setting), if False a failure to stop the process will just be logged as a warning.

waitForFile(file, filedir=None, timeout=30, abortOnError=None)[source]¶

Wait for a file to exist on disk.

This method blocks until a file is created on disk. This is useful for test timing where a component under test creates a file (e.g. for logging) indicating it has performed all initialisation actions and is ready for the test execution steps. If a file is not created on disk within the specified timeout interval, the method returns to the caller.

Parameters

file – The basename of the file used to wait to be created
filedir – The dirname of the file (defaults to the testcase output subdirectory)
timeout – The timeout in seconds to wait for the file to be created
abortOnError – If true abort the test on any failure (defaults to the defaultAbortOnError project setting)

waitForGrep(file, expr='', condition='>=1', timeout=60, poll=0.25, ignores=[], process=None, errorExpr=[], abortOnError=None, encoding=None, detailMessage='', filedir=None, reFlags=0)[source]¶

Wait for a regular expression line to be seen (one or more times) in a text file in the output directory (waitForGrep was formerly known as waitForSignal).

This method provides some parameters that give helpful fail-fast behaviour with a descriptive outcome reason; use these whenever possible:

process= to abort if success becomes impossible due to premature termination of the process that’s generating the output

errorExpr= to abort if an error message/expression is written to the file

This will generate much clearer outcome reasons, which makes test failures easy to triage, and also avoids wasting time waiting for something that will never happen.

Example:

self.waitForGrep('myprocess.log', expr='INFO .*Started successfully', process=myprocess, 
        errorExpr=[' ERROR ', ' FATAL ', 'Failed to start'], encoding='utf-8')

Note that waitForGrep fails the test if the expression is not found (unless abortOnError was set to False, which isn’t recommended), so there is no need to add duplication with an assertGrep to check for the same expression in your validation logic.

The message(s) logged when there is a successful wait can be controlled with the project property verboseWaitForGrep=true/false (or equivalently, verboseWaitForSignal); for best visibility into what is happening set this property to true in your pysysproject.xml.

You can extract information from the matched expression, optionally perform assertions on it, by using one or more (?P<groupName>...) named groups in the expression. A common pattern is to unpack the resulting dict using **kwargs syntax and pass to BaseTest.assertThat. For example:

self.assertThat('username == expected', expected='myuser',
        **self.waitForGrep('myserver.log', expr=r'Successfully authenticated user "(?P<username>[^"]*)"'))

New in version 1.5.1.

Parameters

file (str) – The path of the file to be searched. Usually this is a name/path relative to the self.output directory, but alternatively an absolute path can be specified.
expr (str) – The regular expression to search for in the text file.
condition (str) – The condition to be met for the number of lines matching the regular expression; by default we wait until there is at least one occurrence.
timeout (int) – The number of seconds to wait for the regular expression before giving up and aborting the test with pysys.constants.TIMEDOUT (unless abortOnError=False in which case execution will continue).
process (pysys.process.commonwrapper.CommonProcessWrapper) – The process that is generating the specified file, to allow the wait to fail fast (instead of timing out) if the process dies before the expected signal appears. Can be None if the process is not known or is expected to terminate itself during this period.
errorExpr (list[str]) – Optional list of regular expressions, which if found in the file will cause waiting for the main expression to be aborted with a pysys.constants.BLOCKED outcome. This is useful to avoid waiting a long time for the expected expression when an ERROR is logged that means it will never happen, and also provides much clearer test failure messages in this case.
ignores (list[str]) – A list of regular expressions used to identify lines in the files which should be ignored when matching both expr and errorExpr.
poll (float) – The time in seconds between to poll the file looking for the regular expression and to check against the condition
abortOnError (bool) – If True abort the test on any error outcome (defaults to the defaultAbortOnError project setting, which for a modern project will be True).
encoding (str) – The encoding to use to open the file and convert from bytes to characters. The default value is None which indicates that the decision will be delegated to the getDefaultFileEncoding() method.
detailMessage (str) – An extra string to add to the message logged when waiting to provide extra information about the wait condition. e.g. detailMessage='(downstream message received)'. Added in v1.5.1.
reFlags (int) –
Zero or more flags controlling how the behaviour of regular expression matching, combined together using the | operator, for example reFlags=re.VERBOSE | re.IGNORECASE.

For details see the re module in the Python standard library. Note that re.MULTILINE cannot be used because expressions are matched against one line at a time. Added in PySys 1.5.1.
filedir (str) – Can be used to provide a directory name to add to the beginning of the file parameter; however usually it is clearer just to specify that directory in the file.

Return list[re.Match]

Usually this returns a list of re.Match objects found for the expr, or an empty list if there was no match.

If the expr contains any (?P<groupName>...) named groups, and assuming the condition still the default of “>=1” (i.e. not trying to find multiple matches), then a dict is returned containing dict(groupName: str, matchValue: str or None) (or an empty {} dict if there is no match) which allows the result to be passed to assertThat for further checking of the matched groups (typically unpacked using the ** operator; see example above).

waitForSignal(file, filedir=None, expr='', **waitForGrepArgs)[source]¶

Old alias for waitForGrep; please use waitForGrep in new tests.

All parameters are the same, except that in waitForSignal the (rarely used) filedir argument can be specified as the 2nd positional argument (after file and before expr) whereas in waitForGrep it can only be specified as a filedir= keyword argument.

waitForSocket(port, host='localhost', timeout=60, abortOnError=None, process=None, socketAddressFamily=<AddressFamily.AF_INET: 2>)[source]¶

Wait until it is possible to establish a socket connection to a server running on the specified local or remote port.

This method blocks until connection to a particular host:port pair can be established. This is useful for test timing where a component under test creates a socket for client server interaction - calling of this method ensures that on return of the method call the server process is running and a client is able to create connections to it. If a connection cannot be made within the specified timeout interval, the method returns to the caller, or aborts the test if abortOnError=True.

Changed in version 1.5.1: Added host and socketAddressFamily parameters.

Parameters

port – The port value in the socket host:port pair
host – The host value in the socket host:port pair
timeout – The timeout in seconds to wait for connection to the socket
abortOnError – If true abort the test on any failure (defaults to the defaultAbortOnError project setting)
process – If a handle to a process is specified, the wait will abort if the process dies before the socket becomes available. It is recommended to set this wherever possible.
socketAddressFamily – The socket address family e.g. IPv4 vs IPv6. See Python’s socket module for details.

waitProcess(process, timeout, abortOnError=None)[source]¶

Wait for a background process to terminate, completing on termination or expiry of the timeout.

Timeouts will result in an exception unless the project property defaultAbortOnError==False.

This method does not check the exit code for success, but you can manually check the return value (which is the same as process.exitStatus) using assertThat if you wish to check it succeeded.

Parameters

process – The process handle returned from the startProcess method
timeout – The timeout value in seconds to wait before returning
abortOnError – If True aborts the test with an exception on any error, if False just log it as a warning. (defaults to the defaultAbortOnError project setting)

Returns

The process’s exitStatus. This will be None if the process timed out and abortOnError is disabled.

writeProcess(process, data, addNewLine=True)[source]¶

Write binary data to the stdin of a process.

This method uses pysys.process.commonwrapper.CommonProcessWrapper.write to write binary data to the stdin of a process. This wrapper around the write method of the process helper only adds checking of the process running status prior to the write being performed, and logging to the testcase run log to detail the write.

Parameters

process – The process handle returned from the startProcess() method
data – The data to write to the process stdin. As only binary data can be written to a process stdin, if a character string rather than a byte object is passed as the data, it will be automatically converted to a bytes object using the encoding given by locale.getpreferredencoding().
addNewLine – True if a new line character is to be added to the end of the data string

write_text(file, text, encoding=None)[source]¶

Writes the specified characters to a file in the output directory.

Parameters

file – The path of the file to write, either an absolute path or relative to the self.output directory.
text –
The string to write to the file, with n for newlines (do not use os.linesep as the file will be opened in text mode so platform line separators will be added automatically).

On Python 3 this must be a character string.

On Python 2 this can be a character or byte string containing ASCII characters. If non-ASCII characters are used, it must be a unicode string if there is an encoding specified for this file/type, or else a byte string.
encoding – The encoding to use to open the file. The default value is None which indicates that the decision will be delegated to the getDefaultFileEncoding() method.