Package pysys :: Module basetest :: Class BaseTest
[hide private]
[frames] | no frames]

Class BaseTest

source code

              object --+    
                       |    
process.user.ProcessUser --+
                           |
                          BaseTest
Known Subclasses:

The base class for all PySys testcases.

BaseTest is the parent class of all PySys system testcases. The class provides utility functions for cross-platform process management and manipulation, test timing, and test validation. Any PySys testcase should inherit from the base test and provide an implementation of the abstract execute method defined in this class. Child classes can also overide the setup, cleanup and validate methods of the class to provide custom setup and cleanup actions for a particual test, and to perform all validation steps in a single method should this prove logically more simple.

Execution of a PySys testcase is performed through an instance of the pysys.baserunner.BaseRunner class, or a subclass thereof. The base runner instantiates an instance of the testcase, and then calls the setup, execute, validate and cleanup methods of the instance. All processes started during the test execution are reference counted within the base test, and terminated within the cleanup method.

Validation of the testcase is through the assert* methods. Execution of many methods appends an outcome to the outcome data structure maintained by the ProcessUser base class, thus building up a record of the individual validation outcomes. Several potential outcomes are supported by the PySys framework (SKIPPED, BLOCKED, DUMPEDCORE, TIMEDOUT, FAILED, NOTVERIFIED, and PASSED) and the overall outcome of the testcase is determined using aprecedence order of the individual outcomes.

All assert* methods except for assertThat support variable argument lists for common non-default parameters. Currently this includes the assertMessage parameter, to override the default statement logged by the framework to stdout and the run log, and the abortOnError parameter, to override the defaultAbortOnError project setting.

Instance Methods [hide private]
 
__init__(self, descriptor, outsubdir, runner)
Create an instance of the BaseTest class.
source code
 
setKeywordArgs(self, xargs)
Set the xargs as data attributes of the test class.
source code
 
setup(self)
Setup method which may optionally be overridden to perform custom setup operations prior to test execution.
source code
 
execute(self)
Execute method which must be overridden to perform the test execution steps.
source code
 
validate(self)
Validate method which may optionally be overridden to group all validation steps.
source code
 
cleanup(self)
Cleanup method which performs cleanup actions after execution and validation of the test.
source code
 
addResource(self, resource)
Add a resource which is owned by the test and is therefore cleaned up (deleted) when the test is cleaned up.
source code
handle
startProcessMonitor(self, process, interval, file, **kwargs)
Start a separate thread to log process statistics to logfile, and return a handle to the process monitor.
source code
 
stopProcessMonitor(self, monitor)
Stop a process monitor.
source code
 
startManualTester(self, file, filedir=None, state=11, timeout=1800)
Start the manual tester.
source code
 
stopManualTester(self)
Stop the manual tester if running.
source code
 
waitManualTester(self, timeout=1800)
Wait for the manual tester to be stopped via user interaction.
source code
 
wait(self, interval)
Wait for a specified period of time.
source code
 
assertThat(self, conditionstring, *args)
Perform a validation based on a python eval string.
source code
 
assertTrue(self, expr, **xargs)
Perform a validation assert on the supplied expression evaluating to true.
source code
 
assertFalse(self, expr, **xargs)
Perform a validation assert on the supplied expression evaluating to false.
source code
 
assertDiff(self, file1, file2, filedir1=None, filedir2=None, ignores=[], sort=False, replace=[], includes=[], **xargs)
Perform a validation assert on the comparison of two input text files.
source code
 
assertGrep(self, file, filedir=None, expr='', contains=True, ignores=None, literal=False, **xargs)
Perform a validation assert on a regular expression occurring in a text file.
source code
 
assertLastGrep(self, file, filedir=None, expr='', contains=True, ignores=[], includes=[], **xargs)
Perform a validation assert on a regular expression occurring in the last line of a text file.
source code
 
assertOrderedGrep(self, file, filedir=None, exprList=[], contains=True, **xargs)
Perform a validation assert on a list of regular expressions occurring in specified order in a text file.
source code
 
assertLineCount(self, file, filedir=None, expr='', condition='>=1', ignores=None, **xargs)
Perform a validation assert on the number of lines in a text file matching a specific regular expression.
source code
 
__assertMsg(self, xargs, default)
Return an assert statement requested to override the default value.
source code
 
__abortOnError(self, xargs)
Return an assert statement requested to override the default value.
source code

Inherited from process.user.ProcessUser: __getattr__, abort, addCleanupFunction, addOutcome, allocateUniqueStdOutErr, getInstanceCount, getNextAvailableTCPPort, getOutcome, getOutcomeReason, logFileContents, signalProcess, startProcess, stopProcess, waitForFile, waitForSignal, waitForSocket, waitProcess, writeProcess

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Instance Variables [hide private]
string input
Location for input to any processes (defaults to current working directory)
logging.Logger log
Reference to the logger instance of this class
string mode
The user defined mode the test is running within.
string output
Location for output from any processes (defaults to current working directory)
Project project
Reference to the project details as set on the module load of the launching executable
string reference
Full path to the reference directory of the testcase.
Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, descriptor, outsubdir, runner)
(Constructor)

source code 

Create an instance of the BaseTest class.

Parameters:
  • descriptor - The descriptor for the test giving all test details
  • outsubdir - The output subdirectory the test output will be written to
  • runner - Reference to the runner responsable for executing the testcase
Overrides: object.__init__

setKeywordArgs(self, xargs)

source code 

Set the xargs as data attributes of the test class.

Values in the xargs dictionary are set as data attributes using the builtin setattr method. Thus an xargs dictionary of the form {'foo': 'bar'} will result in a data attribute of the form self.foo with value bar. This is used so that subclasses can define default values of data attributes, which can be overriden on instantiation e.g. using the -X options to the runTest.py launch executable.

Parameters:
  • xargs - A dictionary of the user defined extra arguments

execute(self)

source code 

Execute method which must be overridden to perform the test execution steps.

Raises:
  • NotImplementedError - Raised exeception should the method not be overridden

cleanup(self)

source code 

Cleanup method which performs cleanup actions after execution and validation of the test.

The cleanup method performs actions to stop all processes started in the background and not explicitly killed during the test execution. It also stops all process monitors running in seperate threads, and any instances of the manual tester user interface.

Should a custom cleanup for a subclass be required, use addCleanupFunction instead of overriding this method.

Overrides: process.user.ProcessUser.cleanup

addResource(self, resource)

source code 

Add a resource which is owned by the test and is therefore cleaned up (deleted) when the test is cleaned up.

Deprecated - please use addCleanupFunction instead of this function.

startProcessMonitor(self, process, interval, file, **kwargs)

source code 

Start a separate thread to log process statistics to logfile, and return a handle to the process monitor.

This method uses the pysys.process.monitor module to perform logging of the process statistics, starting the monitor as a seperate background thread. Should the request to log the statistics fail a BLOCKED outcome will be added to the test outcome list. All process monitors not explicitly stopped using the returned handle are automatically stopped on completion of the test via the cleanup method of the BaseTest.

Parameters:
  • process - The process handle returned from the startProcess method
  • interval - The interval in seconds between collecting and logging the process statistics
  • file - The full path to the filename used for logging the process statistics
  • kwargs - Keyword arguments to allow platform specific configurations
Returns: handle
A handle to the process monitor (pysys.process.monitor.ProcessMonitor)

stopProcessMonitor(self, monitor)

source code 

Stop a process monitor.

Parameters:

startManualTester(self, file, filedir=None, state=11, timeout=1800)

source code 

Start the manual tester.

The manual tester user interface (UI) is used to describe a series of manual steps to be performed to execute and validate a test. Only a single instance of the UI can be running at any given time, and can be run either in the FOREGROUND (method will not return until the UI is closed or the timeout occurs) or in the BACKGROUND (method will return straight away so automated actions may be performed concurrently). Should the UI be terminated due to expiry of the timeout, a TIMEDOUT outcome will be added to the outcome list. The UI can be stopped via the stopManualTester method. An instance of the UI not explicitly stopped within a test will automatically be stopped via the cleanup method of the BaseTest.

Parameters:
  • file - The name of the manual test xml input file (see pysys.xml.manual for details on the DTD)
  • filedir - The directory containing the manual test xml input file (defaults to the output subdirectory)
  • state - Start the manual tester either in the FOREGROUND or BACKGROUND (defaults to FOREGROUND)
  • timeout - The timeout period after which to termintate a manual tester running in the FOREGROUND

wait(self, interval)

source code 

Wait for a specified period of time.

Parameters:
  • interval - The time interval in seconds to wait

assertThat(self, conditionstring, *args)

source code 

Perform a validation based on a python eval string.

The eval string should be specified as a format string, with zero or more %s-style arguments. This provides an easy way to check conditions that also produces clear outcome messages.

e.g. self.assertThat('%d >= 5 or "%s"=="foobar"', myvalue, myothervalue)

Parameters:
  • conditionstring - A string will have any following args substituted into it and then be evaluated as a boolean python expression. If your args are strings that could contain double-quotes, put single quotes around the %s in the conditionstring, and vice-versa.
  • args - Zero or more arguments to be substituted into the format string

assertTrue(self, expr, **xargs)

source code 

Perform a validation assert on the supplied expression evaluating to true.

If the supplied expression evaluates to true a PASSED outcome is added to the outcome list. Should the expression evaluate to false, a FAILED outcome is added.

Parameters:
  • expr - The expression, as a boolean, to check for the True | False value
  • xargs - Variable argument list (see class description for supported parameters)

assertFalse(self, expr, **xargs)

source code 

Perform a validation assert on the supplied expression evaluating to false.

If the supplied expression evaluates to false a PASSED outcome is added to the outcome list. Should the expression evaluate to true, a FAILED outcome is added.

Parameters:
  • expr - The expression to check for the true | false value
  • xargs - Variable argument list (see class description for supported parameters)

assertDiff(self, file1, file2, filedir1=None, filedir2=None, ignores=[], sort=False, replace=[], includes=[], **xargs)

source code 

Perform a validation assert on the comparison of two input text files.

This method performs a file comparison on two input files. The files are pre-processed prior to the comparison to either ignore particular lines, sort their constituent lines, replace matches to regular expressions in a line with an alternate value, or to only include particular lines. Should the files after pre-processing be equivalent a PASSED outcome is added to the test outcome list, otherwise a FAILED outcome is added.

Parameters:
  • file1 - The basename of the first file used in the file comparison
  • file2 - The basename of the second file used in the file comparison (often a reference file)
  • filedir1 - The dirname of the first file (defaults to the testcase output subdirectory)
  • filedir2 - The dirname of the second file (defaults to the testcase reference directory)
  • ignores - A list of regular expressions used to denote lines in the files which should be ignored
  • sort - Boolean flag to indicate if the lines in the files should be sorted prior to the comparison
  • replace - List of tuples of the form ('regexpr', 'replacement'). For each regular expression in the list, any occurences in the files is replaced with the replacement value prior to the comparison being carried out. This is often useful to replace timestamps in logfiles etc.
  • includes - A list of regular expressions used to denote lines in the files which should be used in the comparison. Only lines which match an expression in the list are used for the comparison
  • xargs - Variable argument list (see class description for supported parameters)

assertGrep(self, file, filedir=None, expr='', contains=True, ignores=None, literal=False, **xargs)

source code 

Perform a validation assert on a regular expression occurring in a text file.

When the contains input argument is set to true, this method will add a PASSED outcome to the test outcome list if the supplied regular expression is seen in the file; otherwise a FAILED outcome is added. Should contains be set to false, a PASSED outcome will only be added should the regular expression not be seen in the file.

Parameters:
  • file - The basename of the file used in the grep
  • filedir - The dirname of the file (defaults to the testcase output subdirectory)
  • expr - The regular expression to check for in the file (or a string literal if literal=True). If the match fails, the matching regex will be reported as the test outcome
  • contains - Boolean flag to denote if the expression should or should not be seen in the file
  • ignores - Optional list of regular expressions that will be ignored when reading the file.
  • literal - By default expr is treated as a regex, but set this to True to pass in a string literal instead
  • xargs - Variable argument list (see class description for supported parameters)

assertLastGrep(self, file, filedir=None, expr='', contains=True, ignores=[], includes=[], **xargs)

source code 

Perform a validation assert on a regular expression occurring in the last line of a text file.

When the contains input argument is set to true, this method will add a PASSED outcome to the test outcome list if the supplied regular expression is seen in the file; otherwise a FAILED outcome is added. Should contains be set to false, a PASSED outcome will only be added should the regular expression not be seen in the file.

Parameters:
  • file - The basename of the file used in the grep
  • filedir - The dirname of the file (defaults to the testcase output subdirectory)
  • expr - The regular expression to check for in the last line of the file
  • contains - Boolean flag to denote if the expression should or should not be seen in the file
  • ignores - A list of regular expressions used to denote lines in the file which should be ignored
  • includes - A list of regular expressions used to denote lines in the file which should be used in the assertion.
  • xargs - Variable argument list (see class description for supported parameters)

assertOrderedGrep(self, file, filedir=None, exprList=[], contains=True, **xargs)

source code 

Perform a validation assert on a list of regular expressions occurring in specified order in a text file.

When the contains input argument is set to true, this method will append a PASSED outcome to the test outcome list if the supplied regular expressions in the exprList are seen in the file in the order they appear in the list; otherwise a FAILED outcome is added. Should contains be set to false, a PASSED outcome will only be added should the regular expressions not be seen in the file in the order they appear in the list.

Parameters:
  • file - The basename of the file used in the ordered grep
  • filedir - The dirname of the file (defaults to the testcase output subdirectory)
  • exprList - A list of regular expressions which should occur in the file in the order they appear in the list
  • contains - Boolean flag to denote if the expressions should or should not be seen in the file in the order specified
  • xargs - Variable argument list (see class description for supported parameters)

assertLineCount(self, file, filedir=None, expr='', condition='>=1', ignores=None, **xargs)

source code 

Perform a validation assert on the number of lines in a text file matching a specific regular expression.

This method will add a PASSED outcome to the outcome list if the number of lines in the input file matching the specified regular expression evaluate to true when evaluated against the supplied condition.

Parameters:
  • file - The basename of the file used in the line count
  • filedir - The dirname of the file (defaults to the testcase output subdirectory)
  • expr - The regular expression used to match a line of the input file
  • condition - The condition to be met for the number of lines matching the regular expression
  • ignores - A list of regular expressions that will cause lines to be excluded from the count
  • xargs - Variable argument list (see class description for supported parameters)

__assertMsg(self, xargs, default)

source code 

Return an assert statement requested to override the default value.

Parameters:
  • xargs - Variable argument list to an assert method
  • default - Default assert statement to return if a parameter is not supplied

__abortOnError(self, xargs)

source code 

Return an assert statement requested to override the default value.

Parameters:
  • xargs - Variable argument list to an assert method

Instance Variable Details [hide private]

input

Location for input to any processes (defaults to current working directory)
Type:
string

mode

The user defined mode the test is running within. Subclasses can use this in conditional checks to modify the test execution based upon the mode.
Type:
string

output

Location for output from any processes (defaults to current working directory)
Type:
string

reference

Full path to the reference directory of the testcase. This is used both by the class and its subclasses to locate the default directory containing all reference data to the testcase, as defined in the testcase descriptor.
Type:
string