pysys.process.monitor¶
Contains the process monitoring API used by pysys.basetest.BaseTest.startProcessMonitor
.
The main components are the pysys.process.monitor.BaseProcessMonitor
class, ProcessMonitorKey
constants for identifying
columns and the default ProcessMonitorTextFileHandler
class for writing monitoring information to a file.
ProcessMonitorKey¶
- class pysys.process.monitor.ProcessMonitorKey[source]¶
Bases:
object
Contains constants for supported process monitor keys.
Some of these keys are not currently returned on all platforms.
These constants provide the display names used for column headings etc.
Usually
CPU_CORE_UTILIZATION
is the best key for measuring CPU utilization andMEMORY_RESIDENT_KB
is the most useful way to monitor memory usage.- CPU_CORE_UTILIZATION = 'CPU core %'¶
CPU utilization % scaled by the number of cores so that 100% indicates full use of one core, 200% indicates full use of two cores, etc.
The maximum value is 100 multiplied by the number of CPU cores (as given by
multiprocessing.cpu_count()
).The type is
int
.
- CPU_TOTAL_UTILIZATION = 'CPU total %'¶
Total utilization % of all available CPU cores, with a maximum value of 100%.
If you have 2 cores and one of them is 50% utilized, the value would be 25%.
The type is
int
.
- MEMORY_RESIDENT_KB = 'Resident memory kB'¶
Resident / working set memory usage.
This is the non-swapped physical memory, and is usually a good statistic to use for checking for memory leaks and excessive memory usage.
On Unix it is equivalent to
rss
/RES
and on Windows to the working set memory usage.The type is
int
.
- MEMORY_VIRTUAL_KB = 'Virtual memory kB'¶
Virtual memory of the process including memory that is swapped out.
On Unix it is equivalent to
vsz
/VIRT
, and on Windows to the current page file usage.The type is
int
.
- DATE_TIME = 'Time'¶
String representation of the date and local time for this sample in yyyy-mm-dd HH:MM:SS format.
- DATE_TIME_LEGACY = 'Time (legacy)'¶
String representation of the date and local time for this sample in a format compatible with v1.3.0 and earlier versions of PySys.
This is %d/%m/%y %H:%M:%S on Windows and %m/%d/%y %H:%M:%S on Unix.
- Deprecated
Use
DATE_TIME
if possible.
- SAMPLE = 'Sample'¶
A counter starting from 1 and incrementing with each new set of sample data.
The type is
int
.
BaseProcessMonitorHandler¶
- class pysys.process.monitor.BaseProcessMonitorHandler[source]¶
Bases:
object
Interface to be implemented to provide a custom handler that records or processes data from a
BaseProcessMonitor
.- handleData(data, **kwargs)[source]¶
Called on a background thread each time a new sample of monitoring data is available.
- Parameters
data – a dictionary whose keys are from
ProcessMonitorKey
(or any new keys added by custom process monitor implementations). The dictionary values are of typeint
,float
orstring
.kwargs – Reserved for future use.
ProcessMonitorTextFileHandler¶
- class pysys.process.monitor.ProcessMonitorTextFileHandler(file, columns=None, delimiter=None, writeHeaderLine=None)[source]¶
Bases:
pysys.process.monitor.BaseProcessMonitorHandler
Handles process monitor data by writing the data as delimited values to a file, usually with tab separated values (.tsv).
A new line is written every time the process monitor polls for a new sample of data. By default a header line starting with
#
is included at the start of the file to indicate the column headings.If any value cannot be retrieved or is unavailable on this operating system, a -1 value will be written instead.
Uses default values set on the instance unless keyword overrides are provided; see
setDefaults
.- Parameters
file (str) – An absolute path string or open file handle to which process monitor data lines will be written.
columns (list[str]) – An ordered list of the columns from
ProcessMonitorKey
that should be included in the file. If not specified, the columns specified byDEFAULT_COLUMNS
will be used.delimiter (str) – The delimiter string used between each column. If not specified, the string specified by
DEFAULT_DELIMITER
will be used.writeHeaderLine (bool) – Determines whether a header line prefixed by
#
will be written at the start of the file. If not overridden, the default is taken fromDEFAULT_WRITE_HEADER_LINE
.
- DEFAULT_COLUMNS = ['Time', 'CPU core %', 'Resident memory kB', 'Virtual memory kB']¶
The default columns to write to the file.
If using a process monitor that comes with PySys, the values are from
ProcessMonitorKey
.Additional columns may be added to the end of this list in future releases.
See
setDefaults
if you wish to change the defaults.
- DEFAULT_WRITE_HEADER_LINE = True¶
Determines whether a header line prefixed by
#
will be written at the start of the file.See
setDefaults
.
- DEFAULT_DELIMITER = '\t'¶
The delimiter used between each column. Usually a tab character.
See
setDefaults
.
- static setDefaults(columns, delimiter=None, writeHeaderLine=None)[source]¶
Static helper method for setting the default columns or
writeHeaderLine
setting for all tests that use theProcessMonitorTextFileHandler
.This method could be called from a custom runner’s
pysys.baserunner.BaseRunner.setup
method in order to take effect for all tests.Do not call this from within an individual testcase since that could cause unwanted interference between different testcases.
- Parameters
columns – A list of the colums to be included, using values from
ProcessMonitorKey
. Since additional columns may be added to the end ofDEFAULT_COLUMNS
in future releases, when calling this method you should specify all the columns you want explicitly including the current defaults rather than writingDEFAULT_COLUMNS+[...]
.delimiter – The delimiter string used between each column.
writeHeaderLine – Specifies whether a header line beginning with
#
should be written at the start of the file.
BaseProcessMonitor¶
- class pysys.process.monitor.BaseProcessMonitor(owner, process, interval, handlers, **pmargs)[source]¶
Bases:
object
Process monitor for gathering statistics such as CPU and memory usage from a running process using a background thread.
For detail on the statistic keys available from the standard process monitor classes see
ProcessMonitorKey
.The most convenient way to start the default process monitor for this operating system is to use
pysys.basetest.BaseTest.startProcessMonitor
.You can create a custom subclass if you need to add support for a new OS or return additional monitoring statistics. For example, you could create a custom process monitor for Java processes that returned heap size and thread count, or you could create a simple process monitor wrapper around a Python library that provides a wider set of monitoring statistics, such as psutil. The fields and methods starting with a single underscore
_
can be used or overridden by custom subclasses. To start the process monitor from a custom class, simply ensure you have passed the owner basetest to its constructor and then call thestart
method.Monitors are automatically terminated during cleanup at the end of a test, or can be manually stopped before that using the
stop
method.- Parameters
owner – The BaseTest owning this monitor.
process – The process wrapper object. A numeric pid can be specified instead but with reduced functionality, so use a process object if you have one.
interval – The interval in seconds between polling for each data sample.
pmargs – Keyword arguments to allow parameterization of the returned data. An exception will be raised for any arguments not expected by this class.
- process¶
The process object. Can be None if a pid was passed directly.
- pid¶
The pid to be monitored.
- owner¶
The test object that owns this monitor.
- handlers¶
The list of handlers that will be notified each time the process is polled for new data.
- thread¶
The background thread that responsible for monitoring the process.
- start()[source]¶
Called on the main test thread to start monitoring in the background.
Performs any required initialization of data structures then starts the background thread.
- Returns
This instance.
- Return type
- _preprocessData(data)[source]¶
Called in the background thread with the data dictionary from each poll of the process, to allow OS-agnostic pre-processing and addition of derived data keys such as the date and time.
- Parameters
data – The dictionary of process monitoring data. This method may add or modify the contents of this dictionary.
- running()[source]¶
Return the running status of the process monitor.
- Returns
True if the process monitor background thread is still running.
- Return type
bool
- stop(timeout=None)[source]¶
Request the process monitor thread to terminate.
Waits for the termination to complete if possible, but does not throw an exception if it fails.
- Parameters
timeout (float) – Overrides the maximum time we will wait for the monitor to stop. By default this is
['WaitForProcessStop']
fromconstants.TIMEOUTS
.
- _getData(sample)[source]¶
Implement gathering of latest monitoring data.
Called on the background monitoring thread regularly.
- Parameters
sample – An integer starting at 1 and incrementing each time this method is called.
- Returns
A dictionary of (typically numeric) values, keyed by
ProcessMonitorKey
.- Return type
dict