pysys.perf.reporters¶
Contains the built-in reporter classes.
CSVPerformanceReporter¶
- class pysys.perf.reporters.CSVPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)[source]¶
Bases:
pysys.perf.api.BasePerformanceReporter
Performance reporter which writes to a CSV file.
This reporter writes to a UTF-8 file of comma-separated values that is both machine and human readable and easy to view and use in any spreadsheet program, and after the columns containing the information for each result, contains comma-separated metadata containing key=value information about the entire run (e.g. hostname, date/time, etc), and (optionally) associated with each individual test result (e.g. test mode etc). The per-run and per-result metadata is not arranged in columns since the structure differs from row to row.
After tests have run, the summary file is published with category
CSVPerformanceReport
using thepysys.writer.api.ArtifactPublisher
interface.The following properties can be set in the project configuration for this reporter:
- summaryFile = ''¶
The filename pattern used for the summary file(s); see
DEFAULT_SUMMARY_FILE
.For compatibility purposes, if not specified explicitly, the summary file for the CSVPerformanceReporter can be configured with the project property
csvPerformanceReporterSummaryFile
, however this is deprecated. This property can also be accessed and configured under the alternative capitalizationsummaryfile
, however this is discouraged as of PySys 2.1+, wheresummaryFile
is the preferred name.
- aggregateCycles = False¶
Enable this if you want PySys to rewrite the summary file at the end of a multi-cycle test with an aggregated file containing the mean and standard deviation for all the cycles, rather than a separate line for each cycle. This may be easier to consume when triaging performance results and looking for regressions.
New in version 2.1.
- publishArtifactCategory = 'CSVPerformanceReport'¶
If specified, the output file will be published as an artifact using the specified category name.
New in version 2.1.
- DEFAULT_SUMMARY_FILE = '__pysys_performance/${outDirName}_${hostname}/perf_${startDate}_${startTime}.${outDirName}.csv'¶
The default summary file if not overridden by the
csvPerformanceReporterSummaryFile
project property, or thesummaryFile=
attribute. SeegetRunSummaryFile()
. This is relative to the runner output+’/..’ directory (typically testRootDir, unless--outdir
is overridden).
JSONPerformanceReporter¶
- class pysys.perf.reporters.JSONPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)[source]¶
Bases:
pysys.perf.api.BasePerformanceReporter
Performance reporter which writes to a JSON file.
After tests have run, the summary file is published with category
JSONPerformanceReport
using thepysys.writer.api.ArtifactPublisher
interface.New in version 2.1.
The following properties can be set in the project configuration for this reporter:
- summaryFile = ''¶
The
.json
filename pattern used for the summary file(s); seeDEFAULT_SUMMARY_FILE
.
- publishArtifactCategory = 'JSONPerformanceReport'¶
If specified, the output file will be published as an artifact using the specified category name.
- DEFAULT_SUMMARY_FILE = '__pysys_performance/${outDirName}_${hostname}/perf_${startDate}_${startTime}.${outDirName}.json'¶
The default summary file if not overridden by the
summaryFile=
attribute. SeegetRunSummaryFile()
. This is relative to the runner output+’/..’ directory (typically testRootDir, unless--outdir
is overridden).
PrintSummaryPerformanceReporter¶
- class pysys.perf.reporters.PrintSummaryPerformanceReporter(project, summaryfile, testoutdir, runner, **kwargs)[source]¶
Bases:
pysys.perf.api.BasePerformanceReporter
Performance reporter which logs a human-friendly summary of all performance results to the console at the end of the test run.
By setting the
BASELINES_ENV_VAR
environment variable, this reporter will also print out an automatic comparison from the named baseline file(s) to the results from the current test run. This feature is very useful when comparingdifferent strategies for optimizing your application.New in version 2.1.
- BASELINES_ENV_VAR = 'PYSYS_PERFORMANCE_BASELINES'¶
Set this environment variable to a comma-separated list of performance (e.g.
.csv
) files to print out an automatic comparison from the baseline file(s) to the results from the current test run.This feature is very useful when comparing different strategies for optimizing your application.
For best results, use multiple cycles for all test runs so that standard deviation can be calculated.
The filenames can be absolute paths, glob paths using
*
and**
, or relative to the testRootDir. For example:export PYSYS_PERFORMANCE_BASELINES=__pysys_performance/mybaseline*/**/*.csv,__pysys_performance/optimization1*/**/*.csv