pysys.baserunner module¶
Contains the BaseRunner
class which orchestrates the running of all test cases.
Test selection is by default performed through the pysys.py launch script, which locates and
creates a set of class instances representing the tests to be executed. These are passed to the
base runner as a list of object references, so that the base runner can then iterate through the
list to perform the test execution. For more information see the pysys.baserunner.BaseRunner
API documentation.
-
class
pysys.baserunner.
BaseRunner
(record, purge, cycle, mode, threads, outsubdir, descriptors, xargs)[source]¶ Bases:
pysys.process.user.ProcessUser
A single instance of the runner class is responsible for orchestrating execution and outcome reporting of all testcases.
This class provides the default implementation, and can be subclassed if customizations are needed.
BaseRunner is the parent class for running a set of PySys system testcases. The runner is instantiated with a list of
pysys.xml.descriptor.TestDescriptor
objects detailing the set of testcases to be run. The runner iterates through the descriptor list and for each entry imports thepysys.basetest.BaseTest
subclass for the testcase, creates an instance of the test class and then calls the setup, execute, validate and cleanup methods of the test class instance. The runner is responsible for ensuring the output subdirectory of each testcase is purged prior to test execution to remove stale output from a previous run, detects any core files produced during execution of a testcase from processes started via thepysys.process
module, and performs audit trail logging of the test results on completion of running a set of testcases.The base runner contains the hook functions
setup
,testComplete
, andcleanup
to allow a subclass to perform custom operations prior to the execution of a set of testcases, between the execution of each testcase in a set, and on completion of all testcases respectively. Subclasses are typically used should some global conditions need to be setup prior to the set of testcasess being run (i.e. load data into a shared database, start an external process etc), and subsequently cleaned up after test execution.Variables: - mode (string) – Only used if supportMultipleModesPerRun=False; specifies the single mode tests will be run with.
- outsubdir (string) – The directory name for the output subdirectory. Typically a relative path, but can also be an absolute path.
- log (logging.Logger) – Reference to the logger instance of this class
- project (
Project
) – Reference to the project details as set on the module load of the launching executable
-
__init__
(record, purge, cycle, mode, threads, outsubdir, descriptors, xargs)[source]¶ Create an instance of the BaseRunner class.
Parameters: - record – Indicates if the test results should be recorded
- purge – Indicates if the output subdirectory should be purged on
PASSED
result - cycle – The number of times to execute the set of requested testcases
- mode – Only used if supportMultipleModesPerRun=False; specifies the single mode tests will be run with.
- threads – The number of worker threads to execute the requested testcases
- outsubdir – The name of the output subdirectory
- descriptors – List of
pysys.xml.descriptor.TestDescriptor
descriptors specifying the set of testcases to be run - xargs – The dictionary of additional “-X” user-defined arguments to be set as data attributes on the class
-
containerCallback
(thread, container)[source]¶ Callback method on completion of running a test.
Called on completion of running a testcase, either directly by the BaseRunner class (or a sub-class thereof), or from the ThreadPool.wait() when running with more than one worker thread. This method is always invoked from a single thread, even in multi-threaded mode.
The method is responsible for calling of the testComplete() method of the runner, recording of the test result to the result writers, and for deletion of the test container object.
Parameters: - thread – A reference to the calling thread (ignored in 1.3.0 onwards)
- container – A reference to the container object that ran the test
-
containerExceptionCallback
(thread, exc_info)[source]¶ Callback method for unhandled exceptions thrown when running a test.
Parameters: exc_info – The tuple of values as created from sys.exc_info()
-
cycleComplete
()[source]¶ Cycle complete method which can optionally be overridden to perform custom operations between the repeated execution of a set of testcases.
The default implementation of this method does nothing. Note that providing an override for this method will result in disabling concurrent test execution across multiple cycles.
Deprecated: Overriding this method is discouraged as it disables concurrent test execution across cycles. Instead, cleanup should be performed using either BaseTest.cleanup() or BaseRunner.testComplete() instead.
-
handleKbrdInt
(prompt=True)[source]¶ Handle a keyboard exception caught during running of a set of testcases.
-
isPurgableFile
(path)[source]¶ Determine if a file should be purged when empty at the end of a test run.
This method is called by testComplete to provide runners with the ability to veto deletion of non-empty files that should always be left in a test’s output directory even when the test has passed, by returning False from this method. For example this could be used to avoid deleting code coverage files. By default this will return True.
Parameters: path – The absolute path of the file to be purged
-
processCoverageData
()[source]¶ Called after execution of all tests has completed to allow processing of coverage data (if enabled), for example generating reports etc.
The default implementation collates Python coverage data from coverage.py and produces an HTML report. It assumes a project property pythonCoverageDir is set to the directory coverage files are collected into, and that PySys was run with -X pythonCoverage=true. If a property named pythonCoverageArgs exists then its value will be added to the arguments passed to the run and html report coverage commands.
Custom runner subclasses may replace or add to this by processing coverage data from other languages, e.g. Java.
-
setKeywordArgs
(xargs)[source]¶ Set the xargs as data attributes of the class.
Values in the xargs dictionary are set as data attributes using the builtin
setattr()
method. Thus an xargs dictionary of the form{'foo': 'bar'
} will result in a data attribute of the formself.foo
withvalue bar
.Parameters: xargs – A dictionary of the user defined extra arguments
-
setup
()[source]¶ Setup method which may optionally be overridden to perform custom setup operations prior to execution of a set of testcases.
-
start
(printSummary=True)[source]¶ Start the execution of a set of testcases.
The start method is the main method for executing the set of requested testcases. The set of testcases are executed a number of times determined by the
self.cycle
attribute. When executing a testcase all output from the execution is saved in the testcase output subdirectory; shouldself.cycle
be set to more than 1, the output subdirectory is further split into cycle[n] directories to sandbox the output from each iteration.Parameters: printSummary – Ignored, exists only for compatibility reasons. To provide a custom summary printing implementation, specify a BaseSummaryResultsWriter subclass in the <writers> section of your project XML file. Returns: Use of this value is deprecated as of 1.3.0. This method returns a dictionary of testcase outcomes, and for compatibility reasons this will continue in the short term, but will be removed in a future release. Please ignore the return value of start() and use a custom BaseSummaryResultsWriter if you need to customize summarization of results.
-
testComplete
(testObj, dir)[source]¶ Test complete method which performs completion actions after a testcase has finished executing and its final outcome has been determined.
The testComplete method performs purging of the output subdirectory of a testcase on completion of the test execution. Purging involves removing all files with a zero file length in order to only include files with content of interest. Should
self.purge
be set, the purging will remove all files (excluding the run.log) on aPASSED
outcome of the testcase in order to reduce the on-disk memory footprint when running a large number of tests. Should a custom testComplete for a subclass be required, the BaseRunner testComplete method should be called afterwards inside a try…finally block. Do not put logic which could change the test outcome in testComplete - useBaseTest.cleanup
instead.This method is always invoked from a single thread, even in multi-threaded mode.
Parameters: - testObj – Reference to the
pysys.basetest.BaseTest
instance of the test just completed - dir – The directory to perform the purge on
- testObj – Reference to the