pysys.writer package¶
Submodules¶
Module contents¶
Contains API and sample implementations of BaseResultsWriter
and
its subclasses, which are used to output test results during runtime execution.
Output writers are responsible for summarising test results on completion of a test, or on completion of a set of tests. There are currently three distinct types of writers, namely Record, Progress, and Summary, each of which performs output at different stages of a run:
- Record writers output the outcome of a specific test after completion of that test, to allow runtime auditing of the test output, e.g. into a relational database. Several record writers are distributed with the PySys framework, such as the
writer.JUnitXMLResultsWriter
. Best practice is to subclasswriter.BaseRecordResultsWriter
when writing new record writers. By default, record writers are enabled only when the –record flag is given to the PySys launcher, though some writers may enable/disable themselves under different conditions, by overriding thepysys.writer.BaseRecordResultsWriter.isEnabled
method.- Progress writers output a summary of the test progress after completion of each test, to give an indication of how far and how well the run is progressing. A single implementation of a progress writer is distributed with the PySys framework, namely the
writer.ConsoleProgressResultsWriter
, which details the percentage of tests selected to be run and that have executed, and a summary of the recent test failures. Progress writers should extend thewriter.BaseProgressResultsWriter
and are enabled when the –progress flag is given to the PySys launcher, or when PYSYS_PROGRESS=true is set in the local environment.- Summary writers output an overall summary of the status at the end of a test run. A single implementation of a progress writer is distributed with the PySys framework, namely the
writer.ConsoleSummaryResultsWriter
, which details the overall test run outcome and lists any tests that did not pass. A summary writer is always enabled regardless of the flags given to the pysys launcher.
Project configuration of the writers is through the PySys project XML file using the <writer> tag. Multiple
writers may be configured and their individual properties set through the nested <property> tag. Writer
properties are set as attributes to the class through the setattr() function. Custom (site specific) modules
can be created and configured by users of the PySys framework (e.g. to output test results into a relational
database etc), though they must adhere to the interface demonstrated by the implementations demonstrated here.
If no progress writers are explicitly configured in the PySys project XML file, an instance of
writer.ConsoleProgressResultsWriter
is used. If no summary writer is explicitly configured in the PySys project
XML file, an instance of writer.ConsoleSummaryResultsWriter
is used.
The writers are instantiated and invoked by the pysys.baserunner.BaseRunner
class instance. This calls the class
constructors of all configured test writers, and then the setup (prior to executing the set of tests), processResult
(process a test result), and cleanup (upon completion of the execution of all tests). The **kwargs method parameter
is used for variable argument passing in the interface methods to allow modification of the PySys framework without
breaking writer implementations already in existence.
-
class
pysys.writer.
BaseProgressResultsWriter
(logfile=None, **kwargs)[source]¶ Bases:
pysys.writer.BaseResultsWriter
Base class for writers that display progress information while tests are running.
Progress writers are only enabled if the –progress flag is specified.
-
isEnabled
(record=False, **kwargs)[source]¶ Determines whether this writer can be used in the current environment.
If set to False then after construction none of the other methods (including
setup
)) will be called.Parameters: record – True if the user ran PySys with the –record flag, indicating that test results should be recorded. Returns: For record writers, the default to enable only if record==True, but individual writers can use different criteria if desired, e.g. writers for logging output to a CI system may enable themselves based on environment variables indicating that system is present, even if record is not specified explicitly.
-
-
class
pysys.writer.
BaseRecordResultsWriter
(logfile=None, **kwargs)[source]¶ Bases:
pysys.writer.BaseResultsWriter
Base class for writers that record the results of tests, and are enabled only when the –record flag is specified.
For compatibility reasons writers that do not subclass BaseSummaryResultsWriter or BaseProgressResultsWriter are treated as “record” writers even if they do not inherit from this class.
-
class
pysys.writer.
BaseResultsWriter
(logfile=None, **kwargs)[source]¶ Bases:
object
Base class for objects that get notified as and when test results are available.
-
__init__
(logfile=None, **kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Called after all tests have finished executing (or been cancelled).
This is where file headers can be written, and open handles should be closed.
Parameters: kwargs – Additional keyword arguments may be added in a future release.
-
isEnabled
(record=False, **kwargs)[source]¶ Determines whether this writer can be used in the current environment.
If set to False then after construction none of the other methods (including
setup
)) will be called.Parameters: record – True if the user ran PySys with the –record flag, indicating that test results should be recorded. Returns: For record writers, the default to enable only if record==True, but individual writers can use different criteria if desired, e.g. writers for logging output to a CI system may enable themselves based on environment variables indicating that system is present, even if record is not specified explicitly.
-
processResult
(testObj, cycle=0, testTime=0, testStart=0, runLogOutput='', **kwargs)[source]¶ Called when each test has completed.
This method is always invoked from the same thread as setup() and cleanup(), even when multiple tests are running in parallel.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran. - cycle – The cycle number. These start from 0, so please add 1 to this value before using.
- testTime – Duration of the test in seconds as a floating point number.
- testStart – The time when the test started.
- runLogOutput – The logging output written to run.log, as a unicode character string.
- kwargs – Additional keyword arguments may be added in a future release.
- testObj – Reference to an instance of a
-
processTestStarting
(testObj, cycle=-1, **kwargs)[source]¶ Called when a test is just about to begin executing.
Note on thread-safety: unlike the other methods on this interface, this is usually executed on a worker thread, so any data structures accessed in this method and others on this class must be synchronized if performing non-atomic operations.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran. - cycle – The cycle number. These start from 0, so please add 1 to this value before using.
- kwargs – Additional keyword arguments may be added in a future release.
- testObj – Reference to an instance of a
-
setup
(numTests=0, cycles=1, xargs=None, threads=0, testoutdir='', runner=None, **kwargs)[source]¶ Called before any tests begin.
Before this method is called, for each property “PROP” specified for this writer in the project configuration file, the configured value will be assigned to self.PROP.
Parameters: - numTests – The total number of tests (cycles*testids) to be executed
- cycles – The number of cycles.
- xargs – The runner’s xargs
- threads – The number of threads used for running tests.
- testoutdir – The output directory used for this test run (equal to runner.outsubdir), an identifying string which often contains the platform, or when there are multiple test runs on the same machine may be used to distinguish between them. This is usually a relative path but may be an absolute path.
- runner – The runner instance that owns this writer.
- kwargs – Additional keyword arguments may be added in a future release.
-
-
class
pysys.writer.
BaseSummaryResultsWriter
(logfile=None, **kwargs)[source]¶ Bases:
pysys.writer.BaseResultsWriter
Base class for writers that display a summary of test results.
Summary writers are always enabled (regardless of whether –progress or –record are specified). If no “summary” writers are configured, a default ConsoleSummaryResultsWriter instance will be added automatically.
Summary writers are invoked after all other writers, ensuring that their output will be displayed after output from any other writer types.
-
isEnabled
(record=False, **kwargs)[source]¶ Determines whether this writer can be used in the current environment.
If set to False then after construction none of the other methods (including
setup
)) will be called.Parameters: record – True if the user ran PySys with the –record flag, indicating that test results should be recorded. Returns: For record writers, the default to enable only if record==True, but individual writers can use different criteria if desired, e.g. writers for logging output to a CI system may enable themselves based on environment variables indicating that system is present, even if record is not specified explicitly.
-
-
class
pysys.writer.
CSVResultsWriter
(logfile, **kwargs)[source]¶ Bases:
pysys.writer.BaseRecordResultsWriter
Class to log results to logfile in CSV format.
Writing of the test summary file defaults to the working directory. This can be be over-ridden in the PySys project file using the nested <property> tag on the <writer> tag. The CSV column output is in the form;
id, title, cycle, startTime, duration, outcome
Variables: outputDir (string) – Path to output directory to write the test summary files -
__init__
(logfile, **kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Implementation of the cleanup method.
Flushes and closes the file handle to the logfile.
Parameters: kwargs – Variable argument list
-
outputDir
= None¶
-
processResult
(testObj, **kwargs)[source]¶ Implementation of the processResult method.
Writes the test id and outcome to the logfile.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class - kwargs – Variable argument list
- testObj – Reference to an instance of a
-
-
class
pysys.writer.
ConsoleProgressResultsWriter
(**kwargs)[source]¶ Bases:
pysys.writer.BaseProgressResultsWriter
Default progress writer that logs a summary of progress so far to the console, after each test completes.
-
__init__
(**kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
processResult
(testObj, cycle=-1, **kwargs)[source]¶ Called when each test has completed.
This method is always invoked from the same thread as setup() and cleanup(), even when multiple tests are running in parallel.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran. - cycle – The cycle number. These start from 0, so please add 1 to this value before using.
- testTime – Duration of the test in seconds as a floating point number.
- testStart – The time when the test started.
- runLogOutput – The logging output written to run.log, as a unicode character string.
- kwargs – Additional keyword arguments may be added in a future release.
- testObj – Reference to an instance of a
-
processTestStarting
(testObj, cycle=-1, **kwargs)[source]¶ Called when a test is just about to begin executing.
Note on thread-safety: unlike the other methods on this interface, this is usually executed on a worker thread, so any data structures accessed in this method and others on this class must be synchronized if performing non-atomic operations.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran. - cycle – The cycle number. These start from 0, so please add 1 to this value before using.
- kwargs – Additional keyword arguments may be added in a future release.
- testObj – Reference to an instance of a
-
setup
(cycles=-1, numTests=-1, threads=-1, **kwargs)[source]¶ Called before any tests begin.
Before this method is called, for each property “PROP” specified for this writer in the project configuration file, the configured value will be assigned to self.PROP.
Parameters: - numTests – The total number of tests (cycles*testids) to be executed
- cycles – The number of cycles.
- xargs – The runner’s xargs
- threads – The number of threads used for running tests.
- testoutdir – The output directory used for this test run (equal to runner.outsubdir), an identifying string which often contains the platform, or when there are multiple test runs on the same machine may be used to distinguish between them. This is usually a relative path but may be an absolute path.
- runner – The runner instance that owns this writer.
- kwargs – Additional keyword arguments may be added in a future release.
-
-
class
pysys.writer.
ConsoleSummaryResultsWriter
(**kwargs)[source]¶ Bases:
pysys.writer.BaseSummaryResultsWriter
Default summary writer that is used to list a summary of the test results at the end of execution.
-
__init__
(**kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Called after all tests have finished executing (or been cancelled).
This is where file headers can be written, and open handles should be closed.
Parameters: kwargs – Additional keyword arguments may be added in a future release.
-
processResult
(testObj, cycle=-1, testTime=-1, testStart=-1, **kwargs)[source]¶ Called when each test has completed.
This method is always invoked from the same thread as setup() and cleanup(), even when multiple tests are running in parallel.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class. The writer can extract data from this object but should not store a reference to it. The testObj.descriptor.id indicates the test that ran. - cycle – The cycle number. These start from 0, so please add 1 to this value before using.
- testTime – Duration of the test in seconds as a floating point number.
- testStart – The time when the test started.
- runLogOutput – The logging output written to run.log, as a unicode character string.
- kwargs – Additional keyword arguments may be added in a future release.
- testObj – Reference to an instance of a
-
setup
(cycles=0, threads=0, **kwargs)[source]¶ Called before any tests begin.
Before this method is called, for each property “PROP” specified for this writer in the project configuration file, the configured value will be assigned to self.PROP.
Parameters: - numTests – The total number of tests (cycles*testids) to be executed
- cycles – The number of cycles.
- xargs – The runner’s xargs
- threads – The number of threads used for running tests.
- testoutdir – The output directory used for this test run (equal to runner.outsubdir), an identifying string which often contains the platform, or when there are multiple test runs on the same machine may be used to distinguish between them. This is usually a relative path but may be an absolute path.
- runner – The runner instance that owns this writer.
- kwargs – Additional keyword arguments may be added in a future release.
-
-
class
pysys.writer.
JUnitXMLResultsWriter
(**kwargs)[source]¶ Bases:
pysys.writer.BaseRecordResultsWriter
Class to log test results in Apache Ant JUnit XML format (one output file per test per cycle).
Variables: outputDir (string) – Path to output directory to write the test summary files -
__init__
(**kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Implementation of the cleanup method.
Parameters: kwargs – Variable argument list
-
outputDir
= None¶
-
processResult
(testObj, **kwargs)[source]¶ Implementation of the processResult method.
Creates a test summary file in the Apache Ant JUnit XML format.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class - kwargs – Variable argument list
- testObj – Reference to an instance of a
-
purgeDirectory
(dir, delTop=False)[source]¶ Deprecated: Use pysys.utils.fileutils.deletedir
instead.
-
-
class
pysys.writer.
TextResultsWriter
(logfile, **kwargs)[source]¶ Bases:
pysys.writer.BaseRecordResultsWriter
Class to log results to logfile in text format.
Writing of the test summary file defaults to the working directory. This can be be overridden in the PySys project file using the nested <property> tag on the <writer> tag.
Variables: outputDir (string) – Path to output directory to write the test summary files -
__init__
(logfile, **kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Implementation of the cleanup method.
Flushes and closes the file handle to the logfile.
Parameters: kwargs – Variable argument list
-
outputDir
= None¶
-
processResult
(testObj, **kwargs)[source]¶ Implementation of the processResult method.
Writes the test id and outcome to the logfile.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class - kwargs – Variable argument list
- testObj – Reference to an instance of a
-
-
class
pysys.writer.
XMLResultsWriter
(logfile, **kwargs)[source]¶ Bases:
pysys.writer.BaseRecordResultsWriter
Class to log results to logfile in XML format.
The class creates a DOM document to represent the test output results and writes the DOM to the logfile using toprettyxml(). The outputDir, stylesheet, useFileURL attributes of the class can be over-ridden in the PySys project file using the nested <property> tag on the <writer> tag.
Variables: - outputDir (string) – Path to output directory to write the test summary files
- stylesheet (string) – Path to the XSL stylesheet
- useFileURL (string (true | false)) – Indicates if full file URLs are to be used for local resource references
-
__init__
(logfile, **kwargs)[source]¶ Create an instance of the BaseResultsWriter class.
Parameters: - logfile – Optional configuration property specifying a file to store output in. Does not apply to all writers, can be ignored if not needed.
- kwargs – Additional keyword arguments may be added in a future release.
-
cleanup
(**kwargs)[source]¶ Implementation of the cleanup method.
Updates the test run status in the DOM, and re-writes to logfile.
Parameters: kwargs – Variable argument list
-
outputDir
= None¶
-
processResult
(testObj, **kwargs)[source]¶ Implementation of the processResult method.
Adds the results node to the DOM and re-writes to logfile.
Parameters: - testObj – Reference to an instance of a
pysys.basetest.BaseTest
class - kwargs – Variable argument list
- testObj – Reference to an instance of a
-
setup
(**kwargs)[source]¶ Implementation of the setup method.
Creates the DOM for the test output summary and writes to logfile.
Parameters: kwargs – Variable argument list
-
stylesheet
= None¶
-
useFileURL
= 'false'¶
-
class
pysys.writer.
flushfile
(fp)[source]¶ Bases:
object
Utility class to flush on each write operation - for internal use only.
-
fp
= None¶
-
-
pysys.writer.
replaceIllegalXMLCharacters
(unicodeString, replaceWith='?')[source]¶ Utility function that takes a unicode character string and replaces all characters that are not permitted to appear in an XML document.
The XML specification says Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]
Currently Python’s XML libraries do not perform checking or removal of such characters, so if this function is not used then they may generate XML documents that no XML API (including Python’s) can read.
See https://bugs.python.org/issue5166
Parameters: - unicodeString – a unicode character string (not a byte string). Since most XML documents are encoded in utf-8, typical usage would be to decode the UTF-8 bytes into characters before calling this function and then re-encode as UTF-8 again afterwards.
- replaceWith – the unicode character string to replace each illegal character with.