apama.docker.framework

Support for using Docker, Docker Compose, or Docker Swarm from PySys tests.

DockerImage(parent, name[, own])

Helper class for working with Docker images.

DockerContainer([parent, displayName])

Helper class for working with Docker containers.

DockerService([displayName, parent])

Helper class for working with Docker services.

DockerSecret(parent[, file, secretContents, …])

Helper class for working with Docker secrets.

DockerSwarm(parent, file[, imageSubst, …])

Helper class for working with Docker Swarm environments.

DockerComposition(parent, file[, …])

Helper class for working with Docker Compose environments.

DOCKER_COMMAND_RETURN

apama.docker.framework.DOCKER_COMMAND_RETURN

alias of apama.docker.framework.Docker

DockerHelper

class apama.docker.framework.DockerHelper(parent, displayName)[source]

Bases: object

Base class which provides common functionality needed by both Docker and Kubernetes.

This class is not intended to be used directly, but is invoked or subclassed by other classes in the Apama docker framework.

getAllContainers(stdout='__getcontainers.out', **kwargs)[source]

Returns all the service containers for a running environment

Returns

tuple containing (process, stdout, stderr)

getVolumes(name=None, **kwargs)[source]

Returns all the volumes for a running environment

Returns

A map from volume name to tuple of (driver type, full name)

getNetworks(name=None, **kwargs)[source]

Returns all the networks for a running environment

Returns

A map from network name to tuple of (driver type, full name, id)

_gcImp(name)[source]

Garbage-collects volumes and networks for a specific name

static _executeDockerCommand(parent, arguments, command=None, environs=None, state=11, timeout=600, stdout=None, stderr=None, displayName=None, stdouterrConfig=<apama.docker.framework.Config object>, abortOnError=None, ignoreExitStatus=None, displayNamePostfix=None, stdOutErr=None)[source]

Executes docker commands with default values

Parameters
  • parent – Reference to the parent PySys testcase

  • arguments – the arguments to docker command

  • command – the docker command to run. If not provided, defaults to parent.project.DOCKER_EXE. Added this field to execute docker-compose commands in a similar fashion

  • environs – A dictionary of the environment to run the process in. If “DOCKER_HOST” key is absent, default value is added

  • state – Run the process either in the FOREGROUND or BACKGROUND (defaults to FOREGROUND)

  • timeout – The timeout period after which to terminate processes running in the FOREGROUND.

  • stdout – User specified stdout. To be used with Config.MANUAL_CONFIG()

  • stderr – User specified stderr. To be used with Config.MANUAL_CONFIG()

  • stdOutErr – User specified stdouterr. To be used with Config.MANUAL_CONFIG()

  • displayName – Logical name of the process used for display. Defaults to ‘docker <arguments[0]>’

  • stdouterrConfig – Specifies the way to allocate stdout, stderr files

  • abortOnError – If true abort the test on any error outcome

  • ignoreExitStatus – If False, a BLOCKED outcome is added if the process terminates with non-zero exit code

  • displayNamePostfix – Appended to display name before calling startProcess. Used to make displayName more verbose without affecting stdout, stderr filename generation

Returns

tuple containing (process, stdout, stderr)

DockerImage

class apama.docker.framework.DockerImage(parent, name, own=True)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker images.

Sort-of manages the lifetime of the images it wraps; in most cases, it will keep them around for a long time so that the Docker cache can speed up other tests on the same machine, but it knows how to GC them at a later date due to deterministic naming that encodes the ownership of the image.

Create from a named image and parent testcase

static generateUniqueName()[source]

Return a unique name, used as docker image name. The Name starts with string pysys followed by process id, localtime and random number.

Returns

Docker Image name

classmethod fromBlank(parent)[source]

For when you are generating a docker image by some external means, but want the test framework to manage its lifetime. Contains an auto-generated name (use getName()) for you to tag your image with.

Parameters

parent – Reference to the parent PySys testcase

Returns

a DockerImage

classmethod fromExisting(parent, name, own=False, pull=False, **kwargs)[source]

Wrap an existing image

Parameters
  • parent – Reference to the parent PySys testcase

  • name – Name of the existing image

  • own – Whether or not this object owns the image i.e. deletes it later

  • pull – Whether to pull this tag before running

classmethod fromDockerfile(parent, file, context=None, otherfiles=None, imageSubst=None, abortOnError=True, noCache=False, buildArgs=None, timeout=1200)[source]

Construct an image from a Dockerfile

Parameters
  • parent – Reference to the parent PySys testcase

  • file – Absolute location of the Dockerfile

  • context – Directory containing the build context. If it’s a totally context-free Dockerfile, it’ll just run in a blank directory.

  • otherfiles – Other files to copy into the root of the context before build (e.g. a wrapper that we’re packaging). If the path is relative, it’s taken relative to the directory the Dockerfile is taken from. If the path is absolute, it’s treated as absolute.

  • imageSubst – Maps a nice friendly image name in the FROM of a Dockerfile to a DockerImage that should actually be used (so auto-testing doesn’t have namespacing problems). It’s a tuple (name, image).

  • abortOnError – Abort the test if the build fails

  • noCache – Whether or not to apply –no-cache to docker build

  • timeout – The timeout in seconds.

getName()[source]

The name/tag of this image

DockerService

class apama.docker.framework.DockerService(displayName=None, parent=None)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker services. Owns the service that it wraps, and will garbage collect it when this object is deleted (which will only happen when the parent testcase finishes).

Non-returning methods are Fluent - that is, they return ‘self’ to allow method-chaining

classmethod fromImage(image)[source]

Prepare to create and run a service from an image

Parameters

image – a DockerImage to create from

classmethod fromExisting(parent, name, own=True)[source]

Wrap an existing running service

Parameters
  • parent – Reference to the parent PySys testcase

  • name – Name of the existing image

  • own – Whether or not this object owns the service i.e. deletes it later

run(cmd=None, extraArgs=None, foreground=False, abortOnError=True, environs=None, **kwargs)[source]

Run the service. The default UID for processes inside the service will be the current uid to aid with host-volume sharing

Parameters
  • cmd – Optional command + arguments list to run in the service

  • extraArgs – list of extra arguments to pass to “docker run”

  • foreground – True if this method should only return after the command has finished executing

  • abortOnError – Abort the test if starting the service fails

  • environs – A dictionary of the environment to run the process in (defaults to clean environment)

log(logfile, **kwargs)[source]

Stream the docker logs (i.e. the stdout and stderr) for a service that has been run. Sends it live to two files, one for stdout, another for stderr.

Parameters

logfile – A named file prefix in the output directory. Suffixed with ‘.out’ and ‘.err’

DockerSecret

class apama.docker.framework.DockerSecret(parent, file=None, secretContents=None, name=None)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker secrets. Owns the secret that it wraps, and will garbage collect it when this object is deleted (which will only happen when the parent testcase finishes).

Non-returning methods are Fluent - that is, they return ‘self’ to allow method-chaining

classmethod fromFile(parent, name, file)[source]

Create a secret from a file :param parent: Reference to the parent PySys testcase :param file: The path to the file containing the secret contents, in whatever file format is required by the process that will use it :param name: the name of the secret used by Docker. This represents both the canonical name of the secret within a docker server, and the filename through which the secret can be accessed. Ensure the specified name does not conflict with other testcases, for example by using DockerImage.generateUniqueName(), e.g. name=”my-secret-%s.properties”%DockerImage.generateUniqueName().

classmethod fromContent(parent, name, secretContents)[source]

Create a named secret containing th specified string contents :param parent: Reference to the parent PySys testcase :param secretContents: The contents of the secret as a character string, in whatever file format is expected by the process that will use it :param name: the name of the secret used by Docker. This represents both the canonical name of the secret within a docker server, and the filename through which the secret can be accessed. Ensure the specified name does not conflict with other testcases, for example by using DockerImage.generateUniqueName(), e.g. name=”my-secret-%s.properties”%DockerImage.generateUniqueName().

getName()[source]

The name of this secret

DockerContainer

class apama.docker.framework.DockerContainer(parent=None, displayName=None)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker containers. Owns the container that it wraps, and will garbage collect it when this object is deleted (which will only happen when the parent testcase finishes).

Non-returning methods are Fluent - that is, they return ‘self’ to allow method-chaining

classmethod fromImage(image)[source]

Prepare to create and run a container from an image

Parameters

image – a DockerImage to create from

classmethod fromExisting(parent, name, own=True)[source]

Wrap an existing running container

Parameters
  • parent – Reference to the parent PySys testcase

  • name – Name of the existing image

  • own – Whether or not this object owns the container i.e. deletes it later

run(cmd=None, extraArgs=None, foreground=False, abortOnError=True, environs=None)[source]

Run the container. The default UID for processes inside the container will be the current uid to aid with host-volume sharing

Parameters
  • cmd – Optional command + arguments list to run in the container

  • extraArgs – list of extra arguments to pass to “docker run”

  • foreground – True if this method should only return after the command has finished executing

  • abortOnError – Abort the test if starting the container fails

  • environs – A dictionary of the environment to run the process in (defaults to clean environment)

log(logfile, **kwargs)[source]

Stream the docker logs (i.e. the stdout and stderr) for a container that has been run. Sends it live to two files, one for stdout, another for stderr.

Parameters

logfile – A named file prefix in the output directory. Suffixed with ‘.out’ and ‘.err’

volumeFromHost(hostpath, containerpath)[source]

Bind mount a directory from the host to the container i.e. -v Can’t do this to a container that has already been started with run()

Parameters
  • hostpath – Absolute path on the host

  • containerpath – Absolute path in the container

exposePort(containerPort)[source]

Expose a port on the container to a port on the host, only for a container that hasn’t been run yet

Parameters

containerPort – Port within the container

getExternalPort(port)[source]

Return the external port for a port inside the container

Parameters

port – The port inside the container

Returns

The port on the host operating system

commit(**kwargs)[source]

Do a ‘docker commit’ on this container

Returns

A DockerImage covering the newly created image

start(startCount=0, **kwargs)[source]

Do a ‘docker start’ on this container (starts a stopped container)

stop(**kwargs)[source]

Do a ‘docker stop’ on this container

restart(startCount=0, **kwargs)[source]

Do a ‘docker restart’ on this container

wait(timeout=60, **kwargs)[source]

Block until this container has exited

Returns

The container’s return code, or None if not known

cp(file, **kwargs)[source]

‘docker cp’ a file out of the container into your output directory

Parameters

file – Absolute path of the file in the container. The file in your output directory will share its basename.

DockerSwarm

class apama.docker.framework.DockerSwarm(parent, file, imageSubst=None, externalPorts=None, hostPathSubst=None, buildSubst=None, additionalFiles=None, otherSubsts=None)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker Swarm environments.

As with the other Docker classes, it attempts to clean up after itself

Create from a Docker Swarm file. We’re going to be testing sample files that have hardcoded aspects to them; exposing ports to fixed external ports; naming specific images from earlier steps; refering a specific path to build a Dockerfile in. However, this is inappropriate for auto-tests, so this method will tailor the file according to substitutions.

Parameters
  • parent – Reference to the parent PySys testcase

  • file – Absolute path to the docker-compose file

  • imageSubst – dictionary from fixed image names to DockerImages that should be used instead

  • externalPorts – a list of external port numbers mentioned in this compose that get randomly reallocated. Only the DockerContainer will know which to.

  • hostPathSubst – dictionary from the host path in a volume declaration, to the real host path we’d like it to be

  • buildSubst – dictionary from a build: path to a (directory of real context, [Dockerfile names within context])

  • additionalFiles – dictionary from a build path to [absolute paths of additional files to copy into the context] You might use this to copy Dockerfiles into the context if they’re not already there

  • otherSubsts – Any other kind of arbitrary textual substitution not covered by the above options. Mapping of string to replacement string.

deploy(abortOnError=True, environs=None)[source]

Bring up all services in the environment

Parameters
  • abortOnError – Abort the test if this fails

  • environs – A dictionary of the environment to run the process in (defaults to clean environment)

getContainers()[source]

Returns all the service containers for a running environment

Returns

A map from service name to a list of all DockerContainers running that service

rm(services=None)[source]

docker stack rm

Parameters

services – Names of services to rm. If none provided, removes them all.

DockerComposition

class apama.docker.framework.DockerComposition(parent, file, imageSubst=None, externalPorts=None, hostPathSubst=None, buildSubst=None, additionalFiles=None, otherSubsts=None)[source]

Bases: apama.docker.framework.DockerHelper

Helper class for working with Docker Compose environments.

As with the other Docker classes, it attempts to clean up after itself.

Create from a docker-compose file. We’re going to be testing sample files that have hardcoded aspects to them; exposing ports to fixed external ports; naming specific images from earlier steps; refering a specific path to build a Dockerfile in. However, this is inappropriate for auto-tests, so this method will tailor the file according to substitutions.

Parameters
  • parent – Reference to the parent PySys testcase

  • file – Absolute path to the docker-compose file

  • imageSubst – dictionary from fixed image names to DockerImages that should be used instead

  • externalPorts – a list of external port numbers mentioned in this compose that get randomly reallocated. Only the DockerContainer will know which to.

  • hostPathSubst – dictionary from the host path in a volume declaration, to the real host path we’d like it to be

  • buildSubst – dictionary from a build: path to a (directory of real context, [Dockerfile names within context])

  • additionalFiles – dictionary from a build path to [absolute paths of additional files to copy into the context] You might use this to copy Dockerfiles into the context if they’re not already there

  • otherSubsts – Any other kind of arbitrary textual substitution not covered by the above options. Mapping of string to replacement string.

up(abortOnError=True, environs=None)[source]

Bring up all services in the environment

Parameters
  • abortOnError – Abort the test if this fails

  • environs – A dictionary of the environment to run the process in (defaults to clean environment)

getContainers()[source]

Returns all the service containers for a running environment

Returns

A map from service name to a list of all DockerContainers running that service

stop(**kwargs)[source]

docker-compose stop

rm(services=None)[source]

docker-compose rm

Parameters

services – Names of services to rm. If none provided, removes them all.