1   Docker Autotest 0.8.8

1.1   Introduction

Docker Autotest is a sub-framework for standalone testing of docker. It does not depend on docker itself. Functionally, testing occurs within any number of subtest modules, which in some cases also include further nested sub-subtests. It is designed to support both extremely simple, linear, and more complex, nested, iterative testing of arbitrary external docker commands.

Test content and configuration is fully modular, supporting both included and external tests. The included content is primarily focused on continuous-integration testing of the Docker CLI client, with a very limited amount of negative and stress testing.

As with other Autotest Client Tests the main entry point is the test control file. Docker Autotest’s control file utilizes Autotest’s steps-engine. This allows the overall testing state to be stored, recovered, and resumed should a host-kernel panic or if userspace become unresponsive. Consequently, rebooting the host system as a testing step is also supported.

1.2   Prerequisites

  • Docker

    • Clean environment, only test-related images, containers and dependent services (besides docker -d) at the start of every Autotest run.
    • Docker installation (with docker daemon running)
    • Default settings for Docker on your platform/OS, including storage (LVM thin-pool + XFS).
    • Sufficient available temporary storage for multiple copies of the test-related images/container content.
  • Supported Docker OS platform

    • Red Hat Enterprise Linux 7 Server (preferred for development)
    • Red Hat Enterprise Linux Atomic Host (preferred for testing)
    • Fedora 22 or later including Atomic (not all tests may function)
    • Other platforms (such as CentOs, SLES, Ubuntu) un-maintained but possible.
  • Platform Applications/tools

    • Autotest 0.16.0 or later.
    • Coreutils or equivalent (i.e. cat, mkdir, tee, etc.)
    • Tar and supported compression programs (bzip2, gzip, etc.)
    • nfs-utils (nfs-server support daemons)
    • Python 2.6 or greater (not 3.0)
    • libselinux-python 2.2 or later
    • Optional (for building documentation), make, python-sphinx, and docutils or equivalent for your platform.
    • Optional (for running unittests), pylint, pep8, python-unittest2, and python2-mock.
    • Optional, python-bugzilla for test exclusion based upon bug status. See control.ini file comments for details.
  • Any specific requirements for particular subtest modules. In particular:

    • Direct root access for running docker, restarting the daemon, and accessing metadata / storage details. Running through sudo may work to a degree, but is likely to cause problems.
    • Access to a remote (to the host) image registry via docker pull, http, https, and git.
    • External testing dependencies must be usable by tests w/in fixed timeout periods. If an external resource or connection is too slow, it should be made more network-local to the test host.
    • Most tests with external dependencies will have them flagged as values to the special __example__ option. See example values for more details.

1.3   Quickstart

1.3.1   All platforms

  1. Double-check you meet all the requirements in docker_autotest_prereq. For the quickstart, either a RHEL 7 or Fedora 25 system is assumed with the Docker daemon started, device-mapper or overlay2 storage configured, and at least 10gig of registry space is available.

  2. As root, shallow-clone Autotest (non-recursive) into /var/lib/autotest, and set/export the AUTOTEST_PATH environment variable to it’s location.

    [root@docker ~]# cd /var/lib
    
    [root@docker lib]# git clone --single-branch --depth 1 \
    https://github.com/autotest/autotest.git autotest
    
    [root@docker lib]# export AUTOTEST_PATH=/var/lib/autotest
    
  3. Change to the autotest client subdirectory.

  4. Clone autotest-docker repository into the docker subdirectory. Based from a formal release or the latest available.

    [root@docker lib]# cd $AUTOTEST_PATH/client
    
    [root@docker client]# git clone --branch $VERSION \
    https://github.com/autotest/autotest-docker.git tests/docker
    

    Where $VERSION is the docker-autotest release (e.g. “0.8.8”) or to use the current latest release, omit the –branch option:

    [root@docker client]# git clone \
    https://github.com/autotest/autotest-docker.git docker
    
  5. Make a copy of default configuration, then edit as appropriate. Particularly, verify the CSV list of full-image names and container names, preserve_fqins and preserve_cnames are set correctly. All other images/containers will be destroyed! See default configuration options for more details.

    [root@docker client]# cp -abi tests/docker/config_defaults/defaults.ini \
    tests/docker/config_custom/
    
    [root@docker client]# $EDITOR tests/docker/config_custom/defaults.ini
    

1.3.2   Fedora platforms

The Fedora base-images lack some essential tooling necessary for testing. In addition to the steps above, a custom test-image must be configured for building.

  1. Edit the defaults.ini file again, and change the registry settings as follows:

    [root@docker client]# $EDITOR tests/docker/config_custom/defaults.ini
    
    ...
    docker_registry_host =
    docker_registry_user =
    docker_repo_name = fedora_test_image
    docker_repo_tag = latest
    ...
    
  2. Make a copy of the docker_test_images.ini configuration file and configure it to build the test image.

    [root@docker client]# cp -abi tests/docker/config_defaults/docker_test_images.ini \
    tests/docker/config_custom/
    
    [root@docker client]# $EDITOR tests/docker/config_custom/docker_test_images.ini
    
    ...
    build_name = fedora_test_image:latest
    build_dockerfile = https://github.com/autotest/autotest-docker/raw/master/fedora_test_image.tar.gz
    build_opts_csv = --no-cache,--pull,--force-rm
    ...
    

1.3.3   Execute and examine results

For all platforms, use the standalone autotest client to select and execute subtests. The default behavior is to run all subtests. However, the example below only executes the version subtest for demonstration purposes. This will bring in some additional utility “tests”, such as docker_test_images and garbage_check. Other subtests may be selected via the --args parameter or by customizing control.ini.

[root@docker /]# cd $AUTOTEST_PATH/client

[root@docker client]# ./autotest-local tests/docker/control --args=docker_cli/version
Writing results to /var/lib/autotest/client/results/default
START       ----    ----
Subtest/Sub-subtest requested:
        'docker_cli/version'

Subtest/sub-subtest exclude list:
        'subexample'
        'pretest_example'
        'example'
        'intratest_example'
        'posttest_example'

Executing tests:
        'docker/pretests/docker_test_images.1'
        'docker/pretests/log_sysconfig.2'
        'docker/pretests/log_versions.3'
        'docker/subtests/docker_cli/version.4'
        'docker/intratests/garbage_check.4'

    START   docker/pretests/docker_test_images.1
    docker_test_images: initialize()
    docker_test_images: setup() for subtest version 2055
    docker_test_images: Running sub-subtests...
        puller: initialize()
        puller: run_once()
        puller: Pulling registry.access.redhat.com/rhel7/rhel:latest
        puller: Pulling docker.io/stackbrew/centos:latest
        puller: postprocess()
        puller: cleanup()
        builder: initialize()
        builder: run_once()
        builder: postprocess()
        builder: cleanup()
    docker_test_images: postprocess_iteration() #0 of #1
    docker_test_images: full_name:registry.access.redhat.com/rhel7/rhel:latest
    docker_test_images: full_name:docker.io/stackbrew/centos:latest
    docker_test_images: Updated preserve_fqins: docker.io/stackbrew/centos:latest,registry.access.redhat.com/rhel7/rhel:latest
    docker_test_images: Postprocess sub-subtest results...
    docker_test_images: cleanup()
        GOOD        docker/pretests/docker_test_images.1
    END GOOD        docker/pretests/docker_test_images.1
    START   docker/pretests/log_sysconfig.2
    log_sysconfig: initialize()
    log_sysconfig: setup() for subtest version 0
    log_sysconfig: run_once()
    log_sysconfig: postprocess_iteration() #0 of #1
    log_sysconfig: postprocess()
    log_sysconfig: cleanup()
        GOOD        docker/pretests/log_sysconfig.2
    END GOOD        docker/pretests/log_sysconfig.2
    START   docker/pretests/log_versions.3
    log_versions: initialize()
    log_versions: setup() for subtest version 0
    log_versions: run_once()
    log_versions: Found docker version client: 1.12.6 server 1.12.6
    log_versions: postprocess_iteration() #0 of #1
    log_versions: postprocess()
    log_versions: cleanup()
        GOOD        docker/pretests/log_versions.3
    END GOOD        docker/pretests/log_versions.3
    START   docker/subtests/docker_cli/version.4
    version: initialize()
    version: setup() for subtest version 0
    version: run_once()
    version: postprocess_iteration() #0 of #1
    version: postprocess()
    version: docker version client: 1.12.6 server 1.12.6
    version: Docker cli version matches docker client API version
    version: cleanup()
        GOOD        docker/subtests/docker_cli/version.4
    END GOOD        docker/subtests/docker_cli/version.4
    START   docker/intratests/garbage_check.4
        GOOD        docker/intratests/garbage_check.4
    END GOOD        docker/intratests/garbage_check.4
END GOOD    ----    ----

(timestamps and extra inconsequential text removed for clarity)

Examine the test results by changing to the results/default directory. Note: The name “default” is used when no --tag option is given to the autotest-local command.

[root@docker client]# cd $AUTOTEST_PATH/client/results/default

[root@docker default]# ls -1
control          # Copy of the control file used for the run
control.ini      # Runtime configuration from default control file
control.state    # Used to support mid-test reboot / test resumption
debug            # All the client / sydout/stderr recorded by log-level.
docker           # Directory-tree of subtest results by name
job_report.html  # Autogenerated report web-page.
status           # Text-version of test run / results
status.json      # Same thing, but in JSON format.
sysinfo          # Directory of important log-files for the run.

[root@docker default]# ls -1 docker/subtests/docker_cli/version.4/
debug            # Same as above, but ONLY logs for this subtest
keyval           # Copy of subtest configuration, including defaults
profiling        # Not used
results          # Not used
status           # Same as above, but ONLY for this subtest
sysinfo          # Logs captured after this subtest ran.

If you wish jUnit format results, execute the included conversion script.

[root@docker client]# cd $AUTOTEST_PATH/client
[root@docker client]# tests/docker/results2junit --name $HOSTNAME results/default

[root@docker client]# cat results/default/results.junit
<testsuites>
    <testsuite name="localhost" failures="0" tests="5" skipped="0" errors="0">
        <testcase classname="localhost.pretests" name="docker_test_images" time="29"/>
        ...

1.4   Subtests

All subtest modules reside beneath the subtest directory. A subtest module must have the same name as the directory it is in (minus the .py extension). Other files/directories may exist at that level, but they will not be recognized as subtest modules by the Docker client test. This ensures each subtest’s code is kept separate from all others. Finally, every subtest is run in its own process and context. It does not have visibility into any other subtest code, configuration, or runtime.

1.4.1   Organization and Naming

The structure/layout of the subtest directory tree is relevant for reference and configuration. The reference and configuration section names for subtests are formed by its relative location under the subtest directory. For example, the subtest module subtests/docker_cli/version/version.py matches with the [docker_cli/version] configuration section and docker_cli/version subtest name.

1.4.2   Static Content Setup

Subtests may source their own static content from within their directory and below. When content needs to be built, or is in some way test-environment specific, the setup() method should be overridden. Content may be referenced from the subtest’s directory by using its bindir attribute. A version-specific directory to contain build/setup output is available as the srcdir attribute. The setup() method will *only* be called once per version number (including revisions). State may be reset by clearing the autotest client tmp directory.

Note:The setup() method runs after the initialize() method only once. If the configuration version has not changed, the method will not be called in subsequent Docker Autotest runs.

1.4.3   Sub-subtests

There are provisions for tests that contain, or are composed of multiple child tests or dependent operations. They may share content and code between each other, so long as it lives within the subtest directory or below. Optionally, they may use their own configuration sections, named by appending their class name onto the parent subtest’s name. Sub-subtest configuration inherits undefined values from the parent subtest configuration. Additionally, there are multiple methods of executing sub-subtests depending on needs.

1.5   Images

It is assumed that any images required for testing are available and built beforehand. A default image for testing purposes is required by most tests. Its fully-qualified name is configurable, though test results will be directly affected if it is or becomes unavailable. Individual subtests may require specific additional images or content. If so, this will be noted in the subtest documentation’s Prerequisites section.

1.6   Configuration

The default configuration files are all located under the config_defaults subdirectory. These are intended to be bundled with the autotest docker test. To customize any subtest or global default configuration, copies should be made manually into the config_custom subdirectory. Any content within config_custom will override all files and sections from config_defaults.

The subdirectory structure or relative file locations under config_custom is irrelevant. Multiple sections may appear in the same file, even for unrelated tests. The only exception is the config_custom/defaults.ini file and the test control.ini file.

When customizing subtest and sub-subtest configuration options, it is highly recommended that you add the option config_version with the current version number, into each section. This way, you will receive warnings when updating Docker Autotest, if the specific custom configuration doesn’t match the API. Generally this only happens between major/minor version updates.

1.6.1   Organization

1.6.1.1   Sections

Configuration files use the familiar ini style format with separate sections (e.g. [<section name>]) preventing option names from colliding. All configuration files are loaded into a single name-space, divided by each section. Section names can be arbitrary, however those which exactly match a subtest or sub-subtest’s name, will be automatically loaded (see Subtests).

1.6.1.2   Defaults

The Default, global values for all sections are located within the special defaults.ini file’s DEFAULTS section. These option names and values are supplied for all sections which do not contain a identical named option. This file is loaded either from the config_defaults or config_custom directory.

  • preserve_fqins : (registry.access.redhat.com/rhel7/rhel:latest) CSV of possibly existing full image names to preserve.
  • autotest_version : (@!NOVERSIONCHECK!@) Autotest version dependency for framework (or override for individual tests)
  • wait_ready : (60) Default timeout (seconds) for dockercmd.wait_for_ready()
  • verify_enforcing : (yes) Verify the system has SELinux set to enforcing mode.
  • docker_registry_host : (registry.access.redhat.com) remote components (host:port)
  • docker_repo_tag : (latest) Default image settings for testing (blank if not applicable)
  • docker_path : (/usr/bin/docker) Docker default options (before subcommand)
  • docker_repo_name : (rhel) Default registry settings for testing (blank if not applicable)
  • remove_after_test : (yes) Attempt to remove all created containers/images during test
  • docker_registry_user : (rhel7) remote components (username)
  • docker_options : (<None>) Global docker client command options to use (CSV)
  • config_version : (0.8.8) Undocumented Option, please fix!
  • docker_timeout : (300.0) Max runtime in seconds for any docker command (auto-converts to float)
  • __example__ : (docker_repo_name, docker_repo_tag, preserve_fqins) CSV list of options recommended for customization. Tests will issue warnings when left unmodified from default values. This option is compounded from defaults, not overwritten!
  • preserve_cnames : (<None>) CSV of possibly existing full container names to preserve.

1.6.2   Formatting

1.6.2.1   Long values

Long values may be continued onto the next line by prefixing any run of one or more horizontal-whitespace characters with a newline. For example:

option_name = This option value is super duper long,
but will be un-folded into a single string with <—- all this whitespace replaced by a single space.

In this case, the runs of multiple whitespace following the newline will be folded into a single space, and combined with the previous line.

1.6.2.2   Example Values

In order to help dockertest operate w/o customization, many example or demonstration value have been configured. While this is fine for development and informal testing purposes, it adds external dependencies for production testing. Therefore every option with default example values should be specified in a comma-separated list to the special __example__ option.

The __example__ option’s value is parsed specially. It is not inherited from defaults, or from subtest to sub-subtest. Instead, it is compounded at each level then pruned of modified options (as compared to their default/example value). Any unmodified options remaining at the beginning of a test will cause loud warnings to be issued.

1.6.2.3   Value substitution

Within each section, optional inline-value substitution is supported using the %(<option>)s token format. Where <option> is the literal name of another option. The referenced option must reside in the same section or in the special DEFAULTS section. This is useful when multiple option values need to be different but contain a shared element.

1.6.2.4   Type-conversion

The config parser will attempt to parse each item in the following order: integers, booleans, floats, then strings.

  • Integers are in the form of simple numbers, eg: “123”
  • Booleans are in the form ‘yes’ or ‘true’, ‘no’ or ‘false’ (case insensitive)
  • Floats are in the form of numbers with decimals eg: “123.456” or “123.0”
  • All other items will be returned as strings.

1.6.2.5   Documentation

All configuration options must be documented somewhere at least once. Documented options in [DEFAULTS] will automatically be documented in subtests, no need to re-document. Similarly, for subtests with multiple sub-subtests, options that are overridden by sub-subtests only need to be documented in the subtest section.

Configuration options are documented in ReStructuredText format by prefixing each option with one or more comment lines that begin with the special sequence #:. Regular line comments (beginning with just # are not treated specially).

Documentation comments always and only apply to the next option = value sequence encountered and are otherwise ignored. Further, all lines of documentation are concatenated together and stripped of newlines and multiple runs of space characters. This is required to fit every item into a bullet list. Since newlines and indenting is sometimes required (for example, a bullet list or table), special substitution sequences may be embedded w/in the comment text:

  • Use the [unbroken] sequence {n} wherever a newline should be inserted.
  • Use the [unbroken] sequence {t} wherever a four-space indent should be inserted.
  • Use the sequence {{ and }} to escape literal { and } characters.

1.7   Autotest Integration

1.7.1   Control files

Every test in autotest makes use of a control file as the single operational bridge between the Autotest server/client and the underlying testing details. This arrangement allows tests to ignore higher-level details, like intra-test reboots, reporting, iteration. Instead, tests can remain focused on exercising their subjects with a loose set of input variables an provided initial environment. More details about control files and their standards can be found in the Autotest documentation.

1.7.2   Jobs interface

This is the highest level of abstraction exposed to control files. The job object is implicitly passed into the control file, and used to produce test objects. Docker autotest makes use of several standard facilities provided by the job object:

  • The job.resultdir attribute points to the top-level absolute directory where all results and logs will be recorded. This directory contains a copy of the control file used, as well as logs and results for every test object executed.
  • The job.control attribute points to the absolute path of the control file currently in use. Though primarily for reference purposes, any use of this by lower-level tests can only be on an advisory basis. There is no way for tests to predict which control file will be in use nor what its precise behavior will be.
  • The job.args attribute contains a list of space-separated arguments passed to the control file. This can be via the --args option to the Autotest client, or it can come from elsewhere. Other than parsing the value into the list, it’s entirely up to the control file and/or tests to do what they need with this facility.
  • The job.next_step() method is used by the control file to set up and establish the harness-state for each distinct operation. While the state can be arbitrary, Docker Autotest uses this facility to insert test objects for eventual execution. Each step will ultimately be executed in a forked process, so there is no chance they may directly or accidentally influence each other’s python-environment.

1.7.3   Test interface

Autotest test objects are the primary context which contain testing-behavior and implementation details. In Docker autotest, each Subtest is derived from the autotest test.test class. With some help from the control file, this allows otherwise separate tests to be bundled together and executed in sequence, while sharing some common code. The important details/components of the test interface are available in the Subtest Module section.

1.7.4   Docker Autotest Control

The example control file provided with Docker autotest is designed to locate and setup subtests for execution in a particular way. However, this is only an example, you may customize or provide your own control file if alternate behavior is desired.

1.7.4.1   Selecting Sub-test and Sub-subtest modules

To help with the complex task of locating and queuing subtests for execution. The default control file examines two possibly complementary sets of options. One is via the --args autotest client command-line option. The other is via the control.ini file. This file is typically copied from config_defaults and modified in config_custom.

``–args`` Reference:

  • The value is treated as a sequence of space-separated then comma-separated list of values. Any subtests or sub-subtest names listed alone (space separated), or with commas and no intervening spaces, will be added to list of candidates.
  • If no candidate subtest or sub-subtest list are specified, the list will be generated by searching for applicable modules under subtests, pretests, intratests and posttests.
  • If a space-separated component of --args is of the form i=<name>,<name>,.... Then each <name> will be taken from the candidate list, as a test module to include in the run-queue. Unknown names are ignored with a warning.
  • Similarly, a space-separated component of the form x=<thing>,<thing>,... is taken as the list of test modules to exclude from the run-queue. Any conflicts with the include list, will result in the item being excluded.
  • If the string !!! appears as any of the space-separated items to --args, then no tests will be executed. Instead, the run-queue will simply be displayed and logged.

``control.ini`` Reference:

  • A config_custom/control.ini will be loaded in preference to config_defaults/control.ini.
  • The subthings option takes a CSV list of candidate module names, similar to the space-separated sequence from --args (above).
  • The pretests, intratests and posttests items specify the relative path to directories to search for candidates if they’re not specified in the subthings list (or in --args).
  • The include and exclude CSV lists operate just as expected. Either including or excluding items from the candidate list, into the run-queue.
  • All the other options are fully documented within the config_custom/control.ini file.
Note:The $AUTOTEST_PATH/client/results/default/control.ini represents the parsed, run-time copy of the file. It is consumed and used internally by Docker Autotest and should be considered read-only.

1.7.4.2   Filtering out known failures

Sometimes tests fail temporarily: docker regression, or new test that requires a particular not-yet-in-the-repos docker build, or disparity between RHEL/CentOS/Fedora. We want to acknowledge those and silence them for future runs, but without heavyhandedness such as hardcoded conditionals in the tests themselves.

The mechanism to do this is the file known_failures.txt, maintained within ADEPT and copied into the autotest config_custom subdirectory before a run. This file contains mappings of known failures in particular subtests when run on particular docker builds. If a docker-autotest subtest fails, but the NVR/subtest combination is listed in this file, the test is marked as passing. Test logs are unaffected, so a reader looking at .DEBUG logs will see the original failure.

The format of this file is:

<NVRA> <subtest path> <human-friendly description>

Where:

NVRA is a full N-V-R.A of a specific docker or docker-latest
build. As a special case, R.A can be ‘*’ (asterisk) to indicate that this failure is not likely to be fixed in any non-rebase build of this package.

subtest path is the full name of a subtest or subsubtest

description is a string intended to be read by humans. Bugzilla
IDs are especially helpful here. This string will be shown not only when a failure is bypassed but also if there’s a NON-bypassed failure

Fields are separated by one or more spaces; this allows the columns to be aligned for readability. The description field is optional but strongly recommended; spaces in it are (of course) not delimiters.

Examples:

docker-1.12.3-10.el7.centos docker_cli/iptable/iptable_remove bz1406460: fixed in 1.12.5

docker-1.12.6-49.1.hotfix.git90e29fe.el7.x86_64 docker_cli/run_volumes/oci_umount New oci-umount.conf contains spaces and comments; docker-autotest fix for this is in review.

docker-1.12.5-* docker_cli/negativeusage/iv4 bz1393572: expect fix in docker-1.13?

1.8   Versioning Requirements

In order to support external/private subtests and customized configurations, the Docker Autotest API version has been tightly coupled to test content, configuration, and documentation. Version comparison is only significant for the first two numbers (major and minor). The third (revision) number represents insignificant revisions which do not alter the core test or subtest API.

This allows the API to be extended freely, but any changes which could affect external tests or custom configurations will cause automatic test failures when encountered. The most likely cause for version problems is custom and/or outdated configurations. Double-check any customizations within config_custom match any changes to the current API. The same goes for any private or local tests.

Documentation versioning is similarly tied to changes in the API version. While non-fatal, it will introduce a delay in subtest execution. This signal is intended to alert developers a documentation-update pass is required to reflect changes in the API.

1.9   Subtest Modules

The following sections detail specific subtests, their configuration and any prerequisites or setup requirements.

1.9.1   docker_cli/attach Subtest

1.9.1.1   Summary

Test functioning of docker attach command

1.9.1.2   Operational Summary

  1. Run container w/ control over stdin & stdout
  2. Separately attach to the container in another process
  3. Send input to container via stdin, monitor attach process stdout
  4. Verify output matches input

1.9.1.3   Configuration

Sub-subtests:docker_cli/attach/sig_proxy_on, docker_cli/attach/sig_proxy_off, docker_cli/attach/no_stdin, docker_cli/attach/simple
  • run_options_csv : (--foreground,--rm,--attach=stdout) modifies the docker run options
  • check_run_cmd_out : (<None>) expected data on the run process output
  • interactive_cmd_attach : (append_data) data handed over to attach process stdin
  • check_attach_cmd_out : (append_data) expected data on the attach process output
  • docker_timeout : 60 Overrides default value: 300.0
  • attach_options_csv : (<None>) modifies the docker attach options
  • wait_interactive_cmd : (15) wait before checking the output
  • exit_status : (0) expected exit status
  • cmd : (<None>) modifies the container command
  • interactive_cmd_run : (<None>) data handed over to run process stdin
1.9.1.3.1   docker_cli/attach/sig_proxy_on Sub-subtest
  • signal : (10) Used signal to generate output
  • attach_options_csv : (--sig-proxy=true) modifies the docker attach options - inherited
  • run_options_csv : (--interactive) modifies the docker run options - inherited
  • cmd : ('echo READY; rm -f stop; trap "/usr/bin/date > stop" USR1; while ! [ -f stop ]; do :; done; echo DONE') modifies the container command - inherited
1.9.1.3.2   docker_cli/attach/sig_proxy_off Sub-subtest
  • run_options_csv : (--interactive) modifies the docker run options - inherited
  • attach_options_csv : (--sig-proxy=false) modifies the docker attach options - inherited
  • cmd : ('echo READY; rm -f stop; trap "/usr/bin/date > stop" USR1; while ! [ -f stop ]; do :; done; echo DONE') modifies the container command - inherited
  • signal : (10) Used signal to generate output
1.9.1.3.3   docker_cli/attach/no_stdin Sub-subtest
  • attach_options_csv : (--no-stdin=true) modifies the docker attach options - inherited
  • interactive_cmd_run : (run_data) data handed over to run process stdin - inherited
  • check_run_cmd_out : (run_data) expected data on the run process output - inherited
  • cmd : ('echo READY; cat') modifies the container command - inherited
  • run_options_csv : (--interactive) modifies the docker run options - inherited
1.9.1.3.4   docker_cli/attach/simple Sub-subtest
  • cmd : ('echo READY; cat') modifies the container command - inherited
  • attach_options_csv : (<None>) modifies the docker attach options - inherited
  • run_options_csv : (--interactive) modifies the docker run options - inherited
  • interactive_cmd_run : (run_data) data handed over to run process stdin - inherited
  • check_run_cmd_out : (run_data) expected data on the run process output - inherited

1.9.2   docker_cli/build Subtest

1.9.2.1   Summary

Tests the docker build command operation with various options and pre-defined build-content, both local and remote.

1.9.2.2   Operational Summary

  1. Gather source files if not done previously
  2. Gather all test and command options for each build
  3. Run all builds
  4. Check build exit code, output, images, and containers are expected.

1.9.2.3   Prerequisites

  • Behavior in Dockerfiles match expected behavior of sub-subtest & config.

1.9.2.4   Configuration

Sub-subtests:docker_cli/build/bad_quiet, docker_cli/build/onbuild, docker_cli/build/cache, docker_cli/build/bad_force_rm, docker_cli/build/jsonvol, docker_cli/build/git_path, docker_cli/build/local_path, docker_cli/build/bad, docker_cli/build/dockerignore, docker_cli/build/addurl, docker_cli/build/https_file, docker_cli/build/rm_false, docker_cli/build/expose
  • minus_eff : (<None>) Name of dockerfile to use with -f option, blank for no -f (use Dockerfile).

  • docker_build_options : (--rm=true,--no-cache=true,--quiet=false,--force-rm=false) docker build options as CSV list

  • enable_nonprintables_check : (no) Whether or not to examine docker-build output for terminal escape codes as part of the postive() or negative() post-processing commands. Defaults to disabled, Docker-1.13 ignores $TERM for build sub-command (BZ1403326)

  • dockerfile_dir_path : (<None>) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix).

  • source_dirs : (full,part,bad,dockerignore,expose,addurl,jsonvol,onbuild) Customize to match locally available resources Paths to docker build contexts used by tests (built during setup)

  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), cnt_count('0'), img_count('1'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match.
  • use_config_repo : (yes) Set to yes when dockerfile_dir_path is a writeable file, and test should substitute default test repo into FROM statement. Set to no otherwise, or when dockerfile_dir_path is a remote read-only resource.

1.9.2.4.1   docker_cli/build/bad_quiet Sub-subtest
  • dockerfile_dir_path : (/bad) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • postproc_cmd_csv : (negative(), !rx_out('Schazam!$'), !rx_out('foobar$'), !rx_err('Successfully built') rx_err('returned a non-zero code'), cnt_count('1'), img_count('1'), last_cnt(), !img_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • docker_build_options : (--rm=true,--no-cache=true,--quiet=true,--force-rm=false) docker build options as CSV list - inherited

1.9.2.4.2   docker_cli/build/onbuild Sub-subtest
  • postproc_cmd_csv2 : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), rx_out('Schazam!$'), rx_out('Second!$'), cnt_count('0'), img_count('3'), img_exst(), intr_exst()) Second build postproc_cmd_csv.

  • dockerfile_dir_path3 : (/onbuild/third) Name of dockerfile to use for third build.

  • dockerfile_dir_path2 : (/onbuild/second) Name of dockerfile to use for second build.

  • dockerfile_dir_path : (/onbuild/first) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), rx_out('First!$'), !rx_out('Schazam!$'), cnt_count('0'), img_count('3'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • postproc_cmd_csv3 : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), !rx_out('Schazam!$'), rx_out('Third!$'), cnt_count('0'), img_count('3'), img_exst(), intr_exst()) Second build postproc_cmd_csv.

1.9.2.4.3   docker_cli/build/cache Sub-subtest
  • docker_build_options2 : (--rm=true,--no-cache=false,--quiet=false,--force-rm=false) Second build Options CSV

  • dockerfile_dir_path : (/part) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • docker_build_options : (--rm=true,--no-cache=true,--quiet=false,--force-rm=false) docker build options as CSV list - inherited

  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), !rx_out('Using cache') rx_out('Schazam!$'), cnt_count('0'), img_count('2'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • dockerfile_dir_path2 : (/full) Second build uses this path to check caching of first build

  • postproc_cmd_csv2 : (positive(), rx_out('Using cache') rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), !rx_out('Schazam!$'), cnt_count('0'), img_count('2'), img_exst(), intr_exst()) Second build postproc_cmd_csv.

1.9.2.4.4   docker_cli/build/jsonvol Sub-subtest
  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), intr_exst(), dir_exist('/foo'), dir_exist('/foo/bar'), dir_exist('/usr/local/lib/baz')) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • dockerfile_dir_path : (/jsonvol) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

1.9.2.4.5   docker_cli/build/git_path Sub-subtest
  • docker_repo_name : centos Overrides default value: rhel

  • docker_registry_user : stackbrew Overrides default value: rhel7

  • docker_repo_tag : latest Overrides default value: latest

  • dockerfile_dir_path : (git://github.com/autotest/autotest-docker.git) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • docker_registry_host : docker.io Overrides default value: registry.access.redhat.com

  • postproc_cmd_csv : (positive(), rx_out('It works!$'), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), cnt_count('0'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • __example__ : dockerfile_dir_path, postproc_cmd_csv Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins

  • use_config_repo : (no) Set to yes when dockerfile_dir_path is a writeable file, and test should substitute default test repo into FROM statement. Set to no otherwise, or when dockerfile_dir_path is a remote read-only resource. - inherited

1.9.2.4.6   docker_cli/build/bad_force_rm Sub-subtest
  • docker_build_options : (--rm=true,--no-cache=true,--quiet=false,--force-rm=true) docker build options as CSV list - inherited

  • postproc_cmd_csv : (negative(), rx_out('Schazam!$'), !rx_out('foobar$'), !rx_err('Successfully built') rx_err('returned a non-zero code'), cnt_count('0'), img_count('1'), !img_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • dockerfile_dir_path : (/bad) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

1.9.2.4.7   docker_cli/build/expose Sub-subtest
  • dockerfile_dir_path : (/expose) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited
1.9.2.4.8   docker_cli/build/dockerignore Sub-subtest
  • dockerfile_dir_path : (/dockerignore) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), cnt_count('0'), img_count('1'), img_exst(), intr_exst(), !file_exist('/ignored_file'), !dir_exist('/ignored_dir'), !file_exist('/ignored_dir/ignored_file'), !dir_exist('/ignored_dir/ignored_subdir'), !file_exist('/ignored_dir/ignored_subdir/ignored_file'), !file_exist('/not_ignored_dir/ignored_file'), !file_exist('/.dockerignore'), rx_file('/not_ignored_file:%(_rx)s'), rx_file('/not_ignored_dir/not_ignored_file:%(_rx)s')) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • _rx : (05304147-3222-4b4b-ba27-2a8dca10ec81\w*) Substitution regex for content in all files expected to exist

1.9.2.4.9   docker_cli/build/addurl Sub-subtest
  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), img_exst(), rx_file('%(filepath)s:%(fileregex)s')) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • fileregex : (TERMS AND CONDITIONS FOR COPYING) Regular expression to find inside file

  • __example__ : fileurl,filepath,fileregex Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins

  • dockerfile_dir_path : (/addurl) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • filepath : (/LICENSE) Absolute path to filename in container

  • fileurl : (https://raw.githubusercontent.com/autotest/autotest-docker/master/LICENSE) URL to unambiguous filename to sub. in Dockerfile

1.9.2.4.10   docker_cli/build/local_path Sub-subtest
  • dockerfile_dir_path : (/full) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited
1.9.2.4.11   docker_cli/build/rm_false Sub-subtest
  • dockerfile_dir_path : (/full) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • postproc_cmd_csv : (positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), cnt_count('5'), img_count('1'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • docker_build_options : (--rm=false,--no-cache=true,--quiet=false,--force-rm=false) docker build options as CSV list - inherited

1.9.2.4.12   docker_cli/build/https_file Sub-subtest
  • docker_registry_host : registry.access.redhat.com Overrides default value: registry.access.redhat.com

  • docker_repo_tag : latest Overrides default value: latest

  • docker_repo_name : rhel Overrides default value: rhel

  • postproc_cmd_csv : (positive(), rx_out('Schazam!$'), rx_out('foobar$'), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), cnt_count('0'), img_exst(), intr_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited
  • __example__ : dockerfile_dir_path, postproc_cmd_csv Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins

  • use_config_repo : (no) Set to yes when dockerfile_dir_path is a writeable file, and test should substitute default test repo into FROM statement. Set to no otherwise, or when dockerfile_dir_path is a remote read-only resource. - inherited

  • dockerfile_dir_path : (https://raw.githubusercontent.com/autotest/autotest-docker/master/subtests/docker_cli/build/full/Dockerfile) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • docker_registry_user : rhel7 Overrides default value: rhel7

1.9.2.4.13   docker_cli/build/bad Sub-subtest
  • dockerfile_dir_path : (/bad) A / + a relative path from test setup directory, to subdirectory containing a Dockerfile (and it’s context). Alternatively, a http/ftp/git url accepted by the docker build command (with out any / prefix). - inherited

  • postproc_cmd_csv : (negative(), rx_out('Schazam!$'), !rx_out('foobar$'), !rx_err('Successfully built') rx_err('returned a non-zero code'), cnt_count('1'), img_count('1'), intr_exst('2'), last_cnt(), !img_exst()) Comma-separated post-process checks to call, in order, with optional string parameter enclosed in (', '). Check commands are:

    • positive() - Exit code must be zero, no Go panics or usage messages
    • negative() - Exit code must be non-zero, no Go panics or usage messages
    • rx_out('<regex>') - Must match stdout to <regex> on one or more lines
    • !rx_out('<regex>') - Must not match stdout to <regex> on any line
    • rx_err('<regex>') - Must match stderr to <regex> on one or more lines
    • !rx_err('<regex>') - Must not match stderr to <regex> on any line
    • img_count('<number>') - Must be <number> additional images after build
    • cnt_count('<number>') - Must be <number> additional containers after build
    • last_cnt() - Last created container must exist after build
    • img_exst() - Generated/Unique image name must exist
    • !img_exst() - Generated/Unique image name must not exist
    • intr_exst() - All intermediate created images must exist
    • !intr_exst() - No intermediate created images must exist
    • file_exist('<path>') - File named by <path> must exist
    • !file_exist('<path>') - File named by <path> must not exist
    • dir_exist('<path>') - dir named by <path> must exist
    • !dir_exist('<path>') - dir named by <path> must not exist
    • rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must match (no space either side of : please) * !rx_file('<path>:<regex>') - file named by <path> must exist and <regex> must not match. - inherited

1.9.3   docker_cli/commit Subtest

1.9.3.1   Summary

Several variations of running the commit command

1.9.3.2   Operational Summary

  1. Make new image name.
  2. Make changes in image by docker run.
  3. commit changes.
  4. check if committed image exists.
  5. check if values in changed files for image are correct.
  6. remove committed image from local repo.

1.9.3.3   Configuration

Sub-subtests:docker_cli/commit/check_default_cmd, docker_cli/commit/good
  • docker_read_file_cmd : (cat %%s) cmd to print context of given file
  • docker_commit_timeout : (120.0) docker commit deadline
  • docker_data_prepare_cmd : (/bin/bash -c "echo '%%s' | tee /var/i") Command used to prepare the container
1.9.3.3.1   docker_cli/commit/check_default_cmd Sub-subtest
  • commit_author : (Author_name) Author
  • commit_changed_files : (/var/i) Changed files
  • commit_message : (Message) Commit message
1.9.3.3.2   docker_cli/commit/good Sub-subtest
  • commit_author : (Author_name) Author
  • commit_changed_files : (/var/i) Changed files
  • docker_expected_exit_status : (0) expected result
  • commit_message : (Message) Commit message

1.9.4   docker_cli/cp Subtest

1.9.4.1   Summary

Simple tests that check the the docker cp command. The simple subtest verifies content creation and exact match after cp. The every_last subtest verifies copying many hundreds of files from a stopped container to the host. The volume_mount subtest verifies https://github.com/docker/docker/issues/27773

1.9.4.2   Operational Summary

  1. Look for an image or container
  2. Run the docker cp command on it
  3. Make sure the file was successfully copied

1.9.4.3   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket.
  • Docker image with fairly complex, deeply nested directory structure.

1.9.4.4   Configuration

Sub-subtests:docker_cli/cp/simple, docker_cli/cp/every_last
1.9.4.4.1   docker_cli/cp/every_last Sub-subtest
  • python_path : (python) name of python executable. Usually python; need python3 on F26+ docker image
  • max_files : (100) maximum number of files to try
  • exclude_paths : ("/dev", "/proc", "/sys", "/tmp", "/run/secrets") quoted CSV of directory/file prefixes to skip
  • __example__ : max_files Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • exclude_symlinks : (yes) copy symlinks

1.9.5   docker_cli/create Subtest

1.9.5.1   Summary

Test docker create by executing basic commands inside container and checking the results.

1.9.5.2   Operational Summary

  1. Create container with /bin/true command verify zero-exit status
  2. Create container with /bin/false command verify zero exit status
  3. Create container from image that doesn’t exist locally
  4. Create container, send signal to created container, expect failure.

1.9.5.3   Configuration

Sub-subtests:docker_cli/create/create_names, docker_cli/create/create_false, docker_cli/create/create_tmpfs, docker_cli/create/create_signal, docker_cli/create/create_true, docker_cli/create/create_remote_tag
  • bash_cmd : (/bin/bash,-c) csv command prefix (docker run … $bash_cmd $cmd)
  • run_options_csv : (--attach=stdout) Container run options to use with create command
  • docker_timeout : 10 Overrides default value: 300.0
  • cmd : (<None>) Specifies the executed command inside the container
  • exit_status : (0) expected exit status
1.9.5.3.1   docker_cli/create/create_names Sub-subtest
  • cmd : (sleep 2s) Specifies the executed command inside the container - inherited
  • names_count : (1000) Number of –name options to pass to create command
  • run_options_csv : (--net,none,--restart,always) Container run options to use with create command - inherited
  • last_name_sticks : (yes) Select whether or not the final –name value is used or the first
1.9.5.3.2   docker_cli/create/create_false Sub-subtest
  • cmd : (/bin/false) Specifies the executed command inside the container - inherited
1.9.5.3.3   docker_cli/create/create_tmpfs Sub-subtest
  • run_options_csv : (--tmpfs,%(tmpfs_arg)s) Container run options to use with create command - inherited
  • tmpfs_arg : (/run) –tmpfs parameter to create container
1.9.5.3.4   docker_cli/create/create_signal Sub-subtest
  • listen_signal : (SIGUSR1) Name or number of signal to send to created container
  • cmd : ('/bin/command/does/not/exist') Specifies the executed command inside the container - inherited
  • run_options_csv : (<None>) Container run options to use with create command - inherited
1.9.5.3.5   docker_cli/create/create_true Sub-subtest
  • cmd : (/bin/true) Specifies the executed command inside the container - inherited
1.9.5.3.6   docker_cli/create/create_remote_tag Sub-subtest
  • remote_image_fqin : (docker.io/stackbrew/centos:latest) Fully qualified image name of remote repository to test automatic pull.
  • run_options_csv : (<None>) Container run options to use with create command - inherited
  • __example__ : remote_image_fqin, docker_timeout Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • docker_timeout : 240 Overrides default value: 300.0

1.9.6   docker_cli/deferred_deletion Subtest

1.9.6.1   Summary

Test deferred deletion: device mapper should not delete a docker filesystem if it’s still in use.

1.9.6.2   Operational Summary

  1. Run a docker container
  2. Run a test script inside docker mount space that will:
    • Find out the local (host) mountpoint of the container’s root filesystem.
    • cd into that directory
    • Read the Deferred-Deletion count from ‘docker info’.
    • Remove the container.
    • Read the Deferred-Deletion count; confirm that it has grown by one.
    • cd back out of the container rootfs
    • Read the Deferred-Deletion count; confirm that it has gone back down.

1.9.6.3   Operational Detail

We need to do all the important work from inside a helper script, because our autotest process will only have access to the container’s mounts if MountFlags=slave is absent from the docker systemd unit file; by default that option is present, and it is not clear if/when it will be removed. To make sure we have access to the container’s mounts we need to nsenter the docker daemon’s mount namespace, and we can’t do that within autotest (using ctypes.CDLL and setns) because autotest is multithreaded and setns() doesn’t allow that.

1.9.6.4   Prerequisites

This test is only applicable if docker daemon is run with:

–storage-opts dm.use_deferred_deletion=true

(this is detected and enabled by the docker-storage-setup script)

That suffices on Fedora. On RHEL it’s more complicated: the 7.2 version of docker-storage-setup blindly enabled the option even though it didn’t actually work due to a kernel bug. 7.4 introduced a kernel with a fix, but disabled by default. We therefore need to differentiate between systems with (1) enabled incorrectly and correctly. We do so by checking that the sysctl knob /proc/sys/fs/may_detach_mounts (introduced in kernel 3.10.0-632) exists and is 1.

We abort with TestNAError if any precondition fails.

Note:Subtest does not have any default configuration

1.9.7   docker_cli/diff Subtest

1.9.7.1   Summary

Set of tests that modifies files within an image and then asserts the changes are picked up correctly by docker diff

1.9.7.2   Operational Summary

  1. Modify a file in a docker image
  2. Run docker diff on resultant container
  3. Check output against expected changes

1.9.7.3   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket. docker_cli/diff Configuration

1.9.7.4   Configuration

Sub-subtests:docker_cli/diff/diff_delete, docker_cli/diff/diff_change, docker_cli/diff/diff_add
  • files_changed : (<None>) csv list of expected change types and the files/directories that are changed. It is in the form of: <change type 1>,<path 1>,<change type 2>, <path 2> and so on.
  • docker_timeout : 120.0 Overrides default value: 300.0
  • command : (<None>) csv arg list to docker run that specifies how a test will modify a file for the test
1.9.7.4.1   docker_cli/diff/diff_delete Sub-subtest
  • files_changed : (D,/usr/bin/echo,C,/usr/bin) csv list of expected change types and the files/directories that are changed. It is in the form of: <change type 1>,<path 1>,<change type 2>, <path 2> and so on. - inherited
  • command : (/bin/rm,/usr/bin/echo) csv arg list to docker run that specifies how a test will modify a file for the test - inherited
1.9.7.4.2   docker_cli/diff/diff_change Sub-subtest
  • command : (/bin/chmod,777,/home) csv arg list to docker run that specifies how a test will modify a file for the test - inherited
  • files_changed : (C,/home) csv list of expected change types and the files/directories that are changed. It is in the form of: <change type 1>,<path 1>,<change type 2>, <path 2> and so on. - inherited
1.9.7.4.3   docker_cli/diff/diff_add Sub-subtest
  • command : (/bin/touch,/root/doesnotexist) csv arg list to docker run that specifies how a test will modify a file for the test - inherited
  • files_changed : (A,/root/doesnotexist,C,/root) csv list of expected change types and the files/directories that are changed. It is in the form of: <change type 1>,<path 1>,<change type 2>, <path 2> and so on. - inherited

1.9.8   docker_cli/dockerhelp Subtest

1.9.8.1   Summary

Test various odd-ball options/arguments produce usage info / helpful output

1.9.8.2   Operational Summary

  1. Run docker command with various combinations of options/args
  2. Check results are expected

1.9.8.3   Configuration

Sub-subtests:docker_cli/dockerhelp/help_simple
  • generate_subsubtest_list : (yes) Toggls the generation of a subsubtest per each help_command item. New subsubtest will be created for each item in help_commands. The optional subsection for this will be named [docker_cli/dockerhelp/help_<command>]. For example the attach command will have a subsection titled [docker_cli/dockerhelp/help_attach]. Each one of these sections follows the same config format as help_simple. For each section, if success_option_list is not set, it will default to help <command>,<command> --help instead of the docker_cli/dockerhelp section default.
  • help_commands : (attach build commit cp diff events export history images import info inspect kill load login logs port ps pull push restart rm rmi run save search start stop tag top version wait) Space separated list of all the docker commands to check
1.9.8.3.1   docker_cli/dockerhelp/help_simple Sub-subtest
  • success_option_list : (help,--help) CSV list of docker options where a zero-exit code is expected (though a
  • failure_option_list : (--help --quiet,foobar," ",' ') opposite of success_option_list

1.9.9   docker_cli/dockerimport Subtest

1.9.9.1   Summary

Exercize multiple variations of the docker import command. The empty sub-subtest verifies an empty tarball can be imported through a pipe. The truncated sub-subtest checks that a tarbal which abruptly ends, results in an error while importing through a pipe.

1.9.9.2   Operational Summary

  1. Create a tarball
  2. Pipe tarball into a docker command
  3. Check for expected result of docker command

1.9.9.3   Prerequisites

  • The tar, and cat commands exist in $PATH

1.9.9.4   Configuration

Sub-subtests:docker_cli/dockerimport/empty, docker_cli/dockerimport/truncated
  • tar_command : (/usr/bin/tar) Full path to tar command on host
  • image_name_postfix : (:test) value used to automatically generate a unique image name.
  • tar_options : (--numeric-owner --preserve-permissions --acls --selinux --xattrs --verbose --create .) Options to pass to tar while inside context directory

1.9.10   docker_cli/dockerinspect Subtest

1.9.10.1   Summary

Test output of docker inspect command

1.9.10.2   Operational Summary

  1. Create some docker containers
  2. Run docker inspect command on them
  3. Check output

1.9.10.3   Configuration

Sub-subtests:docker_cli/dockerinspect/inspect_all, docker_cli/dockerinspect/inspect_keys, docker_cli/dockerinspect/inspect_container_simple
1.9.10.3.1   docker_cli/dockerinspect/inspect_keys Sub-subtest
  • image_keys : (Id,DockerVersion,Created) which fields to check for in an image inspect
  • key_regex : (^[A-Z][a-zA-Z0-9_-]*$) asserts that each key matches this regex
  • container_keys : (Id,Image,Config,OpenStdin,NetworkSettings) specifies which fields to check for in a container inspect
1.9.10.3.2   docker_cli/dockerinspect/inspect_all Sub-subtest
  • ignore_fields : (<None>) which fields to ignore when checking all fields when running “docker inspect” on a container
1.9.10.3.3   docker_cli/dockerinspect/inspect_container_simple Sub-subtest
  • check_fields : (Id,Image,Config) specifies which fields to check the existence of when running docker inspect on a container

1.9.11   docker_cli/events Subtest

1.9.11.1   Summary

Start up a simple /bin/true container while monitoring output from docker events command. Verify expected events appear after container finishes and is removed.

1.9.11.2   Operational Summary

  1. Listen for events
  2. Run /bin/true container
  3. Check parsing of events and container events present

1.9.11.3   Prerequisites

  • Historical events exist prior to running test (i.e. docker daemon hasn’t been restarted in a while)
  • Host clock is accurate, local timezone setup properly.
  • Host clock does not change drastically during test

1.9.11.4   Configuration

  • rm_after_run : (True) use the docker rm command after the container finishes
  • docker_timeout : 120 Overrides default value: 300.0
  • unparseable_allowance : (2) specifies the number of lines with parse errors to allow
  • expect_events : (create,start,die,destroy) CSV of required events for test to pass
  • run_args : (--detach,--name=${NAME},${IMAGE},/bin/true) modifies the docker run options
  • wait_stop : (5) time in seconds to wait after removing the container, to check events

1.9.12   docker_cli/history Subtest

1.9.12.1   Summary

Test that checks the docker history command.

1.9.12.2   Operational Summary

  1. Prepare new image name.
  2. Make changes in image by docker run
  3. Make changes in image by docker run [dockerand_data_prepare_cmd].
  4. Get docker history using docker image history.
  5. check if image history is correct.

1.9.12.3   Configuration

Sub-subtests:docker_cli/history/simple
  • docker_data_prepare_cmd : (/bin/bash -c "echo '%%s' > /var/i") command used to generate changes in container
  • docker_read_file_cmd : (cat %%s) command to show content of a file
  • docker_history_timeout : (120.0) docker history execution timeout
1.9.12.3.1   docker_cli/history/simple Sub-subtest
  • docker_expected_exit_status : (0) expected results

1.9.13   docker_cli/images Subtest

1.9.13.1   Summary

Test to confirm output table-format of docker CLI ‘images’ command.

1.9.13.2   Operational Summary

  1. Attempt to parse ‘docker images’ table output
  2. Fail if table-format changes or is not parseable

1.9.13.3   Configuration

1.9.14   docker_cli/images_all Subtest

1.9.14.1   Summary

This test is focused on docker images –all specific behavior. It checks the difference between docker images and docker images --all.

1.9.14.2   Operational Summary

  1. Create image test_a
  2. Create image test_a1 with parent test_a
  3. Create image test_b
  4. Create image test_b1 with parent test_b
  5. Untag test_a
  6. Untag test_a1 (verify intermediary images were removed too)
  7. Untag test_b1 (verify test_b was preserved)
  • Between steps 4-7 verify docker images and docker history

1.9.14.3   Configuration

1.9.15   docker_cli/import_export Subtest

1.9.15.1   Summary

Test output of docker import/export command

1.9.15.2   Operational Summary

  1. Prepare container for export
  2. Export image to stdout
  3. Import image from stdin.
  4. Check image.

1.9.15.3   Configuration

Sub-subtests:docker_cli/import_export/simple
  • docker_data_prep_cmd : (/bin/bash -c "echo data > /var/i") command used to generate the image
  • docker_import_export_timeout : (120.0) maximal duration of docker import/export cmd
  • run_options_csv : (<None>) modifies the docker run options
1.9.15.3.1   docker_cli/import_export/simple Sub-subtest
  • export_cmd_args : (%%(container)s,|) docker export arguments (pipe)
  • import_cmd_args : (-,%%(image)s) docker import arguments

1.9.16   docker_cli/import_url Subtest

1.9.16.1   Summary

Tests on tarball contents imported from a URL. The “md5sum” test compares imported image md5sum against a known value.

1.9.16.2   Operational Summary

  1. Import image
  2. Verify if result is expected

1.9.16.3   Prerequisites

The configured URL points to a tarball in an accepted format by docker (plain, bzip, gzip, etc.).

1.9.16.4   Configuration

Sub-subtests:docker_cli/import_url/md5sum
  • __example__ : tar_url Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • tar_url : (https://github.com/autotest/autotest-docker/archive/0.7.6.tar.gz) specifies the URL of a tarball to test
1.9.16.4.1   docker_cli/import_url/md5sum Sub-subtest
  • md5sum : (87c171e2b8cfaebd32f7d871e9a89fd9) specifies the md5sum hash value for the file referenced by in_tar_file option
  • in_tar_file : (/autotest-docker-0.7.6/index.rst) specifies the full path to a test file contained within the tarball.
  • __example__ : in_tar_file Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins

1.9.17   docker_cli/info Subtest

1.9.17.1   Summary

Test output of docker info command and verifies it against values obtained from userspace tools. Currently only supports loop-back files for comparing on-disk size. Other setups just check pool name.

1.9.17.2   Operational Summary

  1. Run docker info command
  2. Check output
  3. Compare output with values obtained in userspace
  4. Disk-size error-margins are large, sanity-check only.

1.9.17.3   Prerequisites

dmsetup, stat and du commands are available on host.

1.9.17.4   Configuration

  • __example__ : data_error, data_total_error, meta_error, meta_total_error Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • meta_total_error : (0.25) Docker reported size may vary +/- by this percent from on-disk sizes
  • meta_error : (1.00) Docker reported size may vary +/- by this percent from on-disk sizes
  • data_total_error : (0.25) Docker reported size may vary +/- by this percent from on-disk sizes
  • data_error : (1.00) Docker reported size may vary +/- by this percent from on-disk sizes

1.9.18   docker_cli/iptable Subtest

1.9.18.1   Summary

This a set of test that check the container’s iptable rules on host.

1.9.18.2   Operational Summary

  1. Run container and parse/gather it’s iptable rules on host.
  2. Check if rules are affected as expected/are handled properly.

1.9.18.3   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket.
  • iptables service is not running, nor other services which change iptables (like libvirtd).
  • Firewalld daemon is running and does not show any errors about fail to add rules (https://bugzilla.redhat.com/show_bug.cgi?id=1101484).
  • Command iptable and brctl are working well.
  • sysctl’s net.bridge.bridge-nf-call-iptables and -ip6tables are set to 1

1.9.18.4   Configuration

  • bash_cmd : (/bin/bash -c 'sleep 3s') container’s shell command
  • run_args_csv : (--expose,1234,--publish,1234:1234) Arguments to pass to run command in addition to -d and –name

1.9.19   docker_cli/kill Subtest

1.9.19.1   Summary

Several variations of running the kill command.

  • random_* - series of random signals
  • run_sigproxy* - instead of docker kill uses kill directly on the docker run process
  • attach_sigproxy* - instead of docker kill uses kill on
    docker attach process

1.9.19.2   Operational Summary

  1. start container with test command
  2. execute docker kill no_iteration times
  3. analyze results

1.9.19.3   Configuration

Sub-subtests:docker_cli/kill/run_sigproxy_ttyoff, docker_cli/kill/random_name_ttyoff, docker_cli/kill/attach_sigproxy_ttyoff, docker_cli/kill/run_sigproxy_stress_ttyoff
  • stress_cmd_timeout : (5) maximal acceptable delay caused by stress command
  • attach_options_csv : (<None>) Additional options to pass when attaching to backgrounded container process
  • skip_signals : (9 13 17 18 19 20 27) which signals should not be used (uncatchable signals)
  • kill_signals : (1 31) range of used signals
  • kill_map_signals : (false) map signal numbers to names (RANDOM,true,false)
  • kill_sigproxy : (<None>) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid
  • no_iterations : (100) how many iterations
  • run_options_csv : (--attach=stdout) modifies the docker run options
  • docker_timeout : 60 Overrides default value: 300.0
  • exec_cmd : ('for NUM in `seq 1 64`; do trap "echo Received $NUM, ignoring..." $NUM; done; echo READY; while :; do sleep 0.1; done') modifies the container command
  • signals_sequence : (<None>) allows you to force given sequence of signals
  • run_container_attached : (false) execute detacched container and attach it in separate process
  • check_stdout : (Received %%s, ignoring...) checking output produced by signal
1.9.19.3.1   docker_cli/kill/run_sigproxy_ttyoff Sub-subtest
  • run_options_csv : (--attach=stdout,--sig-proxy=true) modifies the docker run options - inherited
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited
1.9.19.3.2   docker_cli/kill/random_name_ttyoff Sub-subtest
  • kill_map_signals : (true) map signal numbers to names (RANDOM,true,false) - inherited
1.9.19.3.3   docker_cli/kill/attach_sigproxy_ttyoff Sub-subtest
  • run_options_csv : (--detach=true,--sig-proxy=true) modifies the docker run options - inherited
  • run_container_attached : (true) execute detacched container and attach it in separate process - inherited
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited
  • attach_options_csv : (--sig-proxy=true) Additional options to pass when attaching to backgrounded container process - inherited
1.9.19.3.4   docker_cli/kill/run_sigproxy_stress_ttyoff Sub-subtest
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited
  • attach_options_csv : (--sig-proxy=true) Additional options to pass when attaching to backgrounded container process - inherited
  • run_options_csv : (--detach=true,--sig-proxy=true) modifies the docker run options - inherited
  • run_container_attached : (true) execute detacched container and attach it in separate process - inherited

1.9.20   docker_cli/kill_bad Subtest

1.9.20.1   Summary

Tests bad signals and good signals to nonexisting containers

1.9.20.2   Operational Summary

  1. start container with test command
  2. test all bad_signals using docker kill -s $signal $container and verify it wasn’t killed
  3. try to kill nonexisting container

1.9.20.3   Configuration

Sub-subtests:docker_cli/kill_bad/bad
  • exec_cmd : ('while :; do sleep 0.1; done') modifies the container command
  • wait_start : (1) how long to wait before using the container
  • run_options_csv : (-i) modifies the docker run options
1.9.20.3.1   docker_cli/kill_bad/bad Sub-subtest
  • bad_signals : (0,-1,-78,96,SIGBADSIGNAL,SIG,%%,!,\\,\,"''",'""', ,) csv of bad signals which will be used in test

1.9.21   docker_cli/kill_stopped Subtest

1.9.21.1   Summary

Special scenario - kill stopped container

1.9.21.2   Operational Summary

  1. start container with test command
  2. stop the container using kill -SIGSTOP
  3. send all (safe) signals except SIGCONT
  4. send SIGCONT
  5. check received signals

1.9.21.3   Configuration

Sub-subtests:docker_cli/kill_stopped/sigstop_proxy_ttyoff, docker_cli/kill_stopped/sigstop, docker_cli/kill_stopped/sigstop_ttyoff
  • docker_timeout : 60 Overrides default value: 300.0
  • kill_map_signals : (false) map signal numbers to names (RANDOM,true,false)
  • check_stdout : (Received %%s, ignoring...) checking output produced by signal
  • kill_sigproxy : (<None>) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid
  • run_options_csv : (--attach=stdout) modifies the docker run options
  • run_container_attached : (false) execute detacched container and attach it in separate process
  • skip_signals : (9 17 19 27 18 20) which signals should not be used (uncatchable signals)
  • no_iterations : (1) how many iterations
  • stress_cmd_timeout : (5) maximal acceptable delay caused by stress command
  • kill_signals : (1 31) range of used signals
  • signals_sequence : (<None>) allows you to force given sequence of signals
  • exec_cmd : ('for NUM in `seq 1 64`; do trap "echo Received $NUM, ignoring..." $NUM; done; echo READY; while :; do sleep 0.1; done') modifies the container command
1.9.21.3.1   docker_cli/kill_stopped/sigstop_proxy_ttyoff Sub-subtest
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited

1.9.22   docker_cli/kill_stress Subtest

1.9.22.1   Summary

Sends signals to container very quickly

1.9.22.2   Operational Summary

  1. start container with test command
  2. execute docker kill (or kill $PID) for each signal in signals_sequence one after another without delay (using bash for loop)
  3. analyze results

1.9.22.3   Configuration

Sub-subtests:docker_cli/kill_stress/run_sigproxy_stress_ttyoff, docker_cli/kill_stress/stress_ttyoff, docker_cli/kill_stress/attach_sigproxy_stress_ttyoff
  • kill_map_signals : (false) map signal numbers to names (RANDOM,true,false)
  • stress_cmd_timeout : (5) maximal acceptable delay caused by stress command
  • attach_options_csv : (<None>) Extra arguments to use when re-attaching to test container
  • docker_timeout : 60 Overrides default value: 300.0
  • no_iterations : (100) how many iterations
  • exec_cmd : ('for NUM in `seq 1 64`; do trap "echo Received $NUM, ignoring..." $NUM; done; echo READY; while :; do sleep 0.1; done') modifies the container command
  • kill_signals : (1 31) range of used signals
  • check_stdout : (Received %%s, ignoring...) checking output produced by signal
  • signals_sequence : (<None>) allows you to force given sequence of signals
  • kill_sigproxy : (<None>) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid
  • run_container_attached : (false) execute detacched container and attach it in separate process
  • run_options_csv : (--attach=stdout) modifies the docker run options
  • skip_signals : (9 13 17 18 19 20 27) which signals should not be used (uncatchable signals)
1.9.22.3.1   docker_cli/kill_stress/run_sigproxy_stress_ttyoff Sub-subtest
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited
  • run_container_attached : (true) execute detacched container and attach it in separate process - inherited
  • run_options_csv : (--detach=true,--sig-proxy=true) modifies the docker run options - inherited
  • attach_options_csv : (--sig-proxy=true) Extra arguments to use when re-attaching to test container - inherited
1.9.22.3.2   docker_cli/kill_stress/attach_sigproxy_stress_ttyoff Sub-subtest
  • run_container_attached : (true) execute detacched container and attach it in separate process - inherited
  • run_options_csv : (--detach=true,--sig-proxy=true) modifies the docker run options - inherited
  • kill_sigproxy : (true) used kill command (false -> docker kill $name; true` -> os.kill $docker_cmd.pid - inherited
  • attach_options_csv : (--sig-proxy=true) Extra arguments to use when re-attaching to test container - inherited

1.9.23   docker_cli/liverestore Subtest

1.9.23.1   Summary

Test live-restore: running container should survive docker daemon restart.

1.9.23.2   Operational Summary

  1. Run a docker container
  2. Restart docker daemon
  3. Verify that container is listed under ‘docker ps’ and is still running.

1.9.23.3   Operational Detail

Our test container outputs a string (it doesn’t matter what) every second. We check that it’s running by running docker logs repeatedly until we get a new line; timing out at five seconds.

We restart the docker daemon, not separate stop and then start. (The latter would give us more testing options, such as confirming continued output from the container, but there’s also a lot more possibility of leaving our system in an undefined state if something goes wrong between stop and start.) We then check that docker still lists the container in ‘docker ps’ and that the container is still emitting output lines.

1.9.23.4   Prerequisites

This test is only applicable if docker daemon is run with –live-restore option; we abort with TestNAError if it isn’t.

live-restore also requires KillMode=process in the systemd unit file of the controlling process - typically dockerd but possibly docker-containerd. Without that key, the container is stopped on daemon restart; see rhbz1424709#c40 and rhbz1460266.

1.9.24   docker_cli/load Subtest

1.9.24.1   Summary

Verify loading image from file works from static content.

1.9.24.2   Operational Summary

  1. Verify static content doesn’t already exist in repository
  2. Use docker load on existing tarball
  3. Verify expected ID exists.
  4. Remove loaded image

1.9.24.3   Configuration

  • test_filename : (load_test.tar.gz) Filename of tarball located in source directory, or absolute path to filename to load (first character must be ‘/’).
  • test_id_old : (c2a412b0de222fbdb0ebb3b225181320b618c64c806d28f259cb9ce935dbb7c3) For compatibility with docker < 1.10. FIXME: remove once 1.10 is universal.
  • test_fqin : (load:test) The fully-qualified name of image stored in test_filename
  • test_id : (sha256:5a989dd2c48f406db792ebb7fca50160a67398db0af6f15c806f609b77021e9f) The full ID of the image stored in test_filename

1.9.25   docker_cli/login Subtest

1.9.25.1   Summary

Test variations of ‘docker login’

1.9.25.2   Operational Summary

  1. Create a self-signed SSL certificate
  2. Generate a pseudorandom password
  3. Generate htpasswd file with a test user and this password
  4. Run a registry container using new certificate & credentials
  5. Run individual subtests against this standard setup

1.9.25.3   Configuration

  • push_image : (busybox) docker image to use for the push test. Make it something small, so test can pass/fail quickly and without consuming too many system resources. As of this writing (2016-05-26) busybox is just over 1MB.
  • registry_fqin : (registry-1.docker.io/distribution/registry:2.1) docker image containing a functional registry, including auth checks. Make sure to include this in config_custom/defaults.ini:preserve_fqins !
  • docker_auth_file : (/root/.docker/config.json) config file into which docker login writes credentials. This is not likely to change.
  • wait_for_registry_ready : (10) maximum time, in seconds, to wait for registry container to become ready.

1.9.26   docker_cli/logs Subtest

1.9.26.1   Summary

Monitor various container states and stdio in parallel with the logs command.

1.9.26.2   Operational Summary

  1. Create a container, run a docker logs command
  2. Capture new logs output
  3. Start the container, capture new logs output
  4. Stop container, capture new logs output
  5. Verify produced output with expected output

1.9.26.3   Prerequisites

  • Host clock is accurate, local timezone setup properly.
  • Host clock does not change drastically during test (e.g. from ntpdate)
  • Starting and executing the test container takes under one minute

1.9.26.4   Configuration

  • extra_start_args : (--attach=true,--interactive=true) CSV additional start command arguments besides the container name
  • extra_create_args : (--interactive=false,--tty=false) CSV additional create command arguments besides --name <name>, the FQIN, and subtest specified command+args.

1.9.27   docker_cli/namespaces Subtest

1.9.27.1   Summary

Tests relating to namespaces. As of 2016-08-23 only mount namespaces but it’s likely we’ll need tests for user namespaces.

1.9.27.2   Operational Summary

  1. See individual tests.

1.9.27.3   Configuration

1.9.28   docker_cli/negativeusage Subtest

1.9.28.1   Summary

Negative testing for many different malformed docker CLI sub-commands.

1.9.28.2   Operational Summary

  1. Form a docker sub-command based on configured values, though missing some key, required part (parameter value, option value, out-of-order required options, etc.).
  2. Execute malformed command, verify exit status, stdout, and stderr all match expected values.

1.9.28.3   Operational Detail

Sub-subtests are organized with names according to the following legend, with a number appended:

op:
Omit Positional - leave out a required positional parameter
if:
Invalid Flag - Addition of non-existing flag
ov:
Omit Value - Leave out required value to argument (make it like a flag)
iv:
Invalid Value - Give improper/incorrect value to argument
ip:
Invalid Positional - Give value to non-existing positional parameter

1.9.28.4   Configuration

Sub-subtests:docker_cli/negativeusage/op2, docker_cli/negativeusage/op3, docker_cli/negativeusage/iv4, docker_cli/negativeusage/op1, docker_cli/negativeusage/iv3, docker_cli/negativeusage/iv1, docker_cli/negativeusage/ip1, docker_cli/negativeusage/ip2, docker_cli/negativeusage/if1, docker_cli/negativeusage/ov1
  • subcmd : (<None>) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist
  • stdout : (<None>) Optional regex that must match to stdout (group #1), also accepts same substitutions as subcmd.

  • stderr : (<None>) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd.

  • extcmd : (125) Required exit status integer

  • subarg : (<None>) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd.

1.9.28.4.1   docker_cli/negativeusage/op2 Sub-subtest
  • stderr : (no such file or directory) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • extcmd : (127) Required exit status integer - inherited

  • subarg : (--interactive,%%(FQIN)s,/usr/local/sbin/fail) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • subcmd : (run) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
1.9.28.4.2   docker_cli/negativeusage/op3 Sub-subtest
  • subcmd : (run) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
  • stderr : (Unable to find image|is not a valid repository/tag) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subarg : (--interactive,/usr/local/sbin/fail) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

1.9.28.4.3   docker_cli/negativeusage/iv4 Sub-subtest
  • extcmd : (127) Required exit status integer - inherited

  • subarg : (-e,PATH=/tmp,%%(FQIN)s,true) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • stderr : (executable file not found) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subcmd : (run) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
1.9.28.4.4   docker_cli/negativeusage/op1 Sub-subtest
  • extcmd : (1) Required exit status integer - inherited

  • subcmd : (attach) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
  • stderr : ("(docker )?attach" requires( exactly)? 1 argument) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subarg : (--no-stdin,--sig-proxy) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

1.9.28.4.5   docker_cli/negativeusage/iv3 Sub-subtest
  • subarg : (-p,192.168.9.1:9000,%%(FQIN)s,/bin/true) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • subcmd : (run) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
  • stderr : (Invalid hostPort) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

1.9.28.4.6   docker_cli/negativeusage/iv1 Sub-subtest
  • subarg : (--no-stdin=sig-proxy) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • stderr : (invalid boolean value "sig-proxy" for  --no-stdin|invalid argument "sig-proxy" for --no-stdin=sig-proxy) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subcmd : (attach) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
1.9.28.4.7   docker_cli/negativeusage/ip1 Sub-subtest
  • extcmd : (1) Required exit status integer - inherited

  • subcmd : (tag) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
  • subarg : (%%(RUNCNTR)s,%%(NOFQIN)s) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • stderr : (no such id:) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

1.9.28.4.8   docker_cli/negativeusage/ip2 Sub-subtest
  • docker_options : <None> Overrides default value: <None>
  • extcmd : (1) Required exit status integer - inherited
  • subarg : (daemon -g fdsafsd -b dsca -s fdsafs -e feae -- --name=flag_cGpa) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited
1.9.28.4.9   docker_cli/negativeusage/if1 Sub-subtest
  • stderr : (flag provided but not defined|unknown flag: --authormessage) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subarg : (--authormessage,%%(STPCNTR)s,%%(NOFQIN)s) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • subcmd : (commit) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited
1.9.28.4.10   docker_cli/negativeusage/ov1 Sub-subtest
  • subarg : (--input) Optional docker subcommand arguments (CSV), also accepts same substitutions as subcmd. - inherited

  • stderr : (flag needs an argument) Optional regex that must match to stderr (group #1), also accepts same substitutions as subcmd. - inherited

  • subcmd : (load) Optional docker subcommand (string). Automatically generated substitution keys:

    • %%(FQIN)s - valid, default test image name
    • %%(NOFQIN)s - valid image name that doesn’t exist
    • %%(RUNCNTR)s - ID of a running container
    • %%(STPCNTR)s - ID of a stopped container
    • %%(NAME)s - Name of a container that doesn’t exist - inherited

1.9.29   docker_cli/ps_size Subtest

1.9.29.1   Summary

This test checks function of docker ps –size shows correct sizes

1.9.29.2   Operational Summary

  1. Create couple of containers, each creates file of given size
  2. Execute docker ps -a –size
  3. Check the sizes are in given limit ($size; 1mb + $limit_per_mb * $size)

1.9.29.3   Configuration

Sub-subtests:docker_cli/ps_size/simple
1.9.29.3.1   docker_cli/ps_size/simple Sub-subtest
  • dd_sizes : (0 1 10 100) space separated list of used sizes (in megabytes)
  • limit_per_mb : (0.1) Acceptable increase of the size per 1mb (0.1 = 10%)
  • dd_cmd : (dd if=/dev/zero of=/big_testing_file bs=%%s count=%%s) executed command (which creates the files)

1.9.30   docker_cli/psa Subtest

1.9.30.1   Summary

Verify the table output and formatting of the docker ps -a command.

1.9.30.2   Operational Summary

  1. Attempt to parse ‘docker ps -a –no-trunc –size’ table output
  2. Fail if table-format changes or is not parseable

1.9.30.3   Configuration

  • docker_timeout : 120.0 Overrides default value: 300.0
  • wait_stop : (10) How long to wait for container to stop
  • wait_start : (5) How long to wait before using the container

1.9.31   docker_cli/pull Subtest

1.9.31.1   Summary

Test output of docker Pull command

1.9.31.2   Operational Summary

  1. Try to download repository from registry
  2. Ensure that command exit code == docker_expected_exit_status (usually 0)
  3. if docker_expected_exit_status == 0:
    1. Check if image is in local repository.
    2. Remote image from local repository

1.9.31.3   Prerequisites

  • A remote registry server
  • Image on remote registry with ‘latest’ and some other tag
  • Image on remote should not conflict with default test image

1.9.31.4   Configuration

Sub-subtests:docker_cli/pull/wrong_tag, docker_cli/pull/wrong_registry_addr
  • docker_repo_tag : buildroot-2014.02 Overrides default value: latest
  • docker_registry_user : <None> Overrides default value: rhel7
  • __example__ : docker_repo_name, docker_repo_tag, docker_registry_host, docker_registry_user Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • docker_registry_host : docker.io Overrides default value: registry.access.redhat.com
  • docker_repo_name : busybox Overrides default value: rhel
  • docker_pull_timeout : (120.0) Maximal duration of the docker pull command
  • docker_expected_exit_status : (0) Allow test to pass if actual result matches this PASS/FAIL value
1.9.31.4.1   docker_cli/pull/wrong_tag Sub-subtest
  • docker_expected_exit_status : (1) Allow test to pass if actual result matches this PASS/FAIL value - inherited
  • docker_repo_tag : tag_does_not_exist Overrides default value: latest
1.9.31.4.2   docker_cli/pull/wrong_registry_addr Sub-subtest
  • docker_registry_host : registry.does.not.exist.example.com:3 Overrides default value: registry.access.redhat.com
  • docker_expected_exit_status : (1) Allow test to pass if actual result matches this PASS/FAIL value - inherited

1.9.32   docker_cli/restart Subtest

1.9.32.1   Summary

Test usage of docker ‘restart’ command

1.9.32.2   Operational Summary

  1. start container with test command
  2. check container output for start message
  3. execute docker restart
  4. check container output for restart message
  5. execute docker stop
  6. check container output for stop message
  7. analyze results

1.9.32.3   Configuration

Sub-subtests:docker_cli/restart/zerotime, docker_cli/restart/force, docker_cli/restart/nice, docker_cli/restart/stopped
  • docker_timeout : 60 Overrides default value: 300.0
  • exec_cmd : (<None>) executed command
  • run_options_csv : (--detach) modifies the docker run options
  • restart_duration : (3) docker restart duration
  • restart_check : (<None>) expected string after restart
  • stop_check : (<None>) expected string after exit
  • restart_options_csv : (--time=10) modifies the docker restart options
  • check_output_inverted : (false) True - expected string must not be present in the output
  • start_check : (<None>) expected string after start
  • stop_duration : (3) docker stop duration
1.9.32.3.1   docker_cli/restart/zerotime Sub-subtest
  • restart_badcheck : (Received SIGTERM, finishing) String which must NOT be present after restart
  • stop_options_csv : (--time=0) modifies the docker stop options
  • stop_badcheck : (Received SIGTERM, finishing) String which must NOT be present after stop
  • start_check : (STARTING...) expected string after start - inherited
  • stop_check : (%(restart_check)s) expected string after exit - inherited
  • restart_check : (%(start_check)s\nSTARTING...) expected string after restart - inherited
  • restart_options_csv : (--time=0) modifies the docker restart options - inherited
  • exec_cmd : ("trap 'echo Received SIGTERM, finishing; exit' SIGTERM; echo "STARTING..."; while :; do sleep 1; done") executed command - inherited
1.9.32.3.2   docker_cli/restart/force Sub-subtest
  • start_check : (STARTING...) expected string after start - inherited
  • stop_duration : (13) docker stop duration - inherited
  • exec_cmd : ("trap 'echo SIGTERM ignored' SIGTERM; echo "STARTING..."; while :; do sleep 1; done") executed command - inherited
  • restart_duration : (13) docker restart duration - inherited
  • stop_check : (%(restart_check)s\nSIGTERM ignored) expected string after exit - inherited
  • restart_check : (%(start_check)s\nSIGTERM ignored\nSTARTING...) expected string after restart - inherited
1.9.32.3.3   docker_cli/restart/nice Sub-subtest
  • start_check : (STARTING...) expected string after start - inherited
  • restart_check : (%(start_check)s\nReceived SIGTERM, finishing\nSTARTING...) expected string after restart - inherited
  • stop_check : (%(restart_check)s\nReceived SIGTERM, finishing) expected string after exit - inherited
  • exec_cmd : ("trap 'echo Received SIGTERM, finishing; exit' SIGTERM; echo "STARTING..."; while :; do sleep 1; done") executed command - inherited
1.9.32.3.4   docker_cli/restart/stopped Sub-subtest
  • start_check : (STARTING...\nFINISHING...) expected string after start - inherited
  • exec_cmd : ("echo STARTING...; echo FINISHING...") executed command - inherited
  • restart_check : (%(start_check)s\nSTARTING...\nFINISHING...) expected string after restart - inherited
  • stop_check : (%(restart_check)s) expected string after exit - inherited

1.9.33   docker_cli/rm Subtest

1.9.33.1   Summary

Test for docker rm subcommand

1.9.33.2   Operational Summary

  1. Remember subtest start time
  2. Write subtest start time into sub-subtest tmpdir file
  3. Share sub-subtest tmpdir to container as volume
  4. Container waits for signal to update file in volume
  5. Run sub-subtests to exercize ‘rm’ on container and verify volume file

1.9.33.3   Configuration

Sub-subtests:docker_cli/rm/finished, docker_cli/rm/forced
  • docker_timeout : 120 Overrides default value: 300.0
  • rm_options_csv : (--volumes) modifies the docker rm options
  • listen_signal : (SIGUSR1) Signal used to notify container
  • wait_stop : (30) How long to wait for container to stop
  • wait_start : (5) How long to wait before using the container
  • wait_rm : (5) How long to wait after rm command
  • run_options_csv : (--attach=stderr) modifies the docker attach options
1.9.33.3.1   docker_cli/rm/forced Sub-subtest
  • rm_options_csv : (--volumes,--force) modifies the docker rm options - inherited

1.9.34   docker_cli/rmi Subtest

1.9.34.1   Summary

Test behavior and output of docker rmi command

1.9.34.2   Operational Summary

  1. Create new Image
  2. Try to delete image.
  3. Check if image was deleted.

1.9.34.3   Configuration

Sub-subtests:docker_cli/rmi/delete_wrong_name, docker_cli/rmi/with_blocking_container_by_id, docker_cli/rmi/with_blocking_container_by_tag, docker_cli/rmi/only_tag
  • docker_rmi_force : (no) use docker rmi --force
  • docker_rmi_timeout : (30.0) maximal acceptable docker rmi duration
  • docker_expected_exit_status : (0) Expected results
1.9.34.3.1   docker_cli/rmi/delete_wrong_name Sub-subtest
  • docker_expected_exit_status : (1) Expected results - inherited
1.9.34.3.2   docker_cli/rmi/with_blocking_container_by_id Sub-subtest
  • docker_commit_timeout : (60.0) docker commit deadline
  • docker_data_prepare_cmd : (/bin/bash -c "echo '%%s' > /var/i") Command used to prepare the container
  • commit_author : (Author_name) Author
  • commit_message : (Message) Commit message
1.9.34.3.3   docker_cli/rmi/with_blocking_container_by_tag Sub-subtest
  • commit_author : (Author_name) Author
  • docker_commit_timeout : (60.0) docker commit deadline
  • docker_data_prepare_cmd : (/bin/bash -c "echo '%%s' > /var/i") Command used to prepare the container
  • commit_message : (Message) Commit message
  • commit_changed_files : (/var/i) Changed files

1.9.35   docker_cli/run Subtest

1.9.35.1   Summary

Test docker run by command(s) inside container and checking the results.

1.9.35.2   Operational Summary

  1. Form docker command line from configuration options
  2. Execute docker command
  3. PASS/FAIL based on configuration and executed command output/results

1.9.35.3   Configuration

Sub-subtests:docker_cli/run/run_interactive, docker_cli/run/run_false, docker_cli/run/run_names, docker_cli/run/run_remote_tag, docker_cli/run/run_attach_stdout, docker_cli/run/run_install, docker_cli/run/run_true, docker_cli/run/run_tmpfs, docker_cli/run/run_update, docker_cli/run/run_passwd
  • generate_generic : (no) The most basic sub-subtests can be generated from a generic class at runtime, set ‘yes’ to enable this feature.
  • run_append_name : (yes) Set ‘yes’ to append --name with a test-specific random name.
  • bash_cmd : (/bin/bash,-c) CSV docker run COMMAND argument to prefix cmd option (below)
  • run_options_csv : (--rm) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command.
  • cmd : (<None>) Suffix to append to bash_cmd option (above), careful quoting is highly recommended.
  • exit_status : (0) expected exit status from docker run operation
1.9.35.3.1   docker_cli/run/run_interactive Sub-subtest
  • bash_cmd : (cat) CSV docker run COMMAND argument to prefix cmd option (below) - inherited
  • run_options_csv : (--interactive,--rm) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command. - inherited
  • secret_sauce : (gfjfif8HHYnmn!!) Unique string used to verify output from container
1.9.35.3.2   docker_cli/run/run_false Sub-subtest
  • exit_status : (1) expected exit status from docker run operation - inherited
  • generate_generic : (yes) The most basic sub-subtests can be generated from a generic class at runtime, set ‘yes’ to enable this feature. - inherited
  • cmd : (/bin/false) Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
1.9.35.3.3   docker_cli/run/run_names Sub-subtest
  • last_name_sticks : (yes) Set to ‘yes’ if the last generated --name is used by docker, or ‘no’ if the first. Affects test pass/fail.
  • names_count : (1000) The number of --name options to generate for the command line.
  • cmd : (sleep 2s) Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • run_append_name : (no) Set ‘yes’ to append --name with a test-specific random name. - inherited
  • run_options_csv : (--detach) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command. - inherited
1.9.35.3.4   docker_cli/run/run_remote_tag Sub-subtest
  • cmd : (/bin/true) Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • remote_image_fqin : (docker.io/stackbrew/centos:latest) Fully qualified image name stored on a remote registry, not local.
  • run_options_csv : (<None>) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command. - inherited
  • __example__ : remote_image_fqin Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
1.9.35.3.5   docker_cli/run/run_attach_stdout Sub-subtest
  • run_options_csv : (--detach=true) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command. - inherited
  • cmd : ('sleep 5s && echo "%(secret_sauce)s"') Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • secret_sauce : (4c93bb78d98f) Unique string used to verify output from container
1.9.35.3.6   docker_cli/run/run_install Sub-subtest
  • verify_cmd : (bash -c "echo '[{}]' | json_verify -q") Full command to run in commited container to verify installation
  • install_cmd : (yum install --assumeyes yajl) Full command to execute which installs something inside the container
  • __example__ : install_cmd,verify_cmd Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
1.9.35.3.7   docker_cli/run/run_true Sub-subtest
  • cmd : (/bin/true) Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • generate_generic : (yes) The most basic sub-subtests can be generated from a generic class at runtime, set ‘yes’ to enable this feature. - inherited
1.9.35.3.8   docker_cli/run/run_update Sub-subtest
  • generate_generic : (yes) The most basic sub-subtests can be generated from a generic class at runtime, set ‘yes’ to enable this feature. - inherited
  • cmd : ('yum --assumeyes update') Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • __example__ : cmd Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
1.9.35.3.9   docker_cli/run/run_passwd Sub-subtest
  • generate_generic : (yes) The most basic sub-subtests can be generated from a generic class at runtime, set ‘yes’ to enable this feature. - inherited
  • cmd : ('passwd --status root | grep -qv %(expected_status)s') Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited
  • expected_status : ("Password set") String to search for with grep (see cmd option value)
1.9.35.3.10   docker_cli/run/run_tmpfs Sub-subtest
  • run_options_csv : (--tmpfs,%(tmpfs_path)s) CSV of options and arguments to use on docker command line, with the exception of –name, image, and contained command. - inherited
  • tmpfs_path : (/run) –tmpfs parameter
  • cmd : ('findmnt -n -t tmpfs %(tmpfs_path)s') Suffix to append to bash_cmd option (above), careful quoting is highly recommended. - inherited

1.9.36   docker_cli/run_attach Subtest

1.9.36.1   Summary

This test checks different docker run -a xxx variants.

1.9.36.2   Operational Summary

  1. Starts docker run with defined combination of -a … each subtest executes 6 variants of tty/non-tty vs stdin/out/err.
  2. Analyze results (exit_code, input_handling, correct_output)

1.9.36.3   Configuration

1.9.37   docker_cli/run_cgroup_parent Subtest

1.9.37.1   Summary

Tests for bz1320302 - odd behavior of –cgroup-parent option.

1.9.37.2   Operational Summary

  1. Start container with –cgroup-parent=X, for various values of X
  2. Run container command ‘cat /proc/1/cgroup’
  3. Confirm that each line (controller) lists the correct cgroup path

1.9.37.3   Configuration

Sub-subtests:docker_cli/run_cgroup_parent/run_cgroup_parent_invalid_name
1.9.37.3.1   docker_cli/run_cgroup_parent/run_cgroup_parent_invalid_name Sub-subtest
  • expect_stderr : (.*Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"\.) Regex describing the error message we expect. Passed to re.match().

1.9.38   docker_cli/run_cgroups Subtest

1.9.38.1   Summary

Tests that check output/behavior of docker run wuth -m and -c parameters. It verifies that the container’s cgroup resources match value passed and if the container can handle invalid values properly.

1.9.38.2   Operational Summary

  1. Start container with non-default cgroup-related option.
  2. Verify docker inspect matches param value matches actual cgroup value

1.9.38.3   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket.
  • cgroups subsystem enabled, working, and mounted under standard /sys location

1.9.38.4   Configuration

Sub-subtests:docker_cli/run_cgroups/cpu_zero, docker_cli/run_cgroups/cpu_positive, docker_cli/run_cgroups/cpu_overflow, docker_cli/run_cgroups/memory_positive, docker_cli/run_cgroups/memory_no_cgroup, docker_cli/run_cgroups/cpu_none, docker_cli/run_cgroups/memory_negative
  • cgroup_key_value : (memory.limit_in_bytes) which key is used in this test
  • expect_success : (PASS) Expected results
  • cgroup_path : (/sys/fs/cgroup/memory/system.slice/docker) subsystem path to the docker’s cgroups on host
1.9.38.4.1   docker_cli/run_cgroups/cpu_zero Sub-subtest
  • cpushares_value : (0) Set/expected value of the cgroup_key_value
  • cgroup_path : (/sys/fs/cgroup/cpu/system.slice/docker) subsystem path to the docker’s cgroups on host - inherited
  • cgroup_key_value : (cpu.shares) which key is used in this test - inherited
1.9.38.4.2   docker_cli/run_cgroups/cpu_positive Sub-subtest
  • cpushares_value : (10) Set/expected value of the cgroup_key_value
  • cgroup_path : (/sys/fs/cgroup/cpu/system.slice/docker) subsystem path to the docker’s cgroups on host - inherited
  • cgroup_key_value : (cpu.shares) which key is used in this test - inherited
1.9.38.4.3   docker_cli/run_cgroups/cpu_overflow Sub-subtest
  • cpushares_value : (4294967296) Set/expected value of the cgroup_key_value
  • cgroup_key_value : (cpu.shares) Set/expected value of the cgroup_key_value
  • cgroup_path : (/sys/fs/cgroup/cpu/system.slice/docker) subsystem path to the docker’s cgroups on host - inherited
  • expect_success : (FAIL) The maximum allowed cpu-shares is <smaller number>
1.9.38.4.4   docker_cli/run_cgroups/memory_positive Sub-subtest
  • memory_value : (5246976) Set/expected value of the cgroup_key_value
1.9.38.4.5   docker_cli/run_cgroups/memory_no_cgroup Sub-subtest
  • memory_value : (0) Set/expected value of the cgroup_key_value
1.9.38.4.6   docker_cli/run_cgroups/cpu_none Sub-subtest
  • cgroup_path : (/sys/fs/cgroup/cpu/system.slice/docker) subsystem path to the docker’s cgroups on host - inherited
  • cgroup_key_value : (cpu.shares) Set/expected value of the cgroup_key_value
1.9.38.4.7   docker_cli/run_cgroups/memory_negative Sub-subtest
  • memory_min_invalid : (512) Too low memory limit
  • memory_invalid : (abcd) Incorrect string used as memory limit
  • expect_success : (FAIL) Expected results - inherited
  • memory_max_invalid : (92233720300000000000) Too big memory limit

1.9.39   docker_cli/run_cidfile Subtest

1.9.39.1   Summary

This test is focussed on correct handling of docker run --cidfile

1.9.39.2   Operational Summary

  1. Start container with cidfile
  2. Try to start another container with the same cidfile
  3. Exit the first container and try to start another container with the same cidfile
  4. Restart the original container and check the cidfile
  5. Stop the container and cleanup

1.9.39.3   Configuration

  • run_options_csv : (-i) Additional options to pass when starting container

1.9.40   docker_cli/run_dns Subtest

1.9.40.1   Summary

Set various dns’ and dns-searches and check docker behave well.

1.9.40.2   Operational Summary

  1. Execute couple of correctly set dns/dns-search scenarios
  2. Execute couple of incorrect dns/dns-search scenarios
Note:Subtest does not have any default configuration

1.9.41   docker_cli/run_env Subtest

1.9.41.1   Summary

This subtest checks functionality of varios environment variable related options / operations of docker.

1.9.41.2   Operational Summary

  1. Run container with various environment-var related options.
  2. Verify healthy container exit state/status.
  3. Verfiy container environment variables match expectations.

1.9.41.3   Operational Detail

“spam” sub-subtest:

  1. Generate hundreds of random short-named env. vars with short values (so as to not exceede max command-line length)
  2. Execute docker run command with -e options for list, volume mounting the sub-subtest’s results dir into the container.
  3. Dump out container env into a file in mounted results dir
  4. Verify healthy container exit, no oopses or non-zero exit codes
  5. Verify generated environment vars match number & content in dump file.
  6. Ignore any container environment variables that don’t contain a special identifying prefix/suffix.

1.9.41.4   Configuration

Sub-subtests:docker_cli/run_env/spam
1.9.41.4.1   docker_cli/run_env/spam Sub-subtest
  • n_env : (512) Number of -e options to start container with

1.9.42   docker_cli/run_exec Subtest

1.9.42.1   Summary

Test docker exec by executing basic commands inside already running container and checking the results.

1.9.42.2   Operational Summary

  1. Execute a shell in a container and leave it idling
  2. In parallel, run a command in container with docker exec
  3. Verify expected result from above step
  4. Cleanup shell container

1.9.42.3   Prerequisites

  • Test container contains /bin/true and /bin/false
  • Test container contains find command
  • Test container’s /proc filesystem is accessable by docker exec

1.9.42.4   Configuration

Sub-subtests:docker_cli/run_exec/exec_pid_count, docker_cli/run_exec/exec_false
  • run_options_csv : (--interactive) modifies the execning container options (assume –sig-proxy is enabled by default)
  • exec_options_csv : (<None>) modifies the execning command in started container (Do not add –interactive as it will not close STDIN and strigger timeout)
  • docker_timeout : 60 Overrides default value: 300.0
  • exit_status : (0) expected exit status
  • bash_cmd : (/bin/sh) csv command prefix (docker run … $bash_cmd)
1.9.42.4.1   docker_cli/run_exec/exec_pid_count Sub-subtest
  • pid_count : (2) Expected count of pid in container when command using exec is started.
1.9.42.4.2   docker_cli/run_exec/exec_false Sub-subtest
  • exit_status : (1) expected exit status - inherited

1.9.43   docker_cli/run_flags Subtest

1.9.43.1   Summary

Checks that docker daemon is running with expected command-line options.

1.9.43.2   Operational Summary

Get the full command line of the currently-running docker or docker-latest daemon. Check for presence of expected option flags.

1.9.43.3   Configuration

1.9.44   docker_cli/run_sigproxy Subtest

1.9.44.1   Summary

Test the difference of running docker run/attach with/without ‘–sig-proxy’.

1.9.44.2   Operational Summary

  1. start VM with test command
  2. kill $SIGNAL $test_process
  3. analyze results

1.9.44.3   Configuration

  • kill_signals : (10 12 15) Which signals should be used for testing
  • exec_cmd : ('for NUM in `seq 1 64`; do trap "echo Received $NUM, ignoring..." $NUM; done; echo READY; while :; do sleep 0.1; done') modifies the container command
  • wait_between_kill : (0.1) Workaround of the problem with missing signals BUG: This workaround puts sleep between kill signals (RH bugzilla #1096269)
  • docker_timeout : 60 Overrides default value: 300.0
  • check_stdout : (Received %%s, ignoring...) Check message format generated by container for each signal

1.9.45   docker_cli/run_twice Subtest

1.9.45.1   Summary

Negative test, verify cannot run second container with same name.

1.9.45.2   Operational Summary

  1. Run a container with a certain name
  2. Run a container with a certain name again
  3. Fail to run a container again

1.9.45.3   Configuration

1.9.46   docker_cli/run_user Subtest

1.9.46.1   Summary

This test checks correctness of docker run -u …

1.9.46.2   Operational Summary

  1. get container’s /etc/passwd
  2. generate uid which suits the test needs (nonexisting, existing name, uid..)
  3. execute docker run -u … echo $UID:$GID; whoami
  4. check results (pass/fail/details)

1.9.46.3   Configuration

  • exec_cmd : ('echo UIDCHECK: $UID:$GID; echo WHOAMICHECK: `whoami`') modifies the container command
  • docker_timeout : 60 Overrides default value: 300.0

1.9.47   docker_cli/run_volumes Subtest

1.9.47.1   Summary

Test read & write to various host-paths as container volumes

1.9.47.2   Operational Summary

  1. Setup volumes for testing, use known locations / data.
  2. Start container(2), attempt access / read files
  3. Verify tested output/results match expected values.

1.9.47.3   Operational Detail

  • volumes_rw: Attempt to read, then write a file from a host path volume inside a container. Intended to test NFS, SMB, and other ‘remote’ filesystem mounts.
  • volumes_one_source: Have multiple containers mount a directory and then write to files in that directory simultaneously.

1.9.47.4   Prerequisites

  • Remote filesystems are mounted and accessible on host system.
  • Containers have access to read & write files w/in mountpoints

1.9.47.5   Configuration

Sub-subtests:docker_cli/run_volumes/file_volume, docker_cli/run_volumes/volumes_one_source
  • run_template : (--cidfile="%%(cidfile)s",--volume="%%(host_path)s:%%(cntr_path)s") CSV options to docker command(s), values will be substituted for %%(cidfile)s, %%(host_path)s, %%(cntr_path)s. Random filenames will be produced and set by %%(read_fn)s and %%(write_fn)s for use in the test.
  • pretest_cmd : (<None>) Command-line string, required to succeed prior to testing (if any). Will only be run once, before any sub-subtests execute. e.g. setsebool virt_sandbox_use_nfs on
  • cmd_template : (/bin/bash -c 'cd "%%(cntr_path)s" && cat "%%(read_fn)s" | /usr/bin/md5sum >> "%%(write_fn)s"') Command string to execute inside container, values will be substituted as in run_template (above).
  • __example__ : host_paths, cntr_paths Overrides default value: docker_repo_name, docker_repo_tag, preserve_fqins
  • host_paths : (<None>) CSV list of host paths to check, assumes filesystem is already mounted for testing NFS, SMB, etc. If using SELinux, don’t forget to ‘setsebool virt_sandbox_use_<type> on’ where <type> is nfs, samba, etc.
  • cntr_paths : (<None>) CSV list of destination paths in container corresponding to host_paths above (must exactly match the number of host_paths)
1.9.47.5.1   docker_cli/run_volumes/file_volume Sub-subtest
  • regex : ('^State: ONLINE$') Regular expression that must match stdout on successful execution
  • cmd_template : (/usr/bin/journalctl --header --directory=%%(cntr_path)s) Command string to execute inside container, values will be substituted as in run_template (above). - inherited
  • run_template : (-it,--cidfile="%%(cidfile)s",--volume="%%(host_path)s:%%(cntr_path)s", --privileged) CSV options to docker command(s), values will be substituted for %%(cidfile)s, %%(host_path)s, %%(cntr_path)s. Random filenames will be produced and set by %%(read_fn)s and %%(write_fn)s for use in the test. - inherited
  • cntr_paths : (/root/journal) CSV list of destination paths in container corresponding to host_paths above (must exactly match the number of host_paths) - inherited
  • host_paths : (/var/log/journal) CSV list of host paths to check, assumes filesystem is already mounted for testing NFS, SMB, etc. If using SELinux, don’t forget to ‘setsebool virt_sandbox_use_<type> on’ where <type> is nfs, samba, etc. - inherited
1.9.47.5.2   docker_cli/run_volumes/volumes_one_source Sub-subtest
  • selinux_context : (svirt_sandbox_file_t) If non-empty, set this context type on host paths e.g. svirt_sandbox_file_t
  • num_containers : (5) the number of containers to run concurrently.
  • cntr_path : (/run-n-volume) where to mount the volume inside the container
  • exec_command : (/bin/bash -c 'for i in $(seq 5); do echo "%%(name)s - $i - $(date)" >> "%%(write_path)s"; sleep 1; done && /usr/bin/md5sum "%%(write_path)s"') the command each container should run. This should be an IO command that writes to a file at %%(write_path)s which will be inside the mounted volume. This command should also take time to allow for taking place while the other containers are also writing IO.
  • cmd_timeout : (30) the timeout for each container’s IO command

1.9.48   docker_cli/run_volumes_selinux Subtest

1.9.48.1   Summary

Tests the –volume ::Z and ::z feature (automatic selinux context setting).

1.9.48.2   Operational Summary

  1. Start container using volume with z or Z set
  2. Check context and file creation within the container
  3. Start another container –volumes-from $first_container
  4. Verify context and file creation from first, second, and host

1.9.48.3   Configuration

  • run_options_csv : (-i) Additional options to pass when starting container
  • use_system_tmp : (no) Set yes to force use of system temp directory. This may be necessary if selinux is unable to relabel files in the autotest temporary directory.
  • selinux_host : (public_content_rw_t) SELinux context type to set on volumes source directory

1.9.49   docker_cli/save_load Subtest

1.9.49.1   Summary

Tests the docker save and docker load commands.

1.9.49.2   Operational Summary

  1. Prepare image, save it, remove it, load it back again.
  2. Test simultaneous loading of multiple images in parallel.
  3. Check results

1.9.49.3   Configuration

  • docker_data_prep_cmd : (/bin/sh -c "echo data > /var/i") Command executed to prepare container
  • run_options_csv : (<None>) modifies the docker run options
  • save_cmd : (%%(image)s > /%%(tmpdir)s/%%(image)s) docker save command
  • docker_save_load_timeout : (120.0) Deadline for save/load operations
  • load_cmd : (< /%%(tmpdir)s/%%(image)s) docker load command

1.9.50   docker_cli/selinux_labels Subtest

1.9.50.1   Summary

Verify that docker, daemons, and containers all have expected selinux labels.

1.9.50.2   Operational Summary

  1. Execute bundled test.sh script to perform label checking operations.
  2. Fail subtest if script returns non-zero.

1.9.50.3   Prerequisites

Commands contained w/in test script are available/functional on system. Specifically docker 1.12 is needed for dockerd and containerd.

Note:Subtest does not have any default configuration

1.9.51   docker_cli/start Subtest

1.9.51.1   Summary

Test output of docker start command

1.9.51.2   Operational Summary

  1. Create new container.
  2. Wait till end of container command.
  3. Try to start container.
  4. Check if container was started.

1.9.51.3   Prerequisites

  • A remote registry server

1.9.51.4   Configuration

Sub-subtests:docker_cli/start/rerun_long_term_app, docker_cli/start/simple, docker_cli/start/short_term_app, docker_cli/start/long_term_app
  • docker_start_timeout : (30.0) max time to wait for container to start
  • run_cmd : (<None>) Specifies the executed command inside the container
  • docker_run_timeout : (30.0) max time to wait for container to finish (docker wait)
  • dkrcmd_waittime : (60) time, in seconds, to wait for command to complete
  • docker_interactive : (yes) Run interactive container -i
  • docker_attach : (yes) Run detached container -d
1.9.51.4.1   docker_cli/start/rerun_long_term_app Sub-subtest
  • dkrcmd_waittime : (105) time, in seconds, to wait for command to complete - inherited
  • run_cmd : (sleep 100) Specifies the executed command inside the container - inherited
1.9.51.4.2   docker_cli/start/simple Sub-subtest
  • run_options_csv : (--interactive=true) modifies the running container options
  • missing_msg : (o such) Expected string to see when starting non-running container
  • docker_timeout : 60 Overrides default value: 300.0
1.9.51.4.3   docker_cli/start/short_term_app Sub-subtest
  • run_cmd : (ls -l /etc) Specifies the executed command inside the container - inherited
1.9.51.4.4   docker_cli/start/long_term_app Sub-subtest
  • dkrcmd_waittime : (105) time, in seconds, to wait for command to complete - inherited
  • run_cmd : (sleep 100) Specifies the executed command inside the container - inherited

1.9.52   docker_cli/stop Subtest

1.9.52.1   Summary

Test usage of docker ‘stop’ command

1.9.52.2   Operational Summary

  1. start VM with test command
  2. execute docker stop
  3. analyze results (duration, exit_code)

1.9.52.3   Configuration

Sub-subtests:docker_cli/stop/nice, docker_cli/stop/force, docker_cli/stop/zerotime, docker_cli/stop/stopped
  • check_stdout : (<None>) expected output to be found in stdout (if check_output_inverted = False
  • stop_options_csv : (--time=10) modifies the docker stop options
  • exec_cmd : (<None>) Command executed on the container (exit or ignores the signal)
  • docker_timeout : 60 Overrides default value: 300.0
  • run_options_csv : (--rm,--attach=stdout) modifies the docker run options
  • stop_duration : (<None>) Expected duration before the container is stopped
  • docker_exit_code : (0) Expected container exit code
  • check_output_inverted : (false) expect the check_stdout NOT to be present in the output
1.9.52.3.1   docker_cli/stop/nice Sub-subtest
  • stop_duration : (2) Expected duration before the container is stopped - inherited
  • check_stdout : (Received SIGTERM, finishing) expected output to be found in stdout (if check_output_inverted = False - inherited
  • exec_cmd : ("trap 'echo %(check_stdout)s; exit' SIGTERM; echo READY; while :; do sleep 1; done") Command executed on the container (exit or ignores the signal) - inherited
1.9.52.3.2   docker_cli/stop/stopped Sub-subtest
  • exec_cmd : ("echo READY") Command executed on the container (exit or ignores the signal) - inherited
  • stop_duration : (2) Expected duration before the container is stopped - inherited
  • check_stdout : (<None>) expected output to be found in stdout (if check_output_inverted = False - inherited
  • run_options_csv : (--attach=stdout) modifies the docker run options - inherited
1.9.52.3.3   docker_cli/stop/zerotime Sub-subtest
  • check_stdout : (Received SIGTERM, finishing) expected output to be found in stdout (if check_output_inverted = False - inherited
  • check_output_inverted : (true) expect the check_stdout NOT to be present in the output - inherited
  • exec_cmd : ("trap 'echo %(check_stdout)s; exit' SIGTERM; echo READY; while :; do sleep 1; done") Command executed on the container (exit or ignores the signal) - inherited
  • stop_duration : (2) Expected duration before the container is stopped - inherited
  • docker_exit_code : (137) Expected container exit code - inherited
  • stop_options_csv : (--time=0) modifies the docker stop options - inherited
1.9.52.3.4   docker_cli/stop/force Sub-subtest
  • check_stdout : (SIGTERM ignored) expected output to be found in stdout (if check_output_inverted = False - inherited
  • docker_exit_code : (137) Expected container exit code - inherited
  • exec_cmd : ("trap 'echo %(check_stdout)s' SIGTERM; echo READY; while :; do sleep 1; done") Command executed on the container (exit or ignores the signal) - inherited
  • stop_duration : (12) Expected duration before the container is stopped - inherited

1.9.53   docker_cli/syslog Subtest

1.9.53.1   Summary

Test monitoring containers logs from host via bind-mount /dev/log to containers. Test containers can send log to host

1.9.53.2   Operational Summary

  1. Mount the /dev/log/ directory to container.
  2. Use logger send a message.
  3. Verify that could see the message on host.

1.9.53.3   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket.
  • /dev/log is existing and could be mounted to containers.

1.9.53.4   Configuration

  • syslogfile : (/var/log/messages) Absolute path to file holding main system logs
  • docker_timeout : 60.0 Overrides default value: 300.0

1.9.54   docker_cli/systemd Subtest

1.9.54.1   Summary

This is to test basic container as systemd service using a predefined unit file. Image pulling, building, and running are performed.

1.9.54.2   Operational Summary

  1. Provided unitfile is edited & copied to /etc/systemd/system.
  2. Image is pulled using unitfile
  3. Image is build using provided Dockedfile
  4. Container is started/run using unitfile

1.9.54.3   Operational Detail

1.9.54.3.1   systemd pull
  1. Edit unitfile docker-pull.service & copy it to /etc/systemd/system
  2. Systemd service is started using this file to pull an image.
  3. Systemd service is stoped and image removed
1.9.54.3.2   systemd build
  1. Edit unitfile docker-build.service & copy it to /etc/systemd/system
  2. Edit Dockerfile
  3. Systemd service is started using this file to build an image.
  4. Systemd service is stoped and image removed
1.9.54.3.3   systemd run
  1. Edit unitfile p4321.service and copy to /etc/systemd/system
  2. Edit Dockerfile and copy to test temporary directory
  3. Copy a script p4321-server.py to test temporary directory
  4. Built an image using the Dockerfile and script
  5. Systemd starts a container using this image
  6. The container writes current time to port 4321.
  7. From host, the socket is read and the value is checked
  8. Finally, container is stopped through systemd and system is cleaned up.

1.9.54.4   Prerequisites

  • Docker daemon is running
  • systemd unitfiles can be copied to /etc/systemd/system
  • systemd actions of start/status/stop & daemon-reload can be performed
  • An image stated in systemd.ini can be pulled & build

1.9.54.5   Configuration

Sub-subtests:docker_cli/systemd/systemd_pull, docker_cli/systemd/systemd_build, docker_cli/systemd/systemd_run
  • dockerfile_base_image : (centos:latest) base image for Dockerfile. Not used in every test.
  • sysd_unit_file : (FIXME) systemd unit file (basename only). Each subsubtest has a file by this name in our source dir; we copy it into /etc/systemd/system, replacing a given string foo inside curly braces with the corresponding foo config setting defined below.
1.9.54.5.1   docker_cli/systemd/systemd_pull Sub-subtest
  • sysd_unit_file : (docker-pull.service) systemd unit file (basename only). Each subsubtest has a file by this name in our source dir; we copy it into /etc/systemd/system, replacing a given string foo inside curly braces with the corresponding foo config setting defined below. - inherited
  • image_name : (centos) name of an image to pull
1.9.54.5.2   docker_cli/systemd/systemd_build Sub-subtest
  • image_name : (centos_p4321) name of an image to build
  • sysd_unit_file : (docker-build.service) systemd unit file (basename only). Each subsubtest has a file by this name in our source dir; we copy it into /etc/systemd/system, replacing a given string foo inside curly braces with the corresponding foo config setting defined below. - inherited
  • unit_opts : (--force-rm) build options when building an image
1.9.54.5.3   docker_cli/systemd/systemd_run Sub-subtest
  • sysd_unit_file : (p4321.service) systemd unit file (basename only). Each subsubtest has a file by this name in our source dir; we copy it into /etc/systemd/system, replacing a given string foo inside curly braces with the corresponding foo config setting defined below. - inherited
  • image_name : (centos_p4321) name of an image build and run
  • build_opt : (%(unit_opts)s -t %(image_name)s) build options
  • unit_opts : (--force-rm) unit file build options when building an image

1.9.55   docker_cli/systemd_in_container Subtest

1.9.55.1   Summary

Verify running systemd as ENTRYPOINT in container.

1.9.55.2   Operational Summary

  1. Execute bundled test.sh script to perform label checking operations.
  2. Fail subtest if script returns non-zero.

1.9.55.3   Prerequisites

Commands contained w/in test script are available/functional on system. Specifically docker 1.12 is needed for dockerd and containerd.

Note:Subtest does not have any default configuration

1.9.56   docker_cli/tag Subtest

1.9.56.1   Summary

Test output of docker tag command

1.9.56.2   Operational Summary

  1. Make new image name.
  2. tag changes.
  3. check if tagged image exists.
  4. remote tagged image from local repo.

1.9.56.3   Configuration

Sub-subtests:docker_cli/tag/double_tag, docker_cli/tag/change_user, docker_cli/tag/change_tag, docker_cli/tag/double_tag_force, docker_cli/tag/change_repository, docker_cli/tag/change_registry

1.9.57   docker_cli/top Subtest

1.9.57.1   Summary

Verify output from docker top against a test-controlled container matches content expectations.

1.9.57.2   Operational Summary

  1. start container
  2. execute docker top against container
  3. verify output

1.9.57.3   Configuration

  • run_options_csv : (--detach,--read-only) Options/arguments to pass to docker run command in addition to container name, image name, and commnad.
  • top_options_csv : (-waux) Options/arguments to pass to docker top command in addition to container name.

1.9.58   docker_cli/version Subtest

1.9.58.1   Summary

Test output of docker version command

1.9.58.2   Operational Summary

  1. Run docker version command
  2. Check output
  3. compare to daemon API version

1.9.58.3   Prerequisites

None

Note:Subtest does not have any default configuration

1.9.59   docker_cli/wait Subtest

1.9.59.1   Summary

Test the docker wait operation on containers in various states

1.9.59.2   Operational Summary

  1. Prepare wait target container
  2. Execute docker wait on container
  3. Verify results

1.9.59.3   Configuration

Sub-subtests:docker_cli/wait/Sig, docker_cli/wait/Multi
  • target_run : (--detach) CSV run command line options minus –name & image for each target container

  • target_setups : (RUN) CSV list of strings describing how containers are setup, “CREATE”, “RUN” or “NONE”. Each value represents a separate container and all target_* lists must contain exactly the same number of items.

  • target_cmd : (bash,-c,'trap "echo \"Received Signal 30\"" 30; sleep @SLEEP@s') Quoted command line to pass to each target container

  • stdout : (^0\n$) (optional) Regular expression that much match wait command stdout

  • target_waits : (NONE) CSV list of operations to perform on each container during the test wait command:
    • STOP - Use docker stop on the target container * KILL - Use docker kill on the target container * <n> - Send signal number <n> to target container * REMV - Use docker rm --force on the target container * NONE - Do nothing
  • docker_timeout : 20 Overrides default value: 300.0

  • target_sleeps : (0.5) CSV list of sleep times for each target, must have same number of items as target_setups and target_waits. Do not rely on timing for precise synchronization!

  • target_timeout : (10) Timeout for the target docker command in seconds

  • target_verbose : (no) Run target containers with more verbosity under all conditions

  • wait_verbose : (no) Run wait command with more verbosity

  • exit : (0) Expected exit code from wait command

  • stderr : (<None>) (optional) Regular expression that much match wait command stderr

1.9.59.3.1   docker_cli/wait/Sig Sub-subtest
  • target_setups : (RUN) CSV list of strings describing how containers are setup, “CREATE”, “RUN” or “NONE”. Each value represents a separate container and all target_* lists must contain exactly the same number of items. - inherited

  • target_sleeps : (0.5) CSV list of sleep times for each target, must have same number of items as target_setups and target_waits. Do not rely on timing for precise synchronization! - inherited

  • docker_timeout : 20 Overrides default value: 300.0

  • target_waits : (30) CSV list of operations to perform on each container during the test wait command:
    • STOP - Use docker stop on the target container * KILL - Use docker kill on the target container * <n> - Send signal number <n> to target container * REMV - Use docker rm --force on the target container * NONE - Do nothing - inherited
1.9.59.3.2   docker_cli/wait/Multi Sub-subtest
  • target_waits : (NONE, KILL, NONE) CSV list of operations to perform on each container during the test wait command:
    • STOP - Use docker stop on the target container * KILL - Use docker kill on the target container * <n> - Send signal number <n> to target container * REMV - Use docker rm --force on the target container * NONE - Do nothing - inherited
  • target_sleeps : (1, 5, 3) CSV list of sleep times for each target, must have same number of items as target_setups and target_waits. Do not rely on timing for precise synchronization! - inherited

  • stdout : (^0\n137\n0$) (optional) Regular expression that much match wait command stdout - inherited

  • docker_timeout : 20 Overrides default value: 300.0

  • target_setups : (RUN, RUN, RUN) CSV list of strings describing how containers are setup, “CREATE”, “RUN” or “NONE”. Each value represents a separate container and all target_* lists must contain exactly the same number of items. - inherited

1.9.60   docker_cli/workdir Subtest

1.9.60.1   Summary

Test that checks the docker run --workdir command could set workdir successfully if the dir is a valid path, and fails if it’s not absolute path or not a path, like a file.

1.9.60.2   Operational Summary

  1. Run container w/ valid & invalid workdir
  2. Verify access or failure is as expected

1.9.60.3   Operational Detail

1.9.60.3.1   Positive test
  1. Run a container with an existing workdir
  2. Run a container with an non-existing workdir
1.9.60.3.2   Negative testing
  1. Run a container with an invalid workdir which is not absolute path
  2. Run a container with the workdir which is not a path, but a file

1.9.60.4   Prerequisites

  • Docker daemon is running and accessible by it’s unix socket.

1.9.60.5   Configuration

1.9.61   redhat/rhel_push_plugin Subtest

1.9.61.1   Summary

Test the rhel-push-plugin, which blocks push of RHEL-based images to docker.io

1.9.61.2   Operational Summary

  1. Build an image claiming to be (or not to be) RHEL-based
  2. Tag it, try to push it
  3. If RHEL-based, expect failure message from plugin.

1.9.61.3   Configuration

Sub-subtests:redhat/rhel_push_plugin/push_name_ok, redhat/rhel_push_plugin/push_vendor_ok, redhat/rhel_push_plugin/push_blocked, redhat/rhel_push_plugin/push_registry_ok
  • image_name : (rhelNOTREALLY) string to use as the Name label in the built image
  • base_image : (docker.io/busybox:latest) base image from which to build the image we try to push. We can’t default to full_name_from_defaults, because that may be RHEL itself. We could use CentOS, but that would mean pulling it if not already present. So pick busybox, which is small enough to pull quickly.
  • image_vendor : (Red Hat, Inc.) string to use for the Vendor label in the built image
  • dest_name : (docker.io/myname/myrhel) where we try to push the image
  • expected_stderr : (<None>) What we expect to see in stderr on the push command. Must be filled in by each subsubtest.
1.9.61.3.1   redhat/rhel_push_plugin/push_name_ok Sub-subtest
  • expected_stderr : (unauthorized: authentication required | denied: requested access to the resource is denied) What we expect to see in stderr on the push command. Must be filled in by each subsubtest. - inherited
  • image_name : (free_warez_from_red_hat) string to use as the Name label in the built image - inherited
1.9.61.3.2   redhat/rhel_push_plugin/push_vendor_ok Sub-subtest
  • expected_stderr : (unauthorized: authentication required | denied: requested access to the resource is denied) What we expect to see in stderr on the push command. Must be filled in by each subsubtest. - inherited
  • image_vendor : (Not Red Hat, Ltd.) string to use for the Vendor label in the built image - inherited
1.9.61.3.3   redhat/rhel_push_plugin/push_blocked Sub-subtest
  • expected_stderr : (Error response from daemon: authorization denied by plugin rhel-push-plugin: RHEL based images are not allowed to be pushed to docker.io) What we expect to see in stderr on the push command. Must be filled in by each subsubtest. - inherited
1.9.61.3.4   redhat/rhel_push_plugin/push_registry_ok Sub-subtest
  • expected_stderr : (unable to ping registry endpoint | dial tcp: lookup not-docker.io) What we expect to see in stderr on the push command. Must be filled in by each subsubtest. - inherited
  • dest_name : (not-docker.io/myname/myrhel) where we try to push the image - inherited

1.10   Additional Test Modules

The following section details included pretests, intratests, and posttests modules and any prerequisites or setup requirements.

1.11   Dockertest 0.8 API Reference

Covers 0.8 and all revisions.

1.11.1   Dockertest Package

All modules relating to the docker autotest sub-framework are in this namespace

Through subclassing dockertest.subtest.Subtest(), subtest modules, also have access to this namespace. It is available using the standard python dotted import syntax. For example, from dockertest import output.

1.11.6   Networking Module

Provides helpers for frequently used docker networking operations.

This module defines several independent interfaces using an abstract-base-class pattern. They are extended through a few subclasses to meet basic needs.

Where/when *possible*, both parameters and return values follow this order and defaults:

  • container_port
  • host_port (same as container_port if unspecified)
  • host_ip (literal “0.0.0.0” if unspecified)
  • protocol (literal “tcp” if unspecified
class dockertest.networking.ContainerPort(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')[source]

Bases: object

Represents a private container port mapping to public host ip, and port.

Parameters:
  • container_port – Port number visible inside the container
  • host_port – Port number on host container_Port maps to
  • host_ip – Host interface IP host_port maps to
  • protocol – Name of protocol for map (i.e. tcp or udp)
port_split_p = <_sre.SRE_Pattern object at 0x2f6f030>

Regex to help extract components from string of format <host IP>:<host port>->

static split_to_component(portstr)[source]

Split published into separate component strings/numbers

Parameters:portstr – Port info string
Returns:Iterable of container_port, host_ip, host_port, protocol
static portstr_from_component(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')[source]

Port name string from individual components.

Parameters:
  • container_port – Port number visible inside the container
  • host_port – Port number on host container_Port maps to
  • host_ip – Host interface IP host_port maps to
  • protocol – Name of protocol (i.e. tcp or udp)
Returns:

port string

cmp_portstr_with_component(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')[source]

Boolean compare instance’s portstr to individual components

Parameters:
  • container_port – Port number visible inside the container
  • host_port – Port number on host container_Port maps to
  • host_ip – Host interface IP host_port maps to
  • protocol – Name of protocol (i.e. tcp or udp)
Returns:

True/False on equality

cmp_portstr(portstr)[source]

Compare instance’s portstr to portstr parameter

Parameters:portstr – Port string
Returns:True/False on equality
class dockertest.networking.PortContainer(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')[source]

Bases: dockertest.networking.ContainerPort

Parser of port string output from port command

static split_to_component(portstr)[source]

Split published into separate component strings/numbers

Parameters:portstr – Port info string
Returns:Iterable of container_port, host_ip, host_port, protocol
cmp_portstr(portstr)

Compare instance’s portstr to portstr parameter

Parameters:portstr – Port string
Returns:True/False on equality
cmp_portstr_with_component(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')

Boolean compare instance’s portstr to individual components

Parameters:
  • container_port – Port number visible inside the container
  • host_port – Port number on host container_Port maps to
  • host_ip – Host interface IP host_port maps to
  • protocol – Name of protocol (i.e. tcp or udp)
Returns:

True/False on equality

static portstr_from_component(container_port, host_port=None, host_ip='0.0.0.0', protocol='tcp')[source]

Port name string from individual components.

Parameters:
  • container_port – Port number visible inside the container
  • host_port – Port number on host container_Port maps to
  • host_ip – Host interface IP host_port maps to
  • protocol – Name of protocol (i.e. tcp or udp)
Returns:

port string

1.11.7   Docker_Daemon Module

Docker Daemon interface helpers and utilities

class dockertest.docker_daemon.ClientBase(uri)[source]

Bases: object

Represents a connection with a single interface to Docker Daemon

Parameters:uri – URI understood handled by interface
interface = None

Interface class/instance to use

get(resource)[source]

Get interface-specific value using interface-specific methods.

Parameters:resource – Opaque value specific to interface.
Returns:Opaque value specific to interface.
static value_to_json(value)[source]

Process value returned by get() into a value_to_json object

Parameters:value – Opaque value returned by get().
Raises:ValueError – When value is invalid/unsupported
Returns:value_to_json opaque-object (impl. specific)
get_json(resource)[source]

Process get(resource) result through value_to_json() method, return json object.

class dockertest.docker_daemon.SocketClient(uri='/var/run/docker.sock')[source]

Bases: dockertest.docker_daemon.ClientBase

Connection to docker daemon through a unix socket

class UHTTPConnection(path='/var/run/docker.sock')[source]

Bases: httplib.HTTPConnection

Subclass of Python library HTTPConnection that uses a unix-domain socket

Parameters:path – Path to the existing unix socket
interface

alias of UHTTPConnection

get(resource)[source]

Get interface-specific value using interface-specific methods.

Parameters:resource – Opaque value specific to interface.
Returns:Opaque value specific to interface.
version()[source]

Return version information as a json object

dockertest.docker_daemon.which_docker()[source]

Returns the name of the currently-running docker systemd service, as a string. This is usually ‘docker’ but could be ‘docker-latest’ or the name of a known docker-as-system-container service.

dockertest.docker_daemon.systemd_action(action)[source]

Run the given systemctl action on the current docker service

dockertest.docker_daemon.stop()[source]

stop the docker daemon

dockertest.docker_daemon.start()[source]

start the docker daemon

dockertest.docker_daemon.restart()[source]

restart the docker daemon

dockertest.docker_daemon.systemd_show(prop)[source]

Runs ‘systemctl show –property=ARG’ for given arg. Returns the value as a string.

dockertest.docker_daemon.pid()[source]

returns the process ID of currently-running docker daemon

dockertest.docker_daemon.cmdline(process_id=None)[source]

Returns the command line (argv) for the given process_id, as obtained from ‘ps’. We don’t use ‘systemctl show’ because that includes unexpanded variables. Return value is a list of strings.

Parameters:process_id – PID whose commandline we read (default: docker daemon)
dockertest.docker_daemon.user_namespaces_enabled()[source]

Returns true if docker daemon is running with user namespaces

dockertest.docker_daemon.user_namespaces_uid()[source]

Returns the subordinate UID used for docker processes.

dockertest.docker_daemon.user_namespaces_gid()[source]

Returns the subordinate GID used for docker processes.

1.11.9   Output Module

API compatibility: on 2016-04-29 output.py got split up into a package with four smaller, more maintainable submodules. This init allows existing clients to continue importing as a single unit.

1.11.10   Xceptions Module

Exception subclasses specific to docker subtests

exception dockertest.xceptions.AutotestError[source]

Bases: exceptions.Exception

Root of most test errors coming from autotest

args
message
exception dockertest.xceptions.DockerCommandError[source]

Bases: exceptions.Exception

Errors coming from within dockercmd module classes

args
message
exception dockertest.xceptions.DockerExecError[source]

Bases: exceptions.Exception

Errors occurring from execution of docker commands

args
message
exception dockertest.xceptions.DockerTestNAError[source]

Bases: exceptions.Exception

Test skip from execution of docker autotest subtest/subsubtest

args
message
exception dockertest.xceptions.DockerTestError[source]

Bases: exceptions.Exception

Code Error in execution of docker autotest subtest/subsubtest

args
message
exception dockertest.xceptions.DockerTestFail[source]

Bases: exceptions.Exception

Test failure in execution of docker autotest subtest/subsubtest

args
message
exception dockertest.xceptions.DockerValueError[source]

Bases: exceptions.ValueError, dockertest.xceptions.AutotestError

args
message
exception dockertest.xceptions.DockerAttributeError[source]

Bases: exceptions.AttributeError, dockertest.xceptions.AutotestError

args
message
exception dockertest.xceptions.DockerKeyError[source]

Bases: exceptions.KeyError, dockertest.xceptions.AutotestError

args
message
exception dockertest.xceptions.DockerOSError[source]

Bases: exceptions.OSError, dockertest.xceptions.AutotestError

args
errno

exception errno

filename

exception filename

message
strerror

exception strerror

exception dockertest.xceptions.DockerIOError[source]

Bases: exceptions.IOError, dockertest.xceptions.AutotestError

args
errno

exception errno

filename

exception filename

message
strerror

exception strerror

exception dockertest.xceptions.DockerConfigError(option, section, msg)[source]

Bases: ConfigParser.InterpolationError, dockertest.xceptions.AutotestError

args
message

Getter for ‘message’; needed only to override deprecation in BaseException.

exception dockertest.xceptions.DockerNotImplementedError[source]

Bases: exceptions.NotImplementedError, dockertest.xceptions.AutotestError

args
message
exception dockertest.xceptions.DockerRuntimeError[source]

Bases: exceptions.RuntimeError, dockertest.xceptions.AutotestError

args
message
exception dockertest.xceptions.DockerVersionError(lib_version=None, config_version=None)[source]

Bases: dockertest.xceptions.DockerValueError

args
message
exception dockertest.xceptions.DockerAutotestVersionError(lib_version=None, config_version=None)[source]

Bases: dockertest.xceptions.DockerVersionError

args
message
exception dockertest.xceptions.DockerOutputError(reason)[source]

Bases: dockertest.xceptions.DockerValueError

args
message
exception dockertest.xceptions.DockerFullNameFormatError(name)[source]

Bases: dockertest.xceptions.DockerValueError

args
message
exception dockertest.xceptions.DockerSubSubtestNAError(child_name)[source]

Bases: dockertest.xceptions.DockerTestNAError

args
message

1.11.11   Sphinx Conf Module

Docker Autotest documentation build configuration file, created by

This file is execfile()d with the current directory set to its containing dir.

Note that not all possible configuration values are present in this autogenerated file.

All configuration values have a default; values that are commented out serve to show the default.

conf.version = '0.8.8'

The documentation version for this instance of the test. It is compared to the API version before every test. This ensures any API changes are also reflected in documentation.

The short X.Y version. This MUST be inside single (“’”) quotes for parsing!!

conf.needs_sphinx = '1.1'

If your documentation needs a minimal Sphinx version, state it here.

conf.templates_path = ['docs_templates']

Add any paths that contain templates here, relative to this directory.

conf.source_suffix = '.rst'

The suffix of source filenames.

conf.source_encoding = 'utf-8-sig'

The encoding of source files.

conf.master_doc = 'index'

The master toctree document.

conf.project = u'Docker Autotest'

General information about the project.

conf.release = '0.8'

The full version, including alpha/beta/rc tags.

conf.language = None

The language for content autogenerated by Sphinx. Refer to documentation for a list of supported languages.

conf.exclude_patterns = ['docs_build', 'config.d', 'README.rst', 'config_custom/README.rst']

List of patterns, relative to source directory, that match files and directories to ignore when looking for source files.

conf.default_role = None

The reST default role (used for this markup: text) to use for all documents.

conf.add_function_parentheses = True

If true, ‘()’ will be appended to :func: etc. cross-reference text.

conf.add_module_names = True

If true, the current module name will be prepended to all description unit titles (such as .. function::).

conf.show_authors = False

If true, sectionauthor and moduleauthor directives will be shown in the output. They are ignored by default.

conf.pygments_style = 'sphinx'

The name of the Pygments (syntax highlighting) style to use.

conf.modindex_common_prefix = []

A list of ignored prefixes for module index sorting.

conf.autodoc_default_flags = ['members', 'undoc-members', 'inherited-members', 'show-inheritance']

Allow using a no- prefix with options

conf.autodoc_member_order = 'bysource'

By default, class/module members are ordered alphabeticaly. However, in this project the layout is all logically ordered

conf.html_title = 'Docker Autotest Documentation'

The name for this set of Sphinx documents. If None, it defaults to “<project> v<release> documentation”.

conf.html_short_title = None

A shorter title for the navigation bar. Default is the same as html_title.

The name of an image file (relative to this directory) to place at the top of the sidebar.

conf.html_favicon = None

The name of an image file (within the static path) to use as favicon of the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 pixels large.

conf.html_last_updated_fmt = '%b %d, %Y'

If not ‘’, a ‘Last updated on:’ timestamp is inserted at every page bottom, using the given strftime format.

conf.html_use_smartypants = True

If true, SmartyPants will be used to convert quotes and dashes to typographically correct entities.

conf.html_sidebars = {'**': ['searchbox.html', 'localtoc.html']}

Custom sidebar templates, maps document names to template names.

conf.html_additional_pages = {}

Additional templates that should be rendered to pages, maps page names to template names.

conf.html_domain_indices = True

If false, no module index is generated.

conf.html_use_index = True

If false, no index is generated.

conf.html_split_index = False

If true, the index is split into individual pages for each letter.

If true, links to the reST sources are added to the pages.

conf.html_show_sphinx = True

If true, “Created using Sphinx” is shown in the HTML footer. Default is True.

If true, “(C) Copyright …” is shown in the HTML footer. Default is True.

conf.html_use_opensearch = ''

If true, an OpenSearch description file will be output, and all pages will contain a <link> tag referring to it. The value of this option must be the base URL from which the finished HTML is served.

conf.html_file_suffix = None

This is the file name suffix for HTML files (e.g. “.xhtml”).

conf.htmlhelp_basename = 'DockerAutotestdoc'

Output file base name for HTML help builder.

conf.latex_documents = [('index', 'DockerAutotest.tex', u'Docker Autotest Documentation', u'Chris Evich', 'manual')]

Grouping the document tree into LaTeX files. List of tuples (source start file, target name, title, author, documentclass [howto/manual]).

The name of an image file (relative to this directory) to place at the top of the title page.

conf.latex_use_parts = False

For “manual” documents, if this is true, then toplevel headings are parts, not chapters.

conf.latex_show_pagerefs = False

If true, show page references after internal links.

conf.latex_show_urls = False

If true, show URL addresses after external links.

conf.latex_appendices = []

Documents to append as an appendix to all manuals.

conf.latex_domain_indices = True

If false, no module index is generated.

conf.man_pages = [('index', 'dockerautotest', u'Docker Autotest Documentation', [u'Chris Evich'], 1)]

One entry per manual page. List of tuples (source start file, name, description, authors, manual section).

conf.man_show_urls = False

If true, show URL addresses after external links.

conf.texinfo_documents = [('index', 'DockerAutotest', u'Docker Autotest Documentation', u'Chris Evich', 'DockerAutotest', 'One line description of project.', 'Miscellaneous')]

Grouping the document tree into Texinfo files. List of tuples (source start file, target name, title, author, dir menu entry, description, category)

conf.texinfo_appendices = []

Documents to append as an appendix to all manuals.

conf.texinfo_domain_indices = True

If false, no module index is generated.

conf.texinfo_show_urls = 'footnote'

How to display URL addresses: ‘footnote’, ‘no’, or ‘inline’.

conf.mock(mod_path)[source]

Recursivly inject tree of mocked modules from entire mod_path

conf.html_theme = 'sphinx_rtd_theme'

The theme to use for HTML and HTML Help pages. See the documentation for a list of builtin themes.

conf.html_theme_options = {}

Theme options are theme-specific and customize the look and feel of a theme further. For a list of options available for each theme, see the documentation.

conf.html_theme_path = ['/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/envs/latest/local/lib/python2.7/site-packages']

Add any paths that contain custom themes here, relative to this directory.

1.11.12   Version Module

Module for standardized API version number processing/checking

Subtest modules code, configuration, and documentation are critical to remain in agreement in order to support use of external/private or customized configurations and tests. Therefor version checking is very important. Each subtest must inherit the default ‘config_version’ option with the version string of the dockertest API it was written against. This may come from config_defaults/defaults.ini or config_custom/defaults.ini if it has been customized. Further, the documentation version in the top-level conf.py module must also match (less the REVIS number).

dockertest.version.MAJOR = 0

Major API version number, as an integer (range 0-255).

dockertest.version.MINOR = 8

Minor API version number, as an integer (range 0-255).

dockertest.version.REVIS = 8

API Revision number, as an integer (range 0-255). Not significant! for version comparisons. e.g. 0.0.1 == 0.0.2 != 0.2.2

dockertest.version.FMTSTRING = '%d.%d.%d'

String format representation for MAJOR, MINOR, and REVIS

dockertest.version.STRING = '0.8.8'

String representation of MAJOR, MINOR, and REVIS using FMTSTRING

dockertest.version.NOVERSIONCHECK = '@!NOVERSIONCHECK!@'

If no subtest configuration could be loaded, use this value to signal version checking is impossible

dockertest.version.AUTOTESTVERSION = '0.16.0'

If by chance no autotest_version is set, use this value

dockertest.version.MYDIR = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest/dockertest'

Absolute path to directory containing this module

dockertest.version.PARENTDIR = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest'

Parent directory of directory containing this module

dockertest.version.str2int(version_string)[source]

Convert an ‘x.y.z’ string into binary form

dockertest.version.int2str(version_int)[source]

Convert a binary form into an ‘x.y.z’ string

dockertest.version.compare(lhs, rhs)[source]

Compare lhs version to rhs version (ignoring revision number)

Parameters:
  • lhs – Left-hand-side, string, list, tuple, or number
  • rhs – Right-hand-side, same type as lhs
dockertest.version.get_doc_version()[source]

Parse version string from conf.py module w/o importing it.

Returns:None on error, string version number of success
dockertest.version.check_doc_version()[source]

Compare Dockertest API version to documentation version, fail if greater

dockertest.version.check_version(config_section)[source]

Simple version check that config version == library version

Note: Ignores REVIS, only MAJOR/MINOR compared.

Raises:DockerVersionError – if Major/Minor don’t match
dockertest.version.check_autotest_version(config_section, installed_version)[source]

Raise exception if configured autotest_version < installed_version

1.11.13   Configuration Module

Extension of standard ConfigParser.SafeConfigParser abstracting section names.

The Config class is the main thing here intended for consumption. Possibly the none_if_empty function as well. Everything else is available, and unit-tested but not intended for wide-spread general use.

dockertest.config.MYDIR = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest/dockertest'

Absolute path to directory containing this module

dockertest.config.PARENTDIR = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest'

Parent directory of directory containing this module

dockertest.config.CONFIGDEFAULT = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest/config_defaults'

Directory path relative to PARENTDIR containing default config files

dockertest.config.CONFIGCUSTOMS = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest/config_custom'

Directory path relative to PARENTDIR containing customized config files.

dockertest.config.DEFAULTSUBDIR = 'defaults'

Durectiry path relative to CONFIGDIR containing default config files

dockertest.config.DEFAULTSFILE = 'defaults.ini'

Name of file holding special DEFAULTS section and options

dockertest.config.CONTROLFILE = 'control.ini'

Name of file holding special control script options

class dockertest.config.ConfigSection(defaults, section)[source]

Bases: object

Wraps SafeConfigParser with static section handling

Parameters:
  • defaults – dict-like containing default keys/values
  • section – name of section to initially bind to
Note:

Not an exact interface reproduction, some functionality left out!

defaults()[source]

Returns dictionary of default options

sections()[source]

Returns a list containing this instances section-name

add_section(section)[source]

Not written, do not use!

Raises:NotImplementedError – DO NOT USE!
has_section(section)[source]

Returns True if instance-section == section

Parameters:section – Name of section to check.
Returns:True/False if section exists.
options()[source]

Returns dictionary of all options keys/values

has_option(option)[source]

Returns True if key-named option exists

Parameters:option – Name of the option (key) to check.
Returns:True/False if option (key) exists.
read(filenames)[source]

Replace current contents with content from filename(s)/list

Parameters:filenames – Same as for SafeConfigParser.read method
Returns:List of successfully parsed filenames.
readfp(fp, filename=None)[source]

Replace current contents with content from file

Parameters:
  • fp – Same as for SafeConfigParser.readfp method
  • filename – Same as for SafeConfigParser.readfp method
Returns:

Same as for SafeConfigParser.readfp method

get(option)[source]

Return value assigned to key named option

Parameters:option – Name of the option (key) to check.
Returns:The value assigned to option
getint(option)[source]

Convert/Return value assigned to key named option

Parameters:option – Name of the option (key) to check.
Returns:Value assigned to option converted to an integer.
getfloat(option)[source]

Convert/Return value assigned to key named option

Parameters:option – Name of the option (key) to check.
Returns:Value assigned to option converted to a float.
getboolean(option)[source]

Convert/Return value assigned to key named option

Parameters:option – Name of the option (key) to check.
Returns:True: if value is yes, true. False if no or false.
set(option, value)[source]

Set value assigned to key named option

Parameters:
  • option – Name of the option (key) to set.
  • value – Content to assign to option.
Returns:

Same as for SafeConfigParser.set method.

write(fileobject)[source]

Overwrite current contents of fileobject.name

merge_write(fileobject)[source]

Update section contents of fileobject.name by section only.

remove_option(option)[source]

Remove option-key option

remove_section()[source]

Not implemented, do not use!

Raises:NotImplementedError – DO NOT USE!
items()[source]

Return list of key/value tuples for contents

class dockertest.config.ConfigDict(section, defaults=None, *args, **dargs)[source]

Bases: _abcoll.MutableMapping

Dict-like ConfigSection interface, SafeConfigParser facade.

Parameters:
  • section – Section name string to represent
  • defaults – dict-like of default parameters (lower-case keys)
  • *args – Passed through to dict-like super-class.
  • **dargs – Passed through to dict-like super-class.
get_other(option, other=None)[source]

Regular get() is non-standard, this is defaulting version

read(filelike)[source]

Load configuration from file-like object filelike

static write(filelike)[source]

Raise an IOError exception, instance is read-only

clear() → None. Remove all items from D.
get(k[, d]) → D[k] if k in D, else d. d defaults to None.
items() → list of D's (key, value) pairs, as 2-tuples
iteritems() → an iterator over the (key, value) items of D
iterkeys() → an iterator over the keys of D
itervalues() → an iterator over the values of D
keys() → list of D's keys
pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → list of D's values
class dockertest.config.Config[source]

Bases: dict

Global dict-like of dict-like(s) per section with defaulting values.

Parameters:
  • *args – Same as built-in python dict() params.
  • **dargs – Same as built-in python dict() params.
Returns:

Regular ‘ole python dictionary of global config also as python dictionaries (cached on first load)

defaults_ = None

Public instance attribute cache of defaults parsing w/ non-clashing name

configs_ = None

Public instance attribute cache of configs parsing w/ non-clashing name

prepdict = None

prepared dict for deep copy.

defaults

Read-only cached defaults.ini DEFAULTS section options as a dict.

static load_config_sec(newcd, section, configs_dict)[source]

Load parsed contents and process __example__ options.

Parameters:
  • newcd – New ConfigDict instance loaded with content
  • section – Name of section to process.
  • configs_dict – Destination dict-like to store result
  • defaults_dict – Dict-like containing all default option/values.
static load_config_dir(dirpath, filenames, configs_dict, defaults_dict)[source]

Populate configs_dict with ConfigDict() for sections found in filenames

Parameters:
  • dirpath – Path to directory of ini files to load
  • filenames – List of filenames in directory.
  • configs_dict – Destination dict-like to store result
  • defaults_dict – Dict-like containing all default option/values.
configs

Read-only cached dict of ConfigDict’s by section, aggregating all ini’s

copy()[source]

Return deep-copy/export as a regular dict containing regular dicts

clear() → None. Remove all items from D.
fromkeys(S[, v]) → New dict with keys from S and values equal to v.

v defaults to None.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.
has_key(k) → True if D has a key k, else False
items() → list of D's (key, value) pairs, as 2-tuples
iteritems() → an iterator over the (key, value) items of D
iterkeys() → an iterator over the keys of D
itervalues() → an iterator over the values of D
keys() → list of D's keys
pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem() → (k, v), remove and return some (key, value) pair as a

2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) → None. Update D from dict/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → list of D's values
viewitems() → a set-like object providing a view on D's items
viewkeys() → a set-like object providing a view on D's keys
viewvalues() → an object providing a view on D's values
dockertest.config.get_as_list(value, sep=', ', omit_empty=True)[source]

Return config value as list separated by sep.

Parameters:
  • value – Some string or string-like iterable to parse
  • sep – The character or item to seperate value around
  • omit_empty – When true, skip items that evaluate false
Returns:

List of items from value excluding sep

dockertest.config.none_if_empty(dict_like, key_name=None)[source]

Set empty strings in dict-like to None, if not specific key_name.

Parameters:
  • dict_like – Instance with dict-like interface to examine
  • key_name – Optional single key to check, doesn’t need to exist.

1.11.14   Documentation Module

Documentation building facilities, depends on external Docutils and documentation_deps

This module shouldn’t use any other dockertest modules it may create circular-dependencies. Some familiarity with the docutils documentation is assumed.

class dockertest.documentation.DefaultDoc[source]

Bases: dockertest.docdeps.DocBase

Specialized DocBase singleton that only handles Default configuration

Parameters:ini_path – Optional, full or absolute path to defaults file.
ini_path = '/home/docs/checkouts/readthedocs.org/user_builds/docker-autotest/checkouts/latest/config_defaults/defaults.ini'

Path to ini file, None if not parsed

docitems = None

Contains a tuple of DocItems parsed or None if not

item_fmt = '* ``%(option)s`` : (``%(value)s``) %(desc)s'

String format for each option, desc, value item in a DocItem

default_base_path = '.'

Default base-path to use for all methods requiring one.

singleton = * ``preserve_fqins`` : (``registry.access.redhat.com/rhel7/rhel:latest``) CSV of possibly existing **full** image names to preserve. * ``autotest_version`` : (``@!NOVERSIONCHECK!@``) Autotest version dependency for framework (or override for individual tests) * ``wait_ready`` : (``60``) Default timeout (seconds) for `dockercmd.wait_for_ready()` * ``verify_enforcing`` : (``yes``) Verify the system has SELinux set to enforcing mode. * ``docker_registry_host`` : (``registry.access.redhat.com``) remote components (host:port) * ``docker_repo_tag`` : (``latest``) Default image settings for testing (blank if not applicable) * ``docker_path`` : (``/usr/bin/docker``) Docker default options (before subcommand) * ``docker_repo_name`` : (``rhel``) Default registry settings for testing (blank if not applicable) * ``remove_after_test`` : (``yes``) Attempt to remove all created containers/images during test * ``docker_registry_user`` : (``rhel7``) remote components (username) * ``docker_options`` : (``<None>``) Global docker client command options to use (CSV) * ``config_version`` : (``0.8.8``) Undocumented Option, please fix! * ``docker_timeout`` : (``300.0``) Max runtime in seconds for any docker command (auto-converts to float) * ``__example__`` : (``docker_repo_name, docker_repo_tag, preserve_fqins``) CSV list of options recommended for customization. Tests will issue warnings when left unmodified from default values. **This option is compounded from defaults, not overwritten!** * ``preserve_cnames`` : (``<None>``) CSV of possibly existing **full** container names to preserve.

Class-reference to DefaultDoc instance singleton, if not None.

fmt

Represent entire contents of defaults configuration section.

get_default(option)[source]

Returns docitem for Default option, None if it doesn’t exist

static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
class dockertest.documentation.ConfigDoc(ini_path)[source]

Bases: dockertest.docdeps.DocBase

Specialized DocBase to handle single configuration section + subsection

Parameters:ini_path – Full or absolute path, including filename of config. file
item_fmt = '* ``%(option)s`` : (``%(value)s``) %(desc)s'

String format for each non-default option, desc, value item in a DocItem

def_item_fmt = '* ``%(option)s`` : ``%(value)s`` :ref:`Overrides default value <default configuration options>`: ``%(def_value)s``'

String format for any overridden default option, w/ cross-reference

inherit_fmt = '* ``%(option)s`` : (``%(value)s``) %(desc)s - *inherited*'

String format for any subtest options inherited by sub-subtest

default_base_path = '.'

Default base-path to use for all methods requiring one.

ini_path = None

Path to ini file, None if not parsed

docitems = None

Contains a tuple of DocItems parsed or None if not

classmethod new_by_name(name, base_path=None)[source]

Return instance by searching for name under path

Parameters:
  • name – Subtest name or ‘DEFAULTS’ (NOT sub-subtest names!)
  • base_path – Relative/Absolute path where config_defaults directory can be found. Uses cls.default_base_path if None.
Returns:

New instance of this class

Raises:

ValueError – If subtest name not found.

fmt

Represent entire contents of configuration section (if any)

classmethod ini_filenames(base_path=None)[source]

Return an iterable of absolute paths to all ini files found.

Parameters:base_path – Relative/Absolute path where subtests and config_defaults directories can be found. Uses cls.default_base_path if None.
static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
class dockertest.documentation.SubtestDoc(subtest_path)[source]

Bases: dockertest.docdeps.DocBase

Specialized DocBase to handle a single subtest’s documentation.

Parameters:subtest_path – Full or absolute path, including filename of a subtest module
NoINIString = '\n:Note: Subtest does not have any default configuration\n'

String to use for tests w/o any configuration

ConfigDocClass

Class to use for parsing documentation

alias of ConfigDoc

default_base_path = '.'

Default base-path to use for all methods requiring one.

tld_name = 'subtests'

Top level directory name base and name postfix for instances.

classmethod new_by_name(name, base_path=None)[source]

Return instance by searching for name under path

Parameters:
  • name – Docker autotest standardized name for a subtest
  • base_path – Relative/Absolute path where subtests and config_defaults directories can be found. cls.default_base_path used if None.
Returns:

New instance of this class

Raises:

ValueError – If subtest name not found.

static docstring(subtest_path)[source]

Return cleaned docstring from loading module at subtest_path

Parameters:subtest_path – Relative/absolute path to a subtest module
Returns:Python-parsed docstring from module file
classmethod name(subtest_path)[source]

Return the standardized name for subtest at subtest_path

Parameters:subtest_path – Relative or absolute filename of subtest module
Returns:Standardized docker autotest subtest name
classmethod module_filenames(base_path=None)[source]

Return an iterable of absolute paths to all subtest modules found.

Parameters:base_path – Relative/Absolute path where subtests and config_defaults directories can be found. Uses cls.default_base_path if None.
html(input_string)[source]

(conv method) Render as html snippet.

Parameters:
  • input_string – RST-formatted string
  • visitor – Optional Visitor class (from docutils.nodes.NodeVisitor)
html_summary(input_string)[source]

(conv method) Render as html snippet, w/ only summary info.

Parameters:input_string – RST-formatted string
rst_summary(input_string)[source]

(conv method) Render as RST w/ only summary info.

Parameters:input_string – RST-formatted string
static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
class dockertest.documentation.PretestDoc(subtest_path)[source]

Bases: dockertest.documentation.SubtestDoc

Subclass to represent pretest module documentation

tld_name = 'pretests'

Top level directory name base and name postfix for instances.

ConfigDocClass

alias of ConfigDoc

static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static docstring(subtest_path)

Return cleaned docstring from loading module at subtest_path

Parameters:subtest_path – Relative/absolute path to a subtest module
Returns:Python-parsed docstring from module file
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

html(input_string)

(conv method) Render as html snippet.

Parameters:
  • input_string – RST-formatted string
  • visitor – Optional Visitor class (from docutils.nodes.NodeVisitor)
html_summary(input_string)

(conv method) Render as html snippet, w/ only summary info.

Parameters:input_string – RST-formatted string
classmethod module_filenames(base_path=None)

Return an iterable of absolute paths to all subtest modules found.

Parameters:base_path – Relative/Absolute path where subtests and config_defaults directories can be found. Uses cls.default_base_path if None.
classmethod name(subtest_path)

Return the standardized name for subtest at subtest_path

Parameters:subtest_path – Relative or absolute filename of subtest module
Returns:Standardized docker autotest subtest name
classmethod new_by_name(name, base_path=None)

Return instance by searching for name under path

Parameters:
  • name – Docker autotest standardized name for a subtest
  • base_path – Relative/Absolute path where subtests and config_defaults directories can be found. cls.default_base_path used if None.
Returns:

New instance of this class

Raises:

ValueError – If subtest name not found.

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
rst_summary(input_string)

(conv method) Render as RST w/ only summary info.

Parameters:input_string – RST-formatted string
class dockertest.documentation.IntratestDoc(subtest_path)[source]

Bases: dockertest.documentation.SubtestDoc

Subclass to represent intratest module documentation

tld_name = 'intratests'

Top level directory name base and name postfix for instances.

ConfigDocClass

alias of ConfigDoc

static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static docstring(subtest_path)

Return cleaned docstring from loading module at subtest_path

Parameters:subtest_path – Relative/absolute path to a subtest module
Returns:Python-parsed docstring from module file
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

html(input_string)

(conv method) Render as html snippet.

Parameters:
  • input_string – RST-formatted string
  • visitor – Optional Visitor class (from docutils.nodes.NodeVisitor)
html_summary(input_string)

(conv method) Render as html snippet, w/ only summary info.

Parameters:input_string – RST-formatted string
classmethod module_filenames(base_path=None)

Return an iterable of absolute paths to all subtest modules found.

Parameters:base_path – Relative/Absolute path where subtests and config_defaults directories can be found. Uses cls.default_base_path if None.
classmethod name(subtest_path)

Return the standardized name for subtest at subtest_path

Parameters:subtest_path – Relative or absolute filename of subtest module
Returns:Standardized docker autotest subtest name
classmethod new_by_name(name, base_path=None)

Return instance by searching for name under path

Parameters:
  • name – Docker autotest standardized name for a subtest
  • base_path – Relative/Absolute path where subtests and config_defaults directories can be found. cls.default_base_path used if None.
Returns:

New instance of this class

Raises:

ValueError – If subtest name not found.

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
rst_summary(input_string)

(conv method) Render as RST w/ only summary info.

Parameters:input_string – RST-formatted string
class dockertest.documentation.PosttestDoc(subtest_path)[source]

Bases: dockertest.documentation.SubtestDoc

Subclass to represent posttest module documentation

tld_name = 'posttests'

Top level directory name base and name postfix for instances.

ConfigDocClass

alias of ConfigDoc

static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static docstring(subtest_path)

Return cleaned docstring from loading module at subtest_path

Parameters:subtest_path – Relative/absolute path to a subtest module
Returns:Python-parsed docstring from module file
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

html(input_string)

(conv method) Render as html snippet.

Parameters:
  • input_string – RST-formatted string
  • visitor – Optional Visitor class (from docutils.nodes.NodeVisitor)
html_summary(input_string)

(conv method) Render as html snippet, w/ only summary info.

Parameters:input_string – RST-formatted string
classmethod module_filenames(base_path=None)

Return an iterable of absolute paths to all subtest modules found.

Parameters:base_path – Relative/Absolute path where subtests and config_defaults directories can be found. Uses cls.default_base_path if None.
classmethod name(subtest_path)

Return the standardized name for subtest at subtest_path

Parameters:subtest_path – Relative or absolute filename of subtest module
Returns:Standardized docker autotest subtest name
classmethod new_by_name(name, base_path=None)

Return instance by searching for name under path

Parameters:
  • name – Docker autotest standardized name for a subtest
  • base_path – Relative/Absolute path where subtests and config_defaults directories can be found. cls.default_base_path used if None.
Returns:

New instance of this class

Raises:

ValueError – If subtest name not found.

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
rst_summary(input_string)

(conv method) Render as RST w/ only summary info.

Parameters:input_string – RST-formatted string
class dockertest.documentation.SubtestDocs(base_path=None, exclude=None, subtestdocclass=None, contents=True)[source]

Bases: dockertest.docdeps.DocBase

Combined output from multiple SubtestDoc instances

Parameters:
  • base_path – Relative/Absolute path where searching should begin. Uses self.default_base_path if None.
  • exclude – Customized list of subtests to exclude, None for default
  • SubtestDocClass – Alternate class to use, None for SubtestDoc
  • contents – True to prefix with RST ...contents:: block.
default_base_path = '.'

Default base path to use if base_path is None

contents = '.. contents::\n :depth: 1\n :local:\n\n'

Default contents block to include before all other sections

base_path = None

Absolute path where searching should begin, None if not appropriate

static conv(input_string)

Perform any final output conversion needed after substitution. (nothing by default)

static do_sub_str(input_string, dct)

Substitute in sub_str from dct keys/values.

Parameters:
  • input_string – Format string to substitute into
  • dct – Substitution dictionary to substutute from
static doctree2html(doctree)

Return rendered html fragment from a doctree instance

static doctree2text(doctree)

Return rendered text string from a doctree instance

get_sub_method_args_dct()

Substitute from calls to sub_method_args returned (key, value)

get_sub_method_dct()

Substitute from sub_method, key/method-name results

static rst2doctree(rst, visitor=None)

Returns a (possibly modified) doctree ready for processing/publishing

Parameters:visitor – optional doctutils.nodes.SparseNodeVisitor subclass
Returns:A doctree instance
exclude = ['example', 'subexample', 'pretest_example', 'intratest_example', 'posttest_example']

Names of any subtests to exclude from documentation

stdc

Class to use for instantiating documentation for each subtest

alias of SubtestDoc

fmt

Dynamically represent DocBase.fmt when referenced

Any test names referenced in exclude will be skipped

sub_str

Dynamically represent DocBase.sub_str when referenced

Any test names referenced in exclude will be skipped

names_filenames

Represent mapping of subtest name to absolute filename

Includes test names referenced in exclude

dockertest.documentation.set_default_base_path(base_path)[source]

Modify all relevant classes default_base_path to base_path

1.12   Hacking

1.12.1   Rolling a new minor version

When the time is right, a large number of PRs have been merged, and you’re in the right mood, it’s time for a new minor release. Assuming the last tagged version was 0.8.6, here are the steps required:

  1. Make sure your master branch exactly matches upstream:

    git remote update git checkout master git reset --hard upstream/master

  2. Create a new branch off master with the next number:

    git checkout -tb 0.8.7

  3. Bump the version number up in three places:

    $EDITOR config_defaults/defaults.ini dockertest/version.py conf.py

  4. Create a release commit:

    git commit -asm "Dockertest Version 0.8.7 (NO API Changes)"

  5. Push to your fork, and open a PR targeting the previous milestone, 0.8.7 in this case.

  6. After PR is merged, switch back to master, update it, and tag the version, and push.

    git remote update git checkout master git reset --hard upstream/master git tag 0.8.7 HEAD git push --tags upstream

  7. Create a new release, using the just tagged version and the closed milestone as the title. e.g. “Dockertest Version 0.8.7 (NO API Changes)”. This will cause github to automatically produce a zip and tarball, with URLs for historical reference or use (i.e. somewhere git is not available).

    For the release notes (big text box under the title), link to the github comparison URL formed by the last two tags with ... in-between (end of the url):

    [Changes](https://github.com/autotest/autotest-docker/compare/0.8.6...0.8.7)

    The actual github comparison page (URL above) can also be used as reference for drafting a release-announcement e-mail, providing brownie-points and karma for all the people who helped out.

  8. Next, close the previous milestone (0.8.7 for this example).

  9. Create the next milestone (next version PRs will be attached to), named “Docker Autotest Version 0.8.8 (NO API Changes)”, add a description and due-date if desired.

  10. Move any currently open PRs the new milestone (right side pane of each PRs page).

1.13   Further Reading