NAME

App::Actions::Runner::Containers - start, check and remove a run's containers

SYNOPSIS

my $containers = App::Actions::Runner::Containers->new(
	docker => $docker,
	expression => $expression,
	project_dir => '/home/me/project',
	run_id => $run_id,
);

my $network = $containers->create_network($job);
my @started;
my $services = $containers->start_services($job, $network, $context, \@started);
push @started, $containers->start_job($job, $network, $env, $context, $image, $temp);
...
$containers->cleanup_job($network, \@started);

Make sure nothing is left when a run fails part-way:

my $ok = eval { run_the_jobs($containers); 1 };
$containers->remove_all unless $keep;
die $@ unless $ok;

List what a run that keeps its containers left behind:

my $left = $containers->remaining;
print "container $_\n" for @{ $left->{containers} };
print "network   $_\n" for @{ $left->{networks} };

DESCRIPTION

In short: this module starts and stops the Docker containers of a run, and explains clearly when one will not start or will not stay up. It is a part of App::Actions::Runner; you rarely use it on its own.

Each job gets its own Docker network, its own service containers, and a long-lived job container that its steps are executed in. This class starts them, checks that they stay up, explains why when they do not, and removes them. It remembers every container and network it created, so the run can remove whatever a failed job left behind.

A service has to stay up. One that stops is reported with the status it exited with and the last lines it printed; one that its restart policy keeps bringing back is reported with the restart count, rather than being taken for a working service because Docker says it is running. A container the daemon refuses to start is reported with the daemon's words and, when they name it, the part of the request at fault: the image, the entrypoint, or a published port.

METHODS

new

Creates the container manager for one run.

ARGUMENTS

docker - the App::Actions::Docker object.
expression - the App::Actions::Expression evaluator.
project_dir - the project, mounted at /github/workspace.
run_id - the run's identifier, part of every resource name.
platform, user - optional; passed to docker run.
quiet, explain, keep - optional flags.

RETURNS

An App::Actions::Runner::Containers object.

SIDE EFFECTS

None.

EXAMPLE

my $containers = App::Actions::Runner::Containers->new(
	docker => App::Actions::Docker->new,
	expression => App::Actions::Expression->new,
	project_dir => '/home/me/project',
	run_id => sprintf('%x', time),
	keep => 1,
);

API SPECIFICATION

INPUT

{
	docker      => { type => 'object', can => 'container_start' },
	expression  => { type => 'object', can => 'interpolate' },
	project_dir => { type => 'string', min => 1 },
	run_id      => { type => 'string', min => 1 },
	platform    => { type => 'string', optional => 1 },
	user        => { type => 'string', optional => 1 },
	quiet       => { type => 'boolean', optional => 1 },
	explain     => { type => 'boolean', optional => 1 },
	keep        => { type => 'boolean', optional => 1 },
}

OUTPUT

{ type => 'object', isa => 'App::Actions::Runner::Containers' }

MESSAGES

+---------------------------+------------------------------+-----------------------------+
| Message                   | Meaning                      | Resolution                  |
+---------------------------+------------------------------+-----------------------------+
| Invalid arguments to      | A collaborator is missing or | Pass the documented objects |
| ...Containers::new: REASON| an option has the wrong type |                             |
+---------------------------+------------------------------+-----------------------------+

name

my $name = $containers->name('svc', $job, 'postgres');

Makes a Docker resource name that is unique within the run and says what it is: action-run-RUN-N-KIND-JOB-INDEX[-SUFFIX], with unsafe characters replaced and cut to 63 characters.

EXAMPLE

$containers->name('net', { base_id => 'test', matrix_index => 0 });
# action-run-6553f1a2-1-net-test-0

API SPECIFICATION

INPUT

{
	kind   => { type => 'string', min => 1 },
	job    => { type => 'hashref' },
	suffix => { type => 'string', optional => 1 },
}

OUTPUT

{ type => 'string', min => 1, max => 63 }

MESSAGES

None.

create_network

my $network = $containers->create_network($job);

Creates the job's own network and remembers it for the cleanup.

EXAMPLE

my $network = $containers->create_network($job);

API SPECIFICATION

INPUT

{ job => { type => 'hashref' } }

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

+-----------------------------+------------------------------+-----------------------------+
| Message                     | Meaning                      | Resolution                  |
+-----------------------------+------------------------------+-----------------------------+
| Failed to create Docker     | docker network create failed | Check the daemon; remove    |
| network 'X' (exit N)        |                              | stale action-run networks   |
+-----------------------------+------------------------------+-----------------------------+

start_services

my $services = $containers->start_services($job, $network, $context, \@started);

Starts the job's service containers on its network, each reachable by its id, and checks that each one stays up.

ARGUMENTS

$job, $network, $context - the job, its network and its expression context.
\@started - the job's container list; each started service is added, so the job's cleanup removes it even if a later one fails.

RETURNS

An array reference of [ id, container name ] pairs, for "check_services".

SIDE EFFECTS

Starts containers; prints an explanation block per service in an explain run.

EXAMPLE

my @started;
my $services = $containers->start_services($job, $network, $context, \@started);

API SPECIFICATION

INPUT

{
	job     => { type => 'hashref' },
	network => { type => 'string', min => 1 },
	context => { type => 'hashref' },
	started => { type => 'arrayref' },
}

OUTPUT

{ type => 'arrayref' }

MESSAGES

+-----------------------------+------------------------------+-----------------------------+
| Message                     | Meaning                      | Resolution                  |
+-----------------------------+------------------------------+-----------------------------+
| Service 'X' must be a       | The service entry is neither | Give it an image            |
| mapping / has no image      | a mapping nor an image name  |                             |
| Failed to start service 'X' | docker run failed            | Check the image and options |
| (exit N)                    |                              |                             |
| Service 'X' did not stay    | It exited after starting     | Read the output quoted      |
| running (exit N); ...       |                              | below the message           |
| Service 'X' is restarting   | Its restart policy keeps     | Read the output quoted      |
| (restart count N); ...      | bringing it back             | below the message           |
| action-run: service 'X'     | Registry credentials are not | Log in with docker login    |
| credentials are not         | passed on (a warning)        |                             |
| supported; ...              |                              |                             |
+-----------------------------+------------------------------+-----------------------------+

PSEUDOCODE

for each service, sorted by id:
	check it has an image, and resolve the image name
	build docker run arguments: options, name, network and alias, user,
		volumes, environment, ports, image
	explain it, in an explain run
	start it and remember it, even if it failed to start
	fail if it did not start, is restarting, or has stopped

check_services

$containers->check_services($services);

Fails if any service is restarting, or dead once its restart policy gave up. Docker reports a container between restart attempts as running, so asking only whether it runs would pass one that is backing off. The job checks its services again when its steps finish, because a service that restarted during the run is not the one the steps talked to.

EXAMPLE

$containers->check_services([ [ 'postgres', $name ] ]);

API SPECIFICATION

INPUT

{ services => { type => 'arrayref' } }

OUTPUT

{ type => 'object', isa => 'App::Actions::Runner::Containers' }

MESSAGES

+-----------------------------+------------------------------+-----------------------------+
| Message                     | Meaning                      | Resolution                  |
+-----------------------------+------------------------------+-----------------------------+
| Service 'X' is restarting / | The container is backing off | Read the output quoted      |
| is dead (restart count N);  | or its policy gave up        | below the message           |
| its last output was: ...    |                              |                             |
+-----------------------------+------------------------------+-----------------------------+

start_job

my $container = $containers->start_job($job, $network, $env, $context, $image, $temp_path);

Starts the job container: the project mounted at /github/workspace, the runner directory at /github/actions-runner, the job's environment, and a shell that idles until the steps are run in it.

ARGUMENTS

$job, $network, $env, $context - the job, its network, its resolved environment and its expression context.
$image - the resolved image.
$temp_path - the job's runner directory on the host.

RETURNS

The container name.

SIDE EFFECTS

Starts a container, which is remembered for the cleanup; removes it again if it fails to start or stops at once.

EXAMPLE

my $container = $containers->start_job($job, $network, $env, $context, 'perl:5.40', $temp);

API SPECIFICATION

INPUT

{
	job       => { type => 'hashref' },
	network   => { type => 'string', min => 1 },
	env       => { type => 'hashref' },
	context   => { type => 'hashref' },
	image     => { type => 'string', min => 1 },
	temp_path => { type => 'string', min => 1 },
}

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

+-----------------------------+------------------------------+-----------------------------+
| Message                     | Meaning                      | Resolution                  |
+-----------------------------+------------------------------+-----------------------------+
| Job container 'X' could not | The daemon refused it        | Fix what the quoted daemon  |
| be started (exit N); at     |                              | message names               |
| fault: ...; the daemon said:|                              |                             |
| Job container 'X' did not   | /bin/sh is missing or exited | Use an image with a shell   |
| stay running (exit N); ...  |                              |                             |
+-----------------------------+------------------------------+-----------------------------+

PSEUDOCODE

name the container
build docker run arguments: the job's container options, name, network,
	hostname, user, the workspace and runner mounts, the job's volumes and
	ports, the working directory, the environment, and an idle shell
explain it, in an explain run
start it, capturing what the daemon says
if the start failed, remove it and report the daemon's words
if it is not running, read its output, remove it and report that

inspect

my $facts = $containers->inspect($container);

Asks a running job container what the steps will need to know: its PATH, with the runner's apt shims in front (and the sudo shim when the image has no sudo and the job runs as root), and whether it has bash and node.

EXAMPLE

my $facts = $containers->inspect($container);
print "bash is missing\n" unless $facts->{has_bash};

API SPECIFICATION

INPUT

{ container => { type => 'string', min => 1 } }

OUTPUT

{
	type => 'hashref',
	schema => {
		path     => { type => 'string', min => 1 },
		has_bash => { type => 'boolean' },
		has_node => { type => 'boolean' },
	},
}

MESSAGES

None.

run_action

my $rc = $containers->run_action(%args);

Runs a docker:// action: a container of its own, removed when it exits, on the job's network, with the workspace and runner directory mounted and the step's environment.

ARGUMENTS

image, network, workdir, temp_path - where and what to run.
env - the step's environment.
entrypoint - optional; replaces the image's.
arguments - optional; an array reference of command arguments.
title - the name used in an explain run.

RETURNS

The exit status of the container.

SIDE EFFECTS

Runs a container; prints an explanation block in an explain run.

EXAMPLE

my $rc = $containers->run_action(
	image => 'alpine:3.20', network => $network, workdir => '/github/workspace',
	temp_path => $temp, env => \%env, arguments => [ 'echo', 'hi' ],
	title => "Action 'docker://alpine:3.20' in job 'test'",
);

API SPECIFICATION

INPUT

{
	image      => { type => 'string', min => 1 },
	network    => { type => 'string', min => 1 },
	workdir    => { type => 'string', min => 1 },
	temp_path  => { type => 'string', min => 1 },
	env        => { type => 'hashref' },
	entrypoint => { type => 'string', optional => 1 },
	arguments  => { type => 'arrayref', optional => 1 },
	title      => { type => 'string' },
}

OUTPUT

{ type => 'integer' }

MESSAGES

+---------------------------+------------------------------+-----------------------------+
| Message                   | Meaning                      | Resolution                  |
+---------------------------+------------------------------+-----------------------------+
| Invalid arguments to      | A required argument is       | Pass the documented         |
| ...run_action: REASON     | missing                      | arguments                   |
+---------------------------+------------------------------+-----------------------------+

cleanup_job

$containers->cleanup_job($network, \@started);

Removes a finished job's containers, newest first, and its network, unless the run keeps them. Returns the object.

EXAMPLE

$containers->cleanup_job($network, \@started);

API SPECIFICATION

INPUT

{
	network    => { type => 'string', min => 1 },
	containers => { type => 'arrayref' },
}

OUTPUT

{ type => 'object', isa => 'App::Actions::Runner::Containers' }

MESSAGES

None; a removal that fails is left to the run's final cleanup.

remaining

Returns { containers => [...], networks => [...] }: what the run created and has not removed. A run that keeps its resources lists these.

EXAMPLE

print "  container $_\n" for @{ $containers->remaining->{containers} };

API SPECIFICATION

INPUT

{}

OUTPUT

{ type => 'hashref' }

MESSAGES

None.

remove_all

Removes every container and network the run created and has not removed, newest first. Returns the object.

EXAMPLE

$containers->remove_all unless $keep;

API SPECIFICATION

INPUT

{}

OUTPUT

{ type => 'object', isa => 'App::Actions::Runner::Containers' }

MESSAGES

None.

LIMITATIONS

Service credentials are not used: the local Docker login is.
A container that a parallel job started outlives the run when that job's process is killed, because only the process that started a container knows its name.
The fault named in a start failure comes from matching Docker's English messages. A daemon that reports in another language, or a future Docker that rewords them, gets the quotation without a verdict.

COMMON PITFALLS

Always call "remove_all" at the end of a run, even after an error, unless the run keeps its containers. Otherwise containers and networks are left on the computer.
Pass the same \@started list to the start and clean-up calls. "cleanup_job" removes what is in that list, newest first.
A service must keep running. A database image that exits at once (for example because a required password variable is missing) fails the job, with its last log lines in the message.
Service ports are not published. Steps reach a service by its id on the job's network (postgres:5432), not on localhost. Only ports: mappings in the workflow publish ports on your computer.
Container options from the workflow are passed to Docker as they are. container.options and container.volumes can mount any folder of your computer. Run only workflows you trust.
Names are cut to 63 characters. Very long job ids are shortened, so do not rely on the full id being in the name.

ENCODING

Container and network names are made ASCII: unsafe characters in job and service ids are replaced. Image names are ASCII, as Docker requires. Environment values and options come from the workflow as Perl character strings and are passed to Docker as UTF-8 bytes, so non-ASCII text and emoji work in values. The project path is used as bytes. The log lines quoted in a failure message are what the container printed, as bytes.

AUTHOR

Nigel Horne, njh@nigelhorne.com

FORMAL SPECIFICATION

This section describes each routine in the Z notation, a precise mathematical language for describing programs. You do not need to read it to use the module: everything here is also said in plain words in the sections above. It is for readers who want to check exactly what a routine promises.

How to read it: a box names a routine. The lines above the middle rule declare the values it works with. A name ending in ? is an input, and a name ending in ! is an output. ΞState means the routine changes nothing, and ΔState means it can change the state. The lines below the rule are the conditions that are always true when the routine returns.

new

┌─ Containers ──────────────────────────────────────
│ containers : seq NAME     -- started and not yet removed
│ networks   : seq NAME     -- created and not yet removed
│ counter    : ℕ           -- resource names issued
├───────────────────────────────────────────────────
│ containers and networks have no duplicates
└───────────────────────────────────────────────────

name

┌─ Name ────────────────────────────────────────────
│ ΔContainers
│ kind?, suffix? : seq CHAR
│ name!          : seq CHAR
├───────────────────────────────────────────────────
│ counter' = counter + 1
│ #name! ≤ 63 ∧ name! ¬in; issued
└───────────────────────────────────────────────────

create_network

┌─ CreateNetwork ───────────────────────────────────
│ ΔContainers
│ job?  : JOB
│ net!  : NAME
├───────────────────────────────────────────────────
│ networks' = networks ⁀ ⟨net!⟩
└───────────────────────────────────────────────────

start_services

┌─ StartServices ───────────────────────────────────
│ ΔContainers
│ job?     : JOB
│ started! : seq (ID × NAME)
├───────────────────────────────────────────────────
│ dom started! = sorted(dom job?.services)
│ ∀ (id, n) : ran started! • running(n) ∧ ¬ restarting(n)
│ containers' = containers ⁀ ⟨n : ran started! • second n⟩
└───────────────────────────────────────────────────

check_services

┌─ CheckServices ───────────────────────────────────
│ ΞContainers
│ services? : seq (ID × NAME)
├───────────────────────────────────────────────────
│ (∃ (id, n) : ran services? • status(n) ∈ {restarting, dead} ∨ restarts(n) > 0)
│     ⇒ raises Restarting(id)
└───────────────────────────────────────────────────

start_job

┌─ StartJob ────────────────────────────────────────
│ ΔContainers
│ image? : IMAGE
│ c!     : NAME
├───────────────────────────────────────────────────
│ running(c!) ⇒ containers' = containers
│ ¬ started(c!) ∨ ¬ running(c!) ⇒ raises StartFailure ∧ removed(c!)
└───────────────────────────────────────────────────

inspect

┌─ Inspect ─────────────────────────────────────────
│ ΞContainers
│ c?     : NAME
│ facts! : FACTS
├───────────────────────────────────────────────────
│ facts!.path = aptbin ⁀ ":" ⁀ (if root ∧ ¬ has(c?, sudo) then bin ⁀ ":" else ⟨⟩) ⁀ path(c?)
│ facts!.has_bash ⇔ has(c?, bash) ∧ facts!.has_node ⇔ has(c?, node)
└───────────────────────────────────────────────────

run_action

┌─ RunAction ───────────────────────────────────────
│ ΞContainers
│ image?  : IMAGE
│ rc!     : ℤ
├───────────────────────────────────────────────────
│ rc! = exitStatus(docker run --rm ... image? arguments?)
└───────────────────────────────────────────────────

cleanup_job

┌─ CleanupJob ──────────────────────────────────────
│ ΔContainers
│ net?  : NAME
│ cs?   : seq NAME
├───────────────────────────────────────────────────
│ keep ⇒ θContainers' = θContainers
│ ¬ keep ⇒ containers' = containers ∖ ran cs? ∧ networks' = networks ∖ {net?}
└───────────────────────────────────────────────────

remaining

┌─ Remaining ───────────────────────────────────────
│ ΞContainers
│ r! : RESOURCES
├───────────────────────────────────────────────────
│ r!.containers = containers ∧ r!.networks = networks
└───────────────────────────────────────────────────

remove_all

┌─ RemoveAll ───────────────────────────────────────
│ ΔContainers
├───────────────────────────────────────────────────
│ containers' = ⟨⟩ ∧ networks' = ⟨⟩
└───────────────────────────────────────────────────

STATE DIAGRAM

The object keeps a list of the networks and containers the run created and has not yet removed. One job's resources go through these states:

  create_network($job)
        |
        v
+----------------+
| NETWORK MADE   |  recorded for remove_all
+----------------+
        |
        | start_services($job, $network, ...)
        v
+----------------+  daemon refuses (bad image, entrypoint, port):
| SERVICES       |  error quoting the daemon, naming the fault
| STARTING       |--------------------------------------------+
+----------------+                                            |
        |                                                     |
        | check_services: each still running, no restarts     |
        v                                                     |
+----------------+  exited / dead / restarting:               |
| SERVICES UP    |  error with status, restart count and      |
+----------------+  last log lines --------------------------->+
        |                                                     |
        | start_job($job, $network, $env, ...)                |
        v                                                     |
+----------------+  start refused --------------------------->+
| JOB CONTAINER  |                                            |
| RUNNING (idle) |  inspect: PATH, bash?, node? (for Steps)   |
+----------------+                                            |
        |                                                     |
        | steps run (App::Actions::Runner::Steps);            |
        | run_action for a docker:// step (own container,     |
        | removed when it exits); check_services again at end |
        v                                                     v
+----------------+                                   +----------------+
| JOB FINISHED   |                                   | JOB FAILED     |
+----------------+                                   +----------------+
        |                                                     |
        +---------------------------+-------------------------+
                                    |
                cleanup_job($network, \@started)
                                    |
              keep? ---yes---> left running; listed by remaining()
                |
                no
                v
+----------------------------------+
| REMOVED: containers newest first,|
| then the network                 |
+----------------------------------+

At the end of the run, remove_all() removes anything still recorded
(for example after an error or Ctrl-C), newest first.

LICENSE AND COPYRIGHT

This program is released under the GNU General Public License, version 2. See the LICENSE file that comes with this distribution. If you use it, please let me know.

To install App::Actions, copy and paste the appropriate command in to your terminal.

cpanm

cpanm App::Actions

CPAN shell

perl -MCPAN -e shell
install App::Actions

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)