How to Run a Docker Image as a Background Job Over a REST API

One POST starts an image you nominate as a background job and returns a job id. You find out how it ended by polling that id or by having the job call one of your functions when it finishes. There is no worker fleet to keep warm and nothing accrues between jobs — but it is deliberately not a queue, and that is the first thing to design around.

The launch is the whole API

POST /v1/container/{instance} returns as soon as the machine exists. Nothing is streamed back, there is no body to wait on, and the request does not stay open for the length of the work:

const job = await fetch(`${API}/v1/container/jobs`, {
  method: "POST",
  headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
  body: JSON.stringify({
    image: "ghcr.io/me/transcode:v4",
    cmd: ["/bin/transcode", "--input", "blob:9f3c…:7a1b…"],
    env: { QUALITY: "high" },
    size: "medium",
    timeout_ms: 600000,
  }),
}).then((r) => r.json());

// job.job.id — keep it. That is how you find out what happened.

The caps on that body are small on purpose: cmd is at most 64 strings of 4,096 characters each, env at most 32 variables whose names and values together stay under 8,192 bytes, image at most 512 characters, timeout_ms at least 1,000 and never more than the instance's own maximum. Anything bigger than that belongs in a file whose key you pass in cmd or env, not in the launch.

Reads of the job return the same object the launch did:

{ "job": { "id": "j3f9c…", "status": "running", "image": "ghcr.io/me/transcode:v4", "size": "medium",
           "exit_code": null, "timeout_ms": 600000, "error": null,
           "started_at": 1791561600000, "ended_at": null, "cost_usd": 0 } }

running is the only status that changes. It becomes done on a zero exit, failed on a non-zero one or a timeout, or canceled if you stopped it. Sizes are small (1 CPU, 512 MB), medium (2 CPU, 2 GB) and large (4 CPU, 8 GB), and GET /v1/container/{instance}/sizes tells you what a given instance will actually accept before you try.

Polling, or a callback that can arrive twice

Polling GET /v1/container/{instance}/{id} is fine and cheap. The alternative is to set Call on completion on the instance, naming one of your functions, which then receives a POST with x-ae-trigger: container:

{ "job_id": "j3f9c…", "instance": "6b0e…", "status": "done", "exit_code": 0,
  "image": "ghcr.io/me/transcode:v4", "size": "medium",
  "started_at": 1791561600000, "ended_at": 1791561912000, "cost_usd": 0.0123, "error": null }

Two honest details about that callback. It is not signed — you authenticate it by the presence of x-ae-trigger, which the platform strips from every inbound request, so a stranger hitting the function's public URL cannot claim to be a completion. And it is at-least-once: anything other than a 2xx within 30 seconds is retried about once a minute for five attempts, and an attempt that timed out may well have run already. Key whatever it does on job_id. A job ended by deleting its instance sends no callback at all.

The callback fires only after the result has been recorded, so a callback that never succeeds cannot lose the outcome or cause a re-run. The record is still there to poll.

Nothing is captured for you

A job's stdout is readable while it runs and for about a week after, through the logs endpoint or the console. That is for debugging, not for results. There is no disk that survives the job, so whatever the work produces has to be written somewhere before the command exits — a blob store, your datastore, an API of your own.

For the common case, name a blob instance as the container instance's blob store. Every job then starts with two extra variables and needs no key of its own:

AE_BLOB_URL     https://api.altengine.net/v1/blob/{store}
AE_BLOB_TOKEN   a bearer for that store, and nothing else
# in the job: ask for a link, then PUT the bytes straight to storage
curl -sX POST "$AE_BLOB_URL/uploads" \
  -H "authorization: Bearer $AE_BLOB_TOKEN" -H "content-type: application/json" \
  -d '{"name":"out.csv","size":'$(stat -c%s out.csv)',"content_type":"text/csv"}'

That token puts and gets. It cannot delete, cannot publish a file, and cannot list the store, and it expires with the job's own time limit. It is minted by the platform, which means it is not something you can widen or point at a different store — and also that naming a store here delegates it: anything that can launch a job on the instance can write to that store for the life of the job, including the image itself, whoever wrote it.

It is not a queue, and the concurrency cap is the real bound

This is the limit most likely to surprise you. Each instance has a jobs-at-once setting — 2 by default, up to 50 — and a launch past it is refused with 429 RESOURCE_EXHAUSTED, not queued. There is a launch rate limit too, 120 a minute by default, which you can lower but not raise. Backpressure, retry and ordering are yours to own: if you need work to wait its turn, keep the pending items in your datastore and let a scheduled function drain them at the rate the instance will accept.

That same setting is the spend bound, which is worth doing the arithmetic on once. A large slot running flat out all month is roughly $620 of machine time; the default two slots cap a runaway at about $1,250 a month rather than at infinity. The per-job cost ceiling (default $1.00) is checked before the machine starts, against the worst case for the size and timeout asked for, and the refusal tells you a timeout that would fit — but note that a full hour on a large, the longest job the platform allows at all, comes to about $0.87, so the default ceiling admits almost anything. The concurrency cap is what actually holds the line.

One more default that catches everyone on their first launch: the allowed-images list starts empty, and an empty list runs nothing. An entry can be an exact tag, repo:* for any tag of one repository, or * for anything, up to 50 entries. Images referenced by localhost, by IP address, by a private-network name, or from the platform's own registry are refused on every instance regardless.

What a job cannot do

  • No inbound network. Nothing can connect to a job; it has no address to be reached at. This is for work you hand off, not a service you host.
  • No restart. It runs once and disappears. A crash is not retried for you.
  • No end-user credentials. Containers take an organization API key and reject identity tokens. If a page needs to start a job, have it call a function — grant that function container and it gets env.container, so the decision about whether this user may start this job happens in code you wrote:
const job = await env.container.run({ instance: "jobs" }, {
  image: "ghcr.io/me/export:v3",
  cmd: ["/bin/export", "--user", user.uid],
  timeout_ms: 600000,
});

Every limit above still applies to that path — the allowlist, the timeout, the concurrency cap and the cost ceiling are checked before the machine starts, whoever asked. And there is no cron on containers themselves: a nightly job is a scheduled function whose only work is the run call above, which is also why that function stays well inside the 30 seconds a scheduled invocation is waited for.

Housekeeping, so you are not surprised later: finished jobs are kept for 30 days and then removed, logs for about a week. A running job is never removed on age. Names beginning AE_ are reserved in env and rejected rather than quietly ignored.

What it costs, with numbers

Two lines: machine time at $0.02 per 1,000 machine-seconds, and $1.00 per 1,000 jobs started. Size is a multiplier on the seconds — small ×1, medium ×4, large ×12 — so one rate covers every machine.

Work a two-minute medium job through it. Pulling the image, booting and tearing down take roughly ten seconds that you are charged for, so call it 130 seconds × 4 = 520 machine-seconds, about $0.0104, plus $0.001 for the job. A thousand of those in a month is around $11.40; a hundred is $1.14, which the first $3 of usage each month covers entirely. Nothing accrues between jobs, because there is no machine when there is no job.

That ten-second floor is also the honest reason not to use this for small work. If your task finishes in 40 milliseconds, you are paying for ten seconds of startup to do it, and a function — which starts in single-digit milliseconds and bills CPU time rather than wall-clock — is both cheaper and faster. Containers are for work measured in minutes, where ten seconds is a rounding error: a transcode, a large import, a PDF pipeline, a batch that needs a real toolchain. Functions have a CPU budget measured in tens of milliseconds by default and a hard ceiling of a second or two; the moment your work does not fit that, this is the thing it moves to.

Without a payment method on file, an organization's jobs are capped at $3 a month. Each launch reserves the most the job could cost and releases what it did not use when it ends, so a launch that would pass the cap is refused up front with 402 PAYMENT_REQUIRED rather than failing halfway.

The full reference — every endpoint, every instance setting and the exact refusal for each rule — is in the Containers API docs.

Start free in the console

← Back to the blog