Skip to content
GantryCD

Queue Policy (deployment aging)

A stack can declare an optional queue policy that automatically skips stale deployment-lane work. Every setting is off by default (a stack that declares nothing behaves exactly as before). All three are stored per stack and expressed as Go duration strings ("30m", "2h"); the domain type is domain.QueuePolicy (pkg/domain/queue_policy.go), persisted as three nullable whole-second columns on stacks (confirmation_timeout_secs, queued_timeout_secs, supersede_after_secs).

The three settings

  • Confirmation timeout — a deployment left waiting for confirmation longer than this is auto-skipped. The window is measured from the active confirmation stage’s created_at, which is stamped when the deployment enters waiting and re-created on each replan — so replanning resets the timer. It covers both confirmation gates: the plan→apply gate, and the PR-preview gate (stacks.pr_plan_mode = 'manual'). This is the one setting for which plan_unlocked is in scope (see below) — the timeout means “nobody answered the question in time”, and an unanswered PR-preview confirmation is exactly that. Left alone it would sit waiting forever on a pull request nobody is going to review, holding a pending commit status that blocks the merge.
  • Queued timeout — a deployment left queued behind the stack lane longer than this (measured from deployments.created_at) is auto-skipped. A promoted pending run is not aged out here — that is covered by the global RUNNER_STARTUP_DEADLINE reaper.
  • Supersede-after — when a new lane deployment is enqueued, any existing queued or waiting deployment older than this age is auto-skipped, so only the freshest work survives (“only ever deploy the latest commit”). Purely age-based: an operator-prioritized deployment is superseded like any other.

Scope and invariants

  • Only queued and waiting are ever auto-skipped. pending, planning, and applying (in-flight work) are never touched, so auto-skip never races a runner mid-pickup. The batch UPDATE re-asserts the deployment’s status on the live row, so a deployment that concurrently leaves queued/waiting (a confirm or a promote committing in the same instant) is dropped by the read-committed re-check rather than clobbered.
  • plan_unlocked deployments (PR previews, local plans) never take the lane, so they can neither be superseded by a lane enqueue nor age out of a queue they never joined — queued timeout and supersede-after do not apply to them. The confirmation timeout does, because a PR preview held by pr_plan_mode sits on a real confirmation stage (see above). The reaper’s unlock and run-discard CTEs are simply no-ops for it: it holds no lane and, before confirmation, has no run.
  • Separately from queue policy, an unconfirmed PR preview is superseded unconditionally when a new commit on the same pull request mints its replacement (SupersedeUnconfirmedPRPreviews, skipped_reason = superseded). That is a correctness rule, not an aging one — the PR has moved past that commit — so it ignores supersede_after entirely. See docs/reference/deployment_state_machine.md.
  • raw deployments (the (stack, run_command) state-surgery escape hatch) are exempt from queue policy on both sides: a raw command neither collapses the queue nor is collapsed by it, since it is imperative rather than a desired-state deploy. (An explicit user “skip all” can still discard a raw deployment.)
  • Each auto-skip discards the deployment’s non-terminal runs/stages, releases the stack lane if a skipped waiting deployment held it, and ends with PromoteNextOnStack in the same write transaction — the standard lane-release invariant. Only the current stack incarnation is aged.

How it runs

  • Supersede is event-driven: DeploymentService.CreateDeploymentInTx calls DeploymentRepository.SupersedeOlderLaneDeployments right before promoting the new deployment.
  • The two timeouts are handled by the deployment-timeout scheduler job (internal/backend/jobs/deployment_timeout.go, ~1m cadence), which calls SkipQueuedDeploymentsPastTimeout and SkipWaitingDeploymentsPastConfirmationTimeout across all opted-in stacks.

Observability

An automatic skip records a skipped_reason on the deployment (superseded, confirmation_timeout, or queued_timeout; a user-initiated skip records none), surfaced in the deployment detail UI. It also publishes the terminal SCM commit status (inactive) and fires the deployment_auto_skipped (or, for a preview, preview_auto_skipped) notification via the notification-reconcile sweep — user-initiated skips remain silent.