Skip to content
GantryCD

Cross-Stack Dependency Graph

A derived graph of which stacks and resources depend on which others, used to surface circular dependencies across stack boundaries and to drive dependent redeploys. Stored as a single Postgres edge table in the separate gantrycd_explore database, off the critical primary. Circular dependencies are found by an in-process strongly-connected-component (SCC) pass — see Cycle detection at scale for the theory and the benchmarks behind that choice.

  • Repository: internal/backend/repositories/dependency_edge_repo.go — the DependencyEdgeRepository interface + PostgresDependencyEdgeRepository impl.
  • Schema: schema-explore/tbl_dependency_edges.hcl (one row per directed edge).
  • Domain types: pkg/domain/dependency_graph.go; SCC in pkg/domain/dependency_cycles.go.
  • Config: shares the explore database (EXPLORE_DATABASE_URL), which is required — the backend aborts boot without it. Availability is separate: a valid but unreachable explore DB does not block startup, and the graph paths error at query time (see Wiring status).

For a zoomed-out view of how groups of stacks connect (rather than individual stacks/resources), see Stack Group Topology, which aggregates these same edges by a gantrycd:group:<view> label.

  • Discovery (runner-side): internal/runner/discovery/ + the contract in pkg/contracts/dependencies.go.
  • Consumer (backend): DependencyGraphService reconciles after run completion (asynchronously) and serves the per-stack query (GET /api/v1/orgs/{org}/stacks/{stack}/dependencies).

End-to-end flow

  1. Runner scans gantrycd:dependency: comments and uploads the snapshot in the run’s analysis.json (dependencies section).
  2. Enqueue at completion (DependencyGraphService.EnqueueReconcile, called inside the run-completion write transaction): durably enqueues a dependency_graph_reconcile async task (run-keyed) so it commits atomically with run completion and survives a crash. PR-plan / promoted-PR runs are not enqueued. The edge store is never touched on the completion path. The enqueue itself fires the AsyncTaskPendingChannel NOTIFY when it commits, so the reconcile drains in seconds.
  3. Async reconcile (TaskProcessorJobDependencyGraphService.HandleReconcileTaskReconcileFromRun, best-effort): fetches the snapshot, maps it to edges, ReplaceStackEdges, then reports done. It runs on one of the replica’s worker slots (GANTRYCD_TASK_PROCESSOR_CONCURRENCY, default 4) under an exclusive lease, so no two replicas reconcile the same stack at once. Ordering across a stack’s snapshots is not the processor’s job: ReplaceStackEdges carries a generation and drops a snapshot older than the one already published, which is correct across replicas in a way a per-replica FIFO never was. A failure (e.g. the explore DB briefly down) backs the task off for a later retry.
  4. Query API (GetStackDependenciesDependencyGraphService.GetStackGraph): returns the stack’s bounded dependency neighborhood as {nodes, edges} — nodes carry kind/id/stack (grouped into per-stack boxes by the UI) plus in_cycle, and edges reference node keys and carry on_cycle. Cycle membership is decided server-side and stamped on, so the client renders rather than re-deriving it, and every component the neighborhood renders is flagged — not only the one the central stack belongs to. Two reads. The first is still ORG-WIDE, and it is worth saying plainly: the reduced pair set (OrgStackPairs) scans the org’s edges to group them into distinct stack pairs — ~159-205 ms for an org of 1,500,000 edges reducing to 6,000 pairs (measured, index-only). It is what gives the in-memory BFS its adjacency and the Tarjan SCC pass its components, and it is shared with the cycle banner, trigger demotion and topology. What the page no longer does is read the org’s edge list: EdgesAmongStacks fetches resource-level edges for the stacks the BFS reached and nothing else (~5-12 ms for a 17-stack neighborhood in that same org). Results are redacted per viewer (see Authorization & redaction). The cheap cycles-only counterpart (GET …/dependency-cyclesGetStackCycle) backs the stack-view warning banner.
  5. UI — the stack’s “Dependencies” tab (web/src/components/stacks/DependencyGraph.tsx, React Flow + dagre, lazy-loaded). The model is deliberately minimal: each stack is one box, the only things inside a box are its resource rows, and a dependency on a stack as a whole attaches to the box header itself (no stand-in “whole stack” row). Boxes are laid out by their inter-stack edges with each dependency to the left of what depends on it, so an arrow points leftward from a dependent to the dependency it relies on (org-management ← vpc-prod-billing) — the direction a Terraform from_* reference reads. The central stack is highlighted, and a box/row/edge on a cycle is flagged red with the cycles listed as warnings. A stack the viewer cannot read is drawn as a dashed, locked “Restricted stack” placeholder (its rows likewise locked) so the topology — and any cycle through it — stays visible without revealing its identity. The data→graph layout is the pure, unit-tested dependencyGraphLayout.ts.

Rebuild (backstage + periodic sweep)

The per-run reconcile keeps the graph current, but the graph store is derived and can be wiped/recreated. The backstage per-org “Rebuild Explore metadata” action (POST /api/backstage/v1/orgs/{org}/rebuild-metadataBackstageHandlers.RebuildExploreMetadata) requests the org’s Explore rebuilds on demand. It enqueues three independent async tasks — dependency graph, resource index, and ComputeOrgFindings — and returns 202 {"status":"enqueued"}.

It is not a coordinated refresh, and it guarantees less than its name suggests:

  • the three tasks are separately deduplicated, so a diagnostics task already pending can run before the graph and index rebuilds this call enqueued;
  • the processor moves on to diagnostics even if an earlier rebuild failed;
  • no follow-up diagnostics task is scheduled after the rebuilds land.

So it shortens the wait for the next sweep rather than bounding it, and Insights is not current when the call returns. No HTTP endpoint recomputes findings synchronously — the only synchronous caller of ComputeOrgFindings is the sample-dependencies operator CLI (cmd/cli/operator/advanced/sampledeps), which is a seeding tool, not an operator lever for a real org. A single orchestration task running the three in sequence would be both simpler and actually able to promise what the name implies.

The rebuild itself makes the org’s graph self-healing in both directions:

  1. Forward push — re-derive each stack’s edges from its latest successful configured-branch run (RunRepository.LatestAuthoritativeRunsAfter under AuthorityDeclaredSource, best-effort per run). A run that fails to reconcile is logged at error level and skipped, and the cursor moves past it, so one unreadable stack cannot pin the org. The origin filter (DeploymentOriginType.RunsConfiguredBranch: manual, push, dependency) must stay in lockstep with the inline EnqueueReconcile guard — if the sweep admits fewer origins than the enqueue does, it silently reverts the edges the enqueue just wrote, on a 12h cycle.

    Successful is meant literally here, and it is where this authority parts company with the resource index. The index follows a failed apply too, because a partial apply changed real infrastructure (see Resource Explorer). The graph does not: it is derived from the checked-out HCL, which a failed apply did not change — its declaration is identical to the run that planned the same commit — and the run-completion reconcile only fires for successful runs, so admitting the failure here would make the sweep write edges nothing else writes.

  2. Reverse reap — garbage-collect edges owned by stacks that no longer exist in Postgres (StackRepository.ListIDsByOrg is the authoritative live set → DependencyEdgeRepository.ReapStacksNotIn, a DELETE … WHERE dependent_stack_id NOT IN (live)). The push only touches stacks that still produce a run, so it can never remove these on its own; this closes the gap left when a stack delete’s best-effort DeleteStackEdges never ran. A read error fails the slice rather than passing an empty set (which would wipe the graph) — so a transient DB blip can never be read as “no stacks”, and a reap that never succeeds dead-letters instead of reporting a rebuild that did not happen.

    The reap deletes only edges owned by a dead stack (its dependent endpoint). An edge that merely points at a missing stack is a dangling reference, kept on purpose so the explore diagnostics can surface it as an orphan — not reaped.

The rebuild runs unattended as an eventual-consistency backstop. A dependency-graph-rebuild scheduler job fires every 12h and calls DependencyGraphService.EnqueueRebuildAll, which enqueues one org-keyed dependency_graph_rebuild async task per active org — skipping any org whose prior task is still unfinished, which includes one a worker is running on its final attempt (EnqueueOrgTaskIfNotPending evaluates the check inside its own INSERT), so a slow rebuild never piles up across sweeps.

The TaskProcessorJob drains those tasks via HandleRebuildTask, one two-minute slice per claim. A slice pages the org by keyset in the database’s stack-id order (never re-checked in Go — the two collations disagree on hyphenated slugs), writes how far it got into the task’s own payload and reports continue; that write clears the lease and resets the attempts in the same statement, so any slot on any replica takes the next slice. Only the slice that finds the walk exhausted reaps and reports done. The per-stack walk therefore no longer has a whole-org deadline cliff: the 10-minute ceiling bounds one slice, not one org. The final reverse reap remains whole-org; its residual scaling risk is documented in Resource Explorer: Filtering & Scale. An older payload reads as absent and the org restarts from its first stack, which is always safe because a rebuild is idempotent. A newer payload is refused without being rewritten: an older replica cannot safely interpret a cursor shape written during a rolling upgrade. The payload carries a cursor and nothing else — declared edges come from the stack’s own snapshot, so unlike the resource index there is nothing to revisit once other stacks land. What a slice does when a page read fails part-way, and why a lost lease is an error rather than a continue, is the handler contract in Async Task Queue.

This corrects drift the run-completion reconcile missed: the edge store being recreated, a dropped reconcile task, or an orphan left by a failed delete-time cleanup. Like every graph path it needs the explore database; if that database is unreachable the task fails and the queue retries it with backoff.

Lifecycle cleanup

Nothing else reclaims graph data, so deletes prune it directly. The prune is durable: StackService enqueues an explore_purge async task inside the same write transaction as the stack write, then attempts the removal inline as a fast path and dequeues the task only when it succeeds. A purge failure therefore never fails the stack write (the stores are lower-SLA) and never silently leaves rows behind either — the task processor retries it, instead of the rows lingering until the 12h reap. The inline dequeue is fenced on an EMPTY lease, so it can only remove a task no worker has claimed; if one has, that worker owns finishing it. (Enqueueing is a plain INSERT in that same transaction, so if THAT fails the stack write fails with it, as it should.) A task that exhausts its retries dead-letters, and those rows then wait for the reap exactly as they used to.

That matters because leftover rows are not merely stale display data: they still count as a live participant in cross-stack hazard detection, so a stack that no longer exists can close a cycle for a live one — showing a circular dependency nobody earned and demoting that stack’s always triggers to manual, silently stopping its automatic redeploys — and can qualify a cloud identity as a duplicate only one live stack actually holds.

Only the vacate events are durable (delete, rename away). When a stack is created onto a reused slug or renamed into one, the purge clears a PREVIOUS occupant’s rows while the new stack is live, and the stores carry no owner incarnation to tell the two apart — so a retry that landed after the new stack published would delete its data. Those paths stay inline best-effort, and their window closes on the new stack’s first successful run, which replaces the owned rows wholesale. The handler enforces the same rule: it skips (and retires the task) once a live stack holds the id again, mirroring the prefix_delete handler’s storage-key check.

The stores cleared on each event:

  • Stack delete (StackService.DeleteStackDependencyGraphService.RemoveStackDeleteStackEdges): deletes every edge touching the stack at either endpoint. Edges where the stack was only a dependency target then reappear when the owning stacks next reconcile.
  • Org hard-delete (OrganizationService.HardDeleteOrgDependencyGraphService.RemoveOrgDeleteOrg): deletes every edge for the org.

Discovery (runner-side)

The graph is populated from gantrycd:dependency: comments authors write in their OpenTofu code. On every plan/apply/destroy/refresh, once the repo is fetched the ephemeral runner kicks off the scan in a background goroutine so it overlaps the OpenTofu init/plan/apply instead of delaying it; it resolves the comments against the current stack and, at the end of the run, writes the snapshot into the dependencies section of analysis.json next to the run logs (best-effort, like the rest of that object). The backend consumes it and reconciles the stack’s edges via ReplaceStackEdges.

Comment grammar

A gantrycd:dependency: directive behind a # or // line comment (block comments unsupported):

(#|//) gantrycd:dependency: key=value [key=value ...]

Values are bare (run to the next space) or double-quoted with \"/\\ escapes (templates need this). A second #/// after the directive ends it, so // gantrycd:dependency: from_stack=a // note (or a repeated directive after it) keeps only the first part — but a #/// inside a quoted value is kept. Field order is irrelevant.

dependency fields (identifiers are stack IDs and OpenTofu resource addresses):

fieldmeaning
from_stack / from_stack_tpltarget stack ID (exactly one required)
from_resource / from_resource_tplresource in the target stack (optional; absent ⇒ whole stack). May be an output.<name> reference — the companion to the gantrycd_state_outputs data source, for a stack that consumes another’s output. Scopes triggering too: the dependent only reacts when this resource (or output) changed in the applied plan — see Dependent triggers
resourceresource in the current stack that depends (optional; absent ⇒ the whole stack depends)
stack / stack_regexonly apply when the current stack ID matches (optional)
triggerredeploy policy when from_stack successfully applies: always, manual (default), or never — see Dependent triggers

trigger is a closed enum with no _tpl variant by design: a redeploy policy must be static and reproducible, never derived from rendered stack metadata. An invalid value rejects the whole comment like any other field. Two comments that resolve to the same edge but disagree on trigger are de-duplicated by combining their policies (manual > never > always), so file order never picks the winner.

When resource is omitted, the comment adopts the block it sits directly on top of — the next non-blank, non-comment line — so these are equivalent:

// gantrycd:dependency: from_stack=aws-vpc-dev-euw1 resource=module.this

// gantrycd:dependency: from_stack=aws-vpc-dev-euw1
module "this" { ... }

module "x"module.x, resource "t" "n"t.n, data "t" "n"data.t.n. Only that one next line is inspected; if it is not a resource/data/module block (or there is none), the dependent falls back to the stack itself (a stack-to-stack dependency).

*_tpl fields are Go templates over {{ .stack.ID }}, {{ .stack.Name }}, {{ .stack.Labels.<key> }}, with a curated allowlist of pure sprig string functions (no env/now/random/crypto, and no count-based repeat — output must be deterministic and bounded). The current-stack Name/Labels reach the runner via RunContext fields.

resource-identity comments

The same scanner handles a second verb, the configuration-side twin of the gantrycd:resource-identity-id = "null" tag:

(#|//) gantrycd:resource-identity: [resource=<address>] ignore=<true|false>
fieldmeaning
resourceaddress in the current stack: a resource (with or without its instance key), a data source, or a whole module.x. Optional; absent ⇒ the block the comment sits directly on top of. A comment above nothing is an error — there is no whole-stack form (that is default_tags).
ignorerequired. true: gantrycd asserts no cloud identity for the address — a managed resource is never an owner (no duplicate finding, no inferred dependency onto it) and a data source is never a reader (no inferred dependency from it). false: re-asserts inside a broader ignore=true (the most specific covering directive wins).

It exists for the addresses a tag cannot reach: resource types that take no tags, and data sources, whose recorded tags are the object’s, not the author’s. Two directives for one address are an error naming the first. The directives ride in the same dependencies snapshot and the backend applies them to the state projection of the same run — so a run whose configuration could not be scanned at all (nil snapshot) projects its state without them until the next successful scan, and says so in the log. Holding the projection back instead would freeze the stack’s resources and inferred edges over a missing comment section. A promoted pull request’s apply is a real apply: its comments apply to its own state.

Rules and guarantees

  • Best-effort, never blocks the run. A scan/upload failure is logged and swallowed. The channel startDependencyDiscovery returns carries a *contracts.DependencyDiscovery, so “unknown” is representable — which is what the next rule turns on.
  • Reading the configuration and parsing a comment fail differently. If the scan cannot read what the stack declares — ReadDir fails, a file will not open, a line exceeds the 1 MiB scanner limit, or the scan panics — it reports nil, unknown. It learned nothing about the stack, and by the authoritative-snapshot rule below, the subset that happened to parse would delete every edge declared by the files it never read. A comment that was read and rejected is the opposite case: it is a fact about the configuration, so the snapshot stays authoritative and the comment becomes an Errors entry. Keeping a stale trigger=always because an author mistyped one comment is the worse failure, which is why parse errors do not suppress the snapshot.
  • All-or-nothing per comment. A malformed field, unknown key, bad template, or invalid regex makes that comment an Errors entry — never a partial dependency, never a default. A stack/stack_regex that simply doesn’t match the current stack is a silent skip (not an error). Each entry is printed to the run log as file:line: message; the comment text itself is not echoed.
  • Hostile input is rejected, not recorded. Resolved identifiers are charset-validated (stack ID [A-Za-z0-9._-], resource address adds .[]"), so a value carrying shell/template injection, spaces, or control characters becomes an error rather than data. Field length and rendered-output length are capped.
  • No self-references. A directive that resolves to the very stack or resource that declares it is a meaningless self-loop and becomes an Errors entry, so the author sees it. The backend drops any that slip through anyway.
  • Complete snapshot. The contract is uploaded even when empty, so the backend can reconcile a stack down to zero edges.
  • A missing snapshot is not an empty one. ReplaceStackEdges is destructive: it replaces a stack’s edges wholesale. The upload is best-effort, so a transient storage failure is expected — and if that were read as “this stack has no dependencies”, one failed PUT would delete every edge the stack has. So the dependencies section is a nilable pointer: nil (or no analysis.json at all) means unknown and ReconcileFromRun returns without touching the graph; a non-nil section is authoritative even when its list is empty, and that is what clears a stack’s last gantrycd:dependency: comment. The resource index applies the same rule to state_resources. See contracts.RunAnalysis and run_analysis_absence_integration_test.go.

Known limitations (deliberate)

  • Scanning is line-based, so a # gantrycd:dependency: inside an HCL string/heredoc is a false positive. Accepted for simplicity.
  • Scope is the working directory’s top-level .tf/.tofu files only (non-recursive, mirroring how OpenTofu loads a root module). Comments in nested module directories are not scanned — put them in the root module.
  • The injection charset rejects a few legitimate-but-unusual resource addresses — notably index keys containing spaces (data.x["my key"]). This is a deliberate trade of completeness for a tight guard; such a comment becomes an error.

Why a separate, lower-SLA store

This is an analytical feature, not part of the deployment critical path. Treating it as a derived index buys a lot of simplicity:

  • Postgres stays the source of truth. The graph is rebuildable from runs; staleness, downtime, and full recreation are all acceptable. There is no distributed transaction with the primary and no drift problem to police.
  • It runs off the critical DB. The edges live in the gantrycd_explore database (alongside the resource index), not the primary that runs deployments, so the feature carries a smaller SLA without dragging anything else down.
  • The graph is small. Edges track cross-stack coupling — declared comments, plus one edge per cross-stack data source — not the resource count. The SCC pass consumes the graph already reduced to distinct stack PAIRS (OrgStackPairs), which is smaller again: a million resource-level rows between the same handful of stacks is a few thousand pairs. This is what lets cycle detection run in-process in Go rather than needing a graph engine — see Cycle detection at scale for the measurements (a dedicated graph DB was worse at complex cycles, because the deployed engine has no SCC primitive and its closed-walk explodes on tangled graphs).

The seam (swapping stores later)

This follows the normal repository pattern: callers depend on the DependencyEdgeRepository interface; the only store-specific code is PostgresDependencyEdgeRepository. Everything in and out is a pkg/domain type. Like the resource-index repo it owns its explore-database pool rather than taking a caller transaction — the explore database is a distinct connection that core write paths cannot thread their transaction through.

Data model

dependency_edges (schema-explore/tbl_dependency_edges.hcl), one row per directed edge, org-scoped, no foreign keys (a derived, recreatable store):

  • Endpoints are stored structurally: dependent_{kind,id,stack_id} and dependency_{kind,id,stack_id}. kind is stack or resource; a stack node has id == stack_id, a resource node’s id is its OpenTofu address scoped by its owning stack_id (two stacks containing the same address module.this are distinct nodes). GraphEntity.Key() (stack:<id> / resource:<stack>:<id>) is the in-memory node identity used by the SCC/BFS.
  • One trigger column (always | manual | never), defaulting to manual.
  • One discovered boolean marking provenance: true for an edge gantrycd inferred from the cloud objects the stack reads, false for one a human declared (see Inferred dependencies).
  • Primary key is the full edge tuple, ordered so (org_id, dependent_stack_id) is a prefix — that backs the reconcile delete and any per-dependent scan. discovered sits immediately after it, so declared and inferred rows are disjoint sets each writer can delete-and-replace without touching the other’s.
  • Two further indexes: (org_id, dependency_stack_id) backs inbound (dependent-trigger) lookups, and a partial (org_id, dependent_stack_id, dependency_stack_id) WHERE dependent_stack_id <> dependency_stack_id covers the reduced stack-pair read behind cycle detection and topology. Reordering either one is a DROP + CREATE, and Atlas does not emit CONCURRENTLY: the CREATE holds a SHARE lock on dependency_edges for its duration, blocking every reindex still writing edges — including from old pods mid-rollout. On a large explore store, apply it in a quiet window rather than as part of the deploy.
  • CHECKs enforce the kind/trigger enums and the stack-node identity.

Idempotent per-stack reconciliation

ReplaceStackEdges(org, stackID, edges) is the write primitive for declared edges, and the only one. Callers pass the complete declared edge set originating from a stack every time, so repeated calls converge. Its delete is scoped to NOT discovered and its inserts stamp false, so it never touches the inferred rows the state projection owns — and the reverse holds, scoped the other way.

Edge ownership rule: an edge belongs to the stack of its dependent (source) endpoint. Replacing stackID deletes every row whose dependent_stack_id is stackID, then bulk-inserts the supplied edges (deduplicated) with dependent_stack_id bound to stackID — so the rows are always covered by the delete and the result is a pure function of the input. Edges where stackID is only a dependency target are owned by other stacks and left untouched.

One transaction. The delete + insert run in a single write transaction, so a reader never sees the stack mid-rewrite and a crash cannot leave edges deleted-but-not-recreated. A transaction-scoped pg_advisory_xact_lock on (org, stack) serialises concurrent reconciles of the same stack (the task processor can claim two), so they queue rather than racing into a PK conflict.

Invalid edges are dropped, not fatal. The service validates each edge (DependencyEdge.Validate — well-formed endpoints, no self-loop) before the write and drops + logs any that fail, rather than aborting the reconcile — one bad comment can’t freeze a stack’s dependency view.

Cycle detection

A cycle is a stack-level fact. A stack applies atomically, so the deadlock is “A must be applied after B and B after A” — whichever resources carry it. And the resources cannot carry it alone: the hop that closes the loop runs inside a stack and is never an edge. platform reads infra’s cluster while infra reads platform’s role — four distinct resource nodes, two disjoint edges, no resource-level loop, yet neither stack can be deployed first.

So CyclesFromStackPairs (pkg/domain/dependency_cycles.go) works on the graph already collapsed onto owning stacks (StackPairsFromEdges does that for the one caller still holding rows), then decomposes into strongly-connected components (Tarjan): a circular dependency is exactly an SCC of more than one stack. One O(V+E) pass over the org’s edges yields each entangled group once, and a DependencyCycle is simply that group’s sorted StackIDs.

There is deliberately no path. At stack granularity a single loop through a component can omit members that are equally circular — for {A⇄B, A⇄C} any loop names two of the three — so a stack could be told it is in a cycle whose reported path leaves it out. A component of one stack is never a cycle, which is also what makes an intra-stack declaration (legal, and not a deployment-ordering statement) collapse harmlessly instead of reporting its stack as circular.

The per-stack cycle warning reads the graph already reduced to distinct cross-stack pairs (OrgStackPairs) and filters the SCC result to the stack; the org-wide explore diagnostics run the same SCC over the edge set they load for orphans anyway, falling back to the pair read only when that edge set is over budget. There is one algorithm and no depth bound — SCC detects cycles of any length.

Last-known rows remain part of the graph. Trigger reads return freshness once per child and use it only to demote that child’s automation, so an unreadable scan does not change the org’s cycle answer.

Every read is LIMIT-capped so the backend never materialises an unbounded graph: OrgStackPairs at maxOrgStackPairs, OrgEdges (the diagnostics sweep) at maxOrgEdges, EdgesAmongStacks (one page’s neighborhood) at maxNeighborhoodEdges. The last two are both 200,000 and deliberately separate constants — one is about an org, the other about a page, and a single shared number drifts into meaning whichever the reader assumed.

maxOrgStackPairs is now the org-shape ceiling for all four surfaces that ask what depends on what: the cycle banner, trigger demotion, the topology view, and the Dependencies tab’s neighborhood walk. It was raised from 50,000 to 200,000 when the tab moved onto this read, because at the old number an org of 6,000 stacks with 10 declared dependencies each — 60,000 pairs, 60,000 edges — rendered before the move and would have shown one box after it. 200,000 pairs is ~8 MB materialised, and the limit decides whether the question is answered, not what asking costs: that 60,000-pair org groups in ~16 ms at either limit, and 200,000 pairs in ~75 ms. The raise is a strict improvement for the three surfaces that already shared this read.

The Dependencies view is deliberately not bounded by the org’s edge count: it reads the edges among the stacks it draws, so maxNeighborhoodEdges bounds one neighborhood there (and is spent on the owner-side scan, which is the work). An org can hold many times that number of edges and still render each of its pages. Loading the whole org to keep the slice touching one stack is what made a large org serve every stack as a lone box.

The response is capped separately, and it has to be: the read cap bounds what the backend loads, not what it returns. The neighborhood read keeps every edge between the stacks it reached, and edges are resource-level — so a hub topology (every app declaring many resource dependencies on one shared stack) put the org’s whole edge set into one page load. Measured before the caps: a 200k-edge org returned 51 MB and 202k React Flow nodes for a single stack’s view, with only 101 stacks in the component. The dangerous axis is edge multiplicity between few stacks, not stack count, and generated HCL produces exactly that.

capNeighborhoodEdges bounds it at maxResponseEdges, and is a no-op on any neighborhood that already fits — the ordinary org keeps every edge and sees no notice. Over budget it fills fairly: one edge per (dependent, dependency) stack pair first, then widens each pair up to maxPairEdges. Filling greedily in arrival order instead spends the budget on whichever stacks come first and returns nothing for the rest, and since the node set is derived from these edges, a dropped pair is a stack box that silently disappears — including, potentially, the stack whose page it is.

Order is imposed rather than inherited: the edge read has no ORDER BY, so arrival order is heap order and shifts when a stack reconciles and rewrites its rows. Without the sort, two loads of one page could disagree about which stacks exist. Within a pair, edges carrying a non-manual trigger sort first — an always subscription is the one thing a reader cannot infer from the picture.

TotalEdges and TotalStacks are both counted on the UNTRIMMED neighborhood, the same order the diagnostics sweep uses for its hazard sets and for the same reason: a display cap must not be able to change what the totals say. The view reports “showing N of M dependencies”, and when a neighborhood holds more stack pairs than the whole edge budget — the only case where stacks are lost too — it says that as well instead of implying the drawing is complete.

(For the reduced read the cap bounds memory; the ordered index lets the scan stop early once enough distinct prefixes exist, but a duplicate-heavy org under budget still scans its raw edges.) A pathological org over either cap degrades gracefully rather than risking a serving-pod OOM: the per-stack graph serves the central stack only — now reachable only when the ORG’S SHAPE is over maxOrgStackPairs, or that one neighborhood’s own edges are over maxNeighborhoodEdges — and the diagnostics sweep reports each check separately. Cycles and orphans normally come out of the SAME edge read, so an org within maxOrgEdges gets both from one snapshot and they cannot disagree. The reduced pair read is the FALLBACK: only when the edge read is over budget does the sweep take it, so such an org loses orphans (orphans_complete=false) but keeps its cycles — and loses cycles too only when the pair read is over budget as well. The cycle warning reports unknown rather than nothing, and while cycle status cannot be determined, newly resolved, unskipped always rows fail closed to manual — “could not tell” must not read as “no cycle” (see Cycle demotion). See Cycle detection at scale for why this beats the closed-walk it replaced.

Orphan reference detection

The explore diagnostics flag declared dependencies whose endpoint does not currently exist — a cross-store anti-join of graph edges against the live stacks (core PG) and the resource index (explore DB), in ExploreDiagnosticsService.detectOrphans. Declared only, literally: inferred edges are skipped, because a finding here asks someone to correct a reference and nobody wrote those. A whole-stack endpoint is checked against the live/indexed stack sets; a resource endpoint is checked by address, and how that address is resolved depends on its shape:

  • A resource instance (aws_sns_topic.main, module.network.aws_vpc.this) must match an indexed row’s (stack_id, address) exactly (ResourceIndexRepository.ResourceExists).
  • A whole-module reference (module.network, or nested module.network.module.subnet) points at the module unit, not one address. It exists when any indexed resource lives inside that module subtree (ModuleExists — a hierarchical-coverage match: the resource address continues the module path at a . or [ boundary, the same coverage rule the changeset filter uses). domain.ModuleReferencePath classifies the shape: an address that is purely module.<name> steps is a module reference; anything else is not.
  • An output reference (output.service_endpoint) is a stack-level value the resource index never holds (it stores resources only). It is resolved against the producing stack’s applied state instead: an output exists once its stack has indexed resources, so it only reads as missing (always pending — a removed output can’t be detected without output rows to anti-join, and a gone stack is already caught above) while the stack has no applied state yet. domain.IsOutputReference classifies the output.<name> shape.

This distinction is why module.network and output.service_endpoint are not orphans merely because no resource is literally addressed that way. A malformed or type-only address — a bare resource type with no instance name (aws_sns_topic) or a dangling module keyword (...module) — is neither a module nor an output reference, so it keeps exact-match semantics and stays flagged: it is not a real reference. A flagged reference reports its missing_kind ("output" / "module" / "resource" / "stack") so the UI message reads accurately.

Inferred dependencies

Not every edge is declared. When a data source in one stack reads a cloud object a managed resource in another stack owns, gantrycd records the dependency itself — nobody writes a comment, and the two stacks never mention each other in HCL.

# stack "account" owns it
resource "aws_acm_certificate" "cloudfront_wildcard" { ... }

# stack "taloscluster-ovh-eks-1" reads it — and now depends on "account"
data "aws_acm_certificate" "wildcard" { domain = "..." }

The evidence is resource identity: the same globally-anchored value that backs duplicate detection. Two rows describe the same object when their id, realm and resource type all match, so the match key is exactly that, against managed rows of the same org in other stacks. Every matching owner yields its own edge — when two stacks manage one object (a duplicate) the reader really does depend on both, and collapsing that would name a winner the evidence does not.

Only a provider with a mounted identity rule can be inferred from, and today that is AWS alone — resources from any other provider assert no identity, so they never match and never produce an inferred edge.

  • Resource-scoped on both ends. The dependent endpoint is the data source’s address, the dependency endpoint is the owner’s. So the parent’s changeset scopes it exactly like a declared from_resource= does: the dependent is offered a trigger only when the applied plan actually changed the resource it reads (see Dependent triggers).

  • Always manual. An inference never asks for automation; always is only ever a human’s declaration. It does still carry a policy, and manual combines: a declared always on one of the parent’s resources beside an inference on another — both matching the applied changeset — resolves to manual, because manual is the safe middle ground when the evidence disagrees. The row says so, naming the inferred address, rather than going quiet (domain.HeldByInferenceFormat). Declaring a trigger for that resource is the fix.

  • A declaration wins trigger behavior, by address coverage. domain.EffectiveTriggerEdges ignores an inferred manual policy when a declared edge for the same (dependent stack, dependency stack) has a dependency address that covers the inferred one — the same ResourceAddressCoversPath relation changeset filtering uses. So: the same address, an ancestor of it (a declaration on module.certs speaks for module.certs.aws_acm_certificate.this["prod"]), or the dependency stack as a whole. A sibling resource does not override it; that would let an unrelated declaration change this dependency’s trigger behavior.

    This precedence is applied only inside trigger resolution, before policy combination and changeset filtering. It is not applied to graph or diagnostic reads: the inferred dependency remains a fact even when a broader declaration governs how it triggers.

    Storage retains the declared and inferred rows independently. The Dependencies response (MergeDependencyGraphEdges) unions rows only when their endpoints are exactly equal and reports provenance as declared, inferred, or both. Coverage is not equality, so a module- or stack-scoped declaration that governs a resource-level inference does not merge with it: the view draws two arrows between the same pair of stacks, one declared at the declared scope and one inferred at the resource scope. That is deliberate — the declaration and the inference are different statements about different endpoints — but it is why a stack pair can show more arrows than a reader expects.

  • Counted like any other edge for cycle detection, the graph view, and the org topology. Inferred evidence renders dotted; an exact union says declared + inferred and retains the declared trigger policy.

  • Never an orphan, never unsynced. An orphan finding asks someone to fix a reference they wrote, and nobody wrote these. Nor can an inference be “unsynced”: it only exists because the reader’s lookup and a managed owner were both indexed, so a target that later reads as absent — deleted stack, removed resource, or a stack with no indexed state — is the normal lag between an owner changing and the reader’s next reindex withdrawing the edge. Explore → Insights therefore builds both its Orphan and Unsynced references from declared edges only (the historical findings payload stores both under orphans, distinguished by pending=true). The accepted trade-off: an owner importing a resource into state is not visible as an inferred edge until the periodic rebuild republishes the metadata.

  • Bounded. Both raw addresses of an inferred edge sit in one btree primary key, and a pair past Postgres’ 2704-byte maximum fails the INSERT — which shares the state-projection transaction, so it would wedge that stack’s whole Explore projection on every retry. Either address over maxInferredEdgeAddressLen refuses the inferred-edge update (logged) and retains the last complete set, the same trade maxDeclaredIdentityLen makes for an oversized declared id. Fan-out is data rows × owners, so it scales with the resource fleet rather than with anything a human configured. maxInferredEdgesPerStack bounds all three stages: the identifiable data reads collected before SQL, the cross-stack owner rows SQL may return (LIMIT cap+1), and edge expansion itself. No stage publishes its arbitrary prefix. The latest resource rows still advance, the last complete inferred edges remain visible, and the state generation marks them evidence-unknown so a narrower always cannot fire and a narrower never cannot hide a possible manual trigger (that escalation says “manual deploy offered”, not “auto-trigger disabled” — nothing was withheld). A declared whole-stack policy still wins: it already governs every possible inferred dependency in that parent stack, so incompleteness cannot weaken it.

Who writes them, and when

The reader stack’s reindex does, in the same transaction as its index rows and under the same state generation: ResourceIndexService.ReindexFromRun computes each data row’s identity transiently, resolves the distinct keys through one ResourceIndexRepository.ManagedByIdentities query, and hands domain.InferEdges’ output to ReplaceStackStateProjection. Passing an empty set is how a stack that dropped its last cross-stack data source loses its inferred edges.

Transient is deliberate: resource_index.cloud_identity_id stays NULL for data rows, because a stored data-side identity would group the reader with the resource it reads and report every cross-stack lookup as a duplicate. The identity is derived, matched, and dropped.

The 12h resource-index rebuild uses a bounded two-pass order. The first, stack-id-ordered pass indexes every stack and records which ones made an identifiable data lookup. Once the org’s owner rows are all present, the second pass revisits only those readers. That makes an empty or partially restored Explore store converge in one sweep regardless of whether a reader sorts before its owner, without doubling work for owner-only stacks. There is no inference job or persistent staging projection of its own. The same cadence is the bound on the owner side during ordinary operation: an object newly managed by another stack produces an edge once the reader next reindexes.

Both passes are resumable: the rebuild runs in two-minute slices carrying phase, cursor and the reader set in the task’s own payload. “Visits only the readers” holds only below the payload’s 1,000-id cap — past it the second pass walks the whole org. See Resource Explorer: Filtering & Scale.

Declared and state projections have separate generations. A failed configuration read marks only declarations last-known; bounded or unstoreable inference marks only state. Trigger resolution reads both once per child because either can change the policy result.

Rolling upgrade. Pods predating inference may delete inferred edges during a reindex and do not enforce the newer freshness or budget holds. A mixed-version fleet can therefore briefly auto-trigger a dependent that a new pod would hold for review. This is accepted without a compatibility gate.

What this is not

gantrycd briefly inferred edges from gantrycd_state_outputs lookups, by data source type name plus a syntactically valid stack_id. That was withdrawn and stays withdrawn. Applied state records the producer’s organization, stack and run, but NOT which gantrycd backend served them, and the provider lets any stack point at any endpoint — so two installations sharing a database lineage (a staging clone, the far side of a migration) hold identical ids, and a stack reading the other one’s outputs produced an edge that looked local and was false. It could not deploy anything unattended, but it rendered a Deploy button on the wrong parent’s page and could close a false cycle. “Needs a human to click” is not harmless.

An ARN cannot fail that way: it names a real object, not an installation. What it can fail on is an unverified endpoint — an AWS-compatible API emits well-formed ARNs and the endpoint appears nowhere in state — which is what gantrycd:resource-identity-realm is for, and its remaining bound is that it must be declared. See Resource identity.

Re-enabling gantrycd_state_outputs discovery still needs a stable backend realm identifier recorded by the provider and verified on read. Until then, a gantrycd:dependency: comment remains the way to express a dependency gantrycd cannot see in state — an output reference, an ordering constraint, or any relationship that leaves no shared cloud object behind.

Dependent triggers

The trigger= field turns the graph from purely informational into a (carefully bounded) automation surface: when a plan-mode, non-PR-origin deployment finishes with a successful apply_run stage, the backend resolves the stack’s direct dependents and either redeploys them automatically (always), offers a button on the parent deployment’s page (manual, the default), or just lists them (never). Full lifecycle details (qualifying condition, idempotency claims, async task) live in deployment_state_machine.md — this section owns the resolution semantics.

Resolution rules

DirectDependents(org, stackID) returns the stack’s inbound star (InboundCrossStackEdges): every edge from another stack’s entity onto the stack or one of its resources (intra-stack edges excluded). The execution read returns only whole per-dependent-stack edge sets: at most 10,000 edges for one child and 50,000 in aggregate. A child outside either budget is not dropped and no arbitrary prefix is evaluated; it gets a durable manual row explaining that the relationship was too large to resolve automatically. Oversized children do not consume the aggregate budget, so one pathological stack does not hide ordinary dependents that sort after it. This bounds deployment-path memory while preserving the operator’s Deploy button. The irreducible output remains one stack id and one snapshot row per direct dependent; the cap bounds detailed edge materialization, not the number of real relationships the product must show.

Complete edge sets are grouped by dependent stack and each group collapses to one effective policy (domain.ResolveDependentTrigger):

  1. Changeset filtering. An edge that names a parent resource (from_resource=) only counts when the applied plan actually changed that resource. The apply’s plan-changes artifact (resolved through the apply run’s plan_run_id, so a targeted replan’s narrowed plan is used as-is) is matched against the edge address: equal, or a hierarchical prefix ending at a . or [ boundary — module.vpc covers module.vpc.aws_subnet.a[0], aws_instance.web covers aws_instance.web[0], but not aws_instance.webserver; index keys compare exactly with no quote normalization, and output.<name> entries match output-scoped addresses. Every changeset entry counts, whatever its change type. Edges depending on the whole stack always count — depending on the stack means “anything in it” — and an edge set with no resource-scoped edges never reads the artifact at all. Filtering happens before the policies combine, so a never edge whose resource did not change no longer vetoes an always edge whose resource did. A child whose every edge filtered out keeps an informational skipped row (skipped_reason), with its policy resolved from the unfiltered edges so an all-never child still reads (and gates the button) as never; skipped rows never auto-trigger and never enter automation, but the Deploy button doubles as “deploy anyway” and clears the skip. An unreadable or incomplete artifact does not filter dependencies away, but it also cannot justify unattended automation: a resource-scoped always is retained as a manually actionable row with an explicit changeset-unknown reason. Transient storage errors make resolution retry via the async task / reconcile sweep. Nothing re-reads artifacts later: triggers fire at resolution time, so the decision and the evidence share one moment.

    Why incompleteness is checked, not just readability. An analysis that failed still uploads changes: [] — a well-formed, empty change list that is byte-identical to “this apply changed nothing”. Trusting it filters away every resource-scoped edge, so an apply that really did change things would silently drop the dependents it should have triggered. Readability cannot tell the two apart; only PlanSummary.CountsComplete() can, so the summary is consulted before the detail (appliedPlanChangedPaths). The same applies to a narrowed (-target) apply: its changeset says nothing about the resources it never examined. A complete analysis with an empty changeset is real evidence that nothing changed, and still filters normally.

  2. Declared policy precedence on two explicit scope axes. Declared and inferred edges remain independently stored and visible in the graph; only trigger behavior is narrowed here.

    • On the dependency (parent) side, within the same child/parent stack pair, a declaration suppresses an inferred manual when its target is the same address, an ancestor module, or the whole parent stack. A sibling resource is not covered.
    • On the dependent (child) side, if any surviving declaration scopes the policy to the whole current child stack (a comment with no resource=), those stack-level policies govern instead of child-resource policies. The inbound read is already for one parent stack, so this cannot suppress evidence from a different dependency root.

    These rules let a human override automation deliberately without making the underlying inferred relationship disappear from cycle detection, diagnostics, or the graph.

  3. Combining. Within the surviving set: manual if at least one is manual, else never if at least one is never, else always. Manual is the safe middle ground when declarations disagree; never only suppresses automation when nothing asked to be consulted.

  4. Demotion. At resolution time, an always dependent that was not skipped by changeset filtering is demoted to manual with a stored reason, surfaced as a warning next to the (still present) trigger button. The reasons below are evaluated in this order and the first match wins; demotionFor in pkg/domain/dependent_trigger_resolution.go carries the same order. Facts about what already ran come before inferences from the derived graph.

    #ConditionStored reason
    1The dependent is already on this deployment’s cascade_stack_path — the stacks the cascade that produced it has deployed, carried on the row and inherited hop by hop. A redeploy would re-enter a stack the cascade has been through, whatever the graph says about cycles. There is no depth bound: the path is read, not walked.…: this stack has already been deployed by this cascade
    The child’s complete inbound edge set exceeded the deployment-path budget. This is decided before edge policy can be evaluated; the causal cascade reason above still wins when known.…: this stack has too many dependency edges on the applied stack to resolve automatically
    2The dependent’s declared edges are last-known: its own latest run could not read its dependency declarations, so the always was authored by an earlier revision that may since have dropped or changed it.…: this stack's dependency declarations could not be read on its latest run
    3The dependent’s inferred edges are last-known because its latest state exceeded an inference safety bound or contained unstoreable evidence. A narrower policy resolves conservatively to manual: an omitted inferred sibling could hold always down or make a resource-level never manual. A declared whole-stack policy is exempt because it covers every inferred address in that parent.…: this stack's inferred dependencies could not be fully determined from its latest state
    4The always rests on a resource-scoped edge and the applied plan’s changeset is unreadable, so the resource match cannot be shown to have happened.…: the applied plan's changed resources could not be determined
    5The dependent appears on a dependency cycle through the parent (GetStackCycle, SCC — no depth bound, so loop length does not matter).…: circular dependency
    6Cycle detection could not run at all because the org’s graph exceeded maxOrgStackPairs. “Could not tell” is not “no cycle”, so this fails closed.…: circular dependency status could not be determined

    Every row is the same principle: absence of evidence is not evidence. Each costs a click rather than a deployment.

    Nothing is re-checked later, because nothing waits: triggers fire at resolution time, in the same pass that computed these demotions.

    There is no human-approval backstop behind this. A triggered child is an ordinary plan deployment and follows the dependent stack’s own auto-approve policy, so on an auto-approve stack a missed loop would apply unattended, repeatedly. Demotion is the only thing preventing that.

    The rule is blunt on purpose. What demotes is an always whose endpoints share a stored SCC — whatever kind of edge closes that component, including an inferred one (see Inferred dependencies). It does not try to establish that the loop could actually execute automatically end to end: a manual or never hop would break the chain. That proof would need provenance- and policy-aware cycle detection, and the graph it would run over is asynchronously derived and deliberately retains dangling edges, so it cannot support that precision. Refusing automation inside anything that looks circular costs a human click; getting it wrong the other way costs an unattended redeploy chain.

    A skipped row is left alone: every automatic path already excludes it (skipped_reason IS NULL in the work list), so demoting it would add a cycle warning to a row that already says nothing it depends on changed.

    Demotion applies at RESOLUTION time, which is also when triggers fire — there is no later execution to drift from the decision.

  5. No multi-parent wait — each parent fires its own child. An always dependent of several stacks gets one dependency deployment per parent apply, immediately, each linked to the row of the parent that caused it. Converging a diamond (A → {B, C} → D) into “one redeploy” is the job of D’s own stack lane: the children queue there like any deployments, run back-to-back, and D converges on the last one. A supersede_after queue policy (age-based, off by default) can additionally collapse a queued child that has already waited longer than its threshold — parents applying seconds apart do not trip it, and are not meant to.

    An earlier design parked the trigger instead: a deferred row waited for every other parent to settle, and a minute-ly sweep fired completed joins. It was removed deliberately. A stored executable promise ages — the dependent can edit its declarations, grow a cycle, or change generations while the row waits — and keeping the stored verdict honest grew a revalidation pass, a tri-state verdict, and an authorize-then-fire generation guard, each patching the previous (dependency review, items 23/30/65). Firing immediately deletes the promise and the machinery with it. The honest bill for that: a fan-in of N parents applying together queues up to N children (each a plan; on an auto-approve stack, an apply; on a manual-approve stack, a confirmation prompt each) where the old design ran one — a plan or apply against a half-settled world is corrected by the next parent’s trigger. One class is NOT self-correcting: a parent whose infrastructure changes through a path that fires no trigger of its own (a destroy, a promoted-PR apply) overlapping another parent’s apply can leave the dependent applied against outputs that then changed, with nothing re-firing it — the old sweep converged that window, and now only the dependent’s next ordinary trigger or its Deploy button does. The dependent-trigger-reconcile scheduler job remains, doing only its first duty: resolving finished deployments whose resolution never ran at all (NULL marker — a crash, or completion by a pre-feature instance during a rolling upgrade).

  6. No coalescing. A firing trigger always creates a fresh deployment — it never links an in-flight or queued dependency deployment, which would tie this parent’s trigger to a plan that may predate this parent’s outputs. Concurrent triggers serialize on the child’s stack row lock.

Each snapshot row also pins the dependent stack’s incarnation (dependent_stack_created_at, observed at resolution): a stack deleted and recreated under the same id reads as “stack deleted” on old parents’ pages and both trigger paths refuse it — the new incarnation never declared those dependencies. The auto path runs the same cloud pre-flight as the Deploy button before creating each child, so broken credentials surface as a trigger failure instead of a doomed deployment.

The resolved per-dependent policies are snapshotted into Postgres (deployment_dependent_triggers, one row per dependent stack per parent deployment) — the parent deployment’s page renders that durable point-in-time snapshot, not the live graph. Each row carries the resolved trigger, any demotion reason, the changeset-filtering skip reason when nothing the dependent consumes changed, and — once triggered — the child deployment id, which doubles as the trigger-once claim.

Each row also records which resources put the dependent there (triggering_resources, jsonb): pairs of the parent resource the applied plan changed and the dependent’s own address that reads it, each marked declared or inferred (never both, unlike the row-level evidence: a declaration on an address suppresses the inference for that same address before the pairs are derived). They are derived only from the edges that survived changeset filtering, and only when the changeset was readable — the ones that actually fired — and are sorted, deduplicated, and capped at three pairs with the remainder counted, since the value is rendered verbatim on the parent’s page (“Because account changed aws_acm_certificate.cloudfront_wildcard["gantrycd-prod"], which this stack reads via data.aws_acm_certificate.wildcard.”). A dependent that depends on the whole parent stack records that marker instead of pairs: “anything in that stack” is the relationship, and naming the resources that happened to change would read as the reason.

The column is NULL on every path that did not establish a cause, because the sentence it produces asserts that a resource changed and only the changeset establishes that: the resolution-budget row, a skipped row, an unreadable changeset (reason 4 above — policy falls back to the unfiltered edges, which is right for policy and a lie as provenance), and a fully-filtered child whose inferred set is incomplete (the same fallback, where an omitted sibling may be the real cause). The API additionally withholds it on a redacted row along with the rest of that stack’s identity, because the dependent-side addresses are its configuration.

Triggered deployments

A trigger (automatic or button) creates a normal plan deployment on the dependent stack at the dependent’s own synced commit: it queues into the dependent’s stack lane (PromoteNextOnStack decides queued vs pending) and plans. What happens next follows the dependent stack’s own auto-approve policy, exactly like any other plan deployment: a stack set to auto applies without confirmation. There is no origin filter on that check, which is why cycle demotion is the thing standing between a loop and an unattended redeploy chain. Provenance is recorded as origin_type = dependency with origin_dependency_deployment_id/stack_id pointing back at the parent, so the child page links back and the parent row links forward.

The button (POST …/deployments/{id}/dependents/{dependent_stack_id}/deploy) is gated on (stack, deploy) on the dependent stack — pressing it is exactly “create a deployment there” — plus (stack, read) on the parent. A dependent already triggered conflicts (no re-trigger; the row links to the child instead); a never row is rejected; a row whose earlier auto-trigger failed (dependent stack deleted, commit not synced) is retryable — the button doubles as the retry and clears the recorded error. A human pressing Deploy wants a plan of the world as it is now, so the button always creates a fresh deployment, never links an in-flight one — the same rule the automatic path follows.

Promotion and the configured branch

A trigger runs the dependent’s configured branch, and promotion deliberately does not move it. Promoting a pull request creates a plan/apply deployment at the PR head; the apply is state-bearing (it replaces the stack’s resource rows and the inferred edges derived from them) but it does not run the configured branch, so its dependency discovery is rejected and the stack’s declared edges stay as the last configured-branch run left them.

After a promotion, “current” therefore means two different things for the same stack, and both are intentional:

describes
live infrastructure, outputs, resource rows, inferred edgesthe promoted PR commit
declared dependency policy, and what a trigger deploysthe configured branch

Inferred edges make the promoted head influence trigger outcomes. They are state-derived, so the promoted commit writes them — a pull request that merely adds a data source plants a real inferred edge, before anyone has merged it. The dependent then appears on the parent’s page and can be offered a Deploy button for a dependency its configured branch does not have yet; and if that dependent also declares an always on another of the parent’s resources, the new inference combines with it and holds the automation to manual (the row says so). So that a reader can tell these rows apart from a human-declared dependency, the snapshot records its evidence (declared, inferred, or both, read from the raw edges before declaration precedence; NULL when the edge set exceeded the resolution budget), and the Deploy row says “inferred from live state” and that Deploy runs the configured branch. Removing a data source in a promoted PR does the mirror image: the edge is withdrawn while the configured branch still reads the object.

That follows from the split above rather than contradicting it — inferred edges describe what the stack is, not what it declares — and it self-corrects the moment a configured-branch run reindexes. It is called out because it is the one way a PR that changes no policy can change a trigger outcome.

The consequence is real and it belongs to the user. If a parent applies after a promotion, the dependent’s always enqueues the configured branch, which does not contain the promoted change — so the deployment replaces promoted C1 with configured C0. On a dependent that auto-approves, that happens without another human looking at it.

GantryCD does not arbitrate that. A trigger takes the dependent’s lane like any other deployment and obeys its queue and approval policy, so whether the promotion or the trigger runs last is decided by ordering the operator can see, pause, cancel or reorder. Whoever combines promotion, trigger=always and automatic approval on one stack owns that interaction; the product’s job is to make it legible, not to withhold the automation they configured.

The deployment history is where that is legible. A stack page carried a banner for this state at one point — it named the promoting pull request and said the next configured-branch deployment would replace it — and it was removed: the condition it reported is normal between promoting a fix and merging it back, and it cleared only when a configured-branch apply landed. A merge that plans to no changes, or one whose deployment is still waiting to be approved, applies nothing, so the banner outlived the divergence it described and read as noise on a healthy stack. The deployment list already names each deployment’s origin and pull request.

An earlier implementation went further and demoted the trigger here. It was removed: the check read the latest qualifying run, which cannot tell whether the promotion or the trigger is ahead in the lane. It fired when a promotion resolved first, stayed silent when the trigger resolved first and was merely queued behind the promotion, and cleared itself when any configured-branch refresh or plan became the latest run — fencing some interleavings, missing others, and overriding declared policy in the ones it caught. See item 37 of the dependency review.

Retries follow. Retrying a dependency-origin child deployment repoints every snapshot row that linked the old deployment at its replacement, so parent pages track the live attempt instead of a dead one.

Failures notify. A failed auto-trigger fires the dependent_trigger_failed notification event — once per underlying problem: the failure claims every untriggered always row pointing at the dependent, across parent deployments, so N waiting parents yield one alert, not N — through the org’s notification integrations, anchored to the parent deployment with the dependent stack and reason attached. Broken automation should not wait for someone to open the parent page.

Dependent rows whose stack the viewer cannot read are redacted to the same ~restricted-N tokens as the graph view (name, links, and errors withheld; the policy badge survives). The Dependencies graph view renders the per-edge policy as a small badge on non-default edges (always/never; manual stays unlabelled) and in the edge focus panel, so a subscription is visible before anything ever fires.

Budgets and how to watch them

Every cap above is a cliff: under it nothing changes, over it a feature silently degrades. Three metrics make the approach visible so raising a cap is a planned change rather than an incident.

budget labelCapWhat a user sees when it trips
org_graph_edgesmaxOrgEdges = 200,000The diagnostics sweep stops reporting orphan references — that is a statement about every edge in the org, so no smaller read answers it. Cycles survive: the sweep falls back to the reduced pair read, and only loses them past org_stack_pairs below. Since #278 this cap no longer touches the Dependencies view.
neighborhood_edgesmaxNeighborhoodEdges = 200,000One stack’s Dependencies view renders the central stack alone, which reads as “this stack has no dependencies”. Per page, not per org, so it has no usage gauge — the reading would be one number per (org, stack) neighborhood and no scrape-time query yields it. Watched through hits only.
org_stack_pairsmaxOrgStackPairs = 200,000Cycle status is unknown, and demotion fails closed: every always dependent in the org is demoted to manual. Topology drops its connections, and the Dependencies view — whose neighborhood walk reads the same pairs — renders the central stack alone.
inbound_edges_per_childmaxDirectDependentEdgesPerStack = 10,000One noisy dependent of the deployed stack is demoted to manual with “auto-trigger disabled: this stack has too many dependency edges on the applied stack to resolve automatically”, instead of auto-deploying.
inbound_edges_totalmaxDirectDependentEdges = 50,000The same, for dependents past the aggregate budget of one parent’s fan-out.
inferred_edges_per_stackmaxInferredEdgesPerStack = 5,000One stack keeps its last complete inferred edges; new inference is not published and its evidence is marked unknown. This cap is watched through hits only — it has no usage gauge. It bounds three stages of inference and only the last leaves rows behind, so a stored count would report a stack jammed at the reads stage as near zero: healthy-looking, and refused on every reindex. The hits counter observes every stage.
  • gantrycd_dependency_budget{budget} — the cap itself, read straight from the constant (services.DependencyBudgets). No org label. Publishing it means a dashboard never hard-codes the number, so an alert stays honest after a cap is raised.

  • gantrycd_dependency_usage{org,budget} — how close one org is now, for the four of the six budgets that have an honest, org-wide reading (inferred_edges_per_stack and neighborhood_edges are the exceptions above). Read on the same scrape as gantrycd_explore_dependencies_total (RegisterExploreInventory), one statement for all orgs. org_graph_edges is the org’s total edge count; org_stack_pairs its distinct cross-stack pairs; inbound_edges_total the largest budgeted inbound star on any one stack (children already rejected by the per-child cap contribute nothing, exactly as the deployment path counts it); inbound_edges_per_child the largest bundle from one child onto one parent. These are org-wide maxima on purpose — there is deliberately no per-stack label.

    Cost, measured warm at work_mem=4MB and 32MB (index-only either way, zero temp I/O in all four):

    TableDistinct pairsUsage readThe edge count on the same scrape
    1,089,000 edges14,60093–209 ms78–122 ms
    6,760,000 edges270,4002.0–2.8 s0.87–1.03 s

    The first row is scrape-cheap. The second is not, and no rewrite fixes it — the cost is the 270,400-group aggregation, not the query’s shape. If an org ever approaches that pair count, the follow-up is to stop reading this on the scrape path: have the periodic dependency-graph rebuild sweep compute the numbers once per org and persist them as a usage row, and let the metrics callback read that row back. Not built — a stored row is only worth its staleness once the live read hurts.

  • gantrycd_dependency_budget_hits_total{org,budget} — a counter of the moments a cap actually refused work, incremented once per event (a truncated read, a resolution that dropped dependents, a stack held at last-known), never once per row. The two inbound caps share the label inbound_edges: the read reports its incomplete children without saying which of the two excluded each one, and the two usage gauges above already answer “which am I near”.

Alert rules live in the infra repo. The two worth copying:

# Approaching a cap: over 30% of budget. Each replica reports the same
# scrape-time reading, so aggregate before dividing.
max by (org, budget) (gantrycd_dependency_usage)
  / on (budget) group_left max by (budget) (gantrycd_dependency_budget)
  > 0.3
# A cap actually refused work in the last hour. Any value above zero is a
# user-visible degradation someone should see.
sum by (org, budget) (increase(gantrycd_dependency_budget_hits_total[1h])) > 0

The rebuild’s own runtime

The caps above bound how much a read may return. Nothing above bounds how LONG the periodic rebuilds take, and that used to be its own cliff.

Both rebuilds now run in resumable two-minute slices, so one run is a slice rather than an org and each is registered with a 10-minute ceiling — simultaneously its handler deadline, its lease, and its crash-recovery wait (see Async Task Queue for the model and the measurements). The per-stack walks no longer grow one claim with org size. What can still reach the ceiling is a single stack whose own work outlasts the budget, or the final whole-org reverse reap, which is not yet sliced — see Resource Explorer: Filtering & Scale.

They are watched through gantrycd_async_task_duration_seconds{task_type}, whose boundaries reach 1800 from when the ceiling was thirty minutes:

# A rebuild RUN took over 5 minutes. Either one is now a SLICE at 2.5x its
# two-minute budget — a single stack or read that overran, not a slow org.
# Counted past a bucket rather than with a quantile: the sweep runs twice a day,
# so a rate() window is usually empty and histogram_quantile returns NaN.
sum(increase(gantrycd_async_task_duration_seconds_bucket{
      task_type=~"resource_index_rebuild|dependency_graph_rebuild", le="+Inf"}[6h]))
  - sum(increase(gantrycd_async_task_duration_seconds_bucket{
      task_type=~"resource_index_rebuild|dependency_graph_rebuild", le="300"}[6h]))
  > 0

No org label: the duration histogram is bucketed by task_type alone, and adding org would multiply series by bucket count.

The backstop changed. This rule was introduced alongside gantrycd_job_executions_total{outcome=~"failure|timeout"} firing as job_name="task-processor". That scheduler job no longer exists — the queue runs continuous worker slots, not a periodic batch — so that alert can never fire for async tasks again. What replaces it, per task type rather than for a whole tick: gantrycd_async_task_processed_total{outcome="timeout"} when a handler hits its ceiling, and AsyncTasksDeadLettered once one has given up for good.

Authorization & redaction

The …/dependencies and …/dependency-cycles routes authorize read on the central stack only. But a neighborhood — and especially a cycle — can reach stacks the viewer has no grant on, and node keys carry stack IDs and resource addresses. Dropping those stacks would distort the topology and could hide a cycle, so instead the handler redacts them:

  • StackService.ReadableStackIDs(principal, org, ids) resolves which referenced stacks the viewer may read (an unscoped/super-admin grant short-circuits to all-readable; otherwise each id is resolved to its name and checked with the canonical per-item Allowed, so pattern semantics are never re-implemented).
  • A dependencyRedactor (internal/backend/handlers/dependency_redaction.go) rewrites every entity owned by an unreadable stack into a stable, opaque placeholder — ~restricted-N for the stack, ~hidden-M for its resources — consistently across nodes, edges, and cycle keys, so the shape (including any cycle through a hidden stack) is preserved while names/addresses are not. ~ cannot occur in a real stack id or resource address, so a placeholder can never collide with a real identifier, and the frontend detects the prefix (isRedactedStack / isRedactedDependencyKey in web/src/utils/dependencyKeys.ts) to render a lock. The backend/frontend token prefixes must stay in sync.

Wiring status

The edge repository is built in buildRepos over the explore database pool and held on appRepos.dependencyEdges — always non-nil, because EXPLORE_DATABASE_URL is required and buildExploreDatabase aborts boot without it. Configuration and availability are gated differently: an unset or unparseable URL is fatal at startup, while a valid-but-unreachable database is not (the pool connects lazily and does not ping), so the graph paths degrade at query time and the async queue’s backoff/dead-letter carries the retries (see Datastores). The async handlers are registered on the TaskProcessorJob in cmd/backend/scheduler.go (TaskTypeDependencyReconcileHandleReconcileTask; TaskTypeDependencyGraphRebuildHandleRebuildTask), and the 12h dependency-graph-rebuild scheduler job (EnqueueRebuildAll) is registered alongside the other recurring scans there. The full pipeline above is wired: the runner-side discovery scan, the DependencyGraphService enqueue/reconcile/query/rebuild/cleanup paths, the periodic rebuild sweep, and the Dependencies-tab UI all consume it.