Skip to content
5 changes: 5 additions & 0 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,11 @@ Architecture Decision Records in [adr/](adr/):
- [0008-client-declared-request-priority.md](adr/0008-client-declared-request-priority.md) - Client-declared request priority, cooperatively trusted
- [0009-per-worker-coordination-free-admission.md](adr/0009-per-worker-coordination-free-admission.md) - Per-worker, coordination-free admission capacity
- [0010-generated-user-facing-schema-contract.md](adr/0010-generated-user-facing-schema-contract.md) - Generated user-facing schema contract, hosted in the SDK
- [0011-inline-local-computed-attributes.md](adr/0011-inline-local-computed-attributes.md) - Inline evaluation of local Jinja2 computed attributes during update mutations
- [0012-selective-post-merge-regeneration.md](adr/0012-selective-post-merge-regeneration.md) - Selective post-merge regeneration driven by the captured merge diff
- [0013-webhook-delivery-on-prefect-run-primitives.md](adr/0013-webhook-delivery-on-prefect-run-primitives.md) - Webhook deliveries as retention-bounded Prefect-run objects
- [0014-generic-per-task-recovery-actions.md](adr/0014-generic-per-task-recovery-actions.md) - Generic per-task recovery actions with polymorphic task typing
- [0015-uniform-bounded-webhook-retry.md](adr/0015-uniform-bounded-webhook-retry.md) - Uniform bounded fixed-delay auto-retry for webhook deliveries

## Current Guides

Expand Down
2 changes: 1 addition & 1 deletion dev/adr/0005-account-group-origin-attribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,6 @@ Cheaper still — just emit FR-015 and rely on log queries. Rejected because aud

## Implementation Notes

- Spec: [`specs/infp-556-auto-create-groups/spec.md`](../../specs/infp-556-auto-create-groups/spec.md), specifically FR-012, FR-013, FR-014, FR-021, and Session 2026-05-13 in Clarifications.
- Spec: [`specs/archive/infp-556-auto-create-groups/spec.md`](../../specs/archive/infp-556-auto-create-groups/spec.md), specifically FR-012, FR-013, FR-014, FR-021, and Session 2026-05-13 in Clarifications.
- Schema migration: definition-only (adds the attribute); does **not** run any data migration to populate values on existing rows.
- Read-only enforcement applies even when a user reveals the attribute via the `display: extra` toggle — the UI must still treat it as non-editable.
94 changes: 94 additions & 0 deletions dev/adr/0011-inline-local-computed-attributes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# 11. Inline evaluation of local Jinja2 computed attributes during update mutations

**Status:** Accepted
**Date:** 2026-07-31
**Author:** @opsmill-team

**Source:** `specs/archive/ifc-2273-local-computation-jinja2/research.md` (R1, R3, R5) and the scope
decisions recorded in that spec's `spec.md` Clarifications.

## Context

Jinja2 computed attributes were recomputed exclusively through Prefect background tasks triggered by
change events, except at node creation, which was already handled inline by `_process_macros()`. A
single bulk update of thousands of nodes spawned one background task per node, each re-querying the
database, even when the triggering change and the computed attribute lived on the same node. The
response to the originating mutation did not reflect the recomputed value, and a node could emit two
events: one for the original change and one for the later computed update.

The change had to integrate with the current automation structure, since the placeholder-automation
refactor (INFP-441) was scheduled separately and not available.

## Decision

Split recomputation by the locality of the triggering change.

A local change (the changed attribute or relationship is on the node that owns the computed
attribute) is evaluated inline during `Node._update()`, after attribute and relationship saves and
before HFID and display-label recomputation, and persisted in the same transaction and
`NodeChangelog`. The inline path reuses the template variable-resolution pattern from
`_process_macros()`, and loads the peer attributes that relationship-referencing templates need
through the existing `_collect_extra_filters()` mechanism, so no query is issued beyond the
`resolve_relationships()` the update already runs.

A remote change (a peer node attribute referenced by a computed attribute on another node) keeps the
existing Prefect background-task path unchanged.

The duplicate background path for local changes is suppressed by neutralizing self-targeting
triggers (`targets_self`) into placeholder field matchers that never match a real update event,
rather than deleting them, so the trigger definitions remain available for schema-change detection.
This mirrors the existing HFID and display-label handling.

The optimization is scoped to the update path. Node creation, template instantiation, and Python
transform computed attributes are unchanged. Both optional and mandatory Jinja2 computed attributes
recompute inline on local updates. On inline evaluation failure the error is logged and the value is
left unchanged; the mutation still succeeds, matching the background-task error semantics.

How the four evaluation paths, the `targets_self` neutralization, and the extra-filter peer loading
work is documented in [Computed Attributes](../knowledge/backend/computed-attributes.md).

## Consequences

### Positive

- Mutation responses immediately reflect recomputed local values, with no page refresh.
- Bulk updates of local computed attributes spawn zero background tasks for those recomputes.
- Each local mutation emits a single consolidated event, because inline computed updates are
recorded in the same `NodeChangelog`.

### Negative

- Two evaluation paths now exist for the same attribute kind and must stay consistent: an inline
result must match what the background path would produce for the same inputs.
- Inline evaluation errors are swallowed (logged, value left unchanged), a deliberate exception to
the general "do not catch broadly" convention, justified by parity with the async path.

### Neutral

- The update path does more work per mutation. The cost is bounded by reusing already-loaded node
state and peer data, and is a net reduction against the background-task fan-out it replaces.

## Alternatives Considered

### Hook at `Node.save()` or at the GraphQL `mutate_update()` level

Rejected as too high in the stack. `save()` also covers creation, which already recomputes inline,
and the GraphQL layer misses SDK and other non-GraphQL update paths.

### Delete self-targeting triggers entirely rather than converting them to placeholders

Rejected because it removes schema-change detection for those attributes. Placeholders keep the
definitions available while preventing them from matching real update events.

### Let both the inline and background paths run for local changes

Rejected due to double computation and the race between the two writes.

### Add a `locally_recomputed` flag to event payloads so the trigger can skip

Rejected as more coupling than the placeholder approach for the same effect.

### Re-fetch peer data with an extra query on relationship changes

Rejected as it defeats the performance goal. `_collect_extra_filters()` loads the peers during the
`resolve_relationships()` the update already performs.
87 changes: 87 additions & 0 deletions dev/adr/0012-selective-post-merge-regeneration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# 12. Selective post-merge regeneration driven by the captured merge diff

**Status:** Accepted
**Date:** 2026-07-31
**Author:** @opsmill-team

**Source:** `specs/archive/ifc-2704-incremental-merge-regen/research.md` (D1, D2, D4, D6, D7, D9) and
that spec's `spec.md` (FR-001, FR-008, FR-010, FR-012, SC-001).

## Context

On merge, the follow-up re-ran every generator and regenerated every artifact for every group member
regardless of what the merge changed (the blanket path). On a real dataset this spawned thousands of
background tasks and left the instance effectively unusable for roughly twenty minutes after each
merge (originating incident IFC-2306).

The enriched branch diff that records what a merge changed is available at merge time but
unrecoverable afterward: the freeze marks the diff root merged and rewrites its tracking id, and the
field-summary queries exclude merged roots. The proposed-change pipeline already had
definition-level selection predicates and member-impact analysis, but they were written for a live
source branch, an artifact-id-space comparison, and live-group iteration for new members, none of
which transfer unchanged to a diff-only, member-id-filtered, post-merge target-branch context.

## Decision

Replace the two blanket post-merge triggers with a selective path, gated by
`selective_execution_after_merge` (default on; off restores the blanket path byte-for-byte).

Capture the enriched diff in the merge orchestrator before the freeze, serialized into the SDK
`NodeDiff` summary shape the existing predicates already consume, tagged with the target
(destination) branch, and cache it under a merge-scoped key derived from the stable diff-root uuid.
Thread only that key through the follow-up. The capture is best-effort and split around the merge's
point of no return, so a rolled-back merge writes nothing and a capture failure degrades to the
blanket fallback rather than failing the merge.

In the follow-up, reuse the definition-level gates and member-impact analysis (extracted to a shared
package serving both the proposed-change and merge callers) to select only affected definitions,
reconciled against the live target-branch group so that new members and membership-only additions are
covered. Selection is governed by the over-execution invariant inherited from INFP-409: whenever the
affected set cannot be determined with confidence, the path regenerates everything for that merge.
Every fallback and the flag-off path route to the exact blanket behavior.

The end-to-end flow, selection, cascade, and fallback reasons are documented in
[Selective Merge Regeneration](../knowledge/backend/selective-merge-regeneration.md).

## Consequences

### Positive

- Dispatched-task count drops from proportional to (all definitions x all members) to proportional
to the affected set, so the instance stays responsive after a merge.
- One selection implementation serves both the proposed-change and merge paths.

### Negative

- A composite artifact that inlines another artifact's rendered content is no longer refreshed on
merge (the diff carries no inlining edge), a behavior regression now that the flag defaults on.
- Correct new-member coverage requires bounded per-selected-definition group and subscriber fetches,
so the "no new hot-path Cypher" property holds only at the definition level.
- The direct-merge generator-to-artifact cascade must await generators and capture their output,
because no event machinery regenerates artifacts on generator-produced data mutations.

### Neutral

- A per-merge line records whether the merge took the selective or a named fallback path, with the
dispatched generator and artifact counts, so silent under-execution is observable.

## Alternatives Considered

### Recompute the diff after the merge

Rejected. The freeze makes the recomputed diff return an empty set, causing under-execution.

### Reuse the changelog collector's node set

Rejected. It drops nodes whose only change was a conflict resolved to the base branch, also
under-execution.

### Derive the member filter from the diff alone

Rejected. It cannot enumerate a newly added member or a membership-only change, so member selection
reconciles against the live group instead.

### Dispatch a concurrent full-artifact regeneration for the generator cascade

Rejected. It races the generators' writes and renders artifacts against pre-generator state, so the
cascade sequences artifact regeneration after generator completion.
63 changes: 63 additions & 0 deletions dev/adr/0013-webhook-delivery-on-prefect-run-primitives.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# 13. Webhook deliveries as retention-bounded Prefect-run objects

**Status:** Accepted
**Date:** 2026-07-31
**Author:** @opsmill-team

**Source:** `specs/archive/ifc-2755-webhook-delivery-operability/research.md` (D1, D3, supported by
D4, D5) and that spec's `spec.md` (SC-007, FR-001 through FR-009).

## Context

A webhook delivery was process exhaust: a background run plus log lines, with no record of what was
sent or received, no classified reason, and no way to replay or stop it. Operators needed the
delivery to be a first-class, inspectable, recoverable object. The open question was where a
delivery's state and captured request/response live.

## Decision

Model each delivery as the user-visible `webhook_send` flow run itself, promoted to a registered
CORE workflow so it is resubmittable by id and discriminable by name. All delivery data lives on
Prefect run primitives: frozen parameters (the payload), tags (the webhook node and branch), run
state (the lifecycle), and a single grouped `http` artifact (request, response, and classified error
together) written per run and reflecting the last attempt. No new Neo4j node, attribute,
relationship, or migration. Header redaction and failure classification happen in-process before the
artifact is written, so no raw secret is ever persisted.

The capture and read-back paths are documented in
[Webhooks](../knowledge/backend/webhooks.md).

## Consequences

### Positive

- No migration and no backfill: historical runs are inspectable because the delivery model is
intrinsic to the run, not a separate stored record.
- Read-back mirrors the existing progress-artifact path (one batched read, gated on the GraphQL field
selection), so there is no extra per-task query cost.

### Negative

- Delivery data is bounded by Prefect retention (about 30 days). Older deliveries are neither
inspectable nor retryable, and a retry of an aged-out run fails with a clean "no longer available".
- The capture reflects only the settling (last) attempt, not per-attempt history.

### Neutral

- Delivery history is operational data on Prefect runs, not branch-versioned graph data.

## Alternatives Considered

### Keep `webhook_send` an inline subflow and retry by re-invoking `webhook_process`

Rejected. It re-runs the transform and re-derives the payload (not a frozen replay) and
re-introduces the orchestrator parent the design drops for retries.

### Persist deliveries as a domain node with a migration

Rejected. It adds schema and backfill for data whose lifetime is purely operational, contradicting
the no-migration goal.

### Separate request/response artifacts, or a per-attempt list artifact

Rejected. Two reads for no operator benefit, or unbounded artifact growth.
68 changes: 68 additions & 0 deletions dev/adr/0014-generic-per-task-recovery-actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# 14. Generic per-task recovery actions with polymorphic task typing

**Status:** Accepted
**Date:** 2026-07-31
**Author:** @opsmill-team

**Source:** `specs/archive/ifc-2755-webhook-delivery-operability/research.md` (D6, D7, D8, D9) and
that spec's `spec.md` (FR-016, FR-017, FR-027).

## Context

Retry and cancel needed a GraphQL surface. They could be webhook-specific mutations, or a generic
capability carried by every task. Webhook deliveries are the first, and currently only, task type
that supports them, but other task types may follow, and the surface should not have to change when
they do.

## Decision

Expose recovery actions as a generic capability on every task. `available_actions` (server-computed
from the run's workflow name and current state, as the single source of truth) and the classified
`error` sit on a `TaskNodeInterface`. Concrete task types are discriminated by the run's workflow
name via `resolve_type` against a `TASK_TYPES` map, mirroring the events type hierarchy;
`WebhookDeliveryTask` is the first concrete type. Retry and cancel are generic, task-id-addressable
mutations (`InfrahubTaskRetry` / `InfrahubTaskCancel`, modeled on the bespoke `BranchCreate`
pattern), not webhook-specific. Genericity is confined to the interface: actual support is per task
type, so only `WEBHOOK_SEND` runs are actionable and any other task resolves the actions as
unavailable. Authorization reuses the existing object-level update permission on the target webhook
node; no new global permission is introduced.

The polymorphic task typing is documented in
[Async Tasks](../knowledge/backend/async-tasks.md), and the delivery-specific behavior in
[Webhooks](../knowledge/backend/webhooks.md).

## Consequences

### Positive

- A new actionable task type plugs in by adding a `TASK_TYPES` entry plus an availability rule, with
no new mutation shape.
- `TaskNodes.node` becomes the interface, but `TaskNode` keeps its name, so existing selections, SDK
usage, and `__typename` checks keep resolving with no backfill.

### Negative

- The generic surface can advertise actions a given task type does not support (they resolve
unavailable), so callers must read availability rather than assume it.
- Delivery operability is tied to the webhook node's update permission rather than a dedicated
permission concept.

### Neutral

- Availability is computed once server-side; the frontend renders it and disables controls, and the
mutations re-check availability at execution time to reject a stale action rather than double-send.

## Alternatives Considered

### Webhook-specific mutations (`CoreWebhookRetry` / `CoreWebhookCancel`)

Rejected by the clarified genericity directive.

### A stored `task_type` enum or field, or a tag, as the discriminant

Rejected. The workflow name is intrinsic to every run, so historical runs type correctly with no
backfill and no extra stored field.

### A new `MANAGE_WEBHOOKS` global permission

Rejected. None exists today, and the spec forbids introducing a new permission model.
54 changes: 54 additions & 0 deletions dev/adr/0015-uniform-bounded-webhook-retry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# 15. Uniform bounded fixed-delay auto-retry for webhook deliveries

**Status:** Accepted
**Date:** 2026-07-31
**Author:** @opsmill-team

**Source:** `specs/archive/ifc-2755-webhook-delivery-operability/research.md` (D2) and that spec's
`spec.md` (FR-012, FR-012a, SC-004).

## Context

A failing delivery auto-retries. Two policy choices were open: the backoff shape (fixed versus
exponential), and whether to gate retries by failure class (transient-only versus uniform).

## Decision

Retry every failing delivery uniformly: a fixed delay of about 120s, bounded to 3 retries (4 sends
total), regardless of failure class. No exponential backoff, and no transient-only gating. The
classified failure reason and its per-class remediation hint, not a retry gate, are what tell the
operator whether waiting on the cycle can help. The `transient` flag once carried on the classifier
result was removed; each status class now owns its remediation hint directly.

The retry behavior and its relation to zombie detection are documented in
[Webhooks](../knowledge/backend/webhooks.md) and
[Async Tasks](../knowledge/backend/async-tasks.md).

## Consequences

### Positive

- One flow-level retry policy, with no attempt-level conditional machinery.

### Negative

- A 4xx or configuration failure that cannot succeed on retry still consumes its bounded attempts
before settling.
- A run parks in `AwaitingRetry` between attempts, holding an execution slot; the bounded fixed delay
caps how long.

### Neutral

- The zombie-detection window is sized above this fixed backoff, a relationship that holds because
the delay is fixed and known.

## Alternatives Considered

### Exponential backoff with jitter (the design-doc original)

Rejected. A long back-off parks many runs holding execution slots while waiting on a delayed attempt.

### Transient-only gating (retry timeout / connection / 5xx, fail 4xx / configuration immediately)

Rejected. It requires attempt-level retry-condition machinery whose complexity outweighs the cost of
the bounded extra attempts.
Loading
Loading