You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Track the concrete promotion evidence and offline evaluation artifacts required before the RFC #1770 Memory write evidence-sufficiency policy moves from shadow or advisory to enforcing.
This is follow-up work to RFC #1770 and PR #1921. It does not require replay as a runtime feature for a consumer that remains in shadow or advisory, and it is not a commitment to promote the policy, enable a hosted backend, or add any other decision-policy consumer.
Problem
PR #1921 defines the Memory write policy, defaults new policies to shadow, and persists scope-private observation sidecars with structured references and bounded policy/action/fallback/privacy facts. That establishes an observation baseline without retaining raw candidate or evidence bodies.
The policy's declared promotion_criteria are not measured promotion evidence. The current implementation does not provide a reproducible replay/rescore or re-evaluation workflow, a labeled calibration set, separate held-out evaluation, or a documented promotion and rollback decision. Those missing artifacts mean the observations may inform a future decision but cannot justify enforcing; enabling it merely because configuration permits the mode would treat configuration as evidence.
Acceptance criteria
Define a reproducible, authorization-aware replay or re-evaluation workflow for the Memory write gate, reporting incomplete input coverage separately from negative results.
Provide a labeled calibration set and separate held-out evaluation, including English and Chinese cases where the gate accepts both.
Evaluate candidate outcome mappings and confidence thresholds against false holds, false passes, fallback reasons, latency, request cost, and hold/review friction.
Document the measured promotion decision and rollback procedure, or explicitly retain shadow/advisory when the evidence is insufficient.
Preserve immutable Memory revisions, exact citations, Scope isolation, and the invariant used_fallback=true => coverage=unadjudicated.
Non-goals
No automatic promotion to enforcing.
No dashboard, public HTTP/OpenAPI surface, export API, generic decision workflow engine, hosted-redacted transport, query rewrite, or Handoff/Experience/Skill/duplicate-conflict consumer.
No retention of raw candidate text, evidence text, model rationale, or exception details for convenience.
Two citation issues in this tracking issue, verified against the live repo (2026-10-11):
Tracking issue Tracking Issue: evaluate and integrate Jev with PowerContext #1649 does not exist. The REST API returns 404 for issues/1649 and it is not a discussion either. If this refers to a deleted object the number should be replaced; if it is a typo, please correct the reference, since the acceptance criteria depend on which tracking items are actually outstanding.
Everything else in the issue's premises checked out: RFC #1770 is merged, the three promotion modes and the fallback invariant match the implementation.
Activity
docs(rfc): bound decision policy adoption
lizining1231 commentedon Oct 11, 2026
I'd like to take this issue. Is it okay if I start working on it?
AlexStocks commentedon Oct 11, 2026
Two citation issues in this tracking issue, verified against the live repo (2026-10-11):
Tracking issue Tracking Issue: evaluate and integrate Jev with PowerContext #1649 does not exist. The REST API returns 404 for
issues/1649and it is not a discussion either. If this refers to a deleted object the number should be replaced; if it is a typo, please correct the reference, since the acceptance criteria depend on which tracking items are actually outstanding."Draft PR feat(runtime): add decision policy governance types #1921" is outdated. PR feat(runtime): add decision policy governance types #1921 is now ready for review (
isDraft: false). The factual premises this issue relies on are verifiable in that PR:decision_observations.pyimplements theshadow/advisory/enforcingmodes, and it enforcesfallback => unadjudicatedwith aValueError. Suggest updating the wording to "PR feat(runtime): add decision policy governance types #1921".Everything else in the issue's premises checked out: RFC #1770 is merged, the three promotion modes and the fallback invariant match the implementation.
feat(runtime): add decision policy governance types (#1921)