STATE_ROOTS.md

State Roots Contract

This document defines the canonical locations for all durable state and artifacts in Codex Autorunner (CAR), the retention taxonomy, and the "no shadow state" contract.

Canonical State Roots

All durable artifacts must live under one of these roots:

1. Repo-Local Root

Location: <repo_root>/.codex-autorunner/

Purpose: Per-repository runtime state, tickets, context, and configuration.

Contents:

• tickets/ - Ticket queue (required)
• contextspace/ - Durable context (active_context.md, decisions.md, spec.md)
• config.yml - Generated repo config
• state.sqlite3 - Run state database
• codex-autorunner.log - Runner logs
• lock - Lock file for exclusive access
• runs/ - Run artifacts and dispatch
• flows/ - Flow artifacts
• flows.db - Flow store database
• pma/ - PMA state and queue
• archive/ - Reviewable retained output (worktree snapshots under archive/worktrees/, run archives under archive/runs/). These are not live source of truth; see retention classes below.
• bin/ - Generated helper scripts
• workspace/ - Compatibility-only input (not a canonical store). CAR reads durable context from contextspace/. The workspace/ fallback is used during archive/reset operations only, and has a single owner: _contextspace_source() in core/archive.py. No new runtime code may read from workspace/ directly. If this directory still exists, car doctor may warn until you migrate or remove it. See workspace → contextspace migration.
• app_server_workspaces/ - App-server supervisor/workspace state when the effective destination is docker
• filebox/ - Shared attachment ingress/compatibility root (filebox/inbox, filebox/outbox)

Notable repo-local artifacts:

• flows/<run_id>/chat/inbound.jsonl - Mirrored inbound chat events for a flow run
• flows/<run_id>/chat/outbound.jsonl - Mirrored outbound chat events for a flow run
• tickets/ingest_state.json - Canonical ticket-ingest receipt (ingested, ingested_at, source)
• filebox/outbox/ - Legacy artifact ingress for the active target scope, including car render screenshot/observe/demo outputs; journal-backed delivery records are the source of truth
• usage/opencode_turn_usage.jsonl - Per-turn OpenCode usage snapshots (canonical, high-confidence source for repo usage reports)

Resolution: resolve_repo_state_root(repo_root) in core/state_roots.py

2. Hub Root

Location: <hub_root>/.codex-autorunner/

Purpose: Hub-level state for typed managed resources (repos[]), orchestration metadata, and cross-resource coordination.

Contents:

• manifest.yml - Managed hub resources list (repos: and per-repo destination config)
• hub_state.json - Hub state
• config.yml - Hub config
• codex-autorunner-hub.log - Hub logs
• templates/ - Hub-scoped templates
• runtimes/ - CAR-managed runtime workspace roots under <runtime>/<workspace_id>/
• chat/channel_directory.json - Cross-platform channel directory used for lightweight routing context
• orchestration.sqlite3 - Hub SQLite store for orchestration metadata, bindings, executions, transcript mirrors, and event projections

Resolution: Hub root is typically the hub's repo root, using repo-local patterns.

Orchestration SQLite Database

The orchestration.sqlite3 database is the canonical store for:

• Thread/binding metadata: Durable CAR thread registrations plus external channel bindings (Discord channel IDs, Telegram chat IDs) to consistent thread targets under repos, worktrees, or filesystem-scoped roots
• Execution state: Active/running/queued orchestration work items
• Transcript mirrors: Plain-text user/assistant message copies for search, previews, and debugging
• Event projections: Aggregated views of automation events, flow executions

What orchestration.sqlite3 does NOT store (remains authoritative elsewhere):

• Flow engine internals: Still stored in repo-local flows.db
• Delivery/outbox state: Discord and Telegram state databases remain authoritative for message delivery
• Runtime conversation history: Downstream runtimes (Codex, OpenCode) own their canonical conversation state
• Reasoning traces/tool payloads: Kept in runtime-native stores

Resolution: resolve_hub_orchestration_db_path(hub_root) in core/state_roots.py

Managed runtime roots (hub)

CAR may allocate hub-local directories for runtime-specific durable files under:

• <hub_root>/.codex-autorunner/runtimes/<runtime>/

Use resolve_hub_runtimes_root and resolve_hub_runtime_root for path authority. CAR does not install third-party runtimes as part of the state-root contract.

Resolution:

• resolve_hub_runtimes_root(hub_root)
• resolve_hub_runtime_root(hub_root, runtime=...)

3. Global Root

Location: ~/.codex-autorunner/ (configurable via CAR_GLOBAL_STATE_ROOT)

Purpose: Cross-repo caches, shared resources, update state.

Contents:

• update_cache/ - Cached update artifacts
• update_status.json - Update status
• update_snapshots/ - Timestamped update rollback snapshots, including orchestration DB copies created by macOS safe refresh
• locks/ - Cross-repo locks (e.g., telegram bot lock)
• workspaces/ - App-server workspaces (default for non-docker destinations)

Update snapshot retention:

• Update orchestration (staged install, cutover, restart, health, rollback) lives in the Python UpdateEngine (codex_autorunner.core.update.engine). Both scripts/safe-refresh-local-mac-hub.sh and scripts/safe-refresh-local-linux-hub.sh are thin wrappers that delegate to python -m codex_autorunner.core.update.runner.
• The engine stores update rollback snapshots under ~/.codex-autorunner/update_snapshots/ when UPDATE_STATUS_PATH uses the global state root.
• Snapshot pruning runs after every snapshot-db update phase. The default keeps only the newest snapshot directory needed for rollback.
• Override with UPDATE_SNAPSHOT_MAX_COUNT. Set it to 0 to disable pruning.
• Manual inspection and pruning:

du -sh ~/.codex-autorunner/update_snapshots
find ~/.codex-autorunner/update_snapshots -maxdepth 1 -mindepth 1 -type d -print
python -m codex_autorunner.core.update_transaction prune-snapshots \
  --snapshot-root ~/.codex-autorunner/update_snapshots

The pruning helper only deletes directories with valid orchestration/snapshot.json metadata, skips the current update run when provided, and skips candidate directories when lsof reports open files inside them or cannot prove they are closed.

Docker destination override:

• When a repo/worktree runs with effective destination docker, supervisor state root is forced to:
  • <repo_root>/.codex-autorunner/app_server_workspaces
• Rationale: docker-wrapped commands execute inside the repo bind mount, so state must be writable and visible from that mount.
• This still satisfies the canonical state contract because the override remains under repo-local .codex-autorunner/.

Resolution: resolve_global_state_root() in core/state_roots.py

Config Override: Set state_roots.global in config or CAR_GLOBAL_STATE_ROOT env var.

Retention Taxonomy

Canonical roots define where CAR is allowed to store state. Retention classes define how long that state should be kept.

A path can be canonical without being retained forever. For example, state.sqlite3, lock, and workspace directories are canonical locations but are runtime state, not durable artifacts.

Retention Classes

Class	Description	Cleanup Behavior
durable	Source-of-truth or long-lived artifacts	Never deleted; may be compacted in place
reviewable	Bounded review artifacts	Pruned by explicit age/count/byte policy
ephemeral	Runtime, staging, or transient artifacts	Aggressively pruned when inactive
cache-only	Rebuildable caches	Deleted freely when not in active use

Per-Artifact Retention Mapping

The canonical retention contract is defined in .codex-autorunner/contextspace/spec.md with detailed per-path mappings.

Key durable artifacts (never deleted):

• tickets/ — ticket queue
• contextspace/ — durable context docs (active_context.md, decisions.md, spec.md)
• context_log.md — context log snapshots
• runs/<run_id>/dispatch_history/ and reply_history/ — turn history
• flows.db — flow engine store
• Hub orchestration.sqlite3 — orchestration store
• Stable reports (reports/latest-*, final_report.md)

Key reviewable artifacts (bounded pruning; these are retained output, not live source of truth):

• archive/worktrees/** — worktree snapshots (reviewable retained output)
• archive/runs/** — run archives (reviewable retained output; FlowStore in flows.db is the canonical live run-history store)
• flows/<run_id>/ — flow artifacts
• reports/ history files
• github_context/
• review/runs/ — review run artifacts

Key ephemeral artifacts (aggressive pruning):

• state.sqlite3 — runtime state database
• app_server_workspaces/ — repo-local workspace state
• filebox/inbox/, filebox/outbox/ — attachment staging
• *.log files
• uploads/**
• update-standalone.log
• lock files

Key cache-only artifacts:

• update_cache/ — update artifacts cache

Cleanup Command

Use car cleanup state to reclaim disk space across all retention families:

# Preview what would be deleted
car cleanup state --dry-run

# Clean repo-local state only
car cleanup state --scope repo

# Clean global state only
car cleanup state --scope global

# Clean all scopes
car cleanup state --scope all

For details on the cleanup contract and safety guards, see State Cleanup Operations.

Compatibility-Only Inputs

These paths exist only as fallback or migration inputs. They must not regain runtime ownership, and no new code path may treat them as canonical:

Path	Canonical owner	Fallback behavior
.codex-autorunner/workspace/	contextspace/	_contextspace_source() in core/archive.py falls back to workspace/ during archive/reset only
Legacy numeric run directories under runs/<int>/	flows.db / FlowStore	Not canonical for new runs; retained only for backwards compatibility
Incomplete staging dirs under archive/ without META.json	Ignored	Intentionally invisible to retention pruning

Best-Effort And Skip-On-Partial-Visibility Contracts

Several cleanup and archive operations are intentionally best-effort:

• Retention pruning after archive: worktree/CAR-state archive treats post-archive retention pruning as best-effort follow-up that must not fail the archive itself.
• Run-archive fallback: core/flows/archive_helpers.py falls back to default run-archive retention policy when repo config cannot be loaded.
• Incomplete snapshots ignored: prune_worktree_archive_root() skips snapshot directories that lack META.json, keeping incomplete staging work out of retention logic.
• Stable report preservation: prune_report_directory() always preserves stable-prefix outputs (latest-*, final_report.md) even when the history budget is tight.
• Global workspace cleanup skip: global workspace cleanup skips entirely when hub manifest visibility is too weak to prove which shared workspaces are active.
• Filebox symlinks ignored: prune_filebox_root() ignores symlinks and non-files.

Non-Canonical Locations (Caches Only)

These locations are explicitly non-canonical (ephemeral, disposable):

Location	Purpose	Notes
/tmp/, $TMPDIR	Temporary files	Never durable
$XDG_CACHE_HOME or ~/.cache	Optional caches	Must be rebuildable
__pycache__/	Python bytecode	Auto-generated

Rule: Any location outside the canonical roots must be:

1. A true cache (data is derivable from canonical sources)
2. Explicitly documented here
3. Safe to delete without data loss

Authority Boundaries

This section defines which store is authoritative for which data domain.

Repo-Local flows.db (Flow Engine Internals)

Location: <repo_root>/.codex-autorunner/flows.db

Remains authoritative for:

• Flow definition storage and versioning
• Flow execution state machine (pending, running, completed, failed)
• Flow step orchestration and dependency resolution
• Flow variable and context persistence

The hub orchestration.sqlite3 may contain projections of flow execution status, but flows.db is the source of truth for flow-engine internals.

Transport State Databases (Discord, Telegram)

Discord: <repo_root>/.codex-autorunner/discord_state.sqlite3 Telegram: <repo_root>/.codex-autorunner/telegram_state.sqlite3

These transport-specific databases remain authoritative for:

• Per-channel/topic delivery state and outbox queues
• Platform-specific message metadata
• Webhook state and callback tracking

Binding metadata for Discord and Telegram ordinary turns lives in hub orchestration.sqlite3, while delivery state remains in the transport databases.

PMA Persistence: Canonical vs Compatibility Mirrors

All canonical PMA state lives in orchestration.sqlite3 tables:

Table	Owner	Purpose
orch_thread_targets	PmaThreadStore	Managed thread registrations and lifecycle
orch_thread_executions	PmaThreadStore	Managed turn execution records
orch_thread_actions	PmaThreadStore	Thread-level action records
orch_queue_items	PmaQueue / PmaThreadStore	Lane items and thread-execution queue items
orch_automation_rules	AutomationStore	Unified automation rule definitions, including built-ins
orch_automation_events	AutomationStore	Normalized lifecycle, SCM, schedule, repo, and manual events
orch_automation_jobs	AutomationStore	Durable automation job ledger and execution refs
orch_automation_job_attempts	AutomationStore	Per-attempt execution results and errors
orch_automation_schedules	AutomationStore	Derived schedule state for scheduled rules
orch_automation_subscriptions	Migration diagnostics	Legacy PMA subscription residue, not runtime source of truth
orch_automation_timers	Migration diagnostics	Legacy PMA timer residue, not runtime source of truth
orch_automation_wakeups	Migration diagnostics	Unsupported legacy wakeup residue reported by migration diagnostics
orch_reactive_debounce_state	PmaReactiveStore	Reactive debounce timestamps

Migration blockers for remaining compatibility rows are surfaced by car doctor --json, car hub orchestration status --json, and car automation migration-status --json. The JSON diagnostics report pending schema versions, legacy residue counts, legacy executor-shape blockers, and operator next steps. Normal runtime no longer imports, mirrors, or auto-materializes legacy PMA subscription, timer, or wakeup rows into unified automation rules.

Filesystem mirrors are not the source of truth:

Mirror path	Owner	Notes
.codex-autorunner/pma/queue/*.jsonl	PmaQueue	Rewritten after every queue mutation; replay_pending reads from SQLite
.codex-autorunner/pma/automation_store.json	Migration diagnostics	Obsolete legacy automation artifact; remove after diagnostics are clear
.codex-autorunner/pma/reactive_state.json	PmaReactiveStore	Rewritten after every debounce update
.codex-autorunner/pma/threads.sqlite3	PmaThreadStore	Legacy thread mirror, gated by CAR_LEGACY_MIRROR_ENABLED

Deleting any mirror file does not affect correctness; SQLite remains authoritative and mirrors are regenerated on the next write.

Separate from persistence mirrors, PmaStateStore owns PMA runtime state in .codex-autorunner/pma/state.json. This file is not part of the canonical queue/thread/automation persistence path.

Pre-Orchestration Legacy Artifacts

Locations:

• <repo_root>/.codex-autorunner/pma/deliveries.jsonl
• <repo_root>/.codex-autorunner/pma/thread_*.json
• <repo_root>/.codex-autorunner/pma/queue.json

These are pre-orchestration artifacts used only for initial migration into hub SQLite. No new data should be written to these paths.

No Shadow State Contract

Invariant: All durable state and artifacts must be representable under a canonical root. No "shadow" state directories outside these roots.

What This Means

1. No ad-hoc roots: Don't create new state directories outside the roots
2. Single source of truth: Canonical roots are the source of truth
3. Portable state: All durable state can be moved by relocating the root
4. Testable boundaries: Tests verify no writes outside allowed roots

Enforcement

• core/state_roots.py provides the single authority for root resolution
• Call sites should use resolve_repo_state_root() and resolve_global_state_root()
• Tests verify boundary enforcement (see tests/core/test_state_roots.py)

Path Authority Module

The core/state_roots.py module provides:

def resolve_repo_state_root(repo_root: Path) -> Path:
    """Return the repo-local state root (.codex-autorunner)."""

def resolve_global_state_root(*, config=None, repo_root=None) -> Path:
    """Resolve the global state root for cross-repo caches and locks."""

def resolve_hub_state_root(hub_root: Path) -> Path:
    """Return the hub-scoped state root."""

def resolve_hub_orchestration_db_path(hub_root: Path) -> Path:
    """Return the canonical orchestration SQLite path under the hub state root."""

def resolve_hub_templates_root(hub_root: Path) -> Path:
    """Return the hub-scoped templates root."""

def resolve_hub_runtimes_root(hub_root: Path) -> Path:
    """Return the hub-scoped root for CAR-managed runtime workspaces."""

def resolve_hub_runtime_root(hub_root: Path, *, runtime: str) -> Path:
    """Return the managed root for one runtime under the hub."""

Usage Pattern

from codex_autorunner.core.state_roots import resolve_repo_state_root

state_root = resolve_repo_state_root(repo_root)
db_path = state_root / "state.sqlite3"
log_path = state_root / "codex-autorunner.log"

Migration Notes

When adding new durable artifacts:

1. Determine the appropriate root (repo-local, hub, or global)
2. Use resolve_*_state_root() functions, not ad-hoc path construction
3. Document the artifact in this file
4. Ensure tests cover the boundary