Skip to content
Open
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
771d261
docs: add Context 2.0 design
sklinglernv Aug 19, 2026
0b881af
feat: implement composable context views
sklinglernv Aug 19, 2026
32c83dd
doc: new titlte for context management doc
sklinglernv Aug 19, 2026
8ca9a26
docs: refine context view architecture
sklinglernv Sep 7, 2026
37c2b87
feat: complete composable context views
sklinglernv Sep 7, 2026
057d5c7
docs: clarify default context view policy
sklinglernv Sep 8, 2026
e045cbc
feat: make default context assembly explicit
sklinglernv Sep 8, 2026
aca9570
test: add deterministic context parity experiment
sklinglernv Sep 8, 2026
b1a1d4c
test: assert parity scenario coverage
sklinglernv Sep 8, 2026
00f456c
fix: preserve context observability compatibility
sklinglernv Sep 8, 2026
50096f0
docs: record context parity evidence
sklinglernv Sep 8, 2026
7b1d3e4
test: cover legacy skill context parity
sklinglernv Sep 8, 2026
575aeb2
fix: preserve legacy skill context placement
sklinglernv Sep 8, 2026
fbadf06
docs: update final parity evidence
sklinglernv Sep 8, 2026
4d24bb9
fix: honor disabled skill context keys
sklinglernv Sep 8, 2026
b017e4c
docs: record disabled-skill compatibility fix
sklinglernv Sep 8, 2026
e7bcc20
feat(context): add explicit cache boundaries
sklinglernv Sep 9, 2026
daca2a4
refactor(context): make default view own assembly
sklinglernv Sep 11, 2026
1fddae4
fix(context): address redesign review findings
sklinglernv Sep 11, 2026
42b0c4b
docs(context): refresh verification evidence
sklinglernv Sep 11, 2026
cee1428
fix(context): preserve formatter and call compatibility
sklinglernv Sep 11, 2026
5749ac3
docs(context): record final review evidence
sklinglernv Sep 11, 2026
662b8e2
refactor(context): address review feedback
sklinglernv Sep 11, 2026
813f8cb
Merge origin/main into redesing-context-mechanics
sklinglernv Sep 15, 2026
ae75cdc
docs(context): align design with final implementation
sklinglernv Sep 15, 2026
9ab48d0
Merge remote-tracking branch 'origin/main' into redesing-context-mech…
sklinglernv Sep 15, 2026
3d4a7f2
fix(context): preserve replay and cache boundaries
sklinglernv Sep 16, 2026
0ceba38
fix(context): validate Anthropic message order
sklinglernv Sep 16, 2026
b6ca8b8
fix(context): preserve compatibility guarantees
sklinglernv Sep 18, 2026
1fb7143
Merge remote-tracking branch 'origin/main' into redesing-context-mech…
sklinglernv Sep 29, 2026
e27e6a1
refactor(context): make views own event selection and isolate state c…
sklinglernv Oct 2, 2026
43440c2
Merge current main into context mechanics redesign
sklinglernv Oct 2, 2026
171529f
Merge remote-tracking branch 'origin/main' into redesing-context-mech…
sklinglernv Oct 14, 2026
7e55782
Merge latest main while preserving view-owned context and state compa…
sklinglernv Oct 14, 2026
0ff96d0
fix(context): resolve review feedback on history, skills, and block g…
sklinglernv Oct 14, 2026
867105f
fix(context): isolate state summaries from inherited prompt overrides
sklinglernv Oct 14, 2026
b341ef2
fix(context): preserve omission of archived tool-call batches
sklinglernv Oct 14, 2026
252b3fa
docs: introduce context views in plain language
sklinglernv Oct 14, 2026
76bb9cb
fix(context): unify event provenance and state compaction
sklinglernv Oct 22, 2026
497acfa
fix(context): honor summary middleware and preserve empty-prefix boun…
sklinglernv Oct 23, 2026
436c2de
docs: define strategy-owned context assembly
sklinglernv Oct 27, 2026
edb3f32
Merge main and implement strategy-owned context views
sklinglernv Oct 27, 2026
15c2692
refactor(context): defer summary preparation and clean up view helpers
sklinglernv Oct 28, 2026
257fdc0
test(context): account for framework blocks in calibration budget
sklinglernv Oct 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 0 additions & 94 deletions docs/cache-sharing-summaries.md

This file was deleted.

188 changes: 188 additions & 0 deletions docs/context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
# Context Management in NOOA

A context view projects any owner state into the ordered context for one LLM turn. Agents, skills, and future components use this pattern; one agent view creates the complete context and may explicitly compose other views. The default view is a readable reference implementation. Rendering adds no context sources or placement policy.

## Contract

```python
class Block:
key: str
content: str
role: Role = Role.SYSTEM
metadata: BlockMetadata | None = None


class CacheBoundary:
"""Request caching of the rendered prefix ending here, where supported."""


ContextItem = Block | EventBase | CacheBoundary


class ContextView[Owner](Protocol):
def assemble(
self,
owner: Owner,
call: CurrentCall,
) -> AsyncIterator[ContextItem]: ...
```

`Block.content` is materialized; `metadata` carries optional rendering and budget hints. Events remain typed. `CacheBoundary` is a structural marker: it has no content or token cost and is not evictable. In the default view, `metadata.static` controls prefix placement independently of evaluation timing.

The runtime assembles the selected view for every LLM request and collects it into a `tuple[ContextItem, ...]`. Rendering preserves that item order except when expanding an atomic tool-call replay group. No additional assembled-context type is needed.

`CurrentCall` is a per-request snapshot of an invocation. Its invocation identity and method inputs stay stable; mutable manager and scoped state are captured again for each request. Views may read the resolved strategy, `model`, `context_window`, and `context_budget`. Internal formatting and token-counting data support the helpers. It contains no LLM client or credentials.

## Resolution

Highest precedence wins and replaces the lower view:

```text
Agent: call -> method -> instance -> class -> DefaultAgentView
Skill: instance -> class -> DefaultSkillView
```

```python
class MyAgent(Agent, context_view=AgentView()): ...
agent = MyAgent(context_view=OtherAgentView())
await agent.run(context_view=CallView())


class SearchSkill(Skill, context_view=SearchView()): ...
skill = SearchSkill(context_view=OtherSearchView())
```

`@strategy(..., context_view=view)` is the method override. Skills have no method or call override. Composition is ordinary Python; there is no merge protocol.

`Agent` and `Skill` expose `__context_view__()` as the owner-resolution hook. Registration storage is internal to those classes; generic resolution does not inspect it.

## Sources and helpers

`ContextView` is a projection interface, not a storage API. Agents retain `self.context` and `context_manager`; the default view uses them as one source. Custom views may use them, another state system, or neither.

A native view uses normal Python and yields materialized items directly:

```python
async def assemble(self, agent, call):
state = await agent.compute_state()
yield Block(key="state", content=context_text(state, call=call))
```

Small optional helpers each do one job:

```python
context_text(value, *, call) -> str
async collect_context_items(view, owner, call) -> tuple[ContextItem, ...]
apply_context_budget(items, *, call, evictable) -> tuple[ContextItem, ...]
async evaluate_context_expression(expression, *, owner, call) -> object
is_model_visible(event) -> bool
```

`context_text` formats one value; `collect_context_items` validates and retains yielded order; expression evaluation binds `self` to `owner`; `is_model_visible` identifies nonempty model-facing events without selecting history. Budgeting follows the caller's eviction order and counts blocks only, so it is not a full rendered-request limit. A source helper may retrieve, evaluate, or format one source. It does not select other sources or decide global precedence, placement, or order.

Iterative views must explicitly include the task, model outputs, and execution feedback they need. Strategies produce these as typed events; rendering injects none. `self.events.query()` searches stored history, including archived events, so it is an inspection API rather than prompt selection. `call.invocation_id` stays stable while a strategy may change `call.id`. Tool schemas and execution policy remain strategy-owned.

Views select active history directly. For example, a current-invocation view uses:

```python
for event in agent.event_manager.values():
if event.metadata.get("call_id") == call.invocation_id and is_model_visible(event):
yield event
```

## Defaults

`DefaultAgentView` lives in its own ordinary module and contains the complete default policy. In abbreviated form:

```python
class DefaultAgentView(ContextView[Agent]):
async def assemble(self, agent, call):
disabled = agent.context_manager.disabled()
declared = frozenset(key for key, _ in agent.context_manager.declarations())
blocks = await self._materialize_agent_blocks(
agent, call, disabled=disabled, declaration_keys=declared
)
blocks, skill_items = await self._compose_skill_views(
agent, call, blocks, disabled=disabled, declaration_keys=declared
)
blocks = await self._apply_override_layers(agent, call, blocks, disabled=disabled)
blocks = order_blocks(blocks, call.strategy.get_block_order())
prefix, trailing = partition_blocks(blocks)

history = [e for e in agent.event_manager.values() if is_model_visible(e)]
items = [*prefix, *skill_items, *history]
boundaries = sum(isinstance(item, CacheBoundary) for item in items)
if boundaries > 1:
raise ValueError("multiple cache boundaries")
if items and boundaries == 0:
items.append(CacheBoundary())
items.extend(trailing)
evictable = [*reversed(user_blocks(trailing)), *reversed(framework_blocks(trailing))]
for item in apply_context_budget(items, call=call, evictable=evictable):
yield item
```

The three private phase methods contain source-specific detail. `assemble()` keeps global order, event placement, cache placement, and budgeting visible. `apply_block_overrides` preserves replacement, deletion, inherited placement, and explicit prefix hints.

The three built-in helpers derive content directly from the agent. Protected-key registration selects which defaults apply; disable state and stored overrides are checked before evaluation. The manager stores those controls, user declarations, and the last materialized values for compatible named reads, but does not create defaults.

```python
class DefaultSkillView(ContextView[Skill]):
async def assemble(self, skill, call):
if skill.context_block is not None:
key, expression = skill.context_block
value = await evaluate_context_expression(
expression,
owner=call.agent, # preserves existing agent-scoped semantics
call=call,
)
yield Block(
key=key,
content=context_text(value, call=call),
role=Role.USER,
metadata=BlockMetadata(expr=expression, source_dynamic=True),
)
```

A skill view produces only that skill's ordered contribution; the agent view decides whether and where to include it. Native views normally read `skill` state directly. The fallback preserves the legacy `context_block` expression and agent binding as prompt projection, without registering it in `self.context`; explicit manager declarations win. Custom contributions are inserted intact without manager policy.

Registry skills participate when active; directly attached public skills are active by default. Hidden and inactive skills contribute nothing. `DefaultSkillView` emits only explicitly declared context; it never dumps skill documentation. The default agent view places custom skill contributions before visible events; a custom agent view may place volatile skill state after the cache boundary.

## Model-specific content

Model-specific prompt content remains view policy, not formatter policy. A custom view may collect items, run any application-defined adapter, apply its budget, then yield the final sequence. No framework-level prompt-adapter stage is required.

## Ownership

```text
resolve view
-> view assembles ContextItem items
-> tuple[ContextItem, ...]
-> render_context
-> ProviderFormatter
-> UnifiedLLM
```

- The selected agent view owns content, membership, materialization, filtering, global order, adaptation, and budget policy.
- Views own prompt compression and may call a summarizer on selected content. State compaction operates independently on stored state and replaces only the events it actually summarized. `TokenBudgetSummarizer` uses request usage as a trigger, not the rendered request as summary input.
- A nested view owns the content and local order of its contribution.
- Source-specific helpers translate existing state APIs; they do not choose global placement.
- The renderer expands items in place and emits no boundary text. A canonical assistant tool-call turn and its linked results form one atomic replay group; a boundary cannot split it.
- Provider formatting preserves a cache marker when its output can carry one and otherwise removes it. UnifiedLLM maps explicit boundaries to provider annotations or removes them when unsupported. Without a boundary, NOOA emits no explicit `cache_control` annotation.
- The selected agent view owns cache placement. The default preserves one boundary contributed by a skill, rejects multiple, or adds one after visible history and before trailing context; custom agent views receive none implicitly.
- Provider-managed automatic caching and transport-routing hints such as `prompt_cache_key` are independent of explicit cache breakpoints.
- Bounded serialization is formatting; recovery for missing data or other invented content belongs to event production or view policy.
- Downstream stages preserve view order, except for declared atomic replay groups, or raise `UnsupportedContextLayout`; they do not resolve, evict, repair, or invent content.
- A view omission is prompt policy, not an access-control boundary; tools and generated code may expose data available through other APIs.

## Migration

Keep `agent.context`, context managers, event creation, strategy/scoped overrides, skill activation, and `Skill.context_block` as public state APIs.

These are the default application's context APIs, not requirements of `ContextView`. A custom view may use an independent state API. Native iterative strategies still use NOOA events unless replaced together with the strategy.

Legacy implicit policy is removed: `cache_control_injection_points` raises `ValueError`; `CachedBlockFormatter` no longer chooses placement; and `render_context(context_limit=...)` reports the limit but leaves eviction to the view.

`EventQuery`, agent `event_query=`, `ScopedContext(events=...)`, and manager query overrides are removed. Event membership is view code; event inspection and context-block overrides remain available.

`DefaultAgentView` and its helpers use only public agent, call, manager, strategy, and event interfaces. `ActorRuntime` supplies call facts and budget support, resolves and collects the view, renders it, and handles transport and observability; it does not assemble context content.
18 changes: 8 additions & 10 deletions docs/stable-prefix-caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,8 @@ policies would make their interactions harder to test.
stable input are present. Without them, they send no extra cache fields.
- `cache_breakpoint="anthropic"` explicitly selects the Anthropic Chat mapping,
including gateway aliases that cannot be recognized automatically.
- `cache_breakpoint="openai"` forces the Responses explicit-cache policy, including
its leading-instructions fallback without a boundary. It is an override, not
required for normal use.
- `cache_breakpoint="openai"` forces the Responses explicit-cache policy when a
boundary is present. It is an override, not required for normal use.
- `cache_breakpoint=None` disables NOOA-generated cache markers, not the
provider's implicit cache. Use this opt-out for a Responses endpoint that does
not support explicit-cache fields. Responses clients do not accept the
Expand All @@ -31,9 +30,9 @@ model: use a new client when switching models. The automatic mapping is resolved
against the effective per-call model.

`CacheBoundary` belongs to the UnifiedLLM interface, alongside `LLMResponse`.
The cached renderer inserts it immediately before live context; the formatter
and middleware pass the same object through unchanged. Direct callers use it
in their message list too:
The default context view inserts it immediately before live context; the renderer,
formatter, and middleware pass the same object through unchanged. Direct callers
use it in their message list too:

```python
from nooa.unifiedllm import CacheBoundary
Expand All @@ -46,9 +45,8 @@ Only UnifiedLLM interprets and removes the boundary before sending the request.
NeMo Relay projects it to public metadata JSON at its serialization boundary,
then restores the original object if that entry is unchanged. A raw dictionary
with `nooa_cache_boundary` is rejected with instructions to use `CacheBoundary()`;
there is only one accepted boundary type. Without a boundary, Completion and
forced explicit Responses policies mark only leading system/developer instructions,
not arbitrary history. Automatic Responses adds no cache fields.
there is only one accepted boundary type. Without a boundary, NOOA adds no cache
fields.

## Provider mapping

Expand Down Expand Up @@ -94,7 +92,7 @@ the framework setting must never become a provider request field.
1. `unifiedllm/cache_policy.py` owns the single policy. It consumes the boundary
and changes only the final marker target's containers; unrelated messages
and large strings are shared.
`CacheBoundary` is a small, immutable UnifiedLLM input type. The cached renderer
`CacheBoundary` is a small, immutable UnifiedLLM input type. The default view
places it before live context using the same pass-through path as assistant
responses. The formatter does not translate it or know what it means;
ordinary messages carry no cache flag and are not edited.
Expand Down
Loading