fix(context): align formatter guidance and replay contracts

docs/context.md

@@ -142,6 +142,80 @@ class DecisionContextView(ContextView[Agent]):
 
 Registry skills participate when active; directly attached public skills are active by default. Hidden and inactive skills contribute nothing in the default views.
 
+## Formatting independently of views
+
+Use `RenderConfig` to select presentation without implementing another view:
+
+```python
+from nooa.context_blocks import MarkdownBlockFormatter, RenderConfig
+
+agent = MyAgent(
+    render_config=RenderConfig(block_formatter=MarkdownBlockFormatter()),
+)
+```
+
+`XMLBlockFormatter` is the XML alternative. The legacy
+`nooa.context_blocks.renderers.CachedBlockFormatter` import remains a compatibility
+subclass of `XMLBlockFormatter`, with identical serialization and guidance.
+Neither XML nor Markdown enables caching: both preserve the boundary chosen by
+the view. `RenderConfig` chooses block/event serialization and client message
+shape; it does not choose context sources, history membership, order, or eviction.
+
+`Agent`'s base instructions are format-neutral. The built-in manager-backed
+context views explicitly include the selected formatter's `format_description()`
+in the built-in Agent prompt. User class docstrings, `_resolve_system_prompt()`
+implementations, and manager/view/decorator/scoped prompt overrides retain ownership of their
+instructions. Disabling `system_prompt` also disables default format guidance.
+Guidance is evaluated only after override resolution, when the built-in prompt
+survives; replacing or removing that prompt does not call `format_description()`.
+Custom views receive no implicit guidance; they may explicitly use
+`agent.render_config.block_formatter.format_description()` when desired.
+Rendering itself never injects instructions.
+
+`context_text(value, call=call)` is separate Python-value serialization. It
+applies the call's structural bounds to **non-string** values. Existing strings
+pass through unchanged, even when longer than `max_string`; budget or shorten
+such content explicitly in the view.
+
+For plain event-field serialization, select
+`RenderConfig(block_formatter=PlainBlockFormatter())`, importing
+`PlainBlockFormatter` from `nooa.plain_formatter`. This retains XML wrappers for
+materialized blocks and optional event role/tag metadata. It is not a tag-free
+format; canonical assistant responses retain their original text and replay data.
+
+Block formatters receive one ordered resolved segment at a time, with non-tool
+event text already serialized. They may coalesce adjacent materialized blocks of
+the same role, without partitioning all system blocks ahead of history. The
+renderer handles boundary markers between segments; it does not truncate inputs.
+
+### Atomic tool replay exception
+
+A canonical `LLMResponse` tool-call batch and its linked `ToolCallEvent` results
+form one replay group. Stock formatters emit the response at its original
+position, followed by all results in tool-call order; linked execution sidecars
+emit nothing at their stored positions. Thus a result stored later than an
+unrelated block can move into its response's group. A cache boundary cannot
+split a response from its linked results.
+
+At the formatter level, incomplete canonical groups are safely omitted as a
+group, including orphan linked executions. The renderer rejects a supplied
+`ToolCallEvent` with `result=None` before formatting. This does not authorize
+arbitrary reordering or fabrication of missing results. Default-view recovery of an **existing** execution record
+with an unavailable result is a separate, explicit view policy; absent/archived
+records are not reconstructed. Wrapper-style customizations can override
+`wrap_system_block()` and `wrap_message_block()` on `BlockFormatter` while
+inheriting `format()` to preserve these rules. The first hook formats materialized
+blocks of any role; the second formats ordinary events. Canonical assistant
+replay bypasses the event wrapper.
+
+`RenderedMessage` is the neutral intermediate representation, including a
+private replay carrier. `OpenAIProviderFormatter.format()` returns client input
+containing ordinary dictionaries, original `LLMResponse` objects, and
+`CacheBoundary` markers. It is not final provider wire JSON. UnifiedLLM owns
+provider-specific encoding, native reasoning replay, exact tool argument data,
+and cache annotation mapping. Do not flatten a retained response into a
+mapping merely to satisfy an annotation or change XML/Markdown wrapping.
+
 ## Model-specific content
 
 Model-specific prompt content remains view policy, not formatter policy. A custom view may collect items, run any application-defined adapter, apply its budget, then yield the final sequence. No framework-level prompt-adapter stage is required.

src/nooa/agent.py

@@ -75,9 +75,9 @@ class Agent(metaclass=AgentMeta):
     """You are {type(self).__name__}, a Python agent working in an interactive session.
 
     ## Context blocks
-    Your prompt is organized in XML context blocks: `<name>CONTENT</name>`.
-    Blocks produced by `self.context[...]` carry an `expr="..."` attribute whose value is the Python expression re-evaluated each turn.
-    Event history: system entries in `<sys tag="N">`; reference via `self.events["N"]`.
+    Your prompt contains instructions, context, and selected event history.
+    Expression context is re-evaluated each turn. Event tags identify stored
+    history entries, which can be referenced via `self.events["N"]`.
 
     ## Truncation
     - A bare Python literal (`[1, 2, 3]`, `{{1: 2}}`, `'hello'`) is always complete.

src/nooa/context_blocks/formatter.py

@@ -6,13 +6,14 @@
 
 1. :class:`BlockFormatter` — turns a list of resolved blocks (both system-role
    context blocks and event blocks) into a neutral ``list[RenderedMessage]``.
-   Owns semantic choices: how each block is wrapped (XML / Markdown / plain),
-   how events are serialized, how system blocks are concatenated into a single
-   system message.
+   Preserves view-selected order and membership while wrapping blocks
+   (XML / Markdown / plain), serializing events, and coalescing adjacent
+   materialized blocks with the same role. Canonical tool replay is atomic.
 
 2. :class:`ProviderFormatter` — a thin syntactic adapter that reshapes the
-   neutral message list into the OpenAI-compatible ``list[dict]`` that
-   UnifiedLLM accepts. Provider-specific wire translation happens downstream
+   neutral message list into client input: ordinary message dictionaries,
+   retained ``LLMResponse`` objects, and ``CacheBoundary`` markers.
+   Provider-specific wire translation happens downstream
    (LiteLLM, or the client's own projection), never here.
 
 This split keeps the "format" axis (XML / Markdown / Plain) orthogonal to the
@@ -27,7 +28,7 @@
 
 if TYPE_CHECKING:
     from nooa.config.truncation_config import FormatConfig
-    from nooa.llm_types import LLMResponse
+    from nooa.llm_types import CacheBoundary, LLMResponse
 
 from nooa.agentdoc import pformat
 from nooa.context_blocks.events import CODEACT_INLINE_RETURN, EventBase, ToolCallEvent
@@ -56,7 +57,7 @@ def _is_llm_response(event: EventBase | None) -> TypeGuard["LLMResponse"]:
 
 
 class FormatType(StrEnum):
-    """Format type identifier used for block-level truncation."""
+    """Block/message wrapping style, independent of selection and budgeting."""
 
     XML = "xml"
     MARKDOWN = "markdown"
@@ -76,35 +77,54 @@ class FormatType(StrEnum):
 class BlockFormatter(ABC):
     """Turn resolved blocks into a neutral list of :class:`RenderedMessage`.
 
-    Subclasses implement :meth:`format` to decide both the ordering of messages
-    and the wrapping of each block's content. The stock implementations
-    partition by ``block.role``: SYSTEM blocks concatenate into a single system
-    message, event blocks emit per-event messages.
+    The view owns selection, placement, eviction, and cache boundaries.
+    Subclasses serialize its ordered inputs without repartitioning by role.
+    Stock implementations coalesce adjacent materialized blocks of the same
+    role and expand canonical tool-call responses with their linked results
+    atomically at the response position. Incomplete replay groups may be
+    omitted; missing results are not invented. Wrapper customizations should
+    reuse the stock formatting path to retain these replay rules.
     """
 
     @property
     @abstractmethod
     def format_type(self) -> FormatType:
         """Return format type (FORMAT_XML / FORMAT_MARKDOWN / FORMAT_PLAIN).
 
-        Used by :func:`render_context` to pick the right block-level truncation
-        strategy for oversized content.
+        Identifies wrapper style for message-formatting helpers; it does not
+        select content or control truncation, eviction, or caching.
         """
         ...
 
-    @abstractmethod
     def format(self, blocks: list[ResolvedBlock]) -> list[RenderedMessage]:
         """Convert all resolved blocks into a neutral message list.
 
         Args:
-            blocks: All resolved blocks, already truncated and with event
-                content pre-serialized by :func:`render_context`. System
-                blocks appear before event blocks.
+            blocks: One ordered segment selected by the view, with non-tool
+                event content serialized by :func:`render_context`. Roles may
+                be interleaved. Cache markers delimit segments outside this
+                method; no renderer-owned eviction or truncation occurs.
 
         Returns:
             A list of :class:`RenderedMessage`, in final display order.
         """
-        ...
+        return _build_messages(
+            blocks,
+            wrap_system=self.wrap_system_block,
+            wrap_message=self.wrap_message_block,
+        )
+
+    def wrap_system_block(self, block: ResolvedBlock) -> str:
+        """Wrap one materialized block, regardless of its selected message role.
+
+        Override this hook to customize presentation while retaining shared
+        ordering and canonical tool-replay handling in :meth:`format`.
+        """
+        return block.content
+
+    def wrap_message_block(self, block: ResolvedBlock) -> str:
+        """Wrap an ordinary event; canonical assistant replay bypasses this hook."""
+        return block.content
 
     def format_event(
         self,
@@ -142,8 +162,9 @@ def format_event(
     def format_description(self) -> str:
         """Return a human-readable description of the block format.
 
-        Embedded in the default agent system prompt so the LLM understands
-        how context blocks are structured. Subclasses should override.
+        Built-in manager-backed context views explicitly include this in the
+        built-in Agent prompt. Custom prompts and views choose their own guidance;
+        rendering never injects instructions. Return an empty string for no guidance.
         """
         return ""
 
@@ -156,10 +177,9 @@ def format_description(self) -> str:
 def _xml_system_block(block: ResolvedBlock) -> str:
     """Wrap a system block's content in an XML tag.
 
-    ``expr=`` is only emitted for blocks produced by
-    :meth:`ContextApi.set_dynamic` — static blocks, framework blocks, and
-    strategy overrides render without the attribute so the prompt doesn't
-    leak internal accessor syntax the LLM can't use.
+    ``expr=`` is emitted only when metadata marks a dynamic source, including
+    expression context and default framework sources. Literal declarations
+    and value overrides omit the attribute.
     """
     show_expr = block.metadata.source_dynamic and block.metadata.expr
     attrs = f' expr="{block.metadata.expr}"' if show_expr else ""
@@ -336,8 +356,12 @@ def _event_block_projections(
     """Project each input position, keeping canonical tool batches atomic.
 
     A replayable ``LLMResponse`` expands at its own position to the assistant
-    call batch followed by all linked results. Linked execution sidecars then
-    project to an empty list at their stored positions.
+    call batch followed by all linked results in call order, even when their
+    stored positions are interleaved with other items. Linked execution
+    sidecars then project to an empty list at their stored positions. Missing
+    or incomplete canonical groups are omitted without inventing results.
+    This replay exception does not authorize arbitrary history reordering.
+    Wrapper customizations can reuse this helper through _build_messages().
     """
     replayable_turn_ids = {
         block.event.id
@@ -461,7 +485,9 @@ def flush_blocks() -> None:
 
 
 class XMLBlockFormatter(BlockFormatter):
-    """Wrap each system block in an XML tag; wrap each event in a role tag.
+    """Wrap materialized blocks in XML; annotate event role/tag metadata.
+
+    Canonical assistant responses replay unwrapped.
 
     Example system output::
 
@@ -471,9 +497,9 @@ class XMLBlockFormatter(BlockFormatter):
 
     Example event output::
 
-        <user_message expr='self.events["1"]' tag="1">
+        <sys tag="1">
         Hello world
-        </user_message>
+        </sys>
     """
 
     @property
@@ -483,30 +509,37 @@ def format_type(self) -> FormatType:
     def format_description(self) -> str:
         return (
             "Your prompt is organized in XML context blocks: `<name>CONTENT</name>`.\n"
-            "Blocks produced by `self.context.set_dynamic()` carry an "
+            "Expression context blocks carry an "
             '`expr="..."` attribute whose value is the Python expression '
             "re-evaluated each turn.\n"
             'Event history: system entries in `<sys tag="N">`; '
             'reference via `self.events["N"]`. Your own messages appear '
             "as plain assistant turns."
         )
 
-    def format(self, blocks: list[ResolvedBlock]) -> list[RenderedMessage]:
-        return _build_messages(
-            blocks,
-            wrap_system=_xml_system_block,
-            wrap_message=_xml_message_content,
-        )
+    def wrap_system_block(self, block: ResolvedBlock) -> str:
+        return _xml_system_block(block)
+
+    def wrap_message_block(self, block: ResolvedBlock) -> str:
+        return _xml_message_content(block)
 
 
 class MarkdownBlockFormatter(BlockFormatter):
-    """Render each system block as a Markdown section; wrap events with Markdown headers.
+    """Render materialized blocks as Markdown sections and annotate event metadata.
+
+    Canonical assistant responses retain their original text and replay data.
 
     Example system output::
 
         # Notes `{"expr": "self.context['notes']"}`
 
         Here are my notes...
+
+    Example event output::
+
+        ### User Message `{"tag": "1"}`
+
+        Hello world
     """
 
     @property
@@ -517,26 +550,28 @@ def format_description(self) -> str:
         return (
             "Your prompt is organized in context blocks with markdown headers: "
             "`# Block Name`.\n"
-            "Blocks produced by `self.context.set_dynamic()` carry an inline "
+            "Expression context blocks carry an inline "
             '`{"expr": "..."}` annotation whose value is the Python expression '
-            "re-evaluated each turn."
+            "re-evaluated each turn.\n"
+            'Event history carries `{"tag": "N"}` metadata; '
+            'reference entries via `self.events["N"]`. Your own messages '
+            "retain their original assistant text and replay data."
         )
 
-    def format(self, blocks: list[ResolvedBlock]) -> list[RenderedMessage]:
-        return _build_messages(
-            blocks,
-            wrap_system=_markdown_system_block,
-            wrap_message=_markdown_message_content,
-        )
+    def wrap_system_block(self, block: ResolvedBlock) -> str:
+        return _markdown_system_block(block)
+
+    def wrap_message_block(self, block: ResolvedBlock) -> str:
+        return _markdown_message_content(block)
 
 
 # ---------------------------------------------------------------------------
-# ProviderFormatter — neutral messages → provider wire format
+# ProviderFormatter — neutral messages → client input
 # ---------------------------------------------------------------------------
 
 
 class ProviderFormatter(ABC):
-    """Reshape a neutral :class:`RenderedMessage` list into provider wire format.
+    """Reshape neutral messages into client input, preserving replay carriers.
 
     Thin syntactic adapter: does not wrap content, does not re-order, does not
     decide what goes where. Just translates message shape and tool-call
@@ -545,11 +580,13 @@ class ProviderFormatter(ABC):
 
     @abstractmethod
     def format(self, messages: list[RenderedMessage]) -> Any:
-        """Return provider-specific output for this message list."""
+        """Return client input; final provider wire encoding happens downstream."""
         ...
 
 
-def _append_openai_image_message(out: list[dict], msg: RenderedMessage) -> None:
+def _append_openai_image_message(
+    out: "list[dict[str, Any] | LLMResponse | CacheBoundary]", msg: RenderedMessage
+) -> None:
     content_parts: list[dict] = [{"type": "text", "text": msg.content or ""}]
     content_parts.extend(msg.images or [])
     out.append({"role": msg.role.value, "content": content_parts})
@@ -563,10 +600,16 @@ def _with_reasoning(message: dict[str, Any], reasoning: str | None) -> dict[str,
 
 
 class OpenAIProviderFormatter(ProviderFormatter):
-    """Emit OpenAI-compatible messages (``list[dict]``)."""
+    """Emit UnifiedLLM input while retaining native replay and cache markers.
+
+    This is not final wire encoding. The client owns provider-specific replay
+    projection, including retained reasoning and exact tool argument strings.
+    """
 
-    def format(self, messages: list[RenderedMessage]) -> list[dict]:
-        out: list[dict] = []
+    def format(
+        self, messages: list[RenderedMessage]
+    ) -> "list[dict[str, Any] | LLMResponse | CacheBoundary]":
+        out: list[dict[str, Any] | LLMResponse | CacheBoundary] = []
         for msg in messages:
             if msg.replay_message is not None:
                 out.append(