fix(backend): stop one out-of-vocabulary extractor enum from failing a whole conversation (#11871)

* fix(backend): stop one out-of-vocabulary extractor enum from failing a whole conversation

The conversation structure model is asked for `owner_name` when the owner is known, but the
notes-v2 prompt never restricts `capture_owner` to its vocabulary, so the model sometimes answers
with the speaker's name there instead. `ExtractedActionItem.capture_owner` is a pydantic Literal,
so each such item raised literal_error, PydanticOutputParser turned the batch into
OutputParserException, and `_get_structured` mapped that to HTTP 500 — the user's conversation
never got a summary at all.

In prod on 2026-08-19 one conversation came back with `capture_owner: "Archit"` on three of six
action items; every other field was well formed. Three bad tokens cost the whole note.

`ExtractedActionItem` now normalizes its optional literal fields before validation: a recognized
value in different casing is accepted, an unusable value falls back to the field's own default of
None, and a name where an owner class belongs becomes `other` with the name carried into
`owner_name`, which is where the schema already wanted it. The guard sits on the model, so both
parser targets that embed it — StructuredExtraction and ActionItemsExtraction — are covered.

Verification: backend/tests/unit/test_extracted_action_item_vocabulary.py, 10 tests, all pass and
all 10 fail against origin/main. The first drives the real prod completion through the production
PydanticOutputParser seam. Related suites re-run green: test_conversation_notes_v2,
test_backend_candidate_capture, test_chat_first_proactive_engine, test_action_item_date_validation,
test_action_items_date_coercion, test_conversation_model_split, test_task_transcript_provenance,
test_llm_gateway_openai_compatible, test_llm_gateway_chat_extraction_pilot, test_byok_llm_logging,
test_omi_qos_tiers, test_daily_summary_zero_coordinate_locations.

Failure-Class: new

* chore(failure-classes): register FC-out-of-vocabulary-model-token-discards-extraction

The class this fix repairs has no definition yet. A strict enum on one field of one list element
failed the whole provider completion, so a conversation lost its note over three tokens.

Failure-Class: none

* chore(failure-classes): cite the real PR number as evidence

Failure-Class: none

Author: kodjima33

.github/failure-classes/FC-out-of-vocabulary-model-token-discards-extraction.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": 1,
+  "id": "FC-out-of-vocabulary-model-token-discards-extraction",
+  "violated_contract": "A closed vocabulary a language model was asked to answer in is a preference, not a guarantee, so one out-of-vocabulary token must never discard the whole payload it appeared in. A provider completion is parsed as a batch: when a strict enum on one field of one list element raises, the parser fails the entire response, and every well-formed field the model produced -- the title, the overview, the other list elements -- is lost with it. The user then sees a server error for content that was almost entirely usable.",
+  "canonical_prevention": "Normalize provider-supplied enum fields on the parsed model before validation, not at the call site, so every parser target that embeds the model inherits the guard. Accept a recognized value in any casing or padding; map an unusable value to the field's own declared default, which is what an absent field already means; and where a wrong-but-meaningful answer has an obvious home in the schema, route it there instead of dropping it. Keep the strictness where it is load-bearing -- required fields and fields whose absence changes behavior still fail. The regression test replays a real provider completion through the production parser seam.",
+  "canonical_prevention_artifact": [
+    "backend/models/structured_extraction.py",
+    "backend/tests/unit/test_extracted_action_item_vocabulary.py"
+  ],
+  "evidence_prs": [
+    11871
+  ],
+  "scope_hints": [
+    "backend/models/**",
+    "backend/utils/llm/**"
+  ],
+  "status": "open"
+}

backend/models/structured_extraction.py

@@ -1,11 +1,26 @@
 from datetime import datetime
-from typing import Any, List, Literal, Optional
+from typing import Any, Dict, List, Literal, Optional, Tuple
 
-from pydantic import BaseModel, Field, field_validator
+from pydantic import BaseModel, Field, field_validator, model_validator
 
 from models.conversation_enums import CategoryEnum
 from models.structured import ActionItem, Event, Section, Structured
 
+CAPTURE_OWNERS: Tuple[str, ...] = ('user', 'other', 'unknown')
+
+# The extractor is asked for a fixed vocabulary on these fields, but it answers outside of it often
+# enough to matter: on 2026-08-19 it returned a speaker's name for capture_owner on three items of
+# one conversation, pydantic raised literal_error, and the whole StructuredExtraction failed to
+# parse — so conversation processing returned HTTP 500 and the user got no summary at all. One
+# out-of-vocabulary token must never cost the conversation; an unusable value is worth exactly as
+# much as the field being absent, which is what these Optionals already model.
+_OPTIONAL_LITERAL_VOCABULARIES: Dict[str, Tuple[str, ...]] = {
+    'capture_kind': ('explicit_command', 'clear_commitment', 'direct_request', 'inferred_next_step'),
+    'capture_owner': CAPTURE_OWNERS,
+    'due_certainty': ('confirmed', 'tentative'),
+    'candidate_action': ('create', 'update', 'complete'),
+}
+
 
 class ExtractedActionItem(BaseModel):
     description: str = Field(description="The action item to be completed")
@@ -30,6 +45,31 @@ class ExtractedActionItem(BaseModel):
         description='Transcript segment IDs that directly support this action item',
     )
 
+    @model_validator(mode='before')
+    @classmethod
+    def drop_out_of_vocabulary_literals(cls, data: Any) -> Any:
+        if not isinstance(data, dict):
+            return data
+
+        coerced = dict(data)
+        for field, vocabulary in _OPTIONAL_LITERAL_VOCABULARIES.items():
+            value = coerced.get(field)
+            if value is None or value in vocabulary:
+                continue
+            normalized = value.strip().lower() if isinstance(value, str) else None
+            if normalized in vocabulary:
+                coerced[field] = normalized
+                continue
+            # A name where an owner class belongs still says the owner is somebody other than the
+            # user, and owner_name is where that name is meant to live.
+            if field == 'capture_owner' and normalized:
+                coerced[field] = 'other'
+                if not coerced.get('owner_name'):
+                    coerced['owner_name'] = value.strip()
+                continue
+            coerced[field] = None
+        return coerced
+
     def to_action_item(self) -> ActionItem:
         return ActionItem(
             description=self.description,

backend/tests/unit/test_extracted_action_item_vocabulary.py

@@ -0,0 +1,110 @@
+"""The extractor's out-of-vocabulary literals must not abort a whole conversation extraction.
+
+Prod, 2026-08-19: the structure model returned a speaker's name ("Archit") for `capture_owner` on
+three action items of one conversation. Pydantic raised literal_error for each, PydanticOutputParser
+turned that into OutputParserException, and `_get_structured` mapped it to HTTP 500 — the user's
+conversation never got a summary because of an enum token. The completion below is that prod
+payload, trimmed to the three offending items plus one good one.
+"""
+
+import json
+
+import pytest
+from langchain_core.output_parsers import PydanticOutputParser
+
+from models.structured_extraction import ActionItemsExtraction, ExtractedActionItem, StructuredExtraction
+
+PROD_COMPLETION = json.dumps(
+    {
+        "title": "Archit Defines Omi's Agent Context Strategy",
+        "overview": "Archit and the team discuss Omi's hardware, Mac app, and strategic direction.",
+        "emoji": "🧠",
+        "category": "technology",
+        "action_items": [
+            {
+                "description": "Collect a couple of weeks of performance data for the launch.",
+                "due_at": None,
+                "capture_kind": "clear_commitment",
+                "capture_confidence": 0.9,
+                "ownership_confidence": 0.9,
+                "capture_owner": "Archit",
+                "concrete_deliverable": True,
+                "candidate_action": "complete",
+                "target_task_id": None,
+                "source_segment_ids": [],
+            },
+            {
+                "description": "Download and personally test the Omi macOS app.",
+                "capture_kind": "direct_request",
+                "capture_owner": "user",
+                "candidate_action": "complete",
+                "source_segment_ids": [],
+            },
+            {
+                "description": "Submit the collaboration form discussed during the evaluation.",
+                "capture_kind": "clear_commitment",
+                "capture_owner": "Archit",
+                "candidate_action": "complete",
+                "source_segment_ids": [],
+            },
+            {
+                "description": "Give feedback on the Omi product after trying the device.",
+                "capture_kind": "clear_commitment",
+                "capture_owner": "Archit",
+                "candidate_action": "complete",
+                "source_segment_ids": [],
+            },
+        ],
+        "events": [],
+    }
+)
+
+
+def test_prod_completion_with_named_capture_owner_parses():
+    """The exact prod payload, through the exact production parser seam."""
+    extraction = PydanticOutputParser(pydantic_object=StructuredExtraction).parse(PROD_COMPLETION)
+
+    assert extraction.title == "Archit Defines Omi's Agent Context Strategy"
+    assert [item.capture_owner for item in extraction.action_items] == ['other', 'user', 'other', 'other']
+    assert len(extraction.to_structured().action_items) == 4
+
+
+def test_named_capture_owner_is_kept_as_owner_name():
+    item = ExtractedActionItem.model_validate({'description': 'Send the address', 'capture_owner': 'Archit'})
+
+    assert item.capture_owner == 'other'
+    assert item.owner_name == 'Archit'
+    assert item.to_action_item().owner_name == 'Archit'
+
+
+def test_named_capture_owner_does_not_overwrite_an_explicit_owner_name():
+    item = ExtractedActionItem.model_validate(
+        {'description': 'Send the address', 'capture_owner': 'Archit', 'owner_name': 'Archit Sharma'}
+    )
+
+    assert item.capture_owner == 'other'
+    assert item.owner_name == 'Archit Sharma'
+
+
+@pytest.mark.parametrize('value', ['User', ' other ', 'UNKNOWN'])
+def test_vocabulary_values_survive_casing_and_padding(value):
+    item = ExtractedActionItem.model_validate({'description': 'Ship it', 'capture_owner': value})
+
+    assert item.capture_owner == value.strip().lower()
+
+
+@pytest.mark.parametrize('field', ['capture_kind', 'due_certainty', 'candidate_action'])
+def test_unusable_optional_literal_is_dropped_rather_than_failing_the_item(field):
+    item = ExtractedActionItem.model_validate({'description': 'Ship it', field: 'something_the_model_invented'})
+
+    assert getattr(item, field) is None
+    assert item.description == 'Ship it'
+
+
+def test_action_items_extraction_survives_the_same_payload():
+    """The action-item-only parser shares the model, so it shares the guard."""
+    completion = json.dumps({'action_items': [{'description': 'Send the deck', 'capture_owner': 'Archit'}]})
+
+    extraction = PydanticOutputParser(pydantic_object=ActionItemsExtraction).parse(completion)
+
+    assert extraction.to_action_items()[0].capture_owner == 'other'