Document the warning in the gate guide and enforce that coverage

Closes the three review findings the dedup did not: the message no longer
asserts that both directories already exist (one record does not show that),
datastore-gate.md gains a section for the check, and a test asserts every
REGISTERED_CHECKS entry is named in that guide - replacing the docstring note
that asked the next author to remember. The three low findings on the refresh
tool went away with the tool.

424 passed, 20 skipped with evaleval#190's registry files overlaid.

Author: borgr

.claude/skills/eee-dataset-conversion/reference/datastore-gate.md

@@ -65,6 +65,25 @@ Re-derive this list from `REGISTERED_CHECKS`, `_DEPLOYMENT_TYPES`,
   (`api|local|unknown`, `closed_source|open_weights_deployment|other`). Read the enums
   from the installed validator, not from an existing record or an old bot message.
 
+## §developer — `check_developer_slug` (registered as **warning**)
+- Fires when `model_info.id`'s namespace prefix or `model_info.developer` is a *second
+  name* the eval-card-registry confirms for a publisher — a genuinely different name,
+  model families included (`mistral` for `mistralai`, `glm` for `zai`, `kimi` for
+  `moonshotai`). The datastore takes the publisher directory from that prefix, or from
+  `developer` when the id is flat, so two names for one publisher is two directories.
+- Silent on a canonical id, a HuggingFace namespace the registry records for one
+  (`meta-llama` is Meta), a case or punctuation variant of either (`Anthropic`, `z-ai`),
+  and any name the registry has never seen.
+- **The message names both spellings and picks neither.** Which name is primary is one
+  editorial choice for the whole datastore, and the registry's canonical id is often the
+  *rarer* of the two spellings already published. Match how the publisher is already filed
+  datastore-wide; do not treat the message as an instruction to use the registry's id.
+- One record only proves the alias relation and this record's directory — not that the
+  other directory is already populated. Fix it by choosing the spelling and correcting the
+  adapter's mapping, so every future record agrees.
+- Warning, not error: the affected records are already published and the fix changes join
+  keys. Exit code stays 0.
+
 ## §publish — `publish_evaluation_logs`
 **Mind which root each entry point wants — they differ, and a mismatch is silent until
 the path check rejects the depth:** `publish_evaluation_logs(base_output_dir=…)` takes the

every_eval_ever/validator/validation_core.py

@@ -632,11 +632,11 @@ def check_developer_slug(data: dict[str, Any]) -> list[str]:
     warnings: list[str] = []
     for canonical, (locations, spellings) in declared.items():
         consequence = (
-            'two datastore directories for one publisher, neither listing '
-            'complete'
+            'filing under both puts one publisher in two datastore '
+            'directories, neither listing complete'
             if directory_field in locations
             else 'the id prefix picks the directory here, so this field '
-            'splits none, but anything grouping by developer sees two'
+            'splits none, but anything grouping by developer reads two'
         )
         found = '/'.join(repr(spelling) for spelling in sorted(spellings))
         warnings.append(

tests/test_skill_conversion.py

@@ -9,8 +9,8 @@
   those templates.
 
 Scope: this catches drift in what the templates emit. A new validator rule that the
-reference conversion already satisfies leaves it green, so re-derive
-``reference/datastore-gate.md`` from ``REGISTERED_CHECKS`` when adding a check.
+reference conversion already satisfies leaves it green;
+``test_every_registered_check_is_documented_in_the_gate_guide`` covers that gap.
 
 Regenerate the frozen conversion after a deliberate change:
 
@@ -20,6 +20,7 @@
 from __future__ import annotations
 
 import importlib.util
+import inspect
 import json
 import re
 import shutil
@@ -33,6 +34,7 @@
 from every_eval_ever.helpers import SCHEMA_VERSION
 from every_eval_ever.helpers.io import SourceRecordsError
 from every_eval_ever.validate import main as validate_main
+from every_eval_ever.validator.validation_core import REGISTERED_CHECKS
 
 # --------------------------------------------------------------------------- setup
 
@@ -64,6 +66,38 @@ def _remedy(what_broke: str, *, where: str, regenerate: bool = True) -> str:
     return '\n'.join(lines)
 
 
+# -------------------------------------------------------------------- gate reference
+
+GATE_GUIDE = SKILL_DIR / 'reference' / 'datastore-gate.md'
+
+
+def test_every_registered_check_is_documented_in_the_gate_guide():
+    """A contributor meets the semantic checks through the guide, not the source."""
+    guide = GATE_GUIDE.read_text(encoding='utf-8')
+    documented = {}
+    for check in REGISTERED_CHECKS:
+        names = re.findall(
+            r'return (check_\w+)\(', inspect.getsource(check.run)
+        )
+        assert names, _remedy(
+            f'cannot tell which check {check.name!r} runs, so this test could '
+            'not confirm the guide documents it.',
+            where=(
+                'keep each REGISTERED_CHECKS wrapper a `return check_x(...)`, '
+                'or widen the pattern here'
+            ),
+            regenerate=False,
+        )
+        documented[check.name] = [name for name in names if name not in guide]
+
+    missing = {name: gap for name, gap in documented.items() if gap}
+    assert not missing, _remedy(
+        f'registered checks absent from datastore-gate.md: {missing}',
+        where=f'add a section for each to {GATE_GUIDE.relative_to(REPO_ROOT)}',
+        regenerate=False,
+    )
+
+
 # ------------------------------------------------------------------ skill templates