fix(api): signals endpoint conditional-GET correctness + fail-closed gaps

- Last-Modified only on the unfiltered variant: generated_at is identical
  across tickers variants of one artifact, so a client revalidating a
  filtered request with If-Modified-Since alone (no If-None-Match — legal,
  and Django then never consults the ETag) got a 304 pointing at a
  differently-filtered cached body
- ETag keyed on the normalized (deduped/uppercased/sorted) tickers filter
  shared with the view, each token percent-encoded, '+'-joined: the raw
  query string made equivalent filters miss revalidation; a raw comma in
  the ETag is rejected outright by Django's parse_etags (latent pre-fix bug
  — multi-ticker conditional GET could never 304); and an unencoded '+'
  join collides with a literal %2B token (adversarial-review catch,
  shared-cache poisoning vector under Cache-Control: public)
- _get_artifact: one disk load per request instead of three (@condition
  calls etag/last_modified funcs before the view; glob+stat+read+parse ran
  3x per GET); sorted() of candidates dropped — max() re-derives order
- fail-closed holes in the "never a 500" contract: p.stat() inside the
  try (file pruned between glob and stat -> 404, was an uncaught
  FileNotFoundError), and tz-naive generated_at rejected up front (was a
  TypeError at staleness subtraction)

Tests: 7 new endpoint tests, TDD RED-first (variant Last-Modified, IMS-only
revalidation, ETag normalization + %2B collision, single-load memo, vanish
race, naive timestamp). Endpoint+contract: 19 passed. Spec §4.4 amended.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vVEHtvXuM84SrcjhTVLce

Author: FlyM1ss

Docs/superpowers/specs/2026-07-06-news-to-signals-pipeline-design.md

@@ -160,7 +160,7 @@ A machine-readable JSON Schema ships at `Heartbeat/schemas/signals-v1.schema.jso
 - **200** — public serialization of the newest `signals-*.json` (newest = greatest mtime, filename as a deterministic tiebreak; same-day supplemental stems sort lexicographically before the date-only stem, so stem order alone is not recency): the artifact **minus** `generator`, `model`, `prompt_version` (recon-value stripping), **plus** `"staleness_hours"` computed server-side from `generated_at`.
 - `?tickers=AAPL,MSFT` — optional filter on `signals` keys (case-insensitive; unknown tickers simply absent).
 - **404** `{"error": "no_signals"}` — no artifact exists.
-- Headers: `ETag` (from `generated_at`), `Last-Modified`, `Cache-Control: public, max-age=300`; conditional GET returns 304. Rate-limited via the existing `django_ratelimit` + `api.identity.ratelimit_key` infra.
+- Headers: `ETag` (from `generated_at` + `source_items` + the normalized tickers filter, `+`-joined — Django's `parse_etags()` rejects commas inside an ETag), `Last-Modified` **on the unfiltered variant only** (it is identical across tickers variants, so an `If-Modified-Since`-only revalidation of a filtered request must get a full 200, never a 304 pointing at a differently-filtered cached body), `Cache-Control: public, max-age=300`; conditional GET returns 304. Rate-limited via the existing `django_ratelimit` + `api.identity.ratelimit_key` infra.
 - Serving path: container gets a **runtime-enforced `:ro` mount of `$HEARTBEAT_HOME/signals/` only** — never the whole digests tree (§7.2).
 
 ### 4.5 ATL adapter projection (future session; pinned now)

Main/backend/api/signals_views.py

@@ -13,6 +13,7 @@
 import logging
 from datetime import datetime, timezone
 from pathlib import Path
+from urllib.parse import quote
 
 from django.conf import settings
 from django.http import HttpRequest, JsonResponse
@@ -33,36 +34,71 @@ def _load_latest():
     directory = Path(configured)
     if not directory.is_dir():
         return None
-    candidates = sorted(directory.glob("signals-*.json"))
+    candidates = list(directory.glob("signals-*.json"))
     if not candidates:
         return None
-    # Newest by mtime, filename as a deterministic tiebreak: same-day
-    # supplemental stems (items-<date>-<HHMMSS>.jsonl ->
-    # signals-<date>-<HHMMSS>.json) sort lexicographically BEFORE the
-    # date-only stem ("." > "-" in ASCII), so stem order alone is not
-    # recency — a same-day re-run would otherwise serve the stale artifact.
-    newest = max(candidates, key=lambda p: (p.stat().st_mtime, p.name))
+    newest = None
     try:
+        # Newest by mtime, filename as a deterministic tiebreak: same-day
+        # supplemental stems (items-<date>-<HHMMSS>.jsonl ->
+        # signals-<date>-<HHMMSS>.json) sort lexicographically BEFORE the
+        # date-only stem ("." > "-" in ASCII), so stem order alone is not
+        # recency — a same-day re-run would otherwise serve the stale
+        # artifact. stat() stays inside the try: a file pruned between
+        # glob() and stat() fails closed, never 500s.
+        newest = max(candidates, key=lambda p: (p.stat().st_mtime, p.name))
         artifact = json.loads(newest.read_text(encoding="utf-8"))
-        datetime.fromisoformat(artifact["generated_at"])  # must parse
+        generated = datetime.fromisoformat(artifact["generated_at"])
+        if generated.tzinfo is None:
+            # fromisoformat accepts naive strings; the view subtracts this
+            # from an aware now() for staleness_hours
+            raise ValueError("generated_at must be timezone-aware")
         if not isinstance(artifact["signals"], dict):
             raise ValueError("signals must be a JSON object")
         return artifact
     except (OSError, ValueError, KeyError, TypeError) as exc:
-        logger.error("signals: unreadable artifact %s: %s", newest.name, exc)
+        logger.error("signals: unreadable artifact %s: %s",
+                     newest.name if newest else "<vanished>", exc)
         return None  # fail closed: unreadable == no signals
 
 
+def _get_artifact(request: HttpRequest):
+    """One disk load per request: @condition calls _etag and _last_modified
+    before the view body runs, and all three need the artifact."""
+    if not hasattr(request, "_signals_artifact"):
+        request._signals_artifact = _load_latest()
+    return request._signals_artifact
+
+
+def _tickers_filter(request: HttpRequest):
+    """Normalized tickers filter (deduped, uppercased, sorted) — the single
+    definition shared by the ETag variant key and the view's filtering."""
+    raw = request.GET.get("tickers") or ""
+    return sorted({t.strip().upper() for t in raw.split(",") if t.strip()})
+
+
 def _etag(request: HttpRequest):
-    artifact = _load_latest()
+    artifact = _get_artifact(request)
     if artifact is None:
         return None
-    tickers = (request.GET.get("tickers") or "").upper().replace(" ", "")
+    # Each token percent-encoded so the "+" join is unambiguous (a literal
+    # "+" inside a token — reachable via %2B — must not collide with the
+    # separator), and "+"-joined rather than ","-joined because Django's
+    # parse_etags() rejects an ETag containing a comma (HTTP list separator).
+    tickers = "+".join(quote(t, safe="") for t in _tickers_filter(request))
     return f'"{artifact["generated_at"]}|{artifact.get("source_items", "")}|{tickers}"'
 
 
 def _last_modified(request: HttpRequest):
-    artifact = _load_latest()
+    # Only the unfiltered variant carries Last-Modified: generated_at is
+    # identical across every tickers variant of one artifact, and a client
+    # revalidating a filtered request with If-Modified-Since alone (legal —
+    # and without If-None-Match Django never consults the ETag) would get a
+    # 304 telling it to reuse a differently-filtered cached body. The ETag
+    # is the only validator that can carry the variant.
+    if _tickers_filter(request):
+        return None
+    artifact = _get_artifact(request)
     return (datetime.fromisoformat(artifact["generated_at"])
             if artifact else None)
 
@@ -73,17 +109,16 @@ def _last_modified(request: HttpRequest):
            method=ALL, block=True)
 @condition(etag_func=_etag, last_modified_func=_last_modified)
 def news_signals(request: HttpRequest) -> JsonResponse:
-    artifact = _load_latest()
+    artifact = _get_artifact(request)
     if artifact is None:
         return JsonResponse({'error': 'no_signals'}, status=404)
     body = {k: v for k, v in artifact.items() if k not in _PUBLIC_STRIP}
     generated = datetime.fromisoformat(artifact["generated_at"])
     now = datetime.now(timezone.utc)
     body["staleness_hours"] = round(
         (now - generated).total_seconds() / 3600, 1)
-    raw = request.GET.get("tickers")
-    if raw:
-        wanted = {t.strip().upper() for t in raw.split(",") if t.strip()}
+    wanted = set(_tickers_filter(request))
+    if wanted:
         body["signals"] = {k: v for k, v in body["signals"].items()
                            if k in wanted}
     response = JsonResponse(body)

Main/backend/tests/test_signals_endpoint.py

@@ -7,10 +7,13 @@
 import time
 from datetime import datetime, timedelta, timezone
 from pathlib import Path
+from unittest import mock
 
 from django.core.cache import cache
 from django.test import SimpleTestCase, override_settings
 
+from api import signals_views
+
 URL = "/api/signals/news/"
 
 # Bare pytest (no Django test runner, no pytest-django) never calls
@@ -146,6 +149,105 @@ def test_etag_is_variant_specific_to_tickers_filter(self):
                                      HTTP_IF_NONE_MATCH=filtered_etag)
         self.assertEqual(repeat.status_code, 304)
 
+    def test_last_modified_only_on_unfiltered_variant(self):
+        # Last-Modified (generated_at) is identical across every tickers
+        # variant of one artifact, so it can only be emitted where it
+        # uniquely identifies the variant: the unfiltered response.
+        self._write("2026-07-06", make_artifact(self._recent_iso()))
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC):
+            unfiltered = self.client.get(URL)
+            filtered = self.client.get(URL, {"tickers": "msft"})
+        self.assertTrue(unfiltered.has_header("Last-Modified"))
+        self.assertFalse(filtered.has_header("Last-Modified"))
+
+    def test_if_modified_since_revalidates_unfiltered_but_never_filtered(self):
+        # RFC 9110: with no If-None-Match, the server answers from
+        # If-Modified-Since alone and never consults the ETag — so a 304
+        # here would tell the client to reuse a differently-filtered body.
+        self._write("2026-07-06", make_artifact(self._recent_iso()))
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC):
+            first = self.client.get(URL)
+            ims = first["Last-Modified"]
+            unfiltered = self.client.get(URL, HTTP_IF_MODIFIED_SINCE=ims)
+            filtered = self.client.get(URL, {"tickers": "msft"},
+                                       HTTP_IF_MODIFIED_SINCE=ims)
+        self.assertEqual(unfiltered.status_code, 304)
+        self.assertEqual(filtered.status_code, 200)
+
+    def test_etag_is_stable_across_tickers_order_whitespace_and_dupes(self):
+        # The view filters on a normalized set; the ETag must key on the
+        # same normalization or equivalent requests never revalidate.
+        art = make_artifact(self._recent_iso(), signals={
+            "MSFT": make_artifact("x")["signals"]["MSFT"],
+            "AAPL": dict(make_artifact("x")["signals"]["MSFT"], guid="g2"),
+        })
+        self._write("2026-07-06", art)
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC):
+            first = self.client.get(URL, {"tickers": "MSFT,AAPL"})
+            second = self.client.get(URL, {"tickers": " aapl , msft ,msft"})
+            self.assertEqual(first["ETag"], second["ETag"])
+            revalidated = self.client.get(URL, {"tickers": "AAPL,MSFT"},
+                                          HTTP_IF_NONE_MATCH=first["ETag"])
+        self.assertEqual(revalidated.status_code, 304)
+
+    def test_etag_distinguishes_literal_plus_token_from_plus_joined_list(self):
+        # %2B decodes to a literal '+' inside one token; the ETag's '+' join
+        # must not let {'AAPL','MSFT'} and the single token 'AAPL+MSFT'
+        # collide, or a 304 would point a client (or the shared proxy cache
+        # behind Cache-Control: public) at a differently-filtered body.
+        art = make_artifact(self._recent_iso(), signals={
+            "MSFT": make_artifact("x")["signals"]["MSFT"],
+            "AAPL": dict(make_artifact("x")["signals"]["MSFT"], guid="g2"),
+        })
+        self._write("2026-07-06", art)
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC):
+            joined = self.client.get(URL, {"tickers": "AAPL,MSFT"})
+            literal = self.client.get(URL, {"tickers": "AAPL+MSFT"})
+        self.assertEqual(len(joined.json()["signals"]), 2)
+        self.assertEqual(literal.json()["signals"], {})
+        self.assertNotEqual(joined["ETag"], literal["ETag"])
+
+    def test_artifact_loaded_from_disk_once_per_request(self):
+        # @condition calls _etag and _last_modified before the view runs;
+        # without per-request memoization one GET pays 3x glob+stat+read.
+        self._write("2026-07-06", make_artifact(self._recent_iso()))
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC), \
+             mock.patch.object(signals_views, "_load_latest",
+                               wraps=signals_views._load_latest) as loader:
+            resp = self.client.get(URL)
+        self.assertEqual(resp.status_code, 200)
+        self.assertEqual(loader.call_count, 1)
+
+    def test_artifact_vanishing_between_glob_and_stat_404s_fail_closed(self):
+        # A retention job (or manual cleanup) can unlink a candidate after
+        # glob() lists it and before max() stat()s it — that race must fail
+        # closed to 404, never surface as a 500.
+        self._write("2026-07-06", make_artifact(self._recent_iso()))
+        real_stat = Path.stat
+
+        def racing_stat(self, **kwargs):
+            if self.name.endswith(".json"):
+                raise FileNotFoundError(self.name)
+            return real_stat(self, **kwargs)
+
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC), \
+             mock.patch.object(Path, "stat", autospec=True,
+                               side_effect=racing_stat):
+            resp = self.client.get(URL)
+        self.assertEqual(resp.status_code, 404)
+        self.assertEqual(resp.json(), {"error": "no_signals"})
+
+    def test_naive_generated_at_404s_fail_closed(self):
+        # fromisoformat accepts tz-naive strings; subtracting one from the
+        # aware now() would raise TypeError in the view — the validator must
+        # reject a naive generated_at up front.
+        naive = datetime.now().isoformat(timespec="seconds")
+        self._write("2026-07-06", make_artifact(naive))
+        with override_settings(SIGNALS_DIR=str(self.dir), **_HERMETIC):
+            resp = self.client.get(URL)
+        self.assertEqual(resp.status_code, 404)
+        self.assertEqual(resp.json(), {"error": "no_signals"})
+
     def test_malformed_newest_artifact_404s_fail_closed(self):
         (self.dir / "signals-2026-07-06.json").write_text("{broken",
                                                           encoding="utf-8")