feat(upstream): import open PRs as provisional snapshots

.github/fork-features.yml

@@ -1401,6 +1401,40 @@ features:
       tracking: []
       retire_when: The fork stops shipping as a separately installable application or adopts a shared downstream-namespacing mechanism that preserves the same isolation and hosted-origin cleanup boundary.
 
+  - id: fork-upstream-intake-provenance
+    title: Reconcile provisional upstream imports explicitly
+    status: maintained
+    prs: []
+    invariants:
+      - An open upstream PR is imported as one attributed snapshot commit with PR provenance and a separate tracking commit recording the frozen base and head and the reconciliation plan.
+      - A snapshot remains provisional until its final upstream outcome is reviewed; PR provenance alone cannot skip its eventual integration or advance the intake baseline.
+      - Queue and candidate reports read committed tracking metadata from the selected fork ref, and lag history records completion at reconciliation rather than the initial snapshot.
+    rationale: Open PRs can be amended, rebased, or closed after intake. A reviewed snapshot keeps useful PR provenance without treating unfinished upstream work as fully reconciled.
+    implementation_paths:
+      - .github/upstream-tracked-prs.json
+      - docs/operations/upstream-tracking.md
+      - scripts/upstream/check-intake.ts
+      - scripts/upstream/lag-report.ts
+      - scripts/upstream/lib/tracked-prs.ts
+      - scripts/upstream/queue.ts
+      - scripts/upstream/tracked-prs-report.ts
+    upstream_paths:
+      - .github/upstream-tracked-prs.json
+      - docs/operations/upstream-tracking.md
+      - scripts/upstream/check-intake.ts
+      - scripts/upstream/lag-report.ts
+      - scripts/upstream/lib/tracked-prs.ts
+      - scripts/upstream/queue.ts
+      - scripts/upstream/tracked-prs-report.ts
+    tests:
+      - scripts/upstream/lag-report.test.ts
+      - scripts/upstream/lib/tracked-prs.test.ts
+      - scripts/upstream/queue.test.ts
+    upstream:
+      status: unassessed
+      tracking: []
+      retire_when: The fork stops managing its own upstream intake or adopts another process that preserves explicit reconciliation of provisional imports.
+
   - id: github-issue-thread-context
     title: Start a thread from a GitHub issue
     status: review-needed

docs/operations/upstream-tracking.md

@@ -2,7 +2,7 @@
 
 **Queue → apply in order → validate the batch → promote without a PR → advance the baseline.**
 
-Use an `intake/<batch>` branch based on fork `main`. Preserve upstream authorship, commit messages, and chronological order, adding provenance and the smallest necessary fork adaptations. Publish the branch for CI, then fast-forward `main` to its reviewed tip through **Promote upstream intake**. Do not open a PR for the batch or squash it: retaining upstream history is valuable, and attaching those commits to a PR creates unwanted upstream cross-references and participants.
+Use an `intake/<batch>` branch based on fork `main`. Preserve upstream authorship, commit messages, and chronological order, adding provenance and the smallest necessary fork adaptations. Publish the branch for CI, then fast-forward `main` to its reviewed tip through **Promote upstream intake**. Do not open a PR for the batch or squash merged upstream history: retaining that history is valuable, and attaching those commits to a PR creates unwanted upstream cross-references and participants. An explicitly requested import of an open PR uses one provisional snapshot commit, as described under [early imports](#early-imports).
 
 Nothing automatically syncs upstream or authorizes a promotion. Dispatch only when the maintainer explicitly requests it.
 
@@ -23,7 +23,7 @@ The state in `.github/upstream-intake.json` has three parts:
 - **Target:** the fixed upstream destination for this catch-up. It does not move when upstream receives new commits.
 - **Exceptions:** specific commits already present, intentionally skipped, or reopened as pending, with reasons.
 
-For upstream changes worth watching, add an entry to `.github/upstream-tracked-prs.json` with the PR number and a `reason`: what the fork is waiting on it for, such as a blocked fork change or a bug users hit. Write a fork PR as "fork #123"; a bare number is an upstream PR. Both reports show the reason next to the PR. `status` reads current upstream PR metadata and reports each tracked change as open, pending, recorded, skipped, or awaiting a fresh upstream fetch. Pending entries show how many days their upstream merge is ahead of the fork's last reconciled upstream integration, and whether they fall beyond the fixed target. The report uses the same intake provenance as the queue; a recorded import is evidence of intake, not proof that the behavior still works. Remove an entry once the report shows it recorded, or once its reason no longer applies. This list does not change queue order or authorize early intake.
+For upstream changes worth watching, add an entry to `.github/upstream-tracked-prs.json` with the PR number and a `reason`: what the fork is waiting on it for, such as a blocked fork change or a bug users hit. Write a fork PR as "fork #123"; a bare number is an upstream PR. Both reports show the reason next to the PR. `status` reads current upstream PR metadata and reports each tracked change as open, closed, pending, recorded, skipped, or awaiting a fresh upstream fetch. A provisional import also records a `snapshot` with its reviewed upstream base and head SHAs; a closed PR with a snapshot is reported as "review closed import". Pending entries show how many days their upstream merge is ahead of the fork's last reconciled upstream integration, and whether they fall beyond the fixed target. A recorded import is evidence of intake, not proof that the behavior still works. Remove an ordinary watch entry once recorded or no longer relevant; remove a snapshot only after reviewing its final outcome as described below. Snapshot markers keep the eventual upstream integration pending despite PR provenance. The list does not change queue order or authorize early intake.
 
 The daily **Upstream lag report** workflow includes the tracked PR table in its run summary. **Promote upstream intake** also writes a projected lag report, divergence comparison, and tracked PR table during candidate validation, using the exact candidate SHA. These reports are informational and do not gate promotion. To print the table locally, run `node scripts/upstream/tracked-prs-report.ts` after fetching both repositories.
 
@@ -61,7 +61,7 @@ Decide when the evidence is clear. Ask the maintainer when the intent remains un
 
 A commit whose change differs from upstream's needs a `Fork adaptation:` paragraph saying what differs and why, and a `Fork-Feature: <ledger-id>, <ledger-id>` trailer naming the ledger entries it preserves. The intake audit blocks a differing commit without a note, a commit that matches upstream but cites a feature, and a feature ID the ledger does not list. A differing commit without `Fork-Feature`, such as one shaped by an early import, requires manual review. A note on a matching commit remains useful when it records that a fork change was dropped in favor of upstream's.
 
-Every candidate commit needs an `Upstream-PR: 1234, 5678` and/or `Upstream-Commit: <full lowercase SHA>` trailer. `Upstream-PR` records a PR's own commits, including its squash or merge commit, so never repeat those SHAs in `Upstream-Commit`; the intake audit looks up each `Upstream-Commit` beside `Upstream-PR` on GitHub and blocks repeats. Use `Upstream-Commit` for direct upstream commits, commits from other PRs folded into the same change, and early imports of an unmerged PR, where an `Upstream-PR` trailer would make the queue skip upstream's eventual merge. A verified empty import may use a provenance-only commit; do not infer completeness merely because a cherry-pick is empty.
+Every candidate commit needs an `Upstream-PR: 1234, 5678` and/or `Upstream-Commit: <full lowercase SHA>` trailer. `Upstream-PR` identifies a PR's change, including a provisional open-PR snapshot; it does not complete reconciliation while a snapshot marker remains. Never repeat that PR's own SHAs in `Upstream-Commit`; the intake audit looks up each `Upstream-Commit` beside `Upstream-PR` on GitHub and blocks repeats. Use `Upstream-Commit` for direct upstream commits and commits from other PRs folded into the same change. A verified empty import may use a provenance-only commit; do not infer completeness merely because a cherry-pick is empty.
 
 Read the actual source diffs when reconciling reverts or already-present work; never assume an adjacent commit implements a source. If an exact change/revert pair is accounted for together, verify its net effect and record both sources; do not silently skip either. Existing provenance proves an import was recorded, not that today's tree still has equivalent behavior.
 
@@ -142,7 +142,16 @@ node scripts/upstream/queue.ts early <PR-number> [<PR-number> ...] --through ups
 
 Import a clean plan on its own `intake/upstream-<PR>` branch, including every PR and direct commit the plan added, and follow steps 2 to 4 with the usual `Upstream-PR` trailers. The queue then shows those sources as recorded when chronological intake reaches them.
 
-`early` only accepts PRs merged into upstream `main`. To import an open upstream PR, cherry-pick its commits onto an `intake/upstream-<PR>` branch with `Upstream-Commit` trailers only, as described in step 2. When upstream merges the PR, the queue lists the merge as pending; take upstream's final version at that point.
+`early` only accepts PRs merged into upstream `main`. For an explicitly requested open-PR import:
+
+1. Fetch the PR head and record the exact head and its merge base with the PR's upstream target branch. Review that aggregate diff, including dependencies; importing the whole upstream branch would also import unrelated target-branch changes.
+2. Apply that diff on `intake/upstream-<PR>` as one commit, simulating a squash merge. Preserve the principal upstream author's attribution and credit other contributors with `Co-authored-by` trailers where applicable. Use a concise subject for the complete change and an `Upstream-PR` trailer. Do not list its constituent commits in `Upstream-Commit` or cherry-pick notes. Explain fork adaptations as in step 2.
+3. In a separate commit on the same candidate, update the tracked PR entry with `snapshot: { "base": "<full lowercase base SHA>", "head": "<full lowercase head SHA>" }`. Its `reason` must explain what was imported, any fork adaptations or known gaps, and how to reconcile if the PR merges or closes. Give this tracking commit the same `Upstream-PR` trailer and a `Fork adaptation:` note explaining the provisional tracking metadata. Publish and promote both commits together; the marker must land with the snapshot.
+4. Follow the same final validation, exact-SHA CI, and promotion approval process. The audit compares the implementation with the frozen base-to-head diff, so later upstream amendments or rebases do not change what was reviewed. Fetching a missing snapshot head is read-only upstream access.
+
+When upstream merges the PR, the queue keeps its integrations pending while the marker remains, even if the final patch matches the snapshot. At chronological intake, compare the frozen diff with the final upstream change, reconcile missing or changed behavior and fork adaptations, and validate the result. Remove the marker (or the entire entry when no longer needed) in a separate tracking commit with the same PR provenance, recording the reviewed outcome. Promote that cleanup with the reconciliation; the lag history counts completion at cleanup rather than retroactively at the snapshot import. An identical final change still needs this explicit review and cleanup.
+
+If the PR closes without merging, the report calls for review. Decide whether to remove the imported change or retain it as maintained fork behavior, updating the ledger where appropriate. Record that decision before clearing its tracking marker. If a retained PR reopens, restore the marker so a later merge is reconciled. Closure alone never means that the imported code was reconciled or reverted. Legacy early imports with `Upstream-Commit` trailers keep their existing provenance; they do not need rewriting.
 
 ## T3 Code import check
 
@@ -157,7 +166,7 @@ node apps/server/scripts/generate-t3-import-fixture.ts --server <upstream-checko
 ## Queue reference and exceptions
 
 - `explain <PR-number-or-SHA-prefix>` shows sources and import evidence. SHA prefixes must be unambiguous and at least seven characters.
-- `--fork-ref intake/<batch>` inspects a candidate; `--state` and `--upstream-ref` select alternate local inputs.
+- `--fork-ref intake/<batch>` inspects a candidate, including its committed tracked PR list; draft working-tree tracking edits do not change the queue for `origin/main`. `--state` and `--upstream-ref` select alternate local inputs.
 - `node scripts/upstream/queue.ts --help` lists every command. Modules under `scripts/upstream/lib/` are libraries and do nothing when run directly.
 - `--json` provides structured output for queue commands.
 - PR associations are cached in the primary checkout's ignored `.scratch/`, shared across worktrees and saved incrementally. New targets reuse settled associations from a previous cache and recheck direct or unmerged commits. Use `--refresh-metadata` to rebuild them.

scripts/upstream/check-intake.ts

@@ -3,7 +3,6 @@
 
 import * as NodeChildProcess from "node:child_process";
 import * as NodeFS from "node:fs";
-import * as NodeOS from "node:os";
 import * as NodePath from "node:path";
 import * as NodeURL from "node:url";
 
@@ -35,6 +34,12 @@ import {
   upstreamMigrationManifestPath,
 } from "./lib/migrations.ts";
 import { parseUpstreamProvenance } from "./lib/provenance.ts";
+import {
+  decodeTrackedPRs,
+  fetchTrackedPRMetadata,
+  snapshotTrackingErrors,
+  type TrackedPR,
+} from "./lib/tracked-prs.ts";
 import { makeUpstreamMigrationGit } from "./migration-git.ts";
 import { fetchAssociations } from "./queue.ts";
 
@@ -158,10 +163,10 @@ function commitExists(sha: string): boolean {
   );
 }
 
-function patch(sha: string): string {
+function patch(sha: string, base = `${sha}^`): string {
   const result = NodeChildProcess.spawnSync(
     "git",
-    ["diff", "--no-color", "--no-renames", "--no-ext-diff", "-U0", `${sha}^`, sha],
+    ["diff", "--no-color", "--no-renames", "--no-ext-diff", "-U0", base, sha],
     { cwd: repoRoot, encoding: "utf8", maxBuffer: 256 * 1024 * 1024 },
   );
   if (result.status !== 0) throw new Error(result.stderr.trim() || `git diff ${sha} failed.`);
@@ -207,28 +212,45 @@ function reviewCommits(input: {
   readonly commitMessages: ReadonlyArray<string>;
   readonly ledger: ForkFeatureLedger;
   readonly scratch: string;
+  readonly tracked: readonly TrackedPR[];
 }): ReadonlyArray<CommitReview> {
   return input.commits.map((sha, index) => {
     const message = input.commitMessages[index] ?? "";
     const provenance = parseUpstreamProvenance([message]);
     const sourceCommits = [...new Set([...cherryPickSources(message), ...provenance.commitShas])];
+    const snapshots = input.tracked
+      .filter((pr) => pr.snapshot && provenance.pullRequestNumbers.includes(pr.number))
+      .map((pr) => pr.snapshot!);
     const paths = lines(git(["diff-tree", "--no-commit-id", "--name-only", "-r", sha]));
+    const trackingOnly = paths.length === 1 && paths[0] === ".github/upstream-tracked-prs.json";
     const featureIds = findForkFeatureOverlaps(input.ledger, paths).map(
       ({ feature }) => feature.id,
     );
-    const missing = sourceCommits.filter((source) => !commitExists(source));
+    const missing = [
+      ...sourceCommits,
+      ...snapshots.flatMap((snapshot) => [snapshot.base, snapshot.head]),
+    ].filter((source) => !commitExists(source));
     const comparison: CommitReview["comparison"] =
       paths.length === 0
         ? { status: "provenance-only" }
-        : sourceCommits.length === 0
+        : sourceCommits.length === 0 && snapshots.length === 0
           ? { status: "unavailable", reason: "no cherry-picked or Upstream-Commit source." }
           : missing.length > 0
             ? {
                 status: "unavailable",
                 reason: `upstream ${missing.map((source) => source.slice(0, 10)).join(", ")} not fetched.`,
               }
             : (() => {
-                const upstreamPatches = sourceCommits.map(patch);
+                const sourceHeads = [
+                  ...sourceCommits,
+                  ...snapshots.map((snapshot) => snapshot.head),
+                ];
+                const upstreamPatches = trackingOnly
+                  ? []
+                  : [
+                      ...sourceCommits.map((source) => patch(source)),
+                      ...snapshots.map((snapshot) => patch(snapshot.head, snapshot.base)),
+                    ];
                 const files = compareWithUpstream({
                   upstreamPatches,
                   forkPatch: patch(sha),
@@ -241,7 +263,7 @@ function reviewCommits(input: {
                     );
                     return (
                       lastSource === -1 ||
-                      blobAt(sha, file.path) !== blobAt(sourceCommits[lastSource]!, file.path)
+                      blobAt(sha, file.path) !== blobAt(sourceHeads[lastSource]!, file.path)
                     );
                   })
                   .map((file) => ({
@@ -267,7 +289,18 @@ function reviewCommits(input: {
 }
 
 function withScratchDirectory<A>(use: (directory: string) => A): A {
-  const directory = NodeFS.mkdtempSync(NodePath.join(NodeOS.tmpdir(), "styal-intake-review-"));
+  const scratch = NodePath.join(repoRoot, ".scratch");
+  const ignored = NodeChildProcess.spawnSync("git", ["check-ignore", "-q", ".scratch/"], {
+    cwd: repoRoot,
+  });
+  if (ignored.status === 1) {
+    NodeFS.appendFileSync(
+      git(["rev-parse", "--path-format=absolute", "--git-path", "info/exclude"]),
+      "\n/.scratch/\n",
+    );
+  } else if (ignored.status !== 0) throw new Error("Could not check scratch ignore rules.");
+  NodeFS.mkdirSync(scratch, { recursive: true });
+  const directory = NodeFS.mkdtempSync(NodePath.join(scratch, "upstream-intake-review-"));
   try {
     return use(directory);
   } finally {
@@ -447,6 +480,42 @@ try {
   });
   const commits = lines(git(["rev-list", "--reverse", `${baseSha}..${headSha}`]));
   const commitMessages = commits.map((commit) => git(["show", "-s", "--format=%B", commit]));
+  const tracked = decodeTrackedPRs(git(["show", `${headSha}:.github/upstream-tracked-prs.json`]));
+  const previousTracked = decodeTrackedPRs(
+    git(["show", `${baseSha}:.github/upstream-tracked-prs.json`]),
+  );
+  const sourcePRs = parseUpstreamProvenance(commitMessages).pullRequestNumbers;
+  const snapshotErrors = snapshotTrackingErrors({
+    tracked,
+    previous: previousTracked,
+    sourcePRs,
+    metadata: fetchTrackedPRMetadata(
+      ledger.upstream_repository,
+      sourcePRs.map((number) => ({ number, reason: "Candidate source" })),
+      (command, args) =>
+        NodeChildProcess.execFileSync(command, args, { cwd: repoRoot, encoding: "utf8" }),
+    ),
+  });
+  if (snapshotErrors.length) throw new Error(snapshotErrors.join("\n"));
+  for (const pr of tracked) {
+    if (!pr.snapshot || !sourcePRs.includes(pr.number)) continue;
+    if (!commitExists(pr.snapshot.head)) {
+      NodeChildProcess.execFileSync(
+        "git",
+        [
+          "fetch",
+          "--no-tags",
+          `https://github.com/${ledger.upstream_repository}.git`,
+          pr.snapshot.head,
+        ],
+        { cwd: repoRoot, stdio: "pipe" },
+      );
+    }
+    if (!commitExists(pr.snapshot.base) || !isAncestor(pr.snapshot.base, pr.snapshot.head))
+      throw new Error(
+        `Tracked PR #${pr.number} snapshot base must be an ancestor of its fetched head.`,
+      );
+  }
   const audit = auditUpstreamIntakeCandidate({
     baseSha,
     headSha,
@@ -464,7 +533,7 @@ try {
     migrationErrors: migrations.errors,
     migrationChanges: migrations.changes,
     commitReviews: withScratchDirectory((scratch) =>
-      reviewCommits({ commits, commitMessages, ledger, scratch }),
+      reviewCommits({ commits, commitMessages, ledger, scratch, tracked }),
     ),
   });