Commit 126c5d6
fix(validate): report a Purpose left as the archive placeholder (#1671)
* docs(openspec): propose warn-on-purpose-placeholder
When a delta introduces a capability with no usable `## Purpose`, archive
writes `TBD - created by archiving change <name>. Update Purpose after
archive.` into the new main spec. Three places already tell authors to
replace it -- the `specs` instruction ("including a leftover `TBD`
placeholder"), the sync-specs summary step ("so it gets written now rather
than lingering"), and the cli-archive contract -- but nothing reports that
it is still there.
`--strict` cannot reach it. The check meant to catch a Purpose nobody wrote
is a 50-character floor and the placeholder is 91 characters, so the one
rule that exists to catch a thin Purpose is satisfied by the exact text
meaning nobody wrote one: a Purpose reading "Does stuff." fails --strict
today, while one saying nothing at all passes.
Proposes reporting it as a warning against the spec's Purpose -- silent by
default, failing under --strict, so a project already carrying placeholders
keeps validating until it opts into the stricter gate. Detection is narrow:
the generated sentence wherever it appears, and otherwise only a `TBD`
opening the Purpose, so prose raising an open question is left alone.
Planning artifacts only; no source changes.
Refs #369
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(validate): report a Purpose left as the archive placeholder
When a delta introduces a capability with no usable `## Purpose`, archive
writes a placeholder into the new main spec. Nothing read it afterwards, so
the capability kept a to-do in it while every command reported success.
`--strict` could not reach it. The check that exists to catch a Purpose
nobody wrote is a 50-character floor, and the placeholder clears it: a spec
whose Purpose read "Does stuff." failed --strict, while a spec whose Purpose
said nothing at all passed. #369 reported agents leaving the placeholder
behind and stayed open seven months; every remedy since has been an
instruction, which is the mechanism that report described as unreliable.
validate now reports it as a warning against the Purpose, naming the line
and saying to edit the main spec directly -- a delta's `## Purpose` is read
only when the capability is created, so it cannot replace an existing one.
Warning rather than error, because strict mode already means "warnings
fail": a project carrying placeholders keeps validating by default and only
--strict fails. Archive is untouched -- it validates rebuilt specs without
--strict, so a spec archive writes still passes the validation it would have
passed before, and the text archive writes is byte-identical.
The placeholder is recognised through the same constants the writer composes
it from, so the check cannot drift from the sentence it looks for -- the
failure mode of a second, hand-copied spelling being a check that matches
nothing and looks exactly like a check that found nothing. The one case that
cannot be a lookup is an agent-written placeholder, kept to a `TBD` opening
the Purpose: "the retry budget is TBD pending benchmarks" is authored prose
and is left alone.
Verified: 209 archive tests pass unchanged (the placeholder text is
asserted literally, so the output is provably identical); full suite 138
files / 3993 tests; 36/36 strict spec validations; build, lint and typecheck
clean. Against a project carrying four real placeholders, default mode still
exits 0 and --strict fails exactly those four.
Cross-platform CI is not yet confirmed -- it needs a pushed branch. Line
endings are covered by tests asserting a CRLF spec and an LF spec produce
identical findings, and the module does no path handling.
Refs #369
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(validate): make every placeholder guard load-bearing
A mutation pass over the seven guards -- revert one, see which tests die --
found two that no test held.
The prefix/suffix test did not exercise the guard it named. Its Purpose read
"Explains what happens when archiving change my-change runs twice", which
contains neither half of the generated sentence, so it passed whether or not
the suffix was required. Matching on the prefix alone killed nothing. The
Purpose now embeds the real prefix constant and asserts the suffix is absent,
so the case is the one the name claims; the mutation kills it.
The empty-Purpose early return was genuinely dead. Neither rule matches empty
text, so removing the branch changed no behaviour and failed no test. Rather
than keep a guard nothing can hold, the branch is gone and the comment says
why an empty Purpose still yields null. The tests asserting that behaviour
are unchanged and still pass.
Every guard now dies under mutation:
whole check removed from applySpecRules ......... 6 tests
brevity no longer suppressed (else -> if) ....... 1
word boundary dropped from the TBD marker ....... 1
generated placeholder matched on prefix alone ... 1
line-ending normalisation removed ............... 2
section-boundary guard removed from locator ..... 1
Full suite 138 files / 3993 tests, lint and typecheck clean.
Refs #1670
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(openspec): record the mutation pass in the task list
The mutation work changed the implementation -- a test rewritten and a dead
branch removed -- but no task covered it, so the plan claimed less work than
was done. Added as group 6, marked complete, with why it was not planned.
5.4 now says what blocks it. It needs a pushed branch for the cross-platform
matrix, and the note records that line endings are covered locally by tests
asserting a CRLF spec and an LF spec produce identical findings, so a reader
can tell the difference between unverified and unverifiable-from-here.
The specs, proposal and design are unchanged and were checked: the delta's
empty-Purpose clause constrains behaviour, not structure, and that behaviour
is the same -- the redundant branch went, the rule did not.
26 of 27 tasks complete; the change still validates --strict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(openspec): close 5.4 on a green cross-platform matrix
CI dispatched on the fork against this branch: lint & typecheck, and the test
suite on linux-bash, macos-bash and windows-pwsh -- all green. The Windows job
installed, built and ran the suite rather than short-circuiting, which is the
part 5.4 existed to check, since the placeholder locator counts lines in files
that may carry either ending.
Recorded as a workflow_dispatch run on the fork, not the upstream pull-request
run, because those are not the same gate and the note should not let a reader
assume otherwise. Nix Flake Validation and Validate Release Tracking skipped:
this branch touches neither the flake nor release tracking.
27 of 27 tasks complete.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(validate): name the placeholder's line, not the prose above it
The warning tells you which line to fix, and named the wrong one when the
generated sentence did not open the Purpose:
3 ## Purpose
4 Handles widget retries. <- warning pointed here
5
6 TBD - created by archiving ... <- placeholder is here
The locator asked "what is the first non-blank line after ## Purpose?"
rather than "where is the placeholder?". Those are the same line in five of
the six shapes a placeholder can take -- a leading TBD marker is the first
non-blank line by definition, and archive writes the generated sentence as
the section's only content -- so the two questions only diverge when a human
types prose above a leftover placeholder.
Pointing at that prose is worse than pointing nowhere: the reader sees a
sentence that is plainly fine and concludes the check is broken. design.md
already said a wrong line number is worse than none, and the delta already
required naming the line the placeholder is on, so this is the
implementation meeting a contract that was already written, not a change of
contract.
The locator is now told which rule matched. A leading marker keeps the
first-non-blank behaviour, because that is where it sits; the generated
sentence is located by its own text. When both match the leading marker
wins, being the earlier of the two.
Found by CodeRabbit on #1671. The finding was real despite its own
"Addressed" marker, which only tracked the file changing in a later commit.
Two test gaps let it through. The case that covered this input asserted
only that something was reported, never which line -- so it now asserts the
line, and a table pins every position a placeholder can occupy, each case
first checking that the line it expects really carries the placeholder. The
mutation pass could not have caught it either: mutation proves a test dies
when a guard is broken, and cannot invent an assertion nobody wrote.
Reverting the branch fails exactly the three new expectations. Full suite
4000 tests / 138 files, lint and typecheck clean.
Refs #1670
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(validate): set the placeholder line unconditionally
ValidationIssue.line is optional and the project does not enable
exactOptionalPropertyTypes, so a plain assignment typechecks and
JSON output is unchanged - JSON.stringify drops undefined values.
findPurposePlaceholderIssue already returns the key unconditionally,
and the neighbouring push sites assign line plainly, so the
conditional spread was the odd one out.
* fix(validate): widen the placeholder check to TODO and read fences as quoted
#1670 left two questions open. Both are answered here, against how OpenSpec
already reads a spec.
A `TODO` opening the Purpose now reports as the same finding as a `TBD`.
Nothing OpenSpec writes produces one, but the marker an author leaves behind is
whichever word they reached for, and a Purpose reading `TODO: fill this in` is
as unwritten as one reading `TBD`. Only the opening position counts, as before,
so `TODOs are tracked in the linked issue` is still authored prose.
Fenced code inside a Purpose is now read as quoted material rather than as the
Purpose speaking, through the `buildCodeFenceMask` the requirement and structure
parsers already share. Without it a spec documenting the sentence archive writes
is reported as carrying it, which is the check failing the one document that
explains it - and a warning that fires on the docs teaches people to ignore the
warning. Fenced lines are skipped when locating the placeholder too, so a
`## Purpose` or `## Requirements` quoted in a fence can neither be mistaken for
the section header nor end the section early.
The message now names both what archive writes and a marker left in its place,
since one message covers both. Severity is unchanged: still a warning, so a
project carrying placeholders keeps validating and only --strict fails.
Every new guard is mutation-checked: dropping `TODO` kills 3 tests, unmasking
detection kills 2, unmasking the line locator kills 3, unmasking the header
search kills 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(validate): read the marker boundary in any script, not just ASCII
Review found `\b` reading `TODOé` and `TBD١` as a marker followed by
punctuation, because `\b` only knows ASCII word characters. A Purpose is prose
and prose is not always Latin script, so the rule that a longer word beginning
with those letters is not a marker has to hold in any script.
The lookahead rejects letters, digits, combining marks and `_`, and nothing
else, so `TODO:`, `TBD -` and `TODO(owner):` are still the marker they look
like. Held in both directions: loosening it back to `\b` kills 1 test,
tightening it to reject punctuation kills 4.
Also reworded a task line that opened with `#1670`, which markdownlint reads as
a heading missing its space.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(validate): locate the matched purpose placeholder
* docs(validate): remove trailing task whitespace
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Clay Good <hi@claygood.com>1 parent a7353ae commit 126c5d6
12 files changed
Lines changed: 1185 additions & 3 deletions
File tree
- .changeset
- openspec/changes/warn-on-purpose-placeholder
- specs/cli-validate
- src/core
- validation
- test/core
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
Lines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
Lines changed: 112 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
0 commit comments