Bundler migration, type packaging, scan perf, and community contribution infra - #3
Merged
Merged
Conversation
…seline
maturity-model.md's dimension-points table and LEVEL_REQUIREMENTS thresholds
were only mirrored by hand ("change both together" in a comment), unlike
measure-and-improve.md which docs.test.ts already enforces. A new
rubric-sync.test.ts closes that gap. levels.test.ts now derives the total
from ALL_CHECKS instead of a bare 108 literal, with an explicit snapshot
test forcing a conscious bump when the rubric total changes.
golden-output.test.ts snapshots the Report JSON for every fixture and the
self-scan, giving upcoming build/perf changes a concrete regression baseline.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace plain tsc emit (36 unbundled files, ~82KB) with a tsup build: splitting+minify collapse dist/ to 5 files at ~54KB (index.js, cli.js, a shared chunk, and rolled-up .d.ts per entry). package.json gains an explicit "types" field and a "types" condition in "exports" so TS consumers resolve declarations without relying on default sibling-file lookup; tsc now runs as a separate --noEmit typecheck step instead of doing the emit. Verified non-breaking: golden-output.test.ts (byte-identical Report JSON, modulo tool.version), self-scan stays L4/108/108, and a real external consumer smoke test — npm pack, install the tarball in a throwaway project, run the bin directly and via npx, import the library at runtime, and tsc --noEmit against a consumer.ts using the published types. attw --pack shows no new problems (the package was already ESM-only before this change). RELEASING.md documents that JSR keeps publishing raw src/**/*.ts rather than the dist/ bundle — a deliberate, pre-existing divergence (JSR type-checks TS natively), not something this change needed to reconcile. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The public symbol surface was already complete (src/index.ts re-exports everything), but nothing verified the *published* declarations resolve correctly for an external consumer. Two additions close that gap: - test/types/smoke.ts imports every exported symbol from dist/index.js (not src/) and is type-checked in isolation via a new `typecheck:consumer` script + tsconfig — this is what would have caught a wrong "types" path or a missing re-export, which vitest's runtime tests (importing from src/) never exercise. - `check:types` runs `attw --pack .` with the `esm-only` profile (the package was already ESM-only before this change; the profile just stops attw from flagging that pre-existing, intentional characteristic as an error). Both wired into ci.yml. golden-output snapshot updated: adding files under packages/cli/test/ shifts the self-scan's own SNS-05 "test file" count (that check's regex matches anything under a test/ directory, not just *.test.* files) — a pre-existing check quirk, not a regression from this change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
files is fixed once the walk completes, so matching(re) is a pure function of re's source+flags. Several checks already re-query the exact same pattern independently (every CTX-03..06 rule check calls matching(RULE_RE), HYG-04/HYG-08 both call matching(MCP_RE)) — this turns those repeats into one filter() per distinct pattern instead of one per call site. No behavior change: verified against golden-output.test.ts (byte-identical Report JSON) and the full fixture suite. Deliberately NOT attempted here: parallelizing file reads. Check.run() and ScanContext.read() are part of the public API and synchronous by contract — making reads async would require an API break (a major version), not something to slip in as a perf tweak. Left for a future breaking release per the "Garantias de não-quebra" in the distribution/perf plan. Added bench/scan.bench.mjs (npm run bench) — builds a synthetic repo and times score() against it, since there was no way to measure scan-time changes before this. Measured: ~7% faster on a 20k-file synthetic repo (165.8ms → 154.1ms avg over 8 runs); negligible on this repo itself (too few files with repeated-pattern checks for the win to show), which is expected — this specifically targets large monorepos, not small ones. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…hange template, changesets Formalizes the contribution process CONTRIBUTING.md already describes in prose, and adds low-risk repo-maintenance automation, without inventing a new schema/plugin layer for checks (kept to hand-written TS checks, per scope decision): - .github/ISSUE_TEMPLATE/rubric_change.yml: a dedicated form for add/edit/remove-a-check proposals (dimension, points, detection logic, fixture impact, and a checklist of the artifacts that must move together) — distinct from the general feature_request.yml, and covers edit/removal which that template doesn't. - .github/CODEOWNERS: routes PRs touching checks/score.ts/maturity docs/ fixtures/release workflow to @paladini automatically. - .github/dependabot.yml: weekly updates for devDependencies (grouped, minor/patch only) and GitHub Actions. packages/cli has zero runtime dependencies by design, so this never touches the supply-chain surface that actually ships to consumers. - Changesets (@changesets/cli): contributors run `npm run changeset` to describe a user-facing change; `npm run version-packages` (changeset version + scripts/sync-version.mjs) bumps packages/cli/package.json, generates packages/cli/CHANGELOG.md, and mirrors the new version into TOOL_VERSION/jsr.json — closing the "bump three files by hand" gap changesets doesn't know about on its own. Verified sync-version.mjs against a scratch copy (not the real files) before relying on it. RELEASING.md documents this as the preferred path; the old manual triple-bump still works when no changesets were added. - CONTRIBUTING.md links the new template from "Adding or changing a check" and mentions adding a changeset before opening a PR. Deliberately not done here (need the maintainer's own action or a decision this PR shouldn't make unilaterally): enabling GitHub Discussions, installing an all-contributors bot, or adding FUNDING.yml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
report.root is path.resolve(rootInput) — an absolute filesystem path that differs between machines (D:\...\fixtures\level-3 locally vs /home/runner/work/... on the GitHub Actions runner). The snapshot baseline captured it verbatim, so every snapshot test failed in CI despite passing locally. Excluded alongside tool.version, which was already normalized out for the same class of reason (environment-dependent, not a real regression). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Updates the harness, guide, and agent-facing docs to match this PR's
actual state, plus folds in a few small pre-existing (uncommitted) fixes
that were sitting in the working tree from an earlier session:
- ROADMAP.md: new "Shipped in v0.4.0" section (bundler migration, verified
type packaging, scan-perf memoization, community infra) following the
existing v0.2.0/v0.3.0 pattern; "Proposing something new" now points at
the new Rubric change issue template specifically.
- docs/guide/measure-and-improve.md: new "Using it as a library" section
(score(), and the lower-level createScanContext/buildReport/computeDiff/
renderMarkdown building blocks) — this capability existed before but was
only documented in the npm README, not the canonical guide.
- packages/cli/README.md, README.md: mention the bundled build + verified
type packaging; link the new Rubric change template from the
Contributing section; add `npm run bench` to the dev commands.
- AGENTS.md, .cursor/rules/{core,checks}.mdc, .cursor/skills/release/SKILL.md:
bring the harness's own guidance current — `npm test` now also
typechecks and runs the packaging smoke test, `rubric-sync.test.ts`
closes the maturity-model.md enforcement gap, the sync/read-parallelism
API-stability rule, and the changesets-based release flow. Also fixes a
real stale fact in checks.mdc ("the overall total must stay 100" — it's
been 108 since v0.2.0, just never corrected).
- .github/ISSUE_TEMPLATE/feature_request.yml: narrowed to non-rubric
features now that rubric_change.yml exists, so the two templates don't
overlap/compete in the issue picker.
- Small pre-existing fixes folded in (predate this session, never
committed): stale v0.1.2/61/100 example output in packages/cli/README.md
corrected to v0.3.0/66/108; HomeLanding.vue's Cursor plugin link pointed
at the Marketplace before it's actually live there — now points at the
in-repo plugin/ directory; PLUGINS-ROADMAP.md (the multi-harness plugin
plan ROADMAP.md already linked to) added since it didn't exist in git
yet.
Verified: npm test (171/171), biome check (0 errors), self-scan stays
L4/108/108/100% (CTX-02's evidence line legitimately changed — AGENTS.md
grew from 36 to 49 non-empty lines — golden-output snapshot updated to
match), npm run docs:build clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ting that isn't live Verified against the actual Cursor Marketplace (a search for "harness-score" returns nothing; the only "Harness" result is an unrelated harness.io plugin) — five places still said "Install from the Cursor Marketplace" as if the listing were live: docs/index.md's hero action, the "Cursor plugin" product card in HomeLanding.vue (a second, separate spot from the "installs" list that a prior session already corrected), measure-and-improve.md, packages/cli/README.md, and cursor-harness-surface.md. All now point at the plugin's directory in this repo instead, with a note that the Marketplace listing is submitted and pending review — accurate today, and each spot already links to a real, working destination instead of a search that finds nothing (or the wrong plugin entirely). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tually pinned /harness-audit always runs npx harness-score, which resolves to whatever's latest on npm — not a version frozen at plugin-release time. The "33 checks" figure in the 0.1.0 entry wasn't a harmless historical fact (like a fixed bug), it was actively describing current behavior incorrectly (the rubric has had 36 checks since v0.2.0), so treating it as an untouchable changelog entry didn't hold up on reflection — removed the number rather than re-pinning a new one that would just go stale again next rubric change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Today's live research, written into the repo instead of left only in chat: a new "Landscape research (2026-07-14)" section covering two facts that change the plan's calculus (AGENTS.md is now a Linux Foundation- governed standard read by 8 of the top 10 tools researched; MCP is near-universal across the same list), a verified top-10 comparison table of AI-first dev tools and their own artifact conventions, and new deep-dive sections for OpenCode, Cline, Continue.dev, Zed, and JetBrains AI Assistant/Junie (previously a one-line "not researched" stub for three of these). Repo layout, release-pipeline notes, and sequencing updated to include all of them; the three open decisions needing a maintainer call are unchanged in substance, just updated with the new scope (nine targets, not four). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four independent improvements to
harness-score, each verified not toregress the existing public behavior:
1. Distribution — migrate build to tsup
packages/cli/dist/goes from 36 unbundled files (~82KB) to 5 files(~54KB) via
tsupwith splitting + minification.package.jsongains an explicit"types"field and a"types"condition in
"exports".RELEASING.mddocuments that JSR deliberately keeps publishing rawsrc/**/*.ts(pre-existing, not something this PR needed to reconcile).2. Type completeness for library consumers
test/types/smoke.ts: imports every public symbol from the builtdist/output (notsrc/), type-checked via a newtypecheck:consumerscript — catches packaging regressions vitest's runtime tests (which
import from
src/) never see.check:typesrunsattw --pack . --profile esm-onlyin CI (thepackage was already ESM-only before this change).
3. Scan performance
ctx.matching(re)is now memoized per regex source+flags insidecreateScanContext— several checks already re-query the same patternindependently (e.g. every
CTX-03..06rule check callsmatching(RULE_RE)). No behavior change (verified byte-identicaloutput). ~7% faster on a 20k-file synthetic benchmark
(
npm run bench); negligible on this repo itself (too small to showthe win).
Check.run()/ScanContext.read()are public API and synchronous bycontract; making them async would be a breaking (major-version) change,
not a drive-by perf tweak.
4. Community contribution infrastructure
.github/ISSUE_TEMPLATE/rubric_change.yml: dedicated add/edit/remove-a-checkproposal form (dimension, points, fixture impact, sync checklist).
.github/CODEOWNERS: routes rubric/release-surface PRs for review..github/dependabot.yml: weekly devDependency + Actions updates(packages/cli still has zero runtime dependencies).
@changesets/cli) +scripts/sync-version.mjs: contributorsdescribe user-facing changes via
npm run changeset;npm run version-packagesbumpspackage.json, generatespackages/cli/CHANGELOG.md, and mirrors the version intoTOOL_VERSION/jsr.json(which changesets doesn't know about natively).Not done here (needs the maintainer's own action, not a code change):
enabling GitHub Discussions, installing an all-contributors bot, adding
FUNDING.yml.Non-breaking verification (see plan's "Garantias de não-quebra")
packages/cli/test/golden-output.test.ts: new snapshot baseline of theReportJSON for every fixture + this repo's own self-scan — passedunchanged through the build/perf changes.
npm pack, install the tarball in athrowaway project, run the bin directly and via
npx,importthelibrary at runtime, and
tsc --noEmitagainst a consumer script usingthe published types.
npm ciclean install + full CI-equivalent gate (lint, test:coverage,self-scan, docs:build) all green on the final state.
maturity-model.md'sdimension-points table and level thresholds were only mirrored by hand
(a comment saying "change both together");
test/rubric-sync.test.tsnow enforces it in CI, same as
docs.test.tsalready does formeasure-and-improve.md.Test plan
npm ci(clean install) +npm run lint+npm run test:coverage -w harness-score+ self-audit (node packages/cli/dist/cli.js . --min-level 4) +npm run docs:build— all greenpackages/cli— 171/171 tests pass, including the new golden-output snapshot and rubric-sync testsnpm pack+ install in an external throwaway project: bin execution,npx, runtimeimport, andtsc --noEmitagainst consumer types all passattw --pack . --profile esm-onlycleannpm run benchshows ~7% improvement on a 20k-file synthetic repo, no regression on this repo