Skip to content

fix(cli): count critical Markdoc errors in validate - #3343

Closed
tojacob03 wants to merge 1 commit into
evidence-dev:mainfrom
tojacob03:fix/validate-critical-errors
Closed

tojacob03 wants to merge 1 commit into
evidence-dev:mainfrom
tojacob03:fix/validate-critical-errors

Conversation

@tojacob03

Copy link
Copy Markdown

Summary

evidence validate returned valid: true and exited 0 when a page had a critical Markdoc error, such as an undefined tag.

Markdoc gives structural errors (tag-undefined, missing-opening, tag-placement-invalid, …) the level critical. The validate route only counted error and warning, so these errors were counted as neither. cli/cli/validate.ts already prints critical entries as errors, and serve shows them as a page-level error box. This change makes the JSON counts and the exit code agree with both.

Fixes #3340

Changes

  • cli/src/routes/api/validate/+server.ts: count critical as an error.
  • New validate-route.test.ts next to the route. It runs GET against a temporary project in syntax-only mode, the same way home-page-load.test.ts does, and checks:
    • a clean page is valid: true
    • {% unknown_tag /%} gives errorCount: 1 and valid: false

Testing

  • The new test fails on main (errorCount is 0) and passes with the fix.
  • cli Vitest suite: 389/389 passed.
  • svelte-check on cli: the only error is the missing docs-bundle.generated module, which is created by the bun build step. It is not related to this change.

Markdoc reports structural errors (undefined tags, unclosed blocks) with
level 'critical'. The validate route only counted 'error' and 'warning',
so a page with a critical error returned valid: true and
`evidence validate` exited 0, even though the CLI's human report and
`serve` both treat these as errors.

Fixes evidence-dev#3340

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 29, 2026

Copy link
Copy Markdown

@tojacob03 is attempting to deploy a commit to the Evidence Team on Vercel.

A member of the Team first needs to authorize it.

@hughess

hughess commented Sep 29, 2026

Copy link
Copy Markdown
Member

/upstream

@github-actions

Copy link
Copy Markdown
Contributor

Imported for internal review, this PR will be updated when it merges.

@hughess

hughess commented Sep 29, 2026

Copy link
Copy Markdown
Member

Thanks for the contribution, @tojacob03! Merged upstream, and synced back in 80a719c

@hughess hughess closed this Sep 29, 2026
@tojacob03

Copy link
Copy Markdown
Author

Thanks for the quick review and merge, @hughess !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

evidence validate exits 0 on critical markdown parse errors

2 participants