Skip to content

fix(prompt): resolve contradiction blocking without test output#97

Open
mlugo-apx wants to merge 1 commit into
nizos:mainfrom
mlugo-apx:fix/response-prompt-no-test-output
Open

fix(prompt): resolve contradiction blocking without test output#97
mlugo-apx wants to merge 1 commit into
nizos:mainfrom
mlugo-apx:fix/response-prompt-no-test-output

Conversation

@mlugo-apx

Copy link
Copy Markdown

Summary

Fix prompt contradiction in response.ts that causes incorrect blocking when no test reporter is installed.

  • Remove "No test output captured" from example block reasons (contradicts decision value definition)
  • Add "No test output available" as example approval reason (aligns with null = "insufficient information to determine")

Problem

The response.ts prompt has a contradiction between decision value definitions and example block reasons:

  • Decision values define null as: "Changes follow TDD principles OR insufficient information to determine"
  • Example block reasons include: "No test output captured. Cannot validate TDD compliance without test results."

When no test reporter is installed (custom/browser-based frameworks, Mocha, unittest, etc.), test.json is never created. The validator model sees no test section in its prompt, follows the example block reason, and blocks every implementation edit — even after the developer has manually written and verified a failing test.

This makes tdd-guard unusable for any project without a supported reporter.

Fix

One line removed from example block reasons, one line added to example approval reasons. No code logic changes. Only affects users without a reporter installed — users WITH reporters always have test output and are unaffected.

Test plan

  • Verify existing tests pass (no behavioral change for reporter users)
  • Manual test: project with no reporter installed → implementation edits should not be blocked when test output is absent
  • Manual test: project WITH reporter → blocking behavior unchanged

Related

  • Roadmap mentions Mocha and unittest support — until those reporters ship, this fix prevents incorrect blocking for those users
  • A more robust future enhancement could detect "no reporter installed" in processHookData.ts code (before reaching the AI validator) and return a non-blocking informational message

…st output

Remove "No test output captured" from example block reasons — it contradicts
the decision value definition where null means "insufficient information to
determine." Add matching approval reason for missing test output.

Without this fix, projects using custom/browser-based test frameworks (or any
framework without a supported reporter) get blocked on every implementation
edit because test.json is never created.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@nizos

nizos commented Apr 22, 2026

Copy link
Copy Markdown
Owner

Hi @mlugo-apx,

Thanks for putting this together. I see the issue you're running into.

My concern is that the fix affects more than the "no reporter installed" case. The validator can't distinguish that from "no tests have run yet this session." Both look the same from the prompt's side. Flipping the default would let an agent skip straight to implementation on every new session, before any test has been written or run. That violates TDD.

I also don't read the current prompt as contradictory. The null definition is the general rule, and the block example is a deliberate override for cases where the uncertainty itself is suspicious.

The underlying issue is still worth solving though. I think the right answer is to add the reporter rather than weaken the prompt. Which framework are you hitting this with? Might be worth adding a reporter for it.

Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants