Skip to content

Include auto-generated tscircuit docs in system prompt#67

Open
js360000 wants to merge 1 commit into
tscircuit:mainfrom
js360000:bountybox/issue-45
Open

Include auto-generated tscircuit docs in system prompt#67
js360000 wants to merge 1 commit into
tscircuit:mainfrom
js360000:bountybox/issue-45

Conversation

@js360000
Copy link
Copy Markdown

Summary

feat: include auto-generated tscircuit docs in system prompt (#45)

createLocalCircuitPrompt now fetches the auto-generated docs feed at
https://docs.tscircuit.com/ai.txt and prepends them to the system
prompt, ahead of the existing handwritten overview. The generated
docs are treated as authoritative when they conflict with the
handwritten section.

Network resilience:

  • The two upstream fetches (COMPONENT_TYPES.md and ai.txt) run in
    parallel via Promise.all.
  • The generated-docs fetch is wrapped in fetchOptionalFileContent,
    which catches non-200 responses and network errors and returns an
    empty string. If ai.txt is unavailable, prompt creation still
    succeeds and the section is simply omitted — existing benchmarks
    keep running.

Tests (tests/create-local-circuit-prompt.test.ts) cover all three
paths via a mocked global fetch:

  1. Success — generated docs are included and appear before
    "## tscircuit API overview".
  2. 404 fallback — prompt is still built without the generated-docs
    section; handwritten overview and RULES section remain intact.
  3. Throwing fetch — same fallback, errors swallowed.

Closes #45

Diff

 .../create-local-circuit-prompt.ts                 | 37 ++++++++-
 tests/create-local-circuit-prompt.test.ts          | 93 ++++++++++++++++++++++
 2 files changed, 126 insertions(+), 4 deletions(-)

Claim

/claim #45

…it#45)

`createLocalCircuitPrompt` now fetches the auto-generated docs feed at
https://docs.tscircuit.com/ai.txt and prepends them to the system
prompt, ahead of the existing handwritten overview. The generated
docs are treated as authoritative when they conflict with the
handwritten section.

Network resilience:
- The two upstream fetches (`COMPONENT_TYPES.md` and `ai.txt`) run in
  parallel via `Promise.all`.
- The generated-docs fetch is wrapped in `fetchOptionalFileContent`,
  which catches non-200 responses and network errors and returns an
  empty string. If `ai.txt` is unavailable, prompt creation still
  succeeds and the section is simply omitted — existing benchmarks
  keep running.

Tests (`tests/create-local-circuit-prompt.test.ts`) cover all three
paths via a mocked global `fetch`:
1. Success — generated docs are included and appear before
   "## tscircuit API overview".
2. 404 fallback — prompt is still built without the generated-docs
   section; handwritten overview and RULES section remain intact.
3. Throwing fetch — same fallback, errors swallowed.

Closes tscircuit#45
@js360000
Copy link
Copy Markdown
Author

Heads up on the Bun Test failure showing on this PR: the failing case is tests/utils/generate-random-prompts.test.ts > generateRandomPrompts > should return an array of prompts, which calls the OpenAI API directly without mocking and fails with 401 You didn't provide an API key in CI runs that don't have OPENAI_API_KEY set.

That test is unrelated to this PR's changes — this diff only touches lib/prompt-templates/create-local-circuit-prompt.ts and adds tests/create-local-circuit-prompt.test.ts, which uses a globalThis.fetch stub for both the existing COMPONENT_TYPES.md fetch and the new ai.txt fetch. The 14 other tests in this PR's run (including the 3 new ones) pass. The same Bun Test failure has appeared on other PRs against main over the last few days where the API key wasn't propagated to the runner.

Happy to switch the new tests over to something else if bun:test's global fetch mocking isn't the preferred convention here, but the OpenAI-call test failure isn't something this PR needs to address.

@js360000
Copy link
Copy Markdown
Author

CI is currently red on this PR, but the failure is unrelated to the diff — tests/utils/generate-random-prompts.test.ts calls the OpenAI API and the workflow doesn't have OPENAI_API_KEY available on PR runs:

You didn't provide an API key. ... at node_modules/openai/error.mjs:44:20

The failing test (generateRandomPrompts > should return an array of prompts) lives on main and exercises lib/utils/generate-random-prompts.ts. This PR only touches lib/prompt-templates/create-local-circuit-prompt.ts, which is independent.

If it'd help unblock the gate, options:

  1. Inject the secret on PR runs: add OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} to the test job env, and switch the workflow trigger to pull_request_target for trusted contributors only (or use a manual workflow_dispatch approval gate).
  2. Skip OpenAI-backed tests when the key is absent: small guard at the top of generate-random-prompts.test.ts — e.g. describe.skipIf(!process.env.OPENAI_API_KEY)(...). Cleanest for fork PRs because secrets aren't exposed to forks by GitHub's design.
  3. Keep it as-is and merge on type-check + manual review, since the failure mode is well-known and not introduced here.

Happy to push option 2 as a separate one-line PR if that's the preferred route.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

use the new auto generated docs in the system prompt

2 participants