Skip to content

chore(nx-dev): sync docs kb articles into the pylon knowledge base - #36590

Draft
jaysoo wants to merge 2 commits into
masterfrom
DOC-542
Draft

chore(nx-dev): sync docs kb articles into the pylon knowledge base#36590
jaysoo wants to merge 2 commits into
masterfrom
DOC-542

Conversation

@jaysoo

@jaysoo jaysoo commented Aug 6, 2026

Copy link
Copy Markdown
Member

Current Behavior

Pylon crawls nx.dev, but crawled training data does not feed the support widget's suggested-answer feature - only real KB articles are eligible. The 173 articles left in Pylon from the closed #36277 migration are stale copies of pre-DOC-552 doc paths with no way to refresh them.

Expected Behavior

A nightly job mirrors astro-docs/src/content/docs/kb into Pylon. nx.dev stays canonical - copy only, no source deletion and no redirects.

Pylon stores body_html verbatim, so change detection is a byte compare with no state file. Images carry a data-nx-src content-hash marker so re-runs reuse the uploaded CDN URL. Writes are confined to one collection, keeping the hand-written enterprise articles out of reach, and --prune is opt-in and refuses to delete more than a quarter of the collection.

All 184 articles convert with no warnings. Articles stay unlisted so nx.dev remains SEO-canonical.

Still needed before the nightly can run: PYLON_API_TOKEN in repo secrets.

Related Issue(s)

DOC-542


Polygraph View session ↗

## Current Behavior

Pylon crawls nx.dev, but crawled training data does not feed the support
widget's suggested-answer feature. Only real KB articles are eligible.

The 173 articles left in Pylon from the closed #36277 migration are stale
copies of pre-DOC-552 doc paths, with no way to refresh them.

## Expected Behavior

Nightly job mirrors `astro-docs/src/content/docs/kb` into Pylon. nx.dev stays
canonical - copy only, no source deletion, no redirects.

- `markdoc-to-html.ts` converts the Markdoc AST to Pylon-safe HTML. All 184
  articles convert with no warnings.
- `pylon-client.ts` wraps the REST API with retry, throttle and pagination.
- `sync-kb.ts` diffs source against live and creates/updates. Pylon stores
  `body_html` verbatim, so change detection is a byte compare - no state file.
- Images carry a `data-nx-src` content-hash marker so re-runs reuse the
  uploaded CDN URL instead of re-uploading.
- Writes are confined to one collection, so the hand-written enterprise
  articles beside them cannot be touched.
- `--prune` is opt-in and refuses to delete more than a quarter of the
  collection.

Articles stay unlisted to keep nx.dev SEO-canonical. Needs `PYLON_API_TOKEN`
in CI.

## Related Issue(s)

Fixes DOC-542
@netlify

netlify Bot commented Aug 6, 2026

Copy link
Copy Markdown

Deploy Preview for nx-dev ready!

Name Link
🔨 Latest commit 4afb60f
🔍 Latest deploy log https://app.netlify.com/projects/nx-dev/deploys/6a74e69a540d240008394bea
😎 Deploy Preview https://deploy-preview-36590--nx-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Aug 6, 2026

Copy link
Copy Markdown

Deploy Preview for nx-docs ready!

Name Link
🔨 Latest commit 4afb60f
🔍 Latest deploy log https://app.netlify.com/projects/nx-docs/deploys/6a74e69a6c5ede0008785b54
😎 Deploy Preview https://deploy-preview-36590--nx-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

## Current Behavior

Workflow only runs nightly or via workflow_dispatch, which GitHub exposes
only once the file is on the default branch. No way to exercise it from a PR.

## Expected Behavior

Adds a pull_request trigger scoped to the pylon tooling and kb content, always
forced to --dry-run so a review never writes to Pylon.

- New preview job converts all articles with no credentials, so fork PRs get
  signal too.
- Sync job skips on fork PRs, where secrets are unavailable.
- workflow_dispatch dry-run now defaults to true.
- Slack notification narrows to scheduled runs.

## Related Issue(s)

DOC-542
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant