docs: reorganize RTK documentation into 3-tab structure (guide/reference/architecture)#1029
docs: reorganize RTK documentation into 3-tab structure (guide/reference/architecture)#1029FlorianBruniaux wants to merge 9 commits intodevelopfrom
Conversation
fix(refacto-codebase): technical docs & sub folders
…-components--rtk chore(master): release 0.34.2
fix(refacto): wrappers for standardization, exit codes lexer tokenizer, constants, code clean
…-components--rtk chore(master): release 0.34.3
Creates docs/README.md (interface contract defining 3-tab structure, required frontmatter, and conventions) and 4 stub pages with valid frontmatter for docs/guide/, docs/reference/, and docs/architecture/. These are the prerequisites for the prepare-docs.mjs pipeline (Plan B) and for Adrien's Phase 1 content work. No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Florian BRUNIAUX <florian@bruniaux.com>
Creates the complete user-facing guide for the docs website: Getting started: - quick-start.md: 5-minute walkthrough (init, first commands, rtk gain) - supported-agents.md: Claude Code, Cursor, Copilot, Gemini, Cline, Windsurf, Codex, OpenCode — integration tiers, install commands, graceful degradation Commands (adapted from FEATURES.md, English, --help-first format): - git.md: status/log/diff/show/add/commit/push/pull/branch + gh CLI - cargo.md: test/nextest/build/check/clippy/install + generic test/err wrappers - files.md: ls/read/grep/find/diff/wc/smart with before/after examples - javascript.md: vitest/playwright/tsc/eslint/prettier/next/pnpm/npm/npx/prisma - python.md: pytest/ruff/mypy/pip/deps Reference: - filters/using-filters.md: 8-stage pipeline, lookup priority, TOML DSL reference, Mermaid diagram (adapted from docs/filter-workflow.md) - analytics/gain.md: rtk gain flags, daily/weekly/monthly breakdowns, export formats, token estimation, database management (from docs/AUDIT_GUIDE.md) - configuration.md: full config.toml reference, env vars, tee system, telemetry - troubleshooting.md: common issues and fixes (from docs/TROUBLESHOOTING.md) All pages carry valid frontmatter (title, description, sidebar.order). No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Florian BRUNIAUX <florian@bruniaux.com>
…mmands Guide — 10 new command and analytics pages: - commands/go.md: go test (NDJSON), golangci-lint - commands/ruby.md: rspec, rubocop, rake - commands/dotnet.md: dotnet build/test, binlog, format - commands/containers.md: docker ps/images/logs/compose, kubectl pods/services/logs - commands/github-cli.md: gh pr/issue/run, Graphite gt commands - commands/data.md: json, env, log, curl, wget, aws, psql, summary - commands/utilities.md: proxy passthrough, global flags, RTK_DISABLED - filters/creating-filters.md: TOML DSL guide, field reference, inline tests - analytics/discover.md: rtk discover — missed savings analysis - analytics/economics.md: rtk cc-economics — dollar savings vs ccusage Reference tab (9 pages): - contributing/guide.md: design philosophy, PR process, TOML vs Rust decision - contributing/security.md: vulnerability reporting, dangerous patterns, dep criteria - contributing/coding-standards.md: Rust rules, error handling, lazy_static, fallback pattern - contributing/testing.md: snapshot tests, token accuracy, cross-platform, benchmarks - internals/command-routing.md: 6-phase lifecycle, exit codes, verbosity, module map - internals/filter-pipeline.md: 12 filtering strategies, TOML stages, savings by ecosystem - internals/tracking-system.md: SQLite schema, data flow, token estimation, Tracker API - internals/hook-engine.md: rewrite registry, compound commands, exit code contract, rtk init - toml-dsl/specification.md: complete field reference, pipeline order, build compilation Architecture tab (4 pages): - diagrams/command-flow.md: end-to-end Mermaid — hook → RTK → LLM - diagrams/filter-pipeline.md: build pipeline + 8-stage runtime Mermaid - decisions/why-no-async.md: ADR — single-threaded, <10ms startup constraint - decisions/proxy-architecture.md: ADR — why CLI proxy over aliases/LD_PRELOAD/hooks-only All pages carry valid frontmatter (title, description, sidebar.order). No existing files modified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Florian BRUNIAUX <florian@bruniaux.com>
|
|
i18n docs — two options, need inputBefore this merges, worth aligning on how we handle multi-language docs down the road. The current structure is EN-only (explicit decision in Plan B), but the question is whether we set it up to be migration-friendly. Option A — locale subdirectory at docs/ root
Upside: clean, native Starlight i18n, no URL changes for EN. Option B — EN-only in repo, translations externalized Translations live on Crowdin/Weblate or a separate repo. Upside: zero impact on current structure, zero maintenance burden until there's actual demand. My take: Option B now, Option A when a specific language has contributors ready to maintain it. Migration is a one-liner at that point. No reason to restructure prematurely. @pszymkowiak @aeppling — what's your preference? If you want Option A, better to decide before Adrien starts filling in content so we're not doing a mass rename mid-sprint. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new structured documentation layout under docs/ intended for consumption by the rtk-ai/rtk-website pipeline (Guide / Reference / Architecture), and also includes release/version metadata updates.
Changes:
- Add a large set of new docs pages with required frontmatter, organized into
docs/guide/,docs/reference/, anddocs/architecture/. - Add diagrams (Mermaid) and ADR-style architecture decision pages.
- Update release/version files (
CHANGELOG.md,Cargo.toml,Cargo.lock,.release-please-manifest.json).
Reviewed changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 20 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/README.md | Defines the 3-tab docs structure and frontmatter/linking contract. |
| docs/guide/index.md | Guide landing page linking to key user-facing sections. |
| docs/guide/getting-started/installation.md | Installation + verification instructions. |
| docs/guide/getting-started/quick-start.md | 5-minute walkthrough for initial usage. |
| docs/guide/getting-started/supported-agents.md | Lists supported agents and integration tiers. |
| docs/guide/configuration.md | User-facing config and env var reference. |
| docs/guide/troubleshooting.md | Common issues and fixes. |
| docs/guide/filters/using-filters.md | Explains TOML filters, lookup order, and pipeline (contains schema details). |
| docs/guide/filters/creating-filters.md | Walkthrough for writing project-local TOML filters. |
| docs/guide/commands/git.md | Git command behavior and savings examples. |
| docs/guide/commands/cargo.md | Cargo command behavior and wrappers. |
| docs/guide/commands/files.md | Filesystem/search command behaviors. |
| docs/guide/commands/javascript.md | JS/TS ecosystem commands and examples. |
| docs/guide/commands/python.md | Python ecosystem commands and examples. |
| docs/guide/commands/go.md | Go ecosystem commands and examples. |
| docs/guide/commands/ruby.md | Ruby ecosystem commands and examples. |
| docs/guide/commands/dotnet.md | .NET ecosystem commands and examples. |
| docs/guide/commands/containers.md | Docker/Kubernetes command behaviors. |
| docs/guide/commands/data.md | JSON/env/log/curl/wget/aws/psql/summary commands. |
| docs/guide/commands/github-cli.md | gh/gt behaviors and examples. |
| docs/guide/commands/utilities.md | Proxy passthrough + global flags. |
| docs/guide/analytics/gain.md | rtk gain usage and data model explanation. |
| docs/guide/analytics/discover.md | rtk discover usage and output explanation. |
| docs/guide/analytics/economics.md | rtk cc-economics usage explanation. |
| docs/reference/index.md | Reference tab landing page. |
| docs/reference/contributing/guide.md | Contributor guide distilled for the docs site. |
| docs/reference/contributing/security.md | Security policy + dangerous patterns guidance. |
| docs/reference/contributing/coding-standards.md | RTK-specific Rust coding standards. |
| docs/reference/contributing/testing.md | Testing approaches (insta, savings checks, perf). |
| docs/reference/internals/command-routing.md | High-level routing lifecycle explanation. |
| docs/reference/internals/filter-pipeline.md | Overview of filter strategies + TOML pipeline. |
| docs/reference/internals/tracking-system.md | SQLite tracking system explanation (schema/paths/etc.). |
| docs/reference/internals/hook-engine.md | Hook/rewrite architecture and contracts. |
| docs/reference/toml-dsl/specification.md | TOML DSL “specification” reference page. |
| docs/architecture/index.md | Architecture tab landing page. |
| docs/architecture/diagrams/command-flow.md | Mermaid diagram for end-to-end command flow. |
| docs/architecture/diagrams/filter-pipeline.md | Mermaid diagram for TOML filter build/runtime pipeline. |
| docs/architecture/decisions/why-no-async.md | ADR documenting “no async runtime” rationale. |
| docs/architecture/decisions/proxy-architecture.md | ADR documenting the CLI proxy pattern decision. |
| CHANGELOG.md | Adds release entries for 0.34.2 and 0.34.3. |
| Cargo.toml | Bumps crate version to 0.34.3. |
| Cargo.lock | Updates locked package version to 0.34.3. |
| .release-please-manifest.json | Updates release-please tracked version to 0.34.3. |
| [[filters]] | ||
| name = "my-tool" | ||
| match_command = "^my-tool\\b" | ||
|
|
||
| strip_lines = [ |
There was a problem hiding this comment.
The TOML example uses [[filters]] with a name field and strip_lines, but RTK's TOML engine expects schema_version = 1 at the top and filter definitions under [filters.<filter-name>] with fields like strip_lines_matching / keep_lines_matching (see src/core/toml_filter.rs). As written, this example will fail to parse and the filter will be ignored.
| [[filters]] | |
| name = "my-tool" | |
| match_command = "^my-tool\\b" | |
| strip_lines = [ | |
| schema_version = 1 | |
| [filters.my-tool] | |
| match_command = "^my-tool\\b" | |
| strip_lines_matching = [ |
| S1["1. strip_ansi"] --> S2 | ||
| S2["2. replace"] --> S3 | ||
| S3{"3. match_output\nshort-circuit?"} | ||
| S3 -->|"✅ match"| MSG[["emit on_match\nstop"]] |
There was a problem hiding this comment.
In the runtime pipeline diagram, match_output is shown emitting on_match, but the TOML schema uses match_output = [{ pattern, message, unless? }] and there is no on_match field (see src/core/toml_filter.rs). The diagram should reference the rule’s message instead.
| S3 -->|"✅ match"| MSG[["emit on_match\nstop"]] | |
| S3 -->|"✅ match"| MSG[["emit rule message\nstop"]] |
| **Storage locations:** | ||
| - Linux: `~/.local/share/rtk/tracking.db` | ||
| - macOS: `~/Library/Application Support/rtk/tracking.db` | ||
| - Windows: `%APPDATA%\rtk\tracking.db` | ||
|
|
There was a problem hiding this comment.
The DB filename/location here doesn’t match the current implementation: the tracker uses dirs::data_local_dir()/rtk/history.db by default (not tracking.db), and on Windows it’s under LocalAppData (not %APPDATA%). Please update the paths to match src/core/tracking.rs (HISTORY_DB = "history.db").
| ``` | ||
| estimate_tokens(text) = text.len() / 4 | ||
| ``` | ||
|
|
||
| ~4 characters per token average. Accuracy: ±10% vs actual LLM tokenization. |
There was a problem hiding this comment.
Token estimation is described as text.len() / 4, but the implementation uses ceil(chars/4) (so 1–4 chars still counts as 1 token) in estimate_tokens (src/core/tracking.rs). Please update the formula here to avoid confusion when validating numbers.
| [package] | ||
| name = "rtk" | ||
| version = "0.34.1" | ||
| version = "0.34.3" | ||
| edition = "2021" |
There was a problem hiding this comment.
PR description says “No existing files were modified” and “All changes are additive”, but this PR changes the crate version (and also updates CHANGELOG/Cargo.lock/release manifest). If the version bump is intentional, please update the PR description; otherwise, consider dropping these release/version changes from this docs-only PR.
| [tracking] | ||
| enabled = true # enable/disable token tracking | ||
| history_days = 90 # retention in days (auto-cleanup) | ||
| database_path = "/custom/path/tracking.db" # optional override | ||
|
|
There was a problem hiding this comment.
Docs imply tracking.history_days controls retention, but Tracker::cleanup_old() uses DEFAULT_HISTORY_DAYS and doesn’t read config (src/core/tracking.rs). Consider clarifying this (or implementing config-driven retention) so the config docs match runtime behavior.
|
|
||
| | Variable | Description | | ||
| |----------|-------------| | ||
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | |
There was a problem hiding this comment.
The env var table is missing RTK_DB_PATH, which overrides the tracking DB location with highest priority (src/core/tracking.rs). Adding it here would make configuration docs complete.
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | | |
| | `RTK_DISABLED=1` | Disable RTK for a single command (`RTK_DISABLED=1 git status`) | | |
| | `RTK_DB_PATH` | Override the tracking database path | |
| ELSE 0 END | ||
| ), | ||
| exec_time_ms INTEGER | ||
| ); |
There was a problem hiding this comment.
This schema omits columns that RTK adds via migrations (exec_time_ms and project_path) and also has timestamp populated by RTK (not a DEFAULT). Please update the schema snippet to reflect the actual CREATE TABLE + migrations in src/core/tracking.rs so readers can query the DB correctly.
| ## Thread safety | ||
|
|
||
| Single-threaded execution with `Mutex<Option<Tracker>>` for future-proofing. No multi-threading currently used. |
There was a problem hiding this comment.
The “Thread safety” note mentions Mutex<Option<Tracker>>, but the current implementation doesn’t use that pattern (it opens connections via Tracker::new() and relies on WAL + busy_timeout for concurrent access). Please update this section to match src/core/tracking.rs.
| | `keep_lines_matching` | regex[] | no | Keep only lines matching at least one pattern | | ||
| | `replace` | array | no | Regex substitutions: `{ pattern, replacement }` | | ||
| | `match_output` | array | no | Short-circuit rules: `{ pattern, message }` | | ||
| | `truncate_lines_at` | int | no | Truncate lines longer than N characters | |
There was a problem hiding this comment.
match_output rules also support an optional unless regex to prevent short-circuiting when errors/warnings are present (see MatchOutputRule in src/core/toml_filter.rs). Documenting unless here would make the spec complete.
|
Hey, Thanks for the documentation rework, since PR #869 we already had the technical documentation , this is maintained in each respective folder per responsability, to ensure easy collaboration and work in specific areas. We need a clean separation between user documentation : Usage, How to use, How to configure, and so on.... And the technical documentation: How to contribute, Flow Diagrams, Edge case, Engine This is to keep things simple for users, we do not want noise in those documentation. Current RTK technical doc implementationThe documentation follows a distributed, co-located pattern - each folder contains a README explaining its purpose: Root Level Documentation https://github.com/rtk-ai/rtk/blob/main/docs/contributing/ARCHITECTURE.md (42K) Components level Documentation https://github.com/rtk-ai/rtk/blob/main/src/core/README.md Ecosystem-specific READMEs: https://github.com/rtk-ai/rtk/tree/main/src/cmds/git - Git ecosystem filters Full reviewCore Problems
|
|
Thanks for the detailed review — the points are valid, taking them seriously. What you're right about The The commands pages showing What this PR should actually be User-facing only, minimal scope:
For the "what does RTK optimize" question: a single page listing ecosystems and savings metrics (no Proposed next steps
Does this align with what you had in mind? Happy to rework the PR scope before Adrien starts on content. @adrien-epling — also curious what you'd actually want as a user landing on the docs for the first time. |
What
Reorganizes RTK's documentation from 20+ scattered markdown files into a structured 3-tab layout ready to feed the
rtk-ai/rtk-websiteAstro pipeline.Why
The current
docs/directory is a flat collection of FR/EN mixed files with no hierarchy, no frontmatter, and no structure that a static site generator can consume. This PR establishes the interface contract thatprepare-docs.mjs(rtk-website) validates at build time.What changed
No existing files were modified. All changes are additive.
Interface contract
docs/README.md— defines 3-tab structure, required frontmatter schema, naming conventions, and cross-tab link formatTab 1:
docs/guide/— 23 pagesUser-facing documentation, adapted from existing sources (FEATURES.md, INSTALL.md, AUDIT_GUIDE.md, filter-workflow.md, hooks/*/README.md, TROUBLESHOOTING.md):
getting-started/— installation, quick-start (5-min walkthrough), supported-agents (9 agents, 3 integration tiers)commands/— git, cargo, files, javascript, python, go, ruby, dotnet, containers, github-cli, data, utilitiesfilters/— using-filters (8-stage pipeline, Mermaid), creating-filters (TOML DSL guide)analytics/— gain (rtk gain full reference), discover, economicsconfiguration.md,troubleshooting.mdTab 2:
docs/reference/— 10 pagesContributor and technical documentation, distilled from CONTRIBUTING.md, SECURITY.md, ARCHITECTURE.md, .claude/rules/:
contributing/— guide (design philosophy, PR process, TOML vs Rust decision matrix), security (vulnerability policy, dangerous patterns), coding-standards (Rust rules, fallback pattern), testing (snapshot tests, token accuracy, benchmarks)internals/— command-routing (6-phase lifecycle), filter-pipeline (12 strategies), tracking-system (SQLite schema, Tracker API), hook-engine (rewrite registry, exit code contract)toml-dsl/— specification (complete field reference, pipeline order, build compilation)Tab 3:
docs/architecture/— 5 pagesdiagrams/— command-flow (end-to-end Mermaid: hook → RTK → LLM), filter-pipeline (build + runtime Mermaid)decisions/— why-no-async (ADR: single-threaded, <10ms constraint), proxy-architecture (ADR: CLI proxy vs aliases vs LD_PRELOAD)Validation
Every page carries valid YAML frontmatter (
title,description,sidebar.order) as required by the interface contract. The structure passes theprepare-docs.mjsgate: 3 tabs exist, each has anindex.md, all internal links use relative paths.What this enables
Once merged to develop and then master, the
rtk-websitebuild pipeline can clone this repo, copydocs/tosrc/content/docs/, and serve a full 3-tab Starlight documentation site with zero manual intervention.Out of scope
docs/FEATURES.mddeprecation notice (follow-up PR)defaultLocale: 'en'config will allow adding/fr/,/es/routes later with no URL restructuring needed.🤖 Generated with Claude Code