From 91d465c3fedb1f9b424f06a0328a7a96c704e056 Mon Sep 17 00:00:00 2001 From: Stefan Vacareanu Date: Tue, 28 Jul 2026 19:47:29 +0200 Subject: [PATCH] Add Agent Playbook section for automated updates --- README.md | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/README.md b/README.md index 545f3142e..f77057820 100644 --- a/README.md +++ b/README.md @@ -98,6 +98,91 @@ There are some ESLint rules that will be ignored when running this Docker contai Usually, these rules need an Internet connection and/or to check `node_modules`, and would not run successfully on our Docker container environment. +## Agent Playbook: Updating This Repository End-to-End + +This section is written for an AI coding agent (or a human) tasked with updating this repo — most commonly bumping ESLint itself, one of the ~150 bundled `eslint-plugin-*`/`eslint-config-*`/`@scope/eslint-plugin-*` packages, or a supporting dependency (webpack, typescript, etc.). Follow it top to bottom; it tells you what to change, how to regenerate derived files, how to test locally, and how to interpret CI so you can iterate on failures without guessing. + +### 1. What this repository is + +This is a **Codacy engine**: a Node.js/TypeScript wrapper (built on the `codacy-seed` library, the published package for [codacy-engine-typescript-seed](https://github.com/codacy/codacy-engine-typescript-seed)) that packages [ESLint](https://eslint.org/) plus a very large collection of ESLint plugins and shareable configs as a Docker image Codacy's platform runs against a customer's source code. There is no single "underlying tool version" — this repo bundles ESLint core *and* ~150 separate plugin/config packages side by side, and any of them can be the target of an update. + +The `docs/` directory is machine-consumed configuration, analogous to other Codacy engines' `patterns.json`: + +- `docs/patterns.json` — the full list of ESLint rules ("patterns") across all bundled plugins that Codacy knows about, their parameters/defaults, and which are enabled out of the box. **Generated file, do not hand-edit.** +- `docs/description/description.json` + `docs/description/*.md` — human-readable titles/descriptions per pattern, used in the Codacy UI. **Generated file, do not hand-edit.** +- `docs/multiple-tests/*` — fixtures used by `codacy-plugins-test` (the "multiple tests" mode) to validate the engine actually produces the expected results for real code samples, one directory per scenario/plugin. +- `docs/tool-description.md` — short blurb about the tool, hand-maintained. + +Unlike some Codacy engines (e.g. the Checkstyle wrapper), **there is no external repo clone / tag-scraping step**. The generator (`src/docGenerator.ts`, invoked via `src/docGeneratorMain.ts`) builds the pattern list by directly `require`-ing the ESLint rule modules that are already installed in `node_modules` (see `src/eslintPlugins.ts`, which lists every bundled plugin package name in `packageNames`, and pulls in `@eslint-stylistic/metadata` for stylistic rule metadata). It separately fetches each plugin's rule *documentation* (not its rule logic) as raw Markdown from each plugin's GitHub repo via `axios`, with one `docGenerator.downloadDocs(...)` call per plugin in `src/docGeneratorMain.ts`. This means `npm run build:docs` needs **network access** to GitHub raw content, but does **not** need git or pandoc. + +### 2. Files that encode versions — check all of these on every update + +| File | What it controls | What to check | +|---|---|---| +| `package.json` → `dependencies` | The pinned version of ESLint itself (`"eslint": "^8.57.0"`) and every bundled plugin/config (`eslint-plugin-*`, `eslint-config-*`, `@scope/eslint-plugin-*`, plus supporting libs like `typescript`, `webpack`, `next`, etc.) | Bump the specific package(s) named in the task, e.g. `npm install --legacy-peer-deps --omit=dev eslint-plugin-jsdoc@latest` (see recent commit `c1a4471` "Bump eslint-plugin-jsdoc" for the exact shape: only `package.json`/`package-lock.json` + regenerated `docs/description/*` changed). Respect the `preupgrade` script's excluded packages (`@teambit/react.eslint-config-bit-react`, `eslint` itself pinned to `latest` overall, `eslint-plugin-lodash` pinned to minor) if doing a bulk upgrade. | +| `package-lock.json` | Exact resolved dependency tree | Regenerated automatically by `npm install`; commit it alongside `package.json`. | +| `src/eslintPlugins.ts` → `packageNames` array | Which installed packages' rules get pulled into `docs/patterns.json` | Only touch when **adding/removing** a plugin, not for a plain version bump. | +| `src/docGeneratorMain.ts` | Which plugins get their rule-doc Markdown fetched from GitHub, and from which path/tag-prefix | Only touch when adding a new plugin whose docs should be scraped, or when a plugin's docs moved (a real past fix: `f3b6ac1` "fix: transient version was overriding expected typescript version", `b2677fb` "fix: document generation for some plugins"). | +| `src/blacklist.ts` | Rules disabled because they need internet/`node_modules` access unavailable in the Docker sandbox | Check if a version bump adds/removes rules that need blacklisting. | +| `Dockerfile` → base image (`node:lts-alpine3.21`) | Node runtime the packaged app builds and runs on | Only bump if asked explicitly or if the new package version raises its minimum Node requirement — check `engines.node` (currently `>=18.0.0`) in `package.json` and the target package's own `engines` field. | +| `.circleci/config.yml` → `codacy/base` orb (`12.1.3`) and `codacy/plugins-test` orb (`2.0.11`) | Shared CircleCI steps (checkout, docker build/publish, tagging) and the `codacy-plugins-test` runner | Check the latest published orb versions if asked to bump them; not tied to ESLint plugin bumps. | + +Look at recent bump commits for the shape of a typical diff: `git log --oneline --all | grep -iE "bump|update|upgrade"`, then `git show `. Typical patterns observed in this repo's history: +- A pure dependency bump (e.g. `c1a4471` "Bump eslint-plugin-jsdoc (#4922)") touches only `package.json`/`package-lock.json` plus the regenerated `docs/description/*.md` and `docs/description/description.json` for that plugin's rules. +- Adding a brand-new plugin is a two-commit pattern: first install it (`9fa290b` "feature: Install `eslint-plugin-react-you-might-not-need-an-effect`" — just `package.json`/`package-lock.json`), then run the doc generator in a follow-up commit (`bd4073d` "fix: Run doc generator" — the `docs/description/*` additions). +- Dependabot opens most routine bumps automatically (see `.github/dependabot.yml`) and `.github/workflows/auto-merge.yml` auto-merges dependabot PRs up to `target: major`; a manually-driven bump (like the one this playbook is for) still goes through the same CI gate as those. + +### 3. Step-by-step update procedure + +1. **Install/bump the target package(s)** in `package.json`: + ```bash + npm install --legacy-peer-deps --omit=dev @ + ``` + or, for a broad refresh of everything: `npm run upgrade` (runs `preupgrade` → `npm-check-updates -u` with the exclusions noted above, then installs). +2. **If adding a brand-new plugin/config** (not just bumping an existing one): add it to `packageNames` in `src/eslintPlugins.ts`, and if it has rule docs on GitHub, add a `docGenerator.downloadDocs(...)` call for it in `src/docGeneratorMain.ts` (see the `README.md` "Configuring new plugins" section above for the exact call shape and parameters). +3. **Build the TypeScript sources:** + ```bash + npm run build + ``` +4. **Regenerate the docs** (requires network access to reach `raw.githubusercontent.com` for each plugin's rule Markdown): + ```bash + npm run build:docs + ``` + This rewrites `docs/patterns.json`, `docs/description/description.json`, and `docs/description/*.md`. Review the diff for new/removed/renamed rules and stale `docs/multiple-tests` fixture references. +5. **Run the unit test suite:** + ```bash + npm run lint + npm test + ``` +6. **Build the Docker image** (mirrors CI, which itself runs `npm run build:docs && npm run test` inside the build stage — see `Dockerfile`): + ```bash + npm run build:docker + ``` +7. **Add/update a fixture** in `docs/multiple-tests/` if the bump changes which rules exist or their default output for the plugin in question, so `codacy-plugins-test` exercises the new behavior. +8. **Run `codacy-plugins-test` locally** before pushing — clone https://github.com/codacy/codacy-plugins-test and run its multiple-tests DockerTest command against your locally built image tag (CI runs this as the `plugins_test` job with `run_multiple_tests: true`). +9. **Iterate on failures**, re-running the relevant command after each fix. +10. **Commit** the dependency bump (`package.json`/`package-lock.json`) together with the regenerated `docs/` files in one change. +11. **Push and open a PR.** CI (`.circleci/config.yml`) runs `codacy/checkout_and_version` -> `publish_docker_local` (builds+saves the image) -> `plugins_test` -> `codacy/publish_docker` (default branch only) -> `tag_version`. +12. **Poll the PR's real CI checks until they all pass — local validation is NOT the finish line.** After every push, run `gh pr checks ` and keep re-polling (short sleep while any check is `pending`) until all checks finish. If a check fails, fetch its actual log (CircleCI API/UI for the failing job — don't guess), find the true root cause, fix it, push again (never `--no-verify`, never force-push), and re-poll. Repeat until every check is green. **The CI environment's toolchain can differ from your local one**, so a clean local run does not guarantee CI passes — for example, `npm run build:docs` inside the Docker build stage runs against exactly the `package-lock.json`-resolved tree and Alpine/Node version in `Dockerfile`, which can behave differently from a local `npm install` on a different OS/Node version. Only stop iterating when every check passes, or you hit a genuine product/infra decision that needs a human — in which case explain it in the PR rather than guessing. + +### 4. Common failure modes and fixes + +| Symptom | Likely cause | Fix | +|---|---|---| +| `npm run build:docs` fails to fetch a plugin's rule docs (404/network error) | Plugin moved its docs path, renamed its default branch, or changed its tag-naming scheme since the `downloadDocs(...)` call was written | Update the path/branch/tag-prefix arguments for that plugin in `src/docGeneratorMain.ts` (real past example: `b2677fb` "fix: document generation for some plugins") | +| Wrong/unexpected rule version picked up during doc generation | A transitive dependency shadows the intended plugin version in `node_modules` | Check resolution with `npm ls `; real past example `f3b6ac1` "fix: transient version was overriding expected typescript version" | +| `multiple` DockerTest fails on a specific `docs/multiple-tests/*` fixture | Rule renamed/removed/added or its default output changed upstream between versions | Regenerate docs, update the expected results in that fixture to match the new (verified correct) output | +| A rule needs to be permanently disabled after a bump (needs internet/`node_modules` access unavailable in the sandbox) | New/changed rule requires resources not available inside the Docker container | Add it to `blacklistRegexes` in `src/blacklist.ts` | + +### 5. Definition of done + +- Target package(s) bumped in `package.json`, with `package-lock.json` regenerated and committed. +- `docs/patterns.json` and `docs/description/*` regenerated via `npm run build:docs` and committed, with any stale `docs/multiple-tests` fixtures updated. +- `npm run lint` and `npm test` pass locally. +- Docker image builds successfully via `npm run build:docker` (this also re-runs `build:docs` and `test` inside the build stage). +- `codacy-plugins-test` multiple-tests run passes locally against the freshly built image. +- **After pushing and opening/updating the PR, every CI check on it is green.** Poll `gh pr checks ` and iterate on any failure (fetch the real CI log, fix, push, re-poll) until all pass — a passing local build is not sufficient, because the CI toolchain can differ from your local one (see step 12). + ## What is Codacy [Codacy](https://www.codacy.com/) is an Automated Code Review Tool that monitors your technical debt, helps you improve your code quality, teaches best practices to your developers, and helps you save time in Code Reviews.