diff --git a/.agents/skills/create-map-rule/SKILL.md b/.agents/skills/create-map-rule/SKILL.md index dd12a7e2..901c06bd 100644 --- a/.agents/skills/create-map-rule/SKILL.md +++ b/.agents/skills/create-map-rule/SKILL.md @@ -41,12 +41,12 @@ Current requirements: `wide-event` (40), `audit` (25), `structured-errors` (20), |---|------|--------| | 1 | `packages/cli/src/lib/map/rules/{id}.ts` | Create the rule (one exported const) | | 2 | `packages/cli/src/lib/map/rules/index.ts` | Import + one line in `REGISTRY` | -| 3 | `packages/cli/src/lib/map/types.ts` | Add the id to the `CheckId` union (a type assert in `index.ts` fails the build if the registry and union drift) | +| 3 | `packages/cli/src/lib/map/types.ts` + `packages/evlog/src/shared/define.ts` | Add the id to the `CheckId` union and to `EvlogMapRuleId`, which types `map.rules` in `evlog.config.ts` (type asserts in `index.ts` fail the build if the registry and either union drift) | | 4 | `packages/cli/test/map/rules.test.ts` | Add cases (the file has an ESLint-`RuleTester`-style `Case` harness (`runRuleSet` exercises one rule in isolation)) | | 5 | `apps/docs/content/3.cli/3.rules.md` | Add a row to the Requirements or Opportunities table + a `### {title}` section | | 6 | `apps/docs/content/3.cli/4.scoring.md` | Requirements only: reflect the new weight in the scoring explanation | | 7 | `skills/review-logging-patterns/references/code-review.md` | Add a row to the matching rules table | -| 8 | `.changeset/{id}-map-rule.md` | Changeset for `"@evlog/cli": minor` | +| 8 | `.changeset/{id}-map-rule.md` | Changeset for `"@evlog/cli": minor` and `"evlog": minor` (`map.rules` accepts the new id) | **Important**: Do NOT consider the task complete until all applicable touchpoints have been addressed. @@ -95,10 +95,12 @@ Key rules: - **Weights are a scoring decision**: look at `score.ts` and the existing spread (40 down to 15) and discuss the number in the PR rather than inventing precedent. - Every rule id is also a suppression target (`evlog-map-disable {id}`) and part of the public `evlog.map.json` contract. Renaming later is a breaking change. -## Steps 2 and 3: Registry + CheckId +## Steps 2 and 3: Registry, CheckId and EvlogMapRuleId Add the import and one `REGISTRY` line in `rules/index.ts` (report order matters: requirements before opportunities, heaviest first), and the id to the `CheckId` union in `types.ts`. The `AssertIdsMatch` type in `index.ts` fails the build if you forget either side. +The id also goes in `EvlogMapRuleId` in `packages/evlog/src/shared/define.ts`, the type behind `map.rules` in `evlog.config.ts`. `AssertConfigIdsMatch` in `index.ts` checks it against `CheckId`. The CLI reads that type from the built `evlog` package, so rebuild `evlog` before the CLI typecheck. + ## Step 4: Tests `packages/cli/test/map/rules.test.ts` has a declarative `Case` harness: source code in, expected check results out, with knobs for `kind`, `framework`, `path` (sensitivity), `hasEvlog`, `features`, `pairable`, `dependencies`, `catalogs`, `barrels`. Use `runRuleSet([yourRule], run)` to exercise the rule in isolation. @@ -124,13 +126,14 @@ Read `apps/docs/AGENTS.md` before touching anything under `apps/docs/`. Then in ## Step 8: Changeset -`.changeset/{id}-map-rule.md` with `"@evlog/cli": minor`, written from the user's perspective: what the rule checks, when it fires, whether it moves the score. +`.changeset/{id}-map-rule.md` with `"@evlog/cli": minor` and `"evlog": minor`, written from the user's perspective: what the rule checks, when it fires, whether it moves the score. ## Verification ```bash +pnpm --filter evlog run build # the CLI typechecks against evlog's built types pnpm --filter @evlog/cli run lint -pnpm --filter @evlog/cli run typecheck # catches REGISTRY/CheckId drift +pnpm --filter @evlog/cli run typecheck # catches REGISTRY/CheckId/EvlogMapRuleId drift pnpm --filter @evlog/cli run test ``` diff --git a/.changeset/cli-config.md b/.changeset/cli-config.md new file mode 100644 index 00000000..00be2170 --- /dev/null +++ b/.changeset/cli-config.md @@ -0,0 +1,5 @@ +--- +"@evlog/cli": minor +--- + +`evlog map` and `evlog logs` read `evlog.config.ts` (or `.mts`, `.js`, `.mjs`), the nearest one from the package up to the workspace root, without running it. `map.rules` turns checks off or back on for every entry point, `map.ignore` leaves entry points out by file glob and `--json` reports how many as `ignored`, `map.minScore` and `map.baseline` gate like `--min-score` and `--baseline`, and `logs.dir` and `logs.limit` set the defaults of `--dir` and `--limit`. A flag wins over the config. The config can extend a local file or a published preset, one level deep. `evlog config` prints the settings that apply, grouped into service, sampling, redaction, pipeline and CLI, with the file and line each one comes from and evlog's defaults where the file says nothing, `--json` for the same as JSON, and `evlog doctor` reports a config it cannot read. A misspelt setting, an `extends` written as a path instead of an imported config, a value under `map` or `logs` computed at runtime, or turning off `wide-event` or `context` stops the command with a `cli.CONFIG_*` error. The CLI now needs `evlog` 2.31.0 or later. diff --git a/apps/docs/content/3.cli/0.overview.md b/apps/docs/content/3.cli/0.overview.md index 89326a40..3eca2807 100644 --- a/apps/docs/content/3.cli/0.overview.md +++ b/apps/docs/content/3.cli/0.overview.md @@ -21,7 +21,7 @@ The `evlog` executable ships with the `evlog` package. It runs separately from y Start with [`evlog map`](/cli/map) to check supported entry points for evlog logging patterns and get a static observability score with suggested fixes. It writes `evlog.map.json` unless you pass `--no-write`. The score describes recognized source patterns, not the logs your handlers produce in production. -[`evlog init`](/cli/init) adds evlog configuration to the app. [`evlog logs`](/cli/logs) reads the events the fs drain wrote: the latest requests, the failures, the slowest, or one request in full. [`evlog agents`](/cli/agents) writes logging conventions for the AI agents working in the repository. +[`evlog init`](/cli/init) adds evlog configuration to the app. [`evlog logs`](/cli/logs) reads the events the fs drain wrote: the latest requests, the failures, the slowest, or one request in full. [`evlog agents`](/cli/agents) writes logging conventions for the AI agents working in the repository. [`evlog.config.ts`](/cli/config) holds the settings `evlog map` and `evlog logs` apply, next to the sampling and redaction the app imports. ::warning{icon="i-lucide-flask-conical"} **Early days.** `evlog map` has adapters for Nuxt, Nitro, Next.js App Router, TanStack Start, Hono, Express, and Fastify. Each adapter recognizes specific entry-point shapes. Check the detected framework and entry-point count before interpreting the score. Rules can change between releases, so [pin the version](/cli/ci#pin-the-version) when you gate CI on the number. @@ -98,6 +98,15 @@ The CLI requires Node 22 or later. ::: :::card --- + icon: i-lucide-settings-2 + title: config + to: /cli/config + color: neutral + --- + Show the evlog.config that applies here and where each setting comes from. + ::: + :::card + --- icon: i-lucide-bar-chart-3 title: telemetry to: /cli/telemetry diff --git a/apps/docs/content/3.cli/10.logs.md b/apps/docs/content/3.cli/10.logs.md index 6b0bf7bf..78c4b2cb 100644 --- a/apps/docs/content/3.cli/10.logs.md +++ b/apps/docs/content/3.cli/10.logs.md @@ -65,6 +65,8 @@ Filters compose with any view. A time, level, status or limit that cannot be read stops the command (exit 2) rather than silently widening the query. +`--dir` and `--limit` can be set once for the project in [`evlog.config.ts`](/cli/config), as `logs.dir` and `logs.limit`. The flag wins when both are set. + ```bash [Terminal] evlog logs errors --since 1h evlog logs slow --over 2s --path /api/reports diff --git a/apps/docs/content/3.cli/11.config.md b/apps/docs/content/3.cli/11.config.md new file mode 100644 index 00000000..5889de22 --- /dev/null +++ b/apps/docs/content/3.cli/11.config.md @@ -0,0 +1,238 @@ +--- +title: evlog.config +description: "One evlog.config.ts sets what evlog map checks, where evlog logs reads, and the sampling, redaction and drains your app imports." +navigation: + title: config + icon: i-lucide-settings-2 +links: + - label: Map rules + icon: i-lucide-list-checks + to: /cli/rules + color: neutral + variant: subtle + - label: Sampling + icon: i-lucide-filter + to: /learn/sampling + color: neutral + variant: subtle +--- + +Without a config file, the CI gate is a flag on every `evlog map` run, a check you decided not to care about is disabled file by file, and sampling and redaction live in whichever file calls `initLogger`. `evlog.config.ts` holds all of it, and a preset carries it from one repository to the next. The CLI reads the file without running it, and the app imports it like any other module. + +Here an app builds on a shared preset, turns a check back on, and leaves its dev routes out of the map: + +::code-group +```ts [evlog.config.ts] +import { defineEvlog } from 'evlog' +import { createAxiomDrain } from 'evlog/axiom' +import preset from './evlog.preset' + +export default defineEvlog({ + extends: preset, + service: 'checkout', + drain: createAxiomDrain(), + sampling: { rates: { info: 25 } }, + map: { + rules: { 'audit-coverage': 'on' }, + ignore: ['src/routes/_dev/**'], + }, + logs: { limit: 100 }, +}) +``` + +```ts [evlog.preset.ts] +import { defineEvlog } from 'evlog' + +export default defineEvlog({ + sampling: { rates: { info: 10, debug: 0 } }, + redact: { paths: ['user.password', 'card.number'] }, + map: { rules: { 'error-catalog': 'off', 'audit-coverage': 'off' }, minScore: 70 }, +}) +``` +:: + +`evlog config` prints the merged result grouped by what each setting does, with the line it is written on, and fills in what evlog uses where the file says nothing: + +```bash [Terminal] +evlog config +``` + +```text [Output] +evlog.config.ts · extends ./evlog.preset → evlog.preset.ts + +Service + service checkout evlog.config.ts:7 + environment from NODE_ENV default + +Sampling + trace 0% default + debug 0% evlog.preset.ts:4 + info 25% evlog.config.ts:9 + warn 100% default + error 100% default + fatal always default + +Redaction + redact on evlog.preset.ts:5 + builtins creditCard, email, ipv4, phone, jwt, bearer, iban default + paths user.password evlog.preset.ts:5 + card.number evlog.preset.ts:5 + +Pipeline + drain createAxiomDrain() evlog.config.ts:8 + +CLI · read by evlog map and evlog logs + map.rules.error-catalog 'off' evlog.preset.ts:6 + map.rules.audit-coverage 'on' evlog.config.ts:11 + map.minScore 70 evlog.preset.ts:6 + map.ignore ['src/routes/_dev/**'] evlog.config.ts:12 + logs.limit 100 evlog.config.ts:14 +``` + +Each redaction path keeps the line of the file that adds it, so a path the preset redacts never looks like the app's own. With `routes`, the Service section becomes Services and lists each route's service in the order evlog matches them, the first match winning. A rate that changes nothing gets a warning under its row, such as a `fatal` rate, since fatal events are always kept. + +A value the file computes, like `createAxiomDrain()`, is shown as the code that produces it. `--json` returns the settings written in the files as `cli` and `app` lists of `{ path, value, source }`, one entry per item of `redact.paths`, `redact.patterns` and `sampling.keep`, with a computed value written as `{ "runtime": "createAxiomDrain()" }` and a regular expression as `{ "regexp": "/acct_\\w+/g" }`. + +## Where the CLI finds the file + +The CLI looks for `evlog.config.ts`, `evlog.config.mts`, `evlog.config.js` and `evlog.config.mjs`, in that order, starting in the package it runs on and walking up to the workspace root. The first file found applies on its own. Configs do not cascade, so an app with its own `evlog.config.ts` ignores the one at the root unless it extends it. + +Paths in `map.ignore`, `map.baseline` and `logs.dir` are relative to the package being mapped or read, not to the config file. One config at the root of a monorepo therefore fits every app in it. + +## Gate the map from the config + +`evlog map` reads the `map` section, and a flag passed on the command line wins over the same setting. + +| Setting | Accepts | Flag | What it does | +| --- | --- | --- | --- | +| `map.rules` | `{ [id]: 'on' \| 'off' }` | none | Turns a check off for every entry point, or back on when the preset turned it off | +| `map.ignore` | list of globs | none | Leaves the entry points whose file matches out of the map | +| `map.minScore` | whole number from 0 to 100 | `--min-score` | Exits 1 when the global score is below it | +| `map.baseline` | `true`, a path, or `git:` | `--baseline` | Exits 1 on a regression against the committed map, `true` meaning `evlog.map.json` | + +The ids are the ones on [Rules](/cli/rules). Every check can be turned off except `wide-event` and `context`, because the map sorts entry points into instrumented, partial and dark by them. Leave those entry points out with `map.ignore` instead. + +A check turned off in the config becomes `n/a` on every entry point, with `turned off in evlog.config` as its message in `--json`. The report says what the config changed above the score, and names the setting the gate came from: + +```text [Output] +evlog.config.ts: error-catalog off, 1 entry point ignored +█▀█ ▀▀█ score /100 checkout · Hono +█▀█ ▀█ ▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱▱ 2 entry points scanned +▀▀▀ ▀▀▀ good ▆█ + + GATE score 83 meets map.minScore 70 — exit code 0 +``` + +`evlog map --min-score 95` on the same project gates on 95 and says `--min-score 95`. To turn a check off for one handler rather than the whole project, keep using a [disable comment](/cli/rules#disabling-a-check) next to the code. + +## Read logs from another directory + +`evlog logs` reads the `logs` section, and its flags win the same way. + +| Setting | Accepts | Flag | What it does | +| --- | --- | --- | --- | +| `logs.dir` | a path | `--dir` | Reads this directory instead of `.evlog/logs` | +| `logs.limit` | whole number of 1 or more | `--limit` | Shows at most this many events | + +`--format`, `--verbose`, and `--limit` on `evlog map` stay flags only. They describe one run, not the project. + +## Write values the CLI can read + +The CLI parses `evlog.config.ts` and never runs it, so every value under `map` and `logs` has to be a literal, a `const`, or a value imported from a local file. A call, an environment variable, or a value imported from a package stops the command: + +```text [Output] +logs.limit in evlog.config.ts:14 is computed at runtime +→ Write the value inline, as a const, or import it from a local file +``` + +The rest of the file is for the app and can compute anything: a drain, an `enrich` function, a sampling rate read from `process.env`. The CLI lists those values without evaluating them. + +## Share settings with `extends` + +`extends` takes another config, imported from a local file or from a package. A preset published to npm is an ordinary module whose default export is `defineEvlog({ ... })`, so a team installs it and extends it: + +```ts [evlog.config.ts] +import { defineEvlog } from 'evlog' +import preset from '@acme/evlog-preset' + +export default defineEvlog({ + extends: preset, + service: 'checkout', +}) +``` + +The CLI follows the package's `exports` to the file it ships and reads it the same way, so the preset's `map` and `logs` have to be literals too. + +Settings merge by kind: + +| In the config | Result | +| --- | --- | +| A scalar or a function: `service`, `drain`, `enrich`, `keep` | The config's value replaces the preset's | +| An object: `sampling.rates`, `routes`, `env`, `map.rules` | Merged key by key, the config winning on each key | +| `redact.paths`, `redact.patterns`, `sampling.keep` | The preset's entries, then the config's | +| Any other list: `map.ignore`, `include`, `exclude` | The config's list replaces the preset's | +| `plugins` | Merged by `name`, a config plugin replacing the preset plugin of the same name | +| `redact: false` | Redaction off | +| `redact: true` | The preset's redact settings, unchanged | + +Redaction paths and kept events add up rather than being replaced, so an app that lists its own paths cannot drop the ones the preset redacts. + +A config extends one level only. When `evlog.preset.ts` itself extends a config, extending it fails and names both files: + +```text [Output] +./evlog.preset extends another config, so evlog.config.ts:6 cannot extend it +→ Extend the config it extends directly, or copy the settings you need into one of the two files +``` + +Every setting is then at most one file away from where it applies, and `evlog config` names that file. + +`extends` takes the config itself, the value the app merges at runtime, so a path string is refused rather than resolved: + +```text [Output] +extends in evlog.config.ts:4 is the path './evlog.preset', not a config +→ Import the config from that path as base, then set extends: base +``` + +## Use the config in your app + +Nothing loads `evlog.config.ts` at runtime: the app imports it. `toLoggerConfig` keeps the options `initLogger` takes, `toMiddlewareOptions` keeps the ones a framework middleware takes, and both leave `map` and `logs` out: + +```ts [src/index.ts] +import { Hono } from 'hono' +import { initLogger, toLoggerConfig, toMiddlewareOptions } from 'evlog' +import { evlog, type EvlogVariables } from 'evlog/hono' +import config from '../evlog.config' + +initLogger(toLoggerConfig(config)) + +const app = new Hono() +app.use(evlog(toMiddlewareOptions(config))) +``` + +The Nuxt and Nitro modules do not read `evlog.config.ts`. Their options stay in `nuxt.config.ts` or `nitro.config.ts`, and the file there carries the `map` and `logs` settings the CLI applies. + +## When the config cannot be read + +`evlog map` and `evlog config` exit 1 on a config they cannot read, and `evlog logs` exits 2. [`evlog doctor`](/cli/doctor) reports the same error as a failing `config` check. Each error carries a code from the CLI's catalog: + +| Code | Raised when | +| --- | --- | +| `cli.CONFIG_PARSE_FAILED` | The file has a syntax error | +| `cli.CONFIG_NO_EXPORT` | There is no default export of an object or `defineEvlog({ ... })` | +| `cli.CONFIG_NOT_STATIC` | The default export, `extends`, or a `map` or `logs` value is computed at runtime | +| `cli.CONFIG_INVALID` | A setting is misspelt, has the wrong type, or turns off `wide-event` or `context` | +| `cli.CONFIG_EXTENDS_NOT_FOUND` | The `extends` import does not lead to a file | +| `cli.CONFIG_EXTENDS_DEPTH` | The extended config extends another one | +| `cli.CONFIG_EXTENDS_STRING` | `extends` is a path string instead of an imported config | + +A misspelt key is an error rather than a setting quietly ignored: + +```text [Output] +logs.limt in evlog.config.ts:14 is not a setting; expected dir, limit +→ Use a setting and a value the config reference lists +``` + +## Next + +- [Rules](/cli/rules): the ids `map.rules` takes, and what each check expects +- [CI](/cli/ci): gate a pull request on the score diff --git a/apps/docs/content/3.cli/2.map.md b/apps/docs/content/3.cli/2.map.md index e8b8634b..7285ef39 100644 --- a/apps/docs/content/3.cli/2.map.md +++ b/apps/docs/content/3.cli/2.map.md @@ -296,6 +296,8 @@ export default defineEventHandler(() => ({ ok: true })) The check becomes `n/a` with your reason attached, so it stops costing score, and the report says how many checks the project has disabled, so a high score never hides an app that logs nothing. Full syntax on [Rules](/cli/rules#disabling-a-check). +A check the whole project does without, like `error-catalog` in an app with no error catalog, is turned off once in [`evlog.config.ts`](/cli/config) with `map.rules`. `map.ignore` leaves whole entry points out, by their file. The report says how many it left out, and `--json` carries the count as `ignored`. + ## Flags | Flag | Default | What it does | @@ -313,6 +315,8 @@ The check becomes `n/a` with your reason attached, so it stops costing score, an | `--format ` | human | `github` writes [workflow annotations](/cli/ci#without-the-action) to stdout | | `--limit ` | 10 | Most annotations `--format github` emits; GitHub keeps ten per step | +`--min-score` and `--baseline` can be set once for the project in `evlog.config.ts`, as `map.minScore` and `map.baseline`. The flag wins when both are set. + ## evlog.map.json Every run writes `evlog.map.json` to the project root: the score, the framework, the CLI version and rule-set version that wrote it, and every entry point with its checks, its suggestions, its sensitivity, and its own score. It is the same data `--json` prints. diff --git a/apps/docs/content/3.cli/6.doctor.md b/apps/docs/content/3.cli/6.doctor.md index cb473ba1..a0b9c800 100644 --- a/apps/docs/content/3.cli/6.doctor.md +++ b/apps/docs/content/3.cli/6.doctor.md @@ -41,6 +41,7 @@ EVLOG | `project` | A `package.json` was resolved | None found above the working directory | | `stack` | A framework was detected and evlog is installed | A framework was detected but evlog is not wired to it | | `evlog` | `evlog` resolves from `node_modules` | Declared in `package.json` but not installed, or missing entirely | +| `config` | [`evlog.config.ts`](/cli/config) was read, and the config it extends was found | The file cannot be read: a syntax error, a misspelt setting, or a CLI setting computed at runtime (fail). The check is omitted when there is no config | | `logs` | A local drain exists, or the fs drain is wired (the directory appears on first write) | Never — the check is omitted when no local drain is configured | In a workspace, the header names the workspace kind and `project` shows which package was resolved, the fastest way to notice you are diagnosing the repo root instead of your app. diff --git a/apps/telemetry/server/utils/allowed-tools.ts b/apps/telemetry/server/utils/allowed-tools.ts index d57355c2..8ef08c5e 100644 --- a/apps/telemetry/server/utils/allowed-tools.ts +++ b/apps/telemetry/server/utils/allowed-tools.ts @@ -43,6 +43,8 @@ export const DEFAULT_ALLOWED_CUSTOM_KEYS: Record = { 'workspace', 'doctorEvlogFound', 'doctorLogsSink', + 'doctorConfig', + 'doctorConfigFailed', 'doctorStackDetected', // evlog init — which options were picked 'initFramework', @@ -128,6 +130,9 @@ export const DEFAULT_ALLOWED_CUSTOM_KEYS: Record = { 'mapProjectSuggestions', 'mapGate', 'mapGateFailed', + 'mapConfig', + 'mapRulesOff', + 'mapIgnored', 'mapMinScore', 'mapBaselineDelta', 'mapBaselineRegressions', diff --git a/packages/cli/package.json b/packages/cli/package.json index 66efcb2f..5ea19be4 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -61,7 +61,7 @@ "oxc-parser": "^0.152.0" }, "peerDependencies": { - "evlog": ">=2.29.0" + "evlog": ">=2.31.0" }, "devDependencies": { "evlog": "workspace:*", diff --git a/packages/cli/src/commands/config.ts b/packages/cli/src/commands/config.ts new file mode 100644 index 00000000..e8f9224f --- /dev/null +++ b/packages/cli/src/commands/config.ts @@ -0,0 +1,335 @@ +import type { CliContext } from '../core/context' +import { createStyle } from '../core/output' +import { defineEvlogCommand, failWith } from '../lib/command' +import { formatPath, isAppendedList, loadCliConfig, RuntimeValue } from '../lib/config' +import { isPlainObject } from '../lib/config/read' +import type { CliDebug } from '../lib/debug' +import { createNoopCliDebug } from '../lib/debug' +import { prettyPath, resolveProject } from '../lib/project' + +type ConfigPath = readonly (string | number)[] +type Paint = ReturnType['paint'] + +/** One setting of the resolved config. */ +export interface ConfigEntry { + /** Keys down to the setting, `['sampling', 'rates', 'info']`. Each item of a list `extends` appends to is its own entry. */ + path: ConfigPath + value: unknown + /** Where the setting is written, as `file:line`, or `null` when it comes from a file that only contributes values. */ + source: string | null +} + +/** Typed result of `evlog config`, rendered by {@link formatConfigReport}. */ +export interface ConfigResult { + /** Where the lookup started and stopped, so a missing config says where it was looked for. */ + searched: { from: string, to: string } + /** Relative to the working directory, or `null` when no config applies. */ + file: string | null + extends: { specifier: string | null, file: string } | null + /** Settings the CLI applies itself (`map`, `logs`). */ + cli: ConfigEntry[] + /** Settings the app reads where it imports the config. */ + app: ConfigEntry[] + /** Where each key is written, by {@link formatPath}, for rows that stand for a whole object such as `redact`. */ + sources: ReadonlyMap +} + +/** Sections the CLI reads; everything else in the file is for the app. */ +const CLI_SECTIONS: ReadonlySet = new Set(['map', 'logs']) + +function sourceOf(path: ConfigPath, sources: ReadonlyMap): string | null { + for (let end = path.length; end > 0; end--) { + const source = sources.get(formatPath(path.slice(0, end))) + if (source) return source + } + return null +} + +function entriesOf(value: unknown, path: ConfigPath, sources: ReadonlyMap): ConfigEntry[] { + if (isPlainObject(value) && Object.keys(value).length > 0) { + return Object.entries(value).flatMap(([key, child]) => entriesOf(child, [...path, key], sources)) + } + if (Array.isArray(value) && value.length > 0 && isAppendedList(path)) { + return value.map((item, index) => ({ path: [...path, index], value: item, source: sourceOf([...path, index], sources) })) + } + return [{ path, value, source: sourceOf(path, sources) }] +} + +/** + * Read the `evlog.config` that applies to `ctx.cwd`, merged across `extends`, + * with where each setting is written. Pure with respect to the context. + * + * @throws a `cli.CONFIG_*` error when the file cannot be read statically. + */ +export async function runConfig(ctx: CliContext, log: CliDebug = createNoopCliDebug()): Promise { + const project = await log.step('resolveProject', () => resolveProject(ctx.cwd), p => ({ project: { root: p.root, packageDir: p.packageDir } })) + const searched = { from: prettyPath(ctx.cwd, project.packageDir), to: prettyPath(ctx.cwd, project.root) } + const config = await log.step('loadConfig', () => loadCliConfig(project), c => ({ config: c && { file: c.file, extends: c.extends?.file ?? null } })) + if (!config) return { searched, file: null, extends: null, cli: [], app: [], sources: new Map() } + + const entries = Object.entries(config.resolved).flatMap(([key, value]) => entriesOf(value, [key], config.sources)) + return { + searched, + file: prettyPath(ctx.cwd, config.file), + extends: config.extends && { specifier: config.extends.specifier, file: prettyPath(ctx.cwd, config.extends.file) }, + cli: entries.filter(entry => CLI_SECTIONS.has(entry.path[0])), + app: entries.filter(entry => !CLI_SECTIONS.has(entry.path[0])), + sources: config.sources, + } +} + +/** A value as it reads in source: strings quoted, regexps as literals, runtime values by what they call. */ +export function formatValue(value: unknown): string { + if (value instanceof RuntimeValue) return value.label + if (typeof value === 'string') return `'${value}'` + if (value instanceof RegExp) return String(value) + if (Array.isArray(value)) return `[${value.map(formatValue).join(', ')}]` + if (isPlainObject(value)) { + const fields = Object.entries(value).map(([key, field]) => `${key}: ${formatValue(field)}`) + return fields.length > 0 ? `{ ${fields.join(', ')} }` : '{}' + } + return String(value) +} + +/** JSON can't hold a RegExp; written as `{ regexp }`, the way runtime values are written as `{ runtime }`. */ +function jsonValue(value: unknown): unknown { + if (value instanceof RegExp) return { regexp: String(value) } + if (Array.isArray(value)) return value.map(jsonValue) + if (isPlainObject(value)) return Object.fromEntries(Object.entries(value).map(([key, field]) => [key, jsonValue(field)])) + return value +} + +/** The `--json` payload: entries keyed by path, values JSON-safe. */ +export function configJson(result: ConfigResult): Record { + const settings = (entries: ConfigEntry[]) => entries.map(entry => ({ path: formatPath(entry.path), value: jsonValue(entry.value), source: entry.source })) + return { file: result.file, extends: result.extends, cli: settings(result.cli), app: settings(result.app) } +} + +interface Row { + label: string + value: string + source: string | null + /** Computed when the app starts, painted apart from literals. */ + runtime?: boolean + /** Not a setting, a note about one (`none in this file`). */ + muted?: boolean + /** What the setting covers, under the row. */ + note?: string + warning?: string +} + +interface Section { + title: string + note?: string + rows: Row[] +} + +const DEFAULT = 'default' +const LEVELS = ['trace', 'debug', 'info', 'warn', 'error', 'fatal'] as const +/** evlog's builtin redaction patterns, by name. */ +const BUILTIN_PATTERNS = ['creditCard', 'email', 'ipv4', 'phone', 'jwt', 'bearer', 'iban'] +const PIPELINE = ['drain', 'enrich', 'keep', 'plugins', 'waitUntil'] +const KEEP_FIELDS: ReadonlySet = new Set(['status', 'duration', 'path']) +/** Keys the app sections above lay out; anything else lands in Other. */ +const APP_SECTIONS: ReadonlySet = new Set(['service', 'environment', 'env', 'routes', 'sampling', 'minLevel', 'redact', ...PIPELINE]) + +function startsWith(path: ConfigPath, prefix: ConfigPath): boolean { + return prefix.length <= path.length && prefix.every((key, index) => path[index] === key) +} + +function under(entries: ConfigEntry[], prefix: ConfigPath): ConfigEntry[] { + return entries.filter(entry => startsWith(entry.path, prefix)) +} + +function exact(entries: ConfigEntry[], path: ConfigPath): ConfigEntry | undefined { + return entries.find(entry => entry.path.length === path.length && startsWith(entry.path, path)) +} + +function settingRow(entry: ConfigEntry, label = formatPath(entry.path)): Row { + return { label, value: formatValue(entry.value), source: entry.source, runtime: entry.value instanceof RuntimeValue } +} + +/** A setting that reads as text, a service name or a path, so a string goes unquoted. */ +function textRow(entry: ConfigEntry, label: string): Row { + return typeof entry.value === 'string' ? { label, value: entry.value, source: entry.source } : settingRow(entry, label) +} + +/** Rows for a list, the label on the first item only. */ +function listRows(entries: ConfigEntry[], label: string, describe: (value: unknown) => string, then = ''): Row[] { + return entries.map((entry, index) => ({ + label: index === 0 ? label : then, + value: describe(entry.value), + source: entry.source, + runtime: entry.value instanceof RuntimeValue, + })) +} + +/** The rows for every entry of `scope` a section did not lay out itself. */ +function restRows(scope: ConfigEntry[], used: ReadonlySet): Row[] { + return scope.filter(entry => !used.has(entry)).map(entry => settingRow(entry)) +} + +function serviceSection(app: ConfigEntry[]): Section { + const routes = under(app, ['routes']) + const rows = routes.map(entry => entry.path.length === 3 && entry.path[2] === 'service' ? textRow(entry, String(entry.path[1])) : settingRow(entry)) + const service = exact(app, ['service']) ?? exact(app, ['env', 'service']) + const environment = exact(app, ['environment']) ?? exact(app, ['env', 'environment']) + const fallback = routes.length > 0 ? 'any other route' : 'service' + rows.push( + service ? textRow(service, fallback) : { label: fallback, value: 'from SERVICE_NAME, else app', source: DEFAULT }, + environment ? textRow(environment, 'environment') : { label: 'environment', value: 'from NODE_ENV', source: DEFAULT }, + ...restRows([...under(app, ['service']), ...under(app, ['environment']), ...under(app, ['env'])], new Set([service, environment])), + ) + return routes.length > 0 ? { title: 'Services', note: 'first matching route wins', rows } : { title: 'Service', rows } +} + +function errorWarning(percent: number): string | undefined { + if (percent <= 0) return 'every error is dropped' + if (percent < 100) return `${100 - percent}% of errors are dropped` + return undefined +} + +function describeKeep(value: unknown): string { + if (!isPlainObject(value) || Object.keys(value).length === 0 || !Object.keys(value).every(key => KEEP_FIELDS.has(key))) return formatValue(value) + const parts: string[] = [] + if (value.status !== undefined) parts.push(`status ≥ ${formatValue(value.status)}`) + if (value.duration !== undefined) parts.push(`duration ≥ ${formatValue(value.duration)}ms`) + if (value.path !== undefined) parts.push(`path ${typeof value.path === 'string' ? value.path : formatValue(value.path)}`) + return parts.join(' or ') +} + +function samplingSection(app: ConfigEntry[]): Section { + const used = new Set() + const minLevel = exact(app, ['minLevel']) + const rows: Row[] = [] + + for (const level of LEVELS) { + const rate = exact(app, ['sampling', 'rates', level]) + used.add(rate) + if (level === 'fatal') { + rows.push({ label: level, value: 'always', source: DEFAULT, warning: rate && `rates.fatal at ${rate.source ?? 'the config'} is ignored, fatal events are always kept` }) + } else if (!rate) { + rows.push({ label: level, value: level === 'trace' ? '0%' : '100%', source: DEFAULT }) + } else if (typeof rate.value === 'number') { + rows.push({ label: level, value: `${rate.value}%`, source: rate.source, warning: level === 'error' ? errorWarning(rate.value) : undefined }) + } else { + rows.push(settingRow(rate, level)) + } + } + + if (minLevel) rows.push({ ...textRow(minLevel, 'minLevel'), note: 'applies to the global log API only, not request wide events' }) + used.add(minLevel) + const keep = under(app, ['sampling', 'keep']) + for (const entry of keep) used.add(entry) + rows.push(...listRows(keep, 'keep if', describeKeep, 'or'), ...restRows(under(app, ['sampling']), used)) + return { title: 'Sampling', rows } +} + +function redactionSection(app: ConfigEntry[], sources: ReadonlyMap): Section { + const scope = under(app, ['redact']) + const root = exact(app, ['redact']) + const all = BUILTIN_PATTERNS.join(', ') + if (scope.length === 0) { + return { title: 'Redaction', rows: [ + { label: 'redact', value: 'on in production, off in dev', source: DEFAULT }, + { label: 'builtins', value: all, source: DEFAULT }, + ] } + } + if (root && !isPlainObject(root.value)) { + if (root.value === false) return { title: 'Redaction', rows: [{ label: 'redact', value: 'off', source: root.source }] } + if (root.value === true) return { title: 'Redaction', rows: [{ label: 'redact', value: 'on', source: root.source }, { label: 'builtins', value: all, source: root.source }] } + return { title: 'Redaction', rows: [settingRow(root, 'redact')] } + } + + const source = sourceOf(['redact'], sources) + const builtins = exact(app, ['redact', 'builtins']) + const paths = under(app, ['redact', 'paths']) + const patterns = under(app, ['redact', 'patterns']) + const builtinsRow: Row = !builtins || builtins.value === true + ? { label: 'builtins', value: all, source: builtins?.source ?? DEFAULT } + : builtins.value === false + ? { label: 'builtins', value: 'none', source: builtins.source } + : Array.isArray(builtins.value) && builtins.value.every(name => typeof name === 'string') + ? { label: 'builtins', value: builtins.value.join(', '), source: builtins.source } + : settingRow(builtins, 'builtins') + return { title: 'Redaction', rows: [ + { label: 'redact', value: 'on', source }, + builtinsRow, + ...listRows(paths, 'paths', value => typeof value === 'string' ? value : formatValue(value)), + ...listRows(patterns, 'patterns', formatValue), + ...restRows(scope, new Set([root, builtins, ...paths, ...patterns])), + ] } +} + +function pipelineSection(app: ConfigEntry[]): Section { + const rows = PIPELINE.flatMap(key => under(app, [key]).flatMap((entry): Row[] => Array.isArray(entry.value) && entry.value.length > 0 + ? entry.value.map((item, index) => ({ label: index === 0 ? formatPath(entry.path) : '', value: formatValue(item), source: entry.source, runtime: item instanceof RuntimeValue })) + : [settingRow(entry)])) + if (under(app, ['drain']).length === 0) rows.unshift({ label: 'drain', value: 'none in this file', source: null, muted: true }) + return { title: 'Pipeline', rows } +} + +function renderSection(section: Section, paint: Paint): string[] { + const labelWidth = Math.max(...section.rows.map(row => row.label.length)) + const valueWidth = Math.max(...section.rows.map(row => row.value.length)) + const lines = [paint('bold', section.title) + (section.note ? paint('dim', ` · ${section.note}`) : '')] + for (const row of section.rows) { + const value = row.runtime ? paint('magenta', row.value) : row.muted ? paint('dim', row.value) : row.value + const source = row.source ? `${' '.repeat(valueWidth - row.value.length)} ${paint('dim', row.source)}` : '' + lines.push(` ${paint('cyan', row.label.padEnd(labelWidth))} ${value}${source}`) + if (row.warning) lines.push(` ${' '.repeat(labelWidth)} ${paint('yellow', `⚠ ${row.warning}`)}`) + if (row.note) lines.push(` ${' '.repeat(labelWidth)} ${paint('dim', row.note)}`) + } + return lines +} + +/** + * The config laid out by what it does: services, sampling per level, + * redaction, the pipeline, then the CLI's own settings. Each row says where + * it is written, or `default` for what evlog does when the file is silent. + */ +export function formatConfigReport(ctx: CliContext, result: ConfigResult): string { + const { paint } = createStyle(ctx) + if (!result.file) { + const range = result.searched.from === result.searched.to ? result.searched.from : `${result.searched.from} up to ${result.searched.to}` + return [ + `No evlog.config from ${range}.`, + paint('dim', '→ add an evlog.config.ts exporting defineEvlog({ … }): https://evlog.dev/cli/config'), + '', + ].join('\n') + } + + const parent = result.extends && (result.extends.specifier ? `${result.extends.specifier} → ${result.extends.file}` : result.extends.file) + const lines = [result.file + (parent ? paint('dim', ` · extends ${parent}`) : '')] + const sections: Section[] = [] + if (result.app.length > 0) { + sections.push(serviceSection(result.app), samplingSection(result.app), redactionSection(result.app, result.sources), pipelineSection(result.app)) + } + const other = result.app.filter(entry => !APP_SECTIONS.has(entry.path[0])) + if (other.length > 0) sections.push({ title: 'Other', rows: other.map(entry => settingRow(entry)) }) + if (result.cli.length > 0) sections.push({ title: 'CLI', note: 'read by evlog map and evlog logs', rows: result.cli.map(entry => settingRow(entry)) }) + + if (sections.length === 0) lines.push('', paint('dim', 'The config sets nothing.')) + for (const section of sections) lines.push('', ...renderSection(section, paint)) + lines.push('') + return lines.join('\n') +} + +/** + * `evlog config`: the `evlog.config` that applies here, merged across + * `extends`, and where each setting comes from. + * Logic lives in {@link runConfig}; this file owns the citty surface. + */ +export default defineEvlogCommand('config', { + meta: { name: 'config' }, + async run({ args, cli, log, ui }) { + let result: ConfigResult + try { + result = await runConfig(cli, log) + } catch (error) { + failWith(error, { args, log, ui }) + return + } + ui.done({ jsonMode: args.json, json: configJson(result), human: formatConfigReport(cli, result) }) + }, +}) diff --git a/packages/cli/src/commands/doctor.ts b/packages/cli/src/commands/doctor.ts index bd71fa95..f3ccb292 100644 --- a/packages/cli/src/commands/doctor.ts +++ b/packages/cli/src/commands/doctor.ts @@ -1,4 +1,5 @@ import { telemetry } from '@evlog/telemetry' +import { EvlogError } from 'evlog' import type { CliContext } from '../core/context' import { DOCS_LABEL, @@ -10,6 +11,7 @@ import { } from '../core/output' import type { Check, CheckSummary } from '../core/output' import { defineEvlogCommand } from '../lib/command' +import { loadCliConfig } from '../lib/config' import type { CatalogFindingSource, CliDebug } from '../lib/debug' import { createNoopCliDebug } from '../lib/debug' import { cliErrors } from '../lib/errors' @@ -158,6 +160,26 @@ async function checkLogs(project: ProjectInfo, env: Record>, @@ -231,6 +253,12 @@ export async function runDoctor( s => ({ stack: s }), ) + const config = checkConfig(project) + if (config.error) { + const { code, why, fix, link } = config.error + log.finding({ code: code ?? 'cli.CONFIG_INVALID', why, fix, link }, { id: 'config', status: 'fail' }) + } + const checks = await log.step('checks', async () => { const environment: Check[] = [ checkNode(ctx), @@ -243,6 +271,7 @@ export async function runDoctor( return [ ...environment, checkEvlog(project, resolved), + ...(config.check ? [config.check] : []), ...(logsCheck ? [logsCheck] : []), ] }) @@ -254,7 +283,7 @@ export async function runDoctor( }, { title: 'EVLOG', - checks: checks.filter(c => c.id === 'evlog' || c.id === 'logs'), + checks: checks.filter(c => c.id === 'evlog' || c.id === 'config' || c.id === 'logs'), }, ] @@ -327,6 +356,8 @@ function doctorTelemetryFields(result: DoctorResult): Record import('./doctor'), ), + config: lazyCommand( + { name: 'config', description: 'Show the evlog.config that applies here and where each setting comes from' }, + () => import('./config'), + ), logs: lazyCommand( { name: 'logs', description: 'Read the wide events your app wrote to .evlog/logs' }, () => import('./logs'), diff --git a/packages/cli/src/commands/logs.ts b/packages/cli/src/commands/logs.ts index 71dc15c4..dde66cee 100644 --- a/packages/cli/src/commands/logs.ts +++ b/packages/cli/src/commands/logs.ts @@ -7,6 +7,7 @@ import { readFsLogs, tailFsLogs } from 'evlog/fs' import type { CliContext } from '../core/context' import { createStyle, EXIT_FAIL, EXIT_USAGE } from '../core/output' import { defineEvlogCommand } from '../lib/command' +import { loadCliConfig } from '../lib/config' import { cliErrors } from '../lib/errors' import { buildQuery, computeStats, select } from '../lib/logs/query' import type { LogsArgs, LogsQuery } from '../lib/logs/query' @@ -102,7 +103,11 @@ const timeOf = (event: WideEvent): number => Date.parse(event.timestamp) || 0 /** Read the events the query asks for. Pure with respect to the context: nothing is written. */ export async function runLogs(ctx: CliContext, args: LogsArgs, options: RunLogsOptions = {}): Promise { - const query = buildQuery(args, options.now) + /* `logs.dir` and `logs.limit` stand in for the flags, so a project that + writes somewhere unusual is read without passing `--dir` every time. */ + const config = loadCliConfig(await resolveProject(ctx.cwd)) + const query = buildQuery(args, options.now, config?.logs.limit) + const dir = options.dir ?? config?.logs.dir const fetchFn = options.fetchFn ?? fetch const inRange = (event: WideEvent): boolean => { const at = timeOf(event) @@ -118,7 +123,7 @@ export async function runLogs(ctx: CliContext, args: LogsArgs, options: RunLogsO all = (await fetchEvents(options.url, fetchFn, options.signal)).filter(inRange) sources = [options.url] } else { - const found = await resolveLogsSources(ctx, options.dir) + const found = await resolveLogsSources(ctx, dir) if (!options.follow && !found.some(source => existsSync(source.dir))) throw cliErrors.LOGS_NO_SINK({ cwd: ctx.cwd }) all = [] for (const source of found) { @@ -137,7 +142,7 @@ export async function runLogs(ctx: CliContext, args: LogsArgs, options: RunLogsO same path as the ones still to come. */ for (const event of result.events) options.onEvent?.(event) if (options.url) await followUrl(options.url, all, inRange, options) - else await followDirs(await resolveLogsSources(ctx, options.dir), query, options) + else await followDirs(await resolveLogsSources(ctx, dir), query, options) } return result } diff --git a/packages/cli/src/commands/map.ts b/packages/cli/src/commands/map.ts index bee42e1a..c9c47936 100644 --- a/packages/cli/src/commands/map.ts +++ b/packages/cli/src/commands/map.ts @@ -1,12 +1,14 @@ import { EvlogError } from 'evlog' import type { CliContext } from '../core/context' -import { EXIT_FAIL, EXIT_USAGE } from '../core/output' +import { createStyle, EXIT_FAIL, EXIT_USAGE } from '../core/output' import { defineEvlogCommand } from '../lib/command' +import { loadCliConfig } from '../lib/config' +import type { CliConfig } from '../lib/config' import { FRAMEWORK_IDS, isFramework } from '../lib/frameworks' import type { CliDebug } from '../lib/debug' import { createNoopCliDebug } from '../lib/debug' import { cliErrors } from '../lib/errors' -import { resolveEvlog, resolveProject } from '../lib/project' +import { prettyPath, resolveEvlog, resolveProject } from '../lib/project' import type { ProjectInfo } from '../lib/project' import { checkBaselineVersion, compareToBaseline, hasRegressed, loadBaseline } from '../lib/map/baseline' import type { BaselineComparison } from '../lib/map/baseline' @@ -44,6 +46,8 @@ export interface MapResult { baseline: BaselineComparison | null /** Baseline problems that do not stop the run — a map that predates version reporting. */ baselineWarnings: string[] + /** The `evlog.config` the run applied, if any. */ + config: CliConfig | null } /** @@ -83,21 +87,30 @@ export async function runMap( r => ({ hasEvlog: !!r.install }), ) + const config = await log.step( + 'loadConfig', + () => loadCliConfig(project), + c => ({ config: c && { file: prettyPath(ctx.cwd, c.file), rulesOff: [...c.map.off], ignore: c.map.ignore.length } }), + ) + const scanCtx: ScanContext = { projectRoot: project.packageDir, framework, projectName: project.packageName ?? 'unknown', hasEvlog: !!resolved.install, verbose: options.verbose ?? false, + rulesOff: config?.map.off, + ignore: config?.map.ignore, } /* Read before the scan writes: `writeMapFile` overwrites `evlog.map.json` in place, so loading the baseline afterwards would compare this run against itself and never report a regression. */ - const baselineMap = options.baseline + const baselineSpec = options.baseline ?? config?.map.baseline + const baselineMap = baselineSpec ? await log.step( 'loadBaseline', - () => loadBaseline(project.packageDir, typeof options.baseline === 'string' ? options.baseline : undefined), + () => loadBaseline(project.packageDir, typeof baselineSpec === 'string' ? baselineSpec : undefined), r => ({ baselineSource: r.source.label, baselineScore: r.map.score }), ) : null @@ -144,9 +157,22 @@ export async function runMap( mapPath, baseline, baselineWarnings, + config, } } +/** What the config changed about this run, so a score is never read without it. */ +function formatConfigNote(ctx: CliContext, result: MapResult): string | null { + const { config } = result + if (!config) return null + const changes = [ + config.map.off.size > 0 ? `${[...config.map.off].join(', ')} off` : null, + result.scan.ignored > 0 ? `${result.scan.ignored} entry point${result.scan.ignored === 1 ? '' : 's'} ignored` : null, + ].filter(change => change !== null) + if (changes.length === 0) return null + return createStyle(ctx).paint('dim', `${prettyPath(ctx.cwd, config.file)}: ${changes.join(', ')}`) +} + /** * Pick the view for the flags that were passed. * @@ -158,7 +184,7 @@ export async function runMap( export function formatMapReport( ctx: CliContext, result: MapResult, - options: { all?: boolean, entry?: string, minScore?: number } = {}, + options: { all?: boolean, entry?: string, minScore?: number, minScoreFrom?: string } = {}, ): string { const sections: string[] = [] @@ -169,6 +195,8 @@ export function formatMapReport( if (warnings.length > 0) { sections.push(formatMapWarnings(ctx, warnings)) } + const note = formatConfigNote(ctx, result) + if (note) sections.push(note) if (options.entry) { const route = findEntryPoint(result.scan, options.entry) @@ -186,7 +214,7 @@ export function formatMapReport( } if (options.minScore !== undefined) { - sections.push(formatGate(ctx, result.scan, options.minScore)) + sections.push(formatGate(ctx, result.scan, options.minScore, options.minScoreFrom)) } return sections.join('\n') @@ -285,6 +313,7 @@ export default defineEvlogCommand('map', { let result: MapResult let threshold: number | undefined + let thresholdFrom = '--min-score' let framework: Framework | undefined let format: MapFormat = 'human' let limit = DEFAULT_LIMIT @@ -294,7 +323,7 @@ export default defineEvlogCommand('map', { and writes evlog.map.json before admitting it cannot gate on it. */ format = parseFormatArg(args.format, args.json) limit = parseLimitArg(args.limit) - threshold = parseMinScoreArg(args.minScore) + const minScoreFlag = parseMinScoreArg(args.minScore) framework = parseFrameworkArg(args.framework) result = await runMap(cli, log, { framework, @@ -303,6 +332,8 @@ export default defineEvlogCommand('map', { baseline: parseBaselineArg(args.baseline), baselineLabel: typeof args.baselineLabel === 'string' && args.baselineLabel.length > 0 ? args.baselineLabel : undefined, }) + threshold = minScoreFlag ?? result.config?.map.minScore + if (minScoreFlag === undefined && threshold !== undefined) thresholdFrom = 'map.minScore' } catch (error) { if (error instanceof EvlogError) { log.finding({ code: error.code ?? 'cli.MAP_FAILED', why: error.why, fix: error.fix, link: error.link }, { status: 'fail' }) @@ -328,14 +359,15 @@ export default defineEvlogCommand('map', { baseline: result.baseline, view, wrote: result.mapPath !== null, + config: result.config && { rulesOff: result.config.map.off.size }, }) - const human = formatMapReport(cli, result, { all: args.all, entry, minScore: threshold }) + const human = formatMapReport(cli, result, { all: args.all, entry, minScore: threshold, minScoreFrom: thresholdFrom }) if (format === 'github') { /* Annotations are the stdout contract; the report still goes to stderr so the job log reads the same as a local run. */ const location = { projectRoot: result.projectRoot, workspace: cli.env.GITHUB_WORKSPACE } - ui.stdout(formatGithubAnnotations(result.scan, result.baseline, location, { minScore: threshold, limit })) + ui.stdout(formatGithubAnnotations(result.scan, result.baseline, location, { minScore: threshold, minScoreFrom: thresholdFrom, limit })) ui.human(human) } else { ui.done({ @@ -344,6 +376,7 @@ export default defineEvlogCommand('map', { map: result.scan.map, grade: result.scan.grade, summary: result.scan.summary, + ignored: result.scan.ignored, mapPath: result.mapPath, ...(result.baseline ? { baseline: result.baseline } : {}), }, diff --git a/packages/cli/src/lib/config/index.ts b/packages/cli/src/lib/config/index.ts new file mode 100644 index 00000000..9b3493e2 --- /dev/null +++ b/packages/cli/src/lib/config/index.ts @@ -0,0 +1,206 @@ +import { existsSync } from 'node:fs' +import { dirname, join, resolve } from 'node:path' +import { mergeEvlogConfig } from 'evlog' +import type { EvlogConfig } from 'evlog' +import { cliErrors } from '../errors' +import { RULES } from '../map/rules' +import type { CheckId } from '../map/types' +import { prettyPath } from '../project' +import type { ProjectInfo } from '../project' +import type { ConfigDocument } from './read' +import { formatPath, isPlainObject, readConfig, RuntimeValue } from './read' + +export { formatPath, RuntimeValue } from './read' + +/** Names `evlog.config` is looked up under, in order, in each directory. */ +export const CONFIG_FILES = ['evlog.config.ts', 'evlog.config.mts', 'evlog.config.js', 'evlog.config.mjs'] as const + +/** `map` settings once read and checked. */ +export interface MapSettings { + /** Checks turned off for every entry point. */ + off: ReadonlySet + /** Entry points left out, as globs relative to the package directory. */ + ignore: readonly string[] + minScore?: number + baseline?: true | string +} + +/** `logs` settings once read and checked. */ +export interface LogsSettings { + /** Absolute. */ + dir?: string + limit?: number +} + +/** The `evlog.config` that applies to a project, as the CLI reads it. */ +export interface CliConfig { + file: string + /** The config `file` extends: the import specifier (`null` for one in the same file) and the file it resolved to. */ + extends: { specifier: string | null, file: string } | null + /** Every setting, merged across `extends`. Values computed at runtime are kept as {@link RuntimeValue}. */ + resolved: Record + /** + * Where each setting is written, as `file:line`, by {@link formatPath}. An + * item of a list `extends` appends to has its own entry, `redact.paths[0]`. + */ + sources: ReadonlyMap + map: MapSettings + logs: LogsSettings +} + +const MAP_KEYS = ['rules', 'ignore', 'minScore', 'baseline'] +/** The map sorts entry points into instrumented, partial and dark by these, so neither can be turned off. */ +const CORE_RULES: ReadonlySet = new Set(['wide-event', 'context']) +const LOGS_KEYS = ['dir', 'limit'] + +/** Lists `mergeEvlogConfig` appends the child's items to instead of replacing. */ +const APPENDED_LISTS: readonly (readonly string[])[] = [['redact', 'paths'], ['redact', 'patterns'], ['sampling', 'keep']] + +/** True for a list whose items can come from both files of an `extends`. */ +export function isAppendedList(path: readonly (string | number)[]): boolean { + return APPENDED_LISTS.some(list => list.length === path.length && list.every((key, index) => key === path[index])) +} + +function valueAt(value: unknown, path: readonly string[]): unknown { + let at = value + for (const key of path) at = isPlainObject(at) ? at[key] : undefined + return at +} + +/** + * The nearest `evlog.config` from the package directory up to the workspace + * root. The first one found applies on its own: configs do not cascade. + */ +export function findConfigFile(project: ProjectInfo): string | null { + let dir = project.packageDir + for (;;) { + for (const name of CONFIG_FILES) { + const file = join(dir, name) + if (existsSync(file)) return file + } + const parent = dirname(dir) + if (dir === project.root || parent === dir) return null + dir = parent + } +} + +type At = (path: string) => string + +function staticValue(value: unknown, key: string, at: At): unknown { + if (value instanceof RuntimeValue) throw cliErrors.CONFIG_NOT_STATIC({ key, at: at(key) }) + return value +} + +function section(value: unknown, path: readonly string[], allowed: readonly string[], at: At): Record { + const key = formatPath(path) + if (staticValue(value, key, at) === undefined) return {} + if (!isPlainObject(value)) throw cliErrors.CONFIG_INVALID({ key, at: at(key), problem: 'must be an object' }) + for (const name of Object.keys(value)) { + if (allowed.includes(name)) continue + const child = formatPath([...path, name]) + throw cliErrors.CONFIG_INVALID({ key: child, at: at(child), problem: `is not a setting; expected ${allowed.join(', ')}` }) + } + return value +} + +function wholeNumber(value: unknown, key: string, at: At, { min, max = Number.POSITIVE_INFINITY }: { min: number, max?: number }): number | undefined { + if (staticValue(value, key, at) === undefined) return undefined + if (typeof value !== 'number' || !Number.isInteger(value) || value < min || value > max) { + const range = Number.isFinite(max) ? `from ${min} to ${max}` : `of ${min} or more` + throw cliErrors.CONFIG_INVALID({ key, at: at(key), problem: `must be a whole number ${range}` }) + } + return value +} + +function readMap(value: unknown, at: At): MapSettings { + const map = section(value, ['map'], MAP_KEYS, at) + const ids = RULES.map(rule => rule.id) + const off = new Set() + for (const [id, state] of Object.entries(section(map.rules, ['map', 'rules'], ids, at))) { + const key = `map.rules.${id}` + if (staticValue(state, key, at) !== 'on' && state !== 'off') { + throw cliErrors.CONFIG_INVALID({ key, at: at(key), problem: 'must be \'on\' or \'off\'' }) + } + if (state === 'off' && CORE_RULES.has(id)) { + throw cliErrors.CONFIG_INVALID({ key, at: at(key), problem: 'cannot be turned off: the map classifies entry points by it. Leave entry points out with map.ignore instead' }) + } + if (state === 'off') off.add(id as CheckId) + } + + const ignore = staticValue(map.ignore, 'map.ignore', at) ?? [] + if (!Array.isArray(ignore) || ignore.some(glob => typeof glob !== 'string' || glob.length === 0)) { + throw cliErrors.CONFIG_INVALID({ key: 'map.ignore', at: at('map.ignore'), problem: 'must be a list of globs' }) + } + + const baseline = staticValue(map.baseline, 'map.baseline', at) + if (baseline !== undefined && baseline !== true && (typeof baseline !== 'string' || baseline.length === 0)) { + throw cliErrors.CONFIG_INVALID({ key: 'map.baseline', at: at('map.baseline'), problem: 'must be true, a path, or git:' }) + } + + return { off, ignore, minScore: wholeNumber(map.minScore, 'map.minScore', at, { min: 0, max: 100 }), baseline } +} + +function readLogs(value: unknown, packageDir: string, at: At): LogsSettings { + const logs = section(value, ['logs'], LOGS_KEYS, at) + const dir = staticValue(logs.dir, 'logs.dir', at) + if (dir !== undefined && (typeof dir !== 'string' || dir.length === 0)) { + throw cliErrors.CONFIG_INVALID({ key: 'logs.dir', at: at('logs.dir'), problem: 'must be a path' }) + } + return { + dir: dir === undefined ? undefined : resolve(packageDir, dir), + limit: wholeNumber(logs.limit, 'logs.limit', at, { min: 1 }), + } +} + +/** + * Read the `evlog.config` that applies to `project`, or `null` when there is + * none. The file is parsed, never run, so `map` and `logs` have to be literals; + * anything else in it is kept as a {@link RuntimeValue}. + * + * @throws a `cli.CONFIG_*` error when the file cannot be read that way or a + * `map` / `logs` setting is invalid. + */ +export function loadCliConfig(project: ProjectInfo): CliConfig | null { + const file = findConfigFile(project) + if (!file) return null + const label = (path: string): string => prettyPath(project.cwd, path) + const { config, parent } = readConfig(file, label) + + const documents = parent ? [parent.document, config] : [config] + const sources = new Map() + for (const document of documents) { + for (const [path, line] of document.lines) sources.set(path, `${label(document.file)}:${line}`) + } + const sourceIn = (document: ConfigDocument, path: readonly string[]): string | undefined => { + for (let end = path.length; end > 0; end--) { + const line = document.lines.get(formatPath(path.slice(0, end))) + if (line !== undefined) return `${label(document.file)}:${line}` + } + return undefined + } + for (const list of APPENDED_LISTS) { + let index = 0 + for (const document of documents) { + const items = valueAt(document.value, list) + if (!Array.isArray(items)) continue + const source = sourceIn(document, list) + for (let item = 0; item < items.length; item++, index++) { + if (source) sources.set(formatPath([...list, index]), source) + } + } + } + + const resolved = parent + ? mergeEvlogConfig(parent.document.value as EvlogConfig, config.value as EvlogConfig) as Record + : config.value + const at: At = path => sources.get(path) ?? label(file) + + return { + file, + extends: parent ? { specifier: parent.specifier, file: parent.document.file } : null, + resolved, + sources, + map: readMap(resolved.map, at), + logs: readLogs(resolved.logs, project.packageDir, at), + } +} diff --git a/packages/cli/src/lib/config/modules.ts b/packages/cli/src/lib/config/modules.ts new file mode 100644 index 00000000..a1642d15 --- /dev/null +++ b/packages/cli/src/lib/config/modules.ts @@ -0,0 +1,114 @@ +import { existsSync, readFileSync, statSync } from 'node:fs' +import { dirname, isAbsolute, join, resolve } from 'node:path' + +const EXTENSIONS = ['.ts', '.mts', '.cts', '.js', '.mjs', '.cjs'] + +/** The `exports` conditions an ESM import or a bundler matches. `types` is left out: a `.d.ts` has no values to read. */ +const CONDITIONS = new Set(['import', 'module', 'node', 'default']) + +function isFile(path: string): boolean { + return existsSync(path) && statSync(path).isFile() +} + +/** + * The file an import path points at, trying the spellings TypeScript accepts: + * the path as written, `.js` standing for `.ts`, an added extension, and an + * `index` file. + */ +function resolveFile(base: string): string | null { + const candidates = [base] + const js = /\.([mc]?)js$/.exec(base) + if (js) candidates.push(`${base.slice(0, -js[0].length)}.${js[1]}ts`) + for (const ext of EXTENSIONS) candidates.push(`${base}${ext}`) + for (const ext of EXTENSIONS) candidates.push(join(base, `index${ext}`)) + return candidates.find(isFile) ?? null +} + +/** `@acme/preset/strict` → `@acme/preset` and `./strict`. */ +function splitSpecifier(specifier: string): { name: string, subpath: string } { + const parts = specifier.split('/') + const size = specifier.startsWith('@') ? 2 : 1 + const rest = parts.slice(size).join('/') + return { name: parts.slice(0, size).join('/'), subpath: rest ? `./${rest}` : '.' } +} + +/** The target of an `exports` entry, taking the first matching condition in the package's own key order, as Node does. */ +function pickCondition(entry: unknown): string | null { + if (typeof entry === 'string') return entry + if (Array.isArray(entry)) { + for (const item of entry) { + const target = pickCondition(item) + if (target) return target + } + return null + } + if (typeof entry !== 'object' || entry === null) return null + for (const [condition, value] of Object.entries(entry)) { + if (!CONDITIONS.has(condition)) continue + const target = pickCondition(value) + if (target) return target + } + return null +} + +interface PackageManifest { + exports?: unknown + module?: string + main?: string +} + +function readManifest(path: string): PackageManifest | null { + try { + return JSON.parse(readFileSync(path, 'utf8')) as PackageManifest + } catch { + return null + } +} + +/** The file a package exposes for `subpath`, following `exports` when the package declares it. */ +function packageTarget(manifest: PackageManifest, subpath: string): string | null { + const { exports } = manifest + if (exports === undefined) { + if (subpath !== '.') return subpath + return manifest.module ?? manifest.main ?? 'index.js' + } + const isSubpathMap = typeof exports === 'object' && exports !== null && !Array.isArray(exports) + && Object.keys(exports).some(key => key.startsWith('.')) + if (isSubpathMap) return pickCondition((exports as Record)[subpath]) + return subpath === '.' ? pickCondition(exports) : null +} + +function resolvePackage(specifier: string, fromDir: string): string | null { + const { name, subpath } = splitSpecifier(specifier) + let dir = fromDir + for (;;) { + const packageDir = join(dir, 'node_modules', name) + const manifestPath = join(packageDir, 'package.json') + if (isFile(manifestPath)) { + const manifest = readManifest(manifestPath) + const target = manifest && packageTarget(manifest, subpath) + return target ? resolveFile(join(packageDir, target)) : null + } + const parent = dirname(dir) + if (parent === dir) return null + dir = parent + } +} + +/** Whether an import specifier is a path rather than a package name. */ +export function isRelativeSpecifier(specifier: string): boolean { + return specifier.startsWith('./') || specifier.startsWith('../') || isAbsolute(specifier) +} + +/** + * The source file an import specifier refers to, from `fromDir`, or `null`. + * + * A package is found by walking up `node_modules` and reading its manifest + * rather than through `require.resolve`: an ESM-only package that exports no + * `require` condition cannot be resolved that way, and `import.meta.resolve` + * takes no parent outside an experimental flag. + */ +export function resolveModuleFile(specifier: string, fromDir: string): string | null { + if (isRelativeSpecifier(specifier)) return resolveFile(resolve(fromDir, specifier)) + return resolvePackage(specifier, fromDir) +} diff --git a/packages/cli/src/lib/config/read.ts b/packages/cli/src/lib/config/read.ts new file mode 100644 index 00000000..3038c211 --- /dev/null +++ b/packages/cli/src/lib/config/read.ts @@ -0,0 +1,382 @@ +import { readFileSync } from 'node:fs' +import { dirname } from 'node:path' +import type { Expression, ModuleExportName, Node, ObjectExpression, ObjectProperty, Program } from 'oxc-parser' +import { cliErrors } from '../errors' +import { parseSource } from '../map/parse' +import type { LineIndex } from '../map/parse' +import { isRelativeSpecifier, resolveModuleFile } from './modules' + +/** + * A value the config only has once it runs: a call, a function, a binding + * imported from a package. Kept in place of the value so the rest of the + * config can still be read. + */ +export class RuntimeValue { + /** + * @param code The source that produces the value, whitespace collapsed and cut at 60 characters. + * @param label What the value is, short enough for a column: `createAxiomDrain()`, `function`. + */ + constructor(readonly code: string, readonly label: string = code) {} + + toJSON(): { runtime: string } { + return { runtime: this.code } + } +} + +/** One config object as written, before it is merged with the one it extends. */ +export interface ConfigDocument { + /** The file the object literal is written in. */ + file: string + value: Record + /** Line of each property, by {@link formatPath}. */ + lines: Map +} + +const PLAIN_KEY = /^[\w$-]+$/ + +/** + * A setting's path as it reads in source: `sampling.rates.info`, a key that is + * not a plain name quoted, `routes['/api/v1.2/**'].service`, a list index in + * brackets, `redact.paths[0]`. Keys {@link ConfigDocument.lines}, so a route + * glob holding a dot stays one key. + */ +export function formatPath(path: readonly (string | number)[]): string { + let out = '' + for (const segment of path) { + if (typeof segment === 'number') out += `[${segment}]` + else if (PLAIN_KEY.test(segment)) out += out ? `.${segment}` : segment + else out += `['${segment.replaceAll('\\', '\\\\').replaceAll('\'', '\\\'')}']` + } + return out +} + +export interface ReadConfigResult { + config: ConfigDocument + /** The config `config` extends, with the specifier it was imported from (`null` when it is defined in the same file). */ + parent: { specifier: string | null, document: ConfigDocument } | null +} + +interface Module { + file: string + source: string + program: Program + lines: LineIndex + /** Top-level variable initialisers, by name. */ + bindings: Map + /** Imported bindings, by local name; `imported` is `default`, an export name, or `*`. */ + imports: Map +} + +interface Located { + module: Module + node: T +} + +/** Bounds how far a name is followed through bindings and re-exports, so a cycle ends. */ +const MAX_HOPS = 8 +const DEFINE_SOURCES = new Set(['evlog', 'evlog/toolkit']) +const WRAPPERS = new Set(['ParenthesizedExpression', 'TSAsExpression', 'TSSatisfiesExpression', 'TSNonNullExpression', 'TSTypeAssertion']) +const SNIPPET_LENGTH = 60 + +export function isPlainObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && Object.getPrototypeOf(value) === Object.prototype +} + +function strip(node: Expression): Expression { + let current = node + while (WRAPPERS.has(current.type)) current = (current as { expression: Expression }).expression + return current +} + +function exportName(node: ModuleExportName): string { + return node.type === 'Literal' ? node.value : node.name +} + +function propertyName(prop: ObjectProperty): string | null { + const { key } = prop + if (!prop.computed && key.type === 'Identifier') return key.name + if (key.type === 'Literal' && (typeof key.value === 'string' || typeof key.value === 'number')) return String(key.value) + return null +} + +function indexModule(program: Program): Pick { + const bindings = new Map() + const imports = new Map() + for (const statement of program.body) { + if (statement.type === 'ImportDeclaration') { + if (statement.importKind === 'type') continue + for (const specifier of statement.specifiers) { + if (specifier.type === 'ImportSpecifier' && specifier.importKind === 'type') continue + const imported = specifier.type === 'ImportDefaultSpecifier' + ? 'default' + : specifier.type === 'ImportNamespaceSpecifier' ? '*' : exportName(specifier.imported) + imports.set(specifier.local.name, { source: statement.source.value, imported }) + } + continue + } + const declaration = statement.type === 'ExportNamedDeclaration' ? statement.declaration : statement + if (declaration?.type !== 'VariableDeclaration') continue + for (const declarator of declaration.declarations) { + if (declarator.id.type === 'Identifier' && declarator.init) bindings.set(declarator.id.name, declarator.init) + } + } + return { bindings, imports } +} + +class Reader { + private readonly modules = new Map() + + constructor(private readonly label: (file: string) => string) {} + + module(file: string): Module { + const cached = this.modules.get(file) + if (cached) return cached + const source = readFileSync(file, 'utf8') + const parsed = parseSource(file, source) + if (!parsed) throw cliErrors.CONFIG_NO_EXPORT({ file: this.label(file), name: 'default' }) + if (parsed.errors.length > 0) throw cliErrors.CONFIG_PARSE_FAILED({ file: this.label(file), reason: parsed.errors[0] ?? 'unknown error' }) + const module: Module = { file, source, program: parsed.program, lines: parsed.lines, ...indexModule(parsed.program) } + this.modules.set(file, module) + return module + } + + snippet(module: Module, node: Node): string { + const code = module.source.slice(node.start, node.end).replace(/\s+/g, ' ') + return code.length > SNIPPET_LENGTH ? `${code.slice(0, SNIPPET_LENGTH - 1)}…` : code + } + + /** A call by the function it calls, a function as `function`, anything else as its code. */ + describe(module: Module, node: Node): string { + const expression = node.type === 'SpreadElement' ? node.argument : node + const prefix = node.type === 'SpreadElement' ? '...' : '' + const stripped = WRAPPERS.has(expression.type) ? strip(expression as Expression) : expression + switch (stripped.type) { + case 'CallExpression': + case 'NewExpression': { + const callee = this.snippet(module, stripped.callee) + const args = stripped.arguments.length > 0 ? '…' : '' + return `${prefix}${stripped.type === 'NewExpression' ? 'new ' : ''}${callee}(${args})` + } + case 'AwaitExpression': + return `${prefix}await ${this.describe(module, stripped.argument)}` + case 'ArrowFunctionExpression': + case 'FunctionExpression': + return `${prefix}function` + case 'Property': + return stripped.kind === 'init' ? 'function' : stripped.kind + case 'ArrayExpression': + return `${prefix}[${stripped.elements.map(element => element ? this.describe(module, element) : '').join(', ')}]` + } + return prefix + this.snippet(module, stripped) + } + + runtime(module: Module, node: Node): RuntimeValue { + return new RuntimeValue(this.snippet(module, node), this.describe(module, node)) + } + + /** What `file` exports as `name`, following re-exports. */ + exported(file: string, name: string, hops: number): Located | null { + if (hops > MAX_HOPS) return null + const module = this.module(file) + for (const statement of module.program.body) { + if (statement.type === 'ExportDefaultDeclaration' && name === 'default') { + const { declaration } = statement + const isExpression = declaration.type !== 'FunctionDeclaration' && declaration.type !== 'ClassDeclaration' + && declaration.type !== 'TSInterfaceDeclaration' + return isExpression ? { module, node: declaration as Expression } : null + } + if (statement.type !== 'ExportNamedDeclaration') continue + if (statement.declaration?.type === 'VariableDeclaration') { + for (const declarator of statement.declaration.declarations) { + if (declarator.id.type === 'Identifier' && declarator.id.name === name && declarator.init) return { module, node: declarator.init } + } + } + for (const specifier of statement.specifiers) { + if (exportName(specifier.exported) !== name) continue + const local = exportName(specifier.local) + if (!statement.source) return this.binding(module, local, hops + 1) + const target = resolveModuleFile(statement.source.value, dirname(file)) + return target ? this.exported(target, local, hops + 1) : null + } + } + return null + } + + /** + * What a top-level name refers to: a variable in the module, or an export of + * a file it imports by path. Packages are not followed here, so a value + * imported from one stays a {@link RuntimeValue}. + */ + binding(module: Module, name: string, hops: number): Located | null { + if (hops > MAX_HOPS) return null + const local = module.bindings.get(name) + if (local) return { module, node: local } + const imported = module.imports.get(name) + if (!imported || imported.imported === '*' || !isRelativeSpecifier(imported.source)) return null + const target = resolveModuleFile(imported.source, dirname(module.file)) + return target ? this.exported(target, imported.imported, hops + 1) : null + } + + isDefineEvlog(module: Module, callee: Expression): boolean { + if (callee.type !== 'Identifier') return false + const imported = module.imports.get(callee.name) + if (imported) return DEFINE_SOURCES.has(imported.source) && imported.imported === 'defineEvlog' + return callee.name === 'defineEvlog' + } + + /** The object literal an expression configures, through `defineEvlog(...)` and named bindings. */ + configObject(module: Module, node: Expression, hops: number): Located | null { + if (hops > MAX_HOPS) return null + const expression = strip(node) + if (expression.type === 'ObjectExpression') return { module, node: expression } + if (expression.type === 'CallExpression' && this.isDefineEvlog(module, expression.callee)) { + const [argument] = expression.arguments + if (expression.arguments.length !== 1 || !argument || argument.type === 'SpreadElement') return null + return this.configObject(module, argument, hops + 1) + } + if (expression.type === 'Identifier') { + const target = this.binding(module, expression.name, hops + 1) + return target ? this.configObject(target.module, target.node, hops + 1) : null + } + return null + } + + evaluate({ module, node }: Located, path: readonly string[], lines: Map | undefined, hops: number): unknown { + const expression = strip(node) + switch (expression.type) { + case 'Literal': { + if ('regex' in expression && expression.regex) return new RegExp(expression.regex.pattern, expression.regex.flags) + const { value } = expression + if (value === null || typeof value === 'string' || typeof value === 'number' || typeof value === 'boolean') return value + break + } + case 'TemplateLiteral': { + const [quasi] = expression.quasis + if (expression.expressions.length === 0 && quasi) return quasi.value.cooked ?? quasi.value.raw + break + } + case 'UnaryExpression': { + const operand = this.evaluate({ module, node: expression.argument }, path, undefined, hops) + if (expression.operator === '-' && typeof operand === 'number') return -operand + break + } + case 'ArrayExpression': { + const items: unknown[] = [] + for (const element of expression.elements) { + const item = !element + ? undefined + : element.type === 'SpreadElement' + ? this.evaluate({ module, node: element.argument }, path, undefined, hops) + : this.evaluate({ module, node: element }, path, undefined, hops) + /* An array is static only when every element is: a list half read + at runtime cannot be merged or checked entry by entry. */ + if (item instanceof RuntimeValue || (element?.type === 'SpreadElement' && !Array.isArray(item))) { + return this.runtime(module, expression) + } + if (element?.type === 'SpreadElement') items.push(...(item as unknown[])) + else items.push(item) + } + return items + } + case 'ObjectExpression': + return this.object({ module, node: expression }, path, lines, hops) + case 'Identifier': { + if (expression.name === 'undefined') return undefined + const target = this.binding(module, expression.name, hops + 1) + /* Lines are recorded against the file being read, so an object that + lives in another file contributes its values but not its lines. */ + if (target) return this.evaluate(target, path, target.module === module ? lines : undefined, hops + 1) + break + } + } + return this.runtime(module, expression) + } + + object({ module, node }: Located, prefix: readonly string[], lines: Map | undefined, hops: number): Record | RuntimeValue { + const out: Record = {} + for (const prop of node.properties) { + const line = module.lines.lineAt(prop.start) + if (prop.type === 'SpreadElement') { + const spread = this.evaluate({ module, node: prop.argument }, prefix, undefined, hops) + if (!isPlainObject(spread)) return this.runtime(module, node) + Object.assign(out, spread) + for (const key of Object.keys(spread)) lines?.set(formatPath([...prefix, key]), line) + continue + } + const key = propertyName(prop) + /* A computed key could be any key, so nothing about the object is known. */ + if (key === null) return this.runtime(module, node) + const path = [...prefix, key] + lines?.set(formatPath(path), line) + out[key] = prop.method || prop.kind !== 'init' + ? this.runtime(module, prop) + : this.evaluate({ module, node: prop.value }, path, lines, hops) + } + return out + } + + /** Read a config object; `extends` is returned apart, as written. */ + document(located: Located): { document: ConfigDocument, extendsNode: { node: Expression, line: number } | null } { + const { module, node } = located + let extendsNode: { node: Expression, line: number } | null = null + const properties = node.properties.filter((prop) => { + if (prop.type !== 'Property' || propertyName(prop) !== 'extends') return true + extendsNode = { node: prop.value, line: module.lines.lineAt(prop.start) } + return false + }) + const lines = new Map() + const value = this.object({ module, node: { ...node, properties } }, [], lines, 0) + if (value instanceof RuntimeValue) { + throw cliErrors.CONFIG_NOT_STATIC({ key: 'The config', at: `${this.label(module.file)}:${module.lines.lineAt(node.start)}` }) + } + return { document: { file: module.file, value, lines }, extendsNode } + } + + read(file: string, name: string): ReturnType { + const exported = this.exported(file, name, 0) + const object = exported && this.configObject(exported.module, exported.node, 0) + if (!object) throw cliErrors.CONFIG_NO_EXPORT({ file: this.label(file), name }) + return this.document(object) + } + + /** The config an `extends` value refers to: an import, a config in the same file, or an inline object. */ + parent(module: Module, extendsNode: { node: Expression, line: number }): { specifier: string | null } & ReturnType { + const at = `${this.label(module.file)}:${extendsNode.line}` + const node = strip(extendsNode.node) + const imported = node.type === 'Identifier' ? module.imports.get(node.name) : undefined + if (imported && imported.imported !== '*') { + const target = resolveModuleFile(imported.source, dirname(module.file)) + if (!target) throw cliErrors.CONFIG_EXTENDS_NOT_FOUND({ specifier: imported.source, at }) + return { specifier: imported.source, ...this.read(target, imported.imported) } + } + if (node.type === 'Literal' && typeof node.value === 'string') throw cliErrors.CONFIG_EXTENDS_STRING({ specifier: node.value, at }) + const object = this.configObject(module, node, 0) + if (!object) throw cliErrors.CONFIG_NOT_STATIC({ key: 'extends', at }) + return { specifier: null, ...this.document(object) } + } +} + +/** + * Read `evlog.config` and the config it extends without running either. + * + * Literals become data; anything else becomes a {@link RuntimeValue}, so + * callers can tell what the file says from what it computes. Paths in errors + * go through `label`. + * + * @throws a `cli.CONFIG_*` error when the file cannot be read this way, or + * extends a config that itself extends another. + */ +export function readConfig(file: string, label: (file: string) => string): ReadConfigResult { + const reader = new Reader(label) + const { document, extendsNode } = reader.read(file, 'default') + if (!extendsNode) return { config: document, parent: null } + + const parent = reader.parent(reader.module(document.file), extendsNode) + if (parent.extendsNode) { + throw cliErrors.CONFIG_EXTENDS_DEPTH({ + parent: parent.specifier ?? label(parent.document.file), + at: `${label(document.file)}:${(extendsNode as { line: number }).line}`, + }) + } + return { config: document, parent: { specifier: parent.specifier, document: parent.document } } +} diff --git a/packages/cli/src/lib/errors.ts b/packages/cli/src/lib/errors.ts index 6f9e36ac..800d9a95 100644 --- a/packages/cli/src/lib/errors.ts +++ b/packages/cli/src/lib/errors.ts @@ -312,6 +312,69 @@ export const cliErrors = defineErrorCatalog('cli', { fix: 'Pass a whole number of 1 or more, e.g. --limit 20', tags: ['logs'], }, + CONFIG_PARSE_FAILED: { + status: 400, + message: ({ file, reason }: { file: string, reason: string }) => + `Could not parse ${file}: ${reason}`, + why: 'The CLI reads evlog.config without running it, so the file has to parse on its own', + fix: 'Fix the syntax error and run the command again', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_NO_EXPORT: { + status: 400, + message: ({ file, name }: { file: string, name: string }) => + `${file} has no ${name === 'default' ? 'default export' : `export named ${name}`} the CLI can read`, + why: 'The CLI reads the object literal passed to defineEvlog, through same-file consts and relative imports', + fix: 'Write the config as export default defineEvlog({ ... })', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_NOT_STATIC: { + status: 400, + message: ({ key, at }: { key: string, at: string }) => + `${key} in ${at} is computed at runtime`, + why: 'The CLI reads evlog.config without running it, so map and logs settings have to be literals', + fix: 'Write the value inline, as a const, or import it from a local file', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_INVALID: { + status: 400, + message: ({ key, at, problem }: { key: string, at: string, problem: string }) => + `${key} in ${at} ${problem}`, + why: 'A setting the CLI cannot read would silently fall back to its default', + fix: 'Use a setting and a value the config reference lists', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_EXTENDS_NOT_FOUND: { + status: 404, + message: ({ specifier, at }: { specifier: string, at: string }) => + `Cannot resolve "${specifier}", extended in ${at}`, + why: 'The CLI follows extends to read the parent config, and the import does not lead to a file', + fix: 'Install the package, or correct the import path', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_EXTENDS_DEPTH: { + status: 400, + message: ({ parent, at }: { parent: string, at: string }) => + `${parent} extends another config, so ${at} cannot extend it`, + why: 'A config extends one level only, so every setting is at most one file away from where it applies', + fix: 'Extend the config it extends directly, or copy the settings you need into one of the two files', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, + CONFIG_EXTENDS_STRING: { + status: 400, + message: ({ specifier, at }: { specifier: string, at: string }) => + `extends in ${at} is the path '${specifier}', not a config`, + why: 'extends takes the config itself, the same value the app merges at runtime, so a path is never resolved', + fix: 'Import the config from that path as base, then set extends: base', + link: 'https://evlog.dev/cli/config', + tags: ['config'], + }, }) declare module 'evlog' { diff --git a/packages/cli/src/lib/logs/query.ts b/packages/cli/src/lib/logs/query.ts index ec261b2e..41a00beb 100644 --- a/packages/cli/src/lib/logs/query.ts +++ b/packages/cli/src/lib/logs/query.ts @@ -69,8 +69,8 @@ export function parseOver(value: unknown): number | undefined { const DEFAULT_LIMIT = 50 -export function parseLimit(value: unknown): number { - if (typeof value !== 'string' || value.length === 0) return DEFAULT_LIMIT +export function parseLimit(value: unknown, fallback = DEFAULT_LIMIT): number { + if (typeof value !== 'string' || value.length === 0) return fallback const limit = Number(value) if (!Number.isInteger(limit) || limit < 1) throw cliErrors.LOGS_INVALID_LIMIT({ value }) return limit @@ -232,14 +232,17 @@ export function matchesId(event: WideEvent, id: string): boolean { }) } -/** Turn the flags into a query. Validation happens here, before any file is read. */ -export function buildQuery(args: LogsArgs, now = new Date()): LogsQuery { +/** + * Turn the flags into a query. Validation happens here, before any file is read. + * `defaultLimit` is what `--limit` falls back to, from `logs.limit` in evlog.config. + */ +export function buildQuery(args: LogsArgs, now = new Date(), defaultLimit?: number): LogsQuery { const since = parseTime('since', args.since, now) const until = parseTime('until', args.until, now) const level = parseLevels(args.level) const status = parseStatus(args.status) const over = parseOver(args.over) ?? DEFAULT_OVER - const limit = parseLimit(args.limit) + const limit = parseLimit(args.limit, defaultLimit) const path = typeof args.path === 'string' && args.path.length > 0 ? args.path : undefined const where = parseWheres(args.where) diff --git a/packages/cli/src/lib/map/formats.ts b/packages/cli/src/lib/map/formats.ts index 1135cfcc..ca99f6eb 100644 --- a/packages/cli/src/lib/map/formats.ts +++ b/packages/cli/src/lib/map/formats.ts @@ -27,6 +27,8 @@ export interface Location { export interface AnnotationOptions { minScore?: number + /** The setting `minScore` came from, named in the annotation: `--min-score` or `map.minScore`. */ + minScoreFrom?: string /** * Most findings to emit. GitHub keeps ten annotations per level per step and * drops the rest without a word, so past that the list is not a list. @@ -122,7 +124,7 @@ export function formatGithubAnnotations( hidden > 0 ? `; ${hidden} more finding${hidden === 1 ? '' : 's'} not shown` : ''}` if (options.minScore !== undefined && !passesMinScore(scan.grade, score, options.minScore)) { - lines.push(annotation('error', { title: 'evlog map' }, `${summary}; below --min-score ${options.minScore}`)) + lines.push(annotation('error', { title: 'evlog map' }, `${summary}; below ${options.minScoreFrom ?? '--min-score'} ${options.minScore}`)) } else if (baseline && hasRegressed(baseline)) { lines.push(annotation('error', { title: 'evlog map' }, `${summary}; regressed against ${baseline.source.label}`)) } else { diff --git a/packages/cli/src/lib/map/report.ts b/packages/cli/src/lib/map/report.ts index 36e56381..e97e1842 100644 --- a/packages/cli/src/lib/map/report.ts +++ b/packages/cli/src/lib/map/report.ts @@ -888,7 +888,7 @@ export function formatMapWarnings(ctx: CliContext, warnings: readonly string[]): * Spells out the exit code because this line is most often read in CI logs, * where the reader is looking for why the job went red. */ -export function formatGate(ctx: CliContext, result: ScanResult, threshold: number): string { +export function formatGate(ctx: CliContext, result: ScanResult, threshold: number, from = '--min-score'): string { const style = createReportStyle(ctx) const { paint } = style const { score } = result.map @@ -896,10 +896,10 @@ export function formatGate(ctx: CliContext, result: ScanResult, threshold: numbe const badge = paint(['bold', passed ? 'green' : 'red'], ' GATE ') const verdict = passed - ? `${paint('green', `score ${score} meets --min-score ${threshold}`)} ${paint('dim', '— exit code 0')}` + ? `${paint('green', `score ${score} meets ${from} ${threshold}`)} ${paint('dim', '— exit code 0')}` : result.grade === 'unscored' - ? `${paint('red', `nothing to scan, so --min-score ${threshold} cannot be met`)} ${paint('dim', '— exit code 1')}` - : `${paint('red', `score ${score} is below --min-score ${threshold}`)} ${paint('dim', '— exit code 1')}` + ? `${paint('red', `nothing to scan, so ${from} ${threshold} cannot be met`)} ${paint('dim', '— exit code 1')}` + : `${paint('red', `score ${score} is below ${from} ${threshold}`)} ${paint('dim', '— exit code 1')}` const lines = ['', `${badge} ${verdict}`] if (!passed) { diff --git a/packages/cli/src/lib/map/rules/index.ts b/packages/cli/src/lib/map/rules/index.ts index 67389c55..91487a43 100644 --- a/packages/cli/src/lib/map/rules/index.ts +++ b/packages/cli/src/lib/map/rules/index.ts @@ -1,3 +1,4 @@ +import type { EvlogMapRuleId } from 'evlog' import type { FileFacts } from '../facts' import type { ParseResult } from '../parse' import { walkAst } from '../parse' @@ -55,6 +56,14 @@ type AssertIdsMatch = [RegisteredId] extends [CheckId] const idsMatch: AssertIdsMatch = true void idsMatch +/* `map.rules` in evlog.config is typed by the core package, so its ids must + be the ones this registry runs. */ +type AssertConfigIdsMatch = [EvlogMapRuleId] extends [CheckId] + ? [CheckId] extends [EvlogMapRuleId] ? true : never + : never +const configIdsMatch: AssertConfigIdsMatch = true +void configIdsMatch + /** * Every observability rule, in report order. * @@ -76,6 +85,9 @@ export const RULES: readonly MapRule[] = REGISTRY */ export const RULE_SET_VERSION = 1 +/** Why a check is `n/a` when `evlog.config` turns it off. */ +export const RULE_OFF_MESSAGE = 'turned off in evlog.config' + const RULES_BY_ID = new Map(RULES.map(rule => [rule.id, rule])) /** Look up a rule's metadata — weight, title, docs link, suggested fix. */ @@ -171,12 +183,19 @@ export function runRuleSet(rules: readonly MapRule[], run: RuleRun): RuleResults /* Depends only on the route's path and file, so it holds even for a file we cannot read — an exempt health check stays exempt when it fails to parse. */ const exemption = getRouteExemption(target) + /* A check turned off for the project is `n/a` as a requirement and silence + as an opportunity, the same way a check that does not apply is. */ + const turnedOff = (rule: MapRule): boolean => { + if (!ctx.rulesOff?.has(rule.id)) return false + if (rule.category === 'requirement') results.checks[rule.id] = { status: 'n/a', message: RULE_OFF_MESSAGE } + return true + } if (!parsed || !facts) { /* A file that will not parse is a real failure, but only of requirements — we have no basis to suggest anything about code we could not read. */ for (const rule of relevant) { - if (rule.category !== 'requirement') continue + if (rule.category !== 'requirement' || turnedOff(rule)) continue if (exemption && isSkipped(exemption, rule.id)) { results.checks[rule.id] = { status: 'n/a', message: exemption.reason } continue @@ -199,6 +218,7 @@ export function runRuleSet(rules: readonly MapRule[], run: RuleRun): RuleResults const active: Array<{ rule: MapRule, listeners: RuleListeners, reports: RuleReport[] }> = [] for (const rule of relevant) { + if (turnedOff(rule)) continue if (exemption && isSkipped(exemption, rule.id)) { bucket(rule)[rule.id] = { status: 'n/a', message: exemption.reason } continue diff --git a/packages/cli/src/lib/map/scan.ts b/packages/cli/src/lib/map/scan.ts index 43720eac..8972864c 100644 --- a/packages/cli/src/lib/map/scan.ts +++ b/packages/cli/src/lib/map/scan.ts @@ -1,6 +1,7 @@ import { join } from 'node:path' import { version as CLI_VERSION } from '../../../package.json' import { getFramework } from '../frameworks' +import { glob } from '../glob' import { countSuppressed } from './directives' import { buildFileFacts } from './facts' import { createParseCache, parseFile } from './parse' @@ -20,7 +21,7 @@ import type { ScanContext, ScanResult, } from './types' -import { routeId } from './utils' +import { relativeFromRoot, routeId } from './utils' interface AnalyseInput { ctx: ScanContext @@ -93,7 +94,9 @@ export async function scan(input: ScanContext): Promise { evlogAutoImports: capabilities.evlogAutoImports, }) - const rawRoutes = await adapter.extractRoutes(ctx) + const extracted = await adapter.extractRoutes(ctx) + const ignored = new Set(ctx.ignore?.length ? glob(ctx.ignore, ctx.projectRoot).map(file => relativeFromRoot(ctx.projectRoot, file)) : []) + const rawRoutes = extracted.filter(route => !ignored.has(route.file)) const analysed = rawRoutes.map(raw => analyseRoute({ ctx, raw, project, capabilities })) const routes = analysed.map(entry => entry.route) const warnings = analysed.flatMap(entry => entry.warnings) @@ -125,6 +128,7 @@ export async function scan(input: ScanContext): Promise { project, suggestions, warnings, + ignored: extracted.length - rawRoutes.length, } } diff --git a/packages/cli/src/lib/map/telemetry.ts b/packages/cli/src/lib/map/telemetry.ts index 13a6d08d..9d88d7d5 100644 --- a/packages/cli/src/lib/map/telemetry.ts +++ b/packages/cli/src/lib/map/telemetry.ts @@ -142,6 +142,8 @@ export function mapTelemetryFields(input: { baseline: BaselineComparison | null view: MapView wrote: boolean + /** What the applied `evlog.config` turned off, or `null` when the run had none. */ + config: { rulesOff: number } | null }): Record { const { scan } = input const { routes } = scan.map @@ -164,12 +166,19 @@ export function mapTelemetryFields(input: { mapSuggestions: routes.reduce((total, route) => total + Object.keys(route.suggestions).length, 0), mapProjectSuggestions: scan.suggestions.length, mapGate: input.gate, + mapConfig: input.config !== null, ...kindTallies(routes), ...sensitiveTallies(routes), ...ruleTallies(scan), } if (input.minScore !== undefined) fields.mapMinScore = input.minScore + /* A rule turned off for a whole project is the strongest "bad rule" signal + there is, stronger than a per-line suppression. */ + if (input.config) { + fields.mapRulesOff = input.config.rulesOff + fields.mapIgnored = scan.ignored + } if (input.baseline) { fields.mapBaselineDelta = input.baseline.delta fields.mapBaselineRegressions = input.baseline.regressions.length @@ -213,6 +222,7 @@ export function mapTelemetryFieldNames(): string[] { project: {} as ScanResult['project'], suggestions: [], warnings: [], + ignored: 0, summary: { instrumented: 0, partial: 0, dark: 0, exempt: 0, suppressedChecks: 0 }, } @@ -236,6 +246,7 @@ export function mapTelemetryFieldNames(): string[] { baseline: emptyBaseline, view: 'summary', wrote: false, + config: { rulesOff: 0 }, })) const tallies = [ diff --git a/packages/cli/src/lib/map/types.ts b/packages/cli/src/lib/map/types.ts index 12989cfb..29951213 100644 --- a/packages/cli/src/lib/map/types.ts +++ b/packages/cli/src/lib/map/types.ts @@ -119,6 +119,10 @@ export interface ScanContext { * and each file goes through oxc once. Defaults to an uncached read. */ parse?: ParseFn + /** Checks `evlog.config` turns off: reported `n/a` on every entry point. */ + rulesOff?: ReadonlySet + /** Entry points `evlog.config` leaves out, as globs matched against {@link RawRouteEntry.file}. */ + ignore?: readonly string[] } export interface FrameworkAdapter { @@ -183,6 +187,8 @@ export interface ScanResult { suggestions: ProjectSuggestion[] /** Problems found while scanning — a disable comment naming an unknown check. */ warnings: string[] + /** Entry points `evlog.config` left out through `map.ignore`. */ + ignored: number summary: { instrumented: number partial: number diff --git a/packages/cli/test/config.test.ts b/packages/cli/test/config.test.ts new file mode 100644 index 00000000..61b5d81d --- /dev/null +++ b/packages/cli/test/config.test.ts @@ -0,0 +1,666 @@ +import { cp, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { runCommand } from 'citty' +import { afterEach, describe, expect, it, vi } from 'vitest' +import configCommand, { configJson, formatConfigReport, runConfig } from '../src/commands/config' +import { runDoctor } from '../src/commands/doctor' +import { runLogs } from '../src/commands/logs' +import map, { formatMapReport, runMap } from '../src/commands/map' +import { createContext } from '../src/core/context' +import type { CliContext } from '../src/core/context' +import { loadCliConfig, RuntimeValue } from '../src/lib/config' +import { cliErrors } from '../src/lib/errors' +import { RULE_OFF_MESSAGE } from '../src/lib/map/rules/index' +import { resolveProject } from '../src/lib/project' + +const FIXTURES = join(import.meta.dirname, 'map/fixtures') +const tempDirs: string[] = [] + +async function makeProject(files: Record): Promise { + const dir = await mkdtemp(join(tmpdir(), 'evlog-cli-config-')) + tempDirs.push(dir) + await writeFiles(dir, files) + return dir +} + +async function writeFiles(dir: string, files: Record): Promise { + for (const [path, contents] of Object.entries(files)) { + const full = join(dir, path) + await mkdir(join(full, '..'), { recursive: true }) + await writeFile(full, contents, 'utf-8') + } +} + +async function copyFixture(name: string): Promise { + const dir = await mkdtemp(join(tmpdir(), `evlog-cli-config-${name}-`)) + tempDirs.push(dir) + await cp(join(FIXTURES, name), dir, { recursive: true }) + return dir +} + +function fakeContext(cwd: string): CliContext { + return createContext({ cwd, env: {}, nodeVersion: 'v22.0.0', tty: false, color: false, columns: 120 }) +} + +async function load(cwd: string) { + return loadCliConfig(await resolveProject(cwd)) +} + +function silence(): { stdout: () => string, stderr: () => string } { + const stdout = vi.spyOn(process.stdout, 'write').mockImplementation(() => true) + const stderr = vi.spyOn(process.stderr, 'write').mockImplementation(() => true) + return { + stdout: () => stdout.mock.calls.map(([chunk]) => String(chunk)).join(''), + stderr: () => stderr.mock.calls.map(([chunk]) => String(chunk)).join(''), + } +} + +const PACKAGE = JSON.stringify({ name: 'shop' }) + +/** A preset and a config extending it, with the lines the assertions point at. */ +const EXTENDED = { + 'package.json': PACKAGE, + 'evlog.preset.ts': `import { defineEvlog } from 'evlog' + +export default defineEvlog({ + sampling: { rates: { info: 10, debug: 0 } }, + redact: { paths: ['user.password'] }, + map: { rules: { 'audit': 'off', 'error-catalog': 'off' }, minScore: 90 }, +}) +`, + 'evlog.config.ts': `import { defineEvlog } from 'evlog' +import preset from './evlog.preset' + +export default defineEvlog({ + extends: preset, + sampling: { rates: { info: 50 } }, + redact: { paths: ['card.number'] }, + map: { rules: { audit: 'on' } }, +}) +`, +} + +/** A config extending a package whose package.json is cut off mid-object. */ +const MALFORMED_PRESET = { + 'package.json': PACKAGE, + 'node_modules/@acme/evlog-preset/package.json': '{ "name": "@acme/evlog-preset",', + 'evlog.config.ts': 'import preset from \'@acme/evlog-preset\'\n\nexport default { extends: preset }\n', +} + +afterEach(async () => { + vi.restoreAllMocks() + process.exitCode = undefined + await Promise.all(tempDirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) +}) + +describe('evlog.config lookup', () => { + it('returns null when no evlog.config applies', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE }) + expect(await load(cwd)).toBeNull() + }) + + it('reads map and logs settings from a literal config', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `import { defineEvlog } from 'evlog' + +export default defineEvlog({ + service: 'shop', + map: { + rules: { 'audit': 'off', 'error-catalog': 'off' }, + ignore: ['server/api/internal/**'], + minScore: 80, + baseline: 'git:main', + }, + logs: { dir: 'var/logs', limit: 20 }, +}) +`, + }) + const project = await resolveProject(cwd) + const config = loadCliConfig(project) + + expect([...config!.map.off]).toEqual(['audit', 'error-catalog']) + expect(config!.map.ignore).toEqual(['server/api/internal/**']) + expect(config!.map.minScore).toBe(80) + expect(config!.map.baseline).toBe('git:main') + expect(config!.logs).toEqual({ dir: join(project.packageDir, 'var/logs'), limit: 20 }) + expect(config!.resolved.service).toBe('shop') + }) + + it.each([ + ['evlog.config.mjs', 'const config = { map: { minScore: 60 } }\nexport default config\n'], + ['evlog.config.ts', 'import type { EvlogConfig } from \'evlog\'\n\nexport default { map: { minScore: 60 } } satisfies EvlogConfig\n'], + ['evlog.config.ts', 'import { defineEvlog } from \'evlog\'\n\nconst config = defineEvlog({ map: { minScore: 60 } })\n\nexport default config\n'], + ])('reads %s written as %j', async (name, source) => { + const cwd = await makeProject({ 'package.json': PACKAGE, [name]: source }) + expect((await load(cwd))!.map.minScore).toBe(60) + }) + + it('follows consts and relative imports', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog/map.ts': 'export const rules = { audit: \'off\' } as const\nexport const ignore = [\'server/tasks/**\']\n', + 'evlog.config.ts': `import { defineEvlog } from 'evlog' +import { ignore, rules } from './evlog/map' + +const minScore = 75 + +export default defineEvlog({ map: { rules, ignore, minScore } }) +`, + }) + const config = await load(cwd) + + expect([...config!.map.off]).toEqual(['audit']) + expect(config!.map.ignore).toEqual(['server/tasks/**']) + expect(config!.map.minScore).toBe(75) + }) + + it('uses the nearest config on its own, without cascading', async () => { + const root = await makeProject({ + 'pnpm-workspace.yaml': 'packages:\n - apps/*\n', + 'package.json': JSON.stringify({ name: 'mono', private: true }), + 'evlog.config.ts': 'export default { map: { minScore: 90 }, logs: { limit: 5 } }\n', + 'apps/web/package.json': JSON.stringify({ name: 'web' }), + 'apps/web/evlog.config.ts': 'export default { map: { minScore: 70 } }\n', + }) + const config = await load(join(root, 'apps/web')) + + expect(config!.map.minScore).toBe(70) + expect(config!.logs.limit).toBeUndefined() + }) + + it('applies a workspace root config to every app, with paths relative to the app', async () => { + const root = await makeProject({ + 'pnpm-workspace.yaml': 'packages:\n - apps/*\n', + 'package.json': JSON.stringify({ name: 'mono', private: true }), + 'evlog.config.ts': 'export default { logs: { dir: \'.evlog/logs\' } }\n', + 'apps/web/package.json': JSON.stringify({ name: 'web' }), + }) + const project = await resolveProject(join(root, 'apps/web')) + + expect(loadCliConfig(project)!.logs.dir).toBe(join(project.packageDir, '.evlog/logs')) + }) +}) + +describe('evlog.config extends', () => { + it('merges the parent one level deep, the child winning', async () => { + const cwd = await makeProject(EXTENDED) + const config = await load(cwd) + + expect(config!.resolved.sampling).toEqual({ rates: { info: 50, debug: 0 } }) + expect(config!.resolved.redact).toEqual({ paths: ['user.password', 'card.number'] }) + expect([...config!.map.off]).toEqual(['error-catalog']) + expect(config!.map.minScore).toBe(90) + expect(config!.extends?.specifier).toBe('./evlog.preset') + expect(config!.sources.get('map.minScore')).toBe('evlog.preset.ts:6') + expect(config!.sources.get('map.rules.audit')).toBe('evlog.config.ts:8') + }) + + it('extends a preset published as a package', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'node_modules/@acme/evlog-preset/package.json': JSON.stringify({ + name: '@acme/evlog-preset', + type: 'module', + exports: { '.': { types: './dist/index.d.mts', import: './dist/index.mjs' } }, + }), + 'node_modules/@acme/evlog-preset/dist/index.mjs': `import { defineEvlog } from 'evlog' + +const preset = defineEvlog({ + map: { rules: { 'ai-logging': 'off' }, ignore: ['server/tasks/**'] }, +}) + +export { preset as default } +`, + 'evlog.config.ts': `import preset from '@acme/evlog-preset' +import { defineEvlog } from 'evlog' + +export default defineEvlog({ extends: preset, map: { minScore: 70 } }) +`, + }) + const config = await load(cwd) + + expect([...config!.map.off]).toEqual(['ai-logging']) + expect(config!.map.ignore).toEqual(['server/tasks/**']) + expect(config!.map.minScore).toBe(70) + expect(config!.extends?.specifier).toBe('@acme/evlog-preset') + expect(config!.extends?.file.endsWith(join('dist', 'index.mjs'))).toBe(true) + }) + + it.each([ + ['default before import', { default: './dist/default.mjs', import: './dist/import.mjs' }, 'default.mjs', 60], + ['require before import', { require: './dist/index.cjs', import: './dist/import.mjs' }, 'import.mjs', 70], + ])('resolves package exports in the order the package declares them: %s', async (_, conditions, file, minScore) => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'node_modules/@acme/evlog-preset/package.json': JSON.stringify({ + name: '@acme/evlog-preset', + type: 'module', + exports: { '.': conditions }, + }), + 'node_modules/@acme/evlog-preset/dist/default.mjs': 'export default { map: { minScore: 60 } }\n', + 'node_modules/@acme/evlog-preset/dist/import.mjs': 'export default { map: { minScore: 70 } }\n', + 'node_modules/@acme/evlog-preset/dist/index.cjs': 'module.exports = { map: { minScore: 80 } }\n', + 'evlog.config.ts': 'import preset from \'@acme/evlog-preset\'\n\nexport default { extends: preset }\n', + }) + const config = await load(cwd) + + expect(config!.extends?.file.endsWith(join('dist', file))).toBe(true) + expect(config!.map.minScore).toBe(minScore) + }) + + it('treats a package whose package.json does not parse as unresolved', async () => { + const cwd = await makeProject(MALFORMED_PRESET) + + await expect(load(cwd)).rejects.toMatchObject({ + code: cliErrors.CONFIG_EXTENDS_NOT_FOUND.code, + message: 'Cannot resolve "@acme/evlog-preset", extended in evlog.config.ts:3', + }) + }) + + it('refuses a parent that extends another config', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.base.ts': 'export default { map: { minScore: 50 } }\n', + 'evlog.preset.ts': 'import base from \'./evlog.base\'\n\nexport default { extends: base, map: { minScore: 60 } }\n', + 'evlog.config.ts': 'import preset from \'./evlog.preset\'\n\nexport default { extends: preset }\n', + }) + + await expect(load(cwd)).rejects.toMatchObject({ + code: cliErrors.CONFIG_EXTENDS_DEPTH.code, + message: './evlog.preset extends another config, so evlog.config.ts:3 cannot extend it', + }) + }) + + it('refuses a path where the config itself is expected', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.preset.ts': 'export default { map: { minScore: 50 } }\n', + 'evlog.config.ts': 'export default { extends: \'./evlog.preset\' }\n', + }) + + await expect(load(cwd)).rejects.toMatchObject({ + code: cliErrors.CONFIG_EXTENDS_STRING.code, + message: 'extends in evlog.config.ts:1 is the path \'./evlog.preset\', not a config', + }) + }) + + it('names the import it cannot resolve', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': 'import preset from \'@acme/missing\'\n\nexport default { extends: preset }\n', + }) + + await expect(load(cwd)).rejects.toMatchObject({ + code: cliErrors.CONFIG_EXTENDS_NOT_FOUND.code, + message: 'Cannot resolve "@acme/missing", extended in evlog.config.ts:3', + }) + }) +}) + +describe('evlog.config validation', () => { + it.each([ + ['map: { rules: { \'error-catalogue\': \'off\' } }', 'map.rules.error-catalogue'], + ['map: { rules: { audit: false } }', 'map.rules.audit'], + ['map: { rules: { \'wide-event\': \'off\' } }', 'map.ignore'], + ['map: { minScore: 101 }', 'map.minScore'], + ['map: { ignore: \'server/**\' }', 'map.ignore'], + ['map: { baseline: false }', 'map.baseline'], + ['map: { exclude: [] }', 'map.exclude'], + ['logs: { limit: 0 }', 'logs.limit'], + ])('refuses %s', async (body, mentioned) => { + const cwd = await makeProject({ 'package.json': PACKAGE, 'evlog.config.ts': `export default { ${body} }\n` }) + + const error = await load(cwd).catch((thrown: unknown) => thrown) + expect(error).toMatchObject({ code: cliErrors.CONFIG_INVALID.code }) + expect((error as Error).message).toContain(mentioned) + expect((error as Error).message).toContain('evlog.config.ts:1') + }) + + it('refuses a map or logs setting computed at runtime', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': 'export default {\n map: { minScore: Number(process.env.MIN_SCORE) },\n}\n', + }) + + await expect(load(cwd)).rejects.toMatchObject({ + code: cliErrors.CONFIG_NOT_STATIC.code, + message: 'map.minScore in evlog.config.ts:2 is computed at runtime', + }) + }) + + it('keeps settings computed at runtime outside map and logs', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `import { createAxiomDrain } from 'evlog/axiom' +import { defineEvlog } from 'evlog' + +export default defineEvlog({ + service: process.env.SERVICE_NAME, + drain: createAxiomDrain(), + redact: { patterns: [/acct_\\w+/g] }, + map: { minScore: 50 }, +}) +`, + }) + const config = await load(cwd) + + expect(config!.resolved.service).toEqual(new RuntimeValue('process.env.SERVICE_NAME')) + expect(config!.resolved.drain).toEqual(new RuntimeValue('createAxiomDrain()')) + expect((config!.resolved.redact as { patterns: RegExp[] }).patterns).toEqual([/acct_\w+/g]) + expect(config!.map.minScore).toBe(50) + }) + + it('refuses a file without a default export', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE, 'evlog.config.ts': 'export const config = {}\n' }) + await expect(load(cwd)).rejects.toMatchObject({ code: cliErrors.CONFIG_NO_EXPORT.code }) + }) + + it('refuses a file that does not parse', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE, 'evlog.config.ts': 'export default {\n' }) + await expect(load(cwd)).rejects.toMatchObject({ code: cliErrors.CONFIG_PARSE_FAILED.code }) + }) +}) + +describe('evlog map with evlog.config', () => { + it('turns a check off for every entry point and leaves ignored entry points out', async () => { + const cwd = await copyFixture('nuxt-basic') + const before = await runMap(fakeContext(cwd), undefined, { noWrite: true }) + const docs = before.scan.map.routes.filter(route => route.file.startsWith('server/api/docs/')) + expect(docs.length).toBeGreaterThan(0) + expect(before.scan.map.routes.some(route => route.checks['structured-errors']?.status === 'fail')).toBe(true) + + await writeFile(join(cwd, 'evlog.config.ts'), `export default { + map: { rules: { 'structured-errors': 'off' }, ignore: ['server/api/docs/**'] }, +} +`) + const after = await runMap(fakeContext(cwd), undefined, { noWrite: true }) + + expect(after.scan.ignored).toBe(docs.length) + expect(after.scan.map.routes).toHaveLength(before.scan.map.routes.length - docs.length) + expect(after.scan.map.routes.some(route => route.file.startsWith('server/api/docs/'))).toBe(false) + for (const route of after.scan.map.routes) { + if (route.checks['structured-errors']) { + expect(route.checks['structured-errors']).toEqual({ status: 'n/a', message: RULE_OFF_MESSAGE }) + } + } + expect(formatMapReport(fakeContext(cwd), after)).toContain( + `evlog.config.ts: structured-errors off, ${docs.length} entry point${docs.length === 1 ? '' : 's'} ignored`, + ) + }) + + it('gates on the config minScore, and --min-score wins over it', async () => { + const cwd = await copyFixture('nuxt-basic') + await writeFile(join(cwd, 'evlog.config.ts'), 'export default { map: { minScore: 100 } }\n') + const out = silence() + + await runCommand(map, { rawArgs: ['--cwd', cwd, '--no-header', '--no-write'] }) + expect(process.exitCode).toBe(1) + expect(out.stderr()).toMatch(/score \d+ is below map\.minScore 100/) + + process.exitCode = undefined + await runCommand(map, { rawArgs: ['--cwd', cwd, '--no-header', '--no-write', '--min-score', '0'] }) + expect(process.exitCode).toBeUndefined() + expect(out.stderr()).toMatch(/score \d+ meets --min-score 0/) + }) + + it('names map.minScore in the GitHub annotation it fails on', async () => { + const cwd = await copyFixture('nuxt-basic') + await writeFile(join(cwd, 'evlog.config.ts'), 'export default { map: { minScore: 100 } }\n') + const out = silence() + + await runCommand(map, { rawArgs: ['--cwd', cwd, '--format', 'github', '--no-header', '--no-write'] }) + + expect(process.exitCode).toBe(1) + expect(out.stdout()).toMatch(/^::error title=evlog map::score \d+\/100 .*below map\.minScore 100$/m) + }) + + it('reports the entry points map.ignore left out in --json', async () => { + const cwd = await copyFixture('nuxt-basic') + await writeFile(join(cwd, 'evlog.config.ts'), 'export default { map: { ignore: [\'server/**\', \'pages/**\'] } }\n') + const out = silence() + + await runCommand(map, { rawArgs: ['--cwd', cwd, '--json', '--no-header', '--no-write'] }) + + const raw = JSON.parse(out.stdout()) as { ignored: number, map: { routes: unknown[] } } + expect(raw.map.routes).toHaveLength(0) + expect(raw.ignored).toBeGreaterThan(0) + }) + + it('fails on an invalid config before it scans anything', async () => { + const cwd = await copyFixture('nuxt-basic') + await writeFile(join(cwd, 'evlog.config.ts'), 'export default { map: { minScore: \'high\' } }\n') + const out = silence() + + await runCommand(map, { rawArgs: ['--cwd', cwd, '--json', '--no-header'] }) + + expect(process.exitCode).toBe(1) + expect(out.stdout()).toContain('CONFIG_INVALID') + }) +}) + +describe('evlog logs with evlog.config', () => { + const line = (index: number): string => JSON.stringify({ + timestamp: `2026-10-01T10:0${index}:00.000Z`, + level: 'info', + method: 'GET', + path: `/api/items/${index}`, + status: 200, + durationMs: 5, + requestId: `00000000-0000-4000-8000-00000000000${index}`, + }) + + it('reads logs.dir and logs.limit, the flags winning', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': 'export default { logs: { dir: \'var/events\', limit: 2 } }\n', + 'var/events/2026-10-01.jsonl': `${[1, 2, 3, 4].map(line).join('\n')}\n`, + }) + const now = new Date('2026-10-01T12:00:00.000Z') + + expect((await runLogs(fakeContext(cwd), {}, { now })).events).toHaveLength(2) + expect((await runLogs(fakeContext(cwd), { limit: '3' }, { now })).events).toHaveLength(3) + await expect(runLogs(fakeContext(cwd), {}, { now, dir: join(cwd, 'elsewhere') })).rejects.toThrow() + }) +}) + +describe('evlog doctor with evlog.config', () => { + it('names the config and what it extends', async () => { + const cwd = await makeProject(EXTENDED) + const result = await runDoctor(fakeContext(cwd)) + + expect(result.checks.find(check => check.id === 'config')).toEqual({ + id: 'config', + status: 'ok', + message: 'evlog.config.ts', + hint: 'extends ./evlog.preset', + }) + expect(result.sections.find(section => section.title === 'EVLOG')?.checks.map(check => check.id)).toContain('config') + }) + + it('fails the check with the reader\'s error and fix', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE, 'evlog.config.ts': 'export default { logs: { limit: -1 } }\n' }) + const result = await runDoctor(fakeContext(cwd)) + + expect(result.checks.find(check => check.id === 'config')).toMatchObject({ + status: 'fail', + message: 'logs.limit in evlog.config.ts:1 must be a whole number of 1 or more', + }) + expect(result.summary.fail).toBeGreaterThan(0) + }) + + it('fails the check on a preset whose package.json does not parse', async () => { + const cwd = await makeProject(MALFORMED_PRESET) + const result = await runDoctor(fakeContext(cwd)) + + expect(result.checks.find(check => check.id === 'config')).toMatchObject({ + status: 'fail', + message: 'Cannot resolve "@acme/evlog-preset", extended in evlog.config.ts:3', + }) + }) + + it('says nothing when there is no config', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE }) + const result = await runDoctor(fakeContext(cwd)) + expect(result.checks.find(check => check.id === 'config')).toBeUndefined() + }) +}) + +describe('evlog config', () => { + it('splits CLI and app settings, each with where it is written', async () => { + const cwd = await makeProject(EXTENDED) + const result = await runConfig(fakeContext(cwd)) + + expect(result.file).toBe('evlog.config.ts') + expect(result.extends).toEqual({ specifier: './evlog.preset', file: 'evlog.preset.ts' }) + expect(result.cli).toContainEqual({ path: ['map', 'minScore'], value: 90, source: 'evlog.preset.ts:6' }) + expect(result.cli).toContainEqual({ path: ['map', 'rules', 'audit'], value: 'on', source: 'evlog.config.ts:8' }) + expect(result.app).toContainEqual({ path: ['sampling', 'rates', 'info'], value: 50, source: 'evlog.config.ts:6' }) + expect(result.app).toContainEqual({ path: ['sampling', 'rates', 'debug'], value: 0, source: 'evlog.preset.ts:4' }) + }) + + it('credits each item of an appended list to the file that adds it', async () => { + const cwd = await makeProject(EXTENDED) + const result = await runConfig(fakeContext(cwd)) + + expect(result.app).toContainEqual({ path: ['redact', 'paths', 0], value: 'user.password', source: 'evlog.preset.ts:5' }) + expect(result.app).toContainEqual({ path: ['redact', 'paths', 1], value: 'card.number', source: 'evlog.config.ts:7' }) + }) + + it('lays the settings out by what they do, each with where it is written', async () => { + const cwd = await makeProject(EXTENDED) + const report = formatConfigReport(fakeContext(cwd), await runConfig(fakeContext(cwd))) + + expect(report).toContain('evlog.config.ts · extends ./evlog.preset → evlog.preset.ts') + expect(report).toMatch(/^Sampling$/m) + expect(report).toMatch(/^ {2}info +50% +evlog\.config\.ts:6$/m) + expect(report).toMatch(/^ {2}debug +0% +evlog\.preset\.ts:4$/m) + expect(report).toMatch(/^Redaction$/m) + expect(report).toMatch(/^ {2}paths +user\.password +evlog\.preset\.ts:5$/m) + expect(report).toMatch(/^ {12}card\.number +evlog\.config\.ts:7$/m) + expect(report).toMatch(/^CLI · read by evlog map and evlog logs$/m) + expect(report).toMatch(/^ {2}map\.minScore +90 +evlog\.preset\.ts:6$/m) + }) + + it('fills in what evlog does when the file is silent', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': 'export default { service: \'shop\' }\n', + }) + const report = formatConfigReport(fakeContext(cwd), await runConfig(fakeContext(cwd))) + + expect(report).toMatch(/^Service$/m) + expect(report).toMatch(/^ {2}service +shop +evlog\.config\.ts:1$/m) + expect(report).toMatch(/^ {2}environment +from NODE_ENV +default$/m) + expect(report).toMatch(/^ {2}trace +0% +default$/m) + expect(report).toMatch(/^ {2}fatal +always +default$/m) + expect(report).toMatch(/^ {2}redact +on in production, off in dev +default$/m) + expect(report).toMatch(/^ {2}drain +none in this file$/m) + }) + + it('lists routes in the order they match, a dotted glob kept whole', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `export default { + service: 'shop', + routes: { + '/api/billing/v1.2/**': { service: 'billing' }, + '/api/**': { service: 'api' }, + }, +} +`, + }) + const result = await runConfig(fakeContext(cwd)) + const report = formatConfigReport(fakeContext(cwd), result) + + expect(configJson(result).app).toContainEqual({ path: 'routes[\'/api/billing/v1.2/**\'].service', value: 'billing', source: 'evlog.config.ts:4' }) + expect(report).toMatch(/^Services · first matching route wins$/m) + expect(report).toMatch(/^ {2}\/api\/billing\/v1\.2\/\*\* +billing +evlog\.config\.ts:4\n {2}\/api\/\*\* +api +evlog\.config\.ts:5\n {2}any other route +shop +evlog\.config\.ts:2$/m) + }) + + it('shows sampling per level and warns when errors or fatal rates are lost', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `export default { + minLevel: 'info', + sampling: { + rates: { error: 10, fatal: 50 }, + keep: [{ status: 400 }, { duration: 1000, path: '/api/**' }], + }, +} +`, + }) + const report = formatConfigReport(fakeContext(cwd), await runConfig(fakeContext(cwd))) + + expect(report).toMatch(/^ {2}error +10% +evlog\.config\.ts:4\n +⚠ 90% of errors are dropped$/m) + expect(report).toContain('⚠ rates.fatal at evlog.config.ts:4 is ignored, fatal events are always kept') + expect(report).toMatch(/^ {2}keep if +status ≥ 400 +evlog\.config\.ts:5$/m) + expect(report).toMatch(/^ {2}or +duration ≥ 1000ms or path \/api\/\*\* +evlog\.config\.ts:5$/m) + }) + + it('keeps the rates below minLevel, which only gates the global log API', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `export default { + minLevel: 'warn', + sampling: { rates: { info: 50 } }, +} +`, + }) + const report = formatConfigReport(fakeContext(cwd), await runConfig(fakeContext(cwd))) + + expect(report).toMatch(/^ {2}debug +100% +default$/m) + expect(report).toMatch(/^ {2}info +50% +evlog\.config\.ts:3$/m) + expect(report).toMatch(/^ {2}minLevel +warn +evlog\.config\.ts:2\n +applies to the global log API only, not request wide events$/m) + }) + + it('names runtime values by what they call', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': `import { createAxiomDrain } from 'evlog/axiom' +import { createUserAgentEnricher } from 'evlog/enrichers' + +export default { + drain: createAxiomDrain({ dataset: 'logs' }), + enrich: [createUserAgentEnricher(), ctx => ctx], + keep: ctx => ctx, +} +`, + }) + const report = formatConfigReport(fakeContext(cwd), await runConfig(fakeContext(cwd))) + + expect(report).toMatch(/^ {2}drain +createAxiomDrain\(…\) +evlog\.config\.ts:5$/m) + expect(report).toMatch(/^ {2}enrich +\[createUserAgentEnricher\(\), function\] +evlog\.config\.ts:6$/m) + expect(report).toMatch(/^ {2}keep +function +evlog\.config\.ts:7$/m) + }) + + it('writes regexps and runtime values in a JSON-safe form', async () => { + const cwd = await makeProject({ + 'package.json': PACKAGE, + 'evlog.config.ts': 'import { createAxiomDrain } from \'evlog/axiom\'\n\nexport default { drain: createAxiomDrain(), redact: { patterns: [/acct_\\w+/g] } }\n', + }) + const json = JSON.parse(JSON.stringify(configJson(await runConfig(fakeContext(cwd))))) as { app: { path: string, value: unknown }[] } + + expect(json.app).toContainEqual(expect.objectContaining({ path: 'drain', value: { runtime: 'createAxiomDrain()' } })) + expect(json.app).toContainEqual(expect.objectContaining({ path: 'redact.patterns[0]', value: { regexp: '/acct_\\w+/g' } })) + }) + + it('says where it looked when there is no config', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE }) + const result = await runConfig(fakeContext(cwd)) + + expect(result.file).toBeNull() + expect(formatConfigReport(fakeContext(cwd), result)).toContain('No evlog.config from .') + }) + + it('exits 1 with the catalog error on a config it cannot read', async () => { + const cwd = await makeProject({ 'package.json': PACKAGE, 'evlog.config.ts': 'export default { map: { minScore: 101 } }\n' }) + const out = silence() + + await runCommand(configCommand, { rawArgs: ['--cwd', cwd, '--json', '--no-header'] }) + + expect(process.exitCode).toBe(1) + expect(JSON.parse(out.stdout())).toMatchObject({ error: { code: cliErrors.CONFIG_INVALID.code } }) + }) +}) diff --git a/packages/cli/test/map.command.test.ts b/packages/cli/test/map.command.test.ts index a3324258..c313da0a 100644 --- a/packages/cli/test/map.command.test.ts +++ b/packages/cli/test/map.command.test.ts @@ -367,6 +367,7 @@ describe('map command', () => { environment: string map: { version: number, framework: string, routes: unknown[] } summary: { instrumented: number, partial: number, dark: number, exempt: number } + ignored: number mapPath: string | null } @@ -377,6 +378,7 @@ describe('map command', () => { expect(raw.mapPath).toBeNull() expect(raw.summary.instrumented + raw.summary.partial + raw.summary.dark + raw.summary.exempt) .toBe(raw.map.routes.length) + expect(raw.ignored).toBe(0) }) it('exits 1 under --min-score when there are no entry points to score', async () => { diff --git a/packages/cli/test/map/telemetry.test.ts b/packages/cli/test/map/telemetry.test.ts index 3da3820f..bdb3b6df 100644 --- a/packages/cli/test/map/telemetry.test.ts +++ b/packages/cli/test/map/telemetry.test.ts @@ -44,6 +44,7 @@ function scan(overrides: Partial = {}): ScanResult { project: {} as ScanResult['project'], suggestions: [], warnings: [], + ignored: 0, summary: { instrumented: 1, partial: 0, dark: 0, exempt: 0, suppressedChecks: 0 }, ...overrides, } @@ -57,6 +58,7 @@ function fields(overrides: Partial[0]> = { baseline: null, view: 'summary', wrote: true, + config: null, ...overrides, }) } diff --git a/skills/review-logging-patterns/SKILL.md b/skills/review-logging-patterns/SKILL.md index ab86e209..3e4d0230 100644 --- a/skills/review-logging-patterns/SKILL.md +++ b/skills/review-logging-patterns/SKILL.md @@ -56,7 +56,7 @@ npm install evlog ## Use the CLI (recommended on Nuxt, Nitro, Next.js, TanStack Start, Hono, Express, Fastify) -The `evlog` executable ships with the `evlog` package: `pnpm evlog`, `npx evlog`, `bunx evlog`. It runs `@evlog/cli` when installed and fetches it on demand otherwise (no dependency added to the project), so it works on a project with nothing yet. Early but worth trying. It reads the project on disk (no traffic, no config). On the seven supported frameworks it covers the whole loop: **wire evlog in** (`init`), **score coverage** (`map`), **lock the score in CI** (`--min-score`, `--baseline`). If the CLI is unavailable, the framework has no adapter yet, or the user declines, continue with the manual sections below; the skill does not depend on it. **Ask before installing anything**. +The `evlog` executable ships with the `evlog` package: `pnpm evlog`, `npx evlog`, `bunx evlog`. It runs `@evlog/cli` when installed and fetches it on demand otherwise (no dependency added to the project), so it works on a project with nothing yet. Early but worth trying. It reads the project on disk (no traffic, no config needed). On the seven supported frameworks it covers the whole loop: **wire evlog in** (`init`), **score coverage** (`map`), **lock the score in CI** (`--min-score`, `--baseline`). If the CLI is unavailable, the framework has no adapter yet, or the user declines, continue with the manual sections below; the skill does not depend on it. **Ask before installing anything**. ### 1. Setup: `evlog init` @@ -108,6 +108,8 @@ pnpm exec evlog map --baseline # ratchet: exits 1 if this PR made things w `--baseline` compares the fresh scan against the committed `evlog.map.json`, **per entry point and per requirement**, so a refactor that instruments one route and breaks another fails even if the total score is unchanged. Disabling a passing check with a comment counts as a regression too. New uninstrumented routes are listed as `NEW AND DARK` without failing. Workflow: commit `evlog.map.json` once, add the `--baseline` run to CI, then re-run `map` without `--baseline` to accept an intentional change. Docs: https://www.evlog.dev/cli/ci +When the project has an `evlog.config.ts`, the gate can live there instead of in the CI step: `map: { minScore: 80, baseline: true }` makes a bare `evlog map` gate the same way, and a flag still wins. The same `map` block turns checks off for every entry point (`rules: { 'error-catalog': 'off' }`) and leaves entry points out by file glob (`ignore`). The CLI reads these values without running the file, so they must be literals. Docs: https://www.evlog.dev/cli/config + Early days: adapters and rules are still evolving; expect scores to move between releases. Docs: https://www.evlog.dev/cli/map · Rules: https://www.evlog.dev/cli/rules ---