Skip to content

fix: report the failure stage and a real version in action telemetry - #9

Merged
HugoRCD merged 2 commits into
mainfrom
fix/telemetry-error-stage
Oct 6, 2026
Merged

HugoRCD merged 2 commits into
mainfrom
fix/telemetry-error-stage

Conversation

@evlogai

@evlogai evlogai Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

What

Two telemetry fixes so a red run in the dashboard actually says something.

Failures carry the stage where the action died. When the action's work throws, the error is now tagged with the stage it reached and errorStage is set on the telemetry event before the fixed ACTION_EXECUTION_FAILED code is reported. A failure in baseline checkout, in the CLI run, in the check run or in the comment step no longer all look identical. The original error is still what the caller sees (unchanged), so the CI log keeps showing the real message. The ingest allowlist needs errorStage on the server side: evloghq/evlog#786.

The version is real. GITHUB_ACTION_REF is empty for composite actions on real runners, so all 9 action runs this week reported version: "local". resolveVersion() now prefers GITHUB_ACTION_REF, falls back to the version field of the action's own package.json (the runner checks the action out at the pinned ref into GITHUB_ACTION_PATH), and only then says local. package.json now carries "version": "1.3.0", matching the latest tag; releases must keep bumping it or the fallback goes stale.

Evidence

The failed run that prompted this (id 331131, 2026-10-06 09:46 UTC) recorded only:

{ "outcome": "error", "errorCode": "ACTION_EXECUTION_FAILED", "custom": { "baselineMode": "base", "packages": 1 } }

With this change, the same failure records errorStage (for example "baseline" if git fetch of the base died) and a version like "1.3.0" instead of "local". New tests cover both paths; full test run:

node --test test/*.test.mjs
88 tests, 88 pass, 0 fail

npm run check passes.

ELI5

The action reports every failure to a dashboard as just "something failed", and calls itself version "local" even in production. This makes it say which step broke (setup, scanning, or reporting) and which release it is, so the next failure can be diagnosed from the dashboard without access to the repo's own logs.

evlogai Bot added 2 commits October 6, 2026 17:04
The telemetry layer replaced every action error with a fixed
ACTION_EXECUTION_FAILED code, so the ingest could not tell where a run
broke. main.mjs now stamps errors with the stage they came from
(inputs, baseline, cli, check, comment) and the telemetry layer passes
that stage to the SDK as errorStage, unknown when the error carries
none.
GitHub runners leave GITHUB_ACTION_REF empty for composite actions, so
every production run reported version local. resolveVersion now falls
back to the package.json the runner checked out at GITHUB_ACTION_PATH,
and the action carries its own version (1.3.0, the latest release tag;
bump it at release time) so the fallback stays meaningful.
@evlogai
evlogai Bot requested a review from HugoRCD October 6, 2026 17:07
@coderabbitai

coderabbitai Bot commented Oct 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0e970c7d-1571-4e71-bfe3-62135025e0d9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown

Observability score · 64

needs work across 3 packages · unchanged against main · no regression

Package Score Δ Instrumented Partial Dark Gate
test/fixtures/express-app (express) 100 excellent 0 2 0 0 passed
test/fixtures/hono-app (hono) 100 excellent 0 2 0 0 passed
test/fixtures/nuxt-app (nuxt) 64 needs work 0 2 0 2 passed

Note

No min-score set. Add min-score: 64 to keep today's score from dropping.

Fix first (2)

evlog map v0.8.0 · how the score works · what each check expects

@HugoRCD
HugoRCD merged commit 9e0020f into main Oct 6, 2026
11 checks passed
@HugoRCD
HugoRCD deleted the fix/telemetry-error-stage branch October 6, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant