Skip to content

Add metrics manifest for dashboard and rule change tracking#367

Open
mhotan wants to merge 1 commit intomainfrom
mike/metrics-manifest
Open

Add metrics manifest for dashboard and rule change tracking#367
mhotan wants to merge 1 commit intomainfrom
mike/metrics-manifest

Conversation

@mhotan
Copy link
Copy Markdown
Contributor

@mhotan mhotan commented Apr 29, 2026

Summary

  • Adds metrics-manifest.yaml — an auto-generated list of all metric names from shipped Grafana dashboards and PrometheusRule CRDs
  • Adds scripts/extract-metrics.py to extract metrics from dashboard JSON and PrometheusRule templates
  • Adds make generate-metrics-manifest target
  • Updates CLAUDE.md with instructions for maintaining the metrics glossary

Why

When dashboard panels or PrometheusRules change (metrics added, removed, renamed), the customer-facing metrics glossary in unionai-docs needs to be updated. Today there's no signal in PRs that this is needed.

The manifest makes metric changes visible in PR diffs, so Copilot and reviewers can flag when docs updates are needed.

How it works

  1. scripts/extract-metrics.py parses dashboard JSON expr fields and PrometheusRule record:/alert:/expr: blocks
  2. Outputs a sorted YAML manifest of all metric names, recording rules, and alert names
  3. When a PR modifies dashboards or rules, the contributor runs make generate-metrics-manifest and commits the diff
  4. Reviewers (and Copilot) see exactly which metrics changed

Test plan

  • make generate-metrics-manifest produces clean output
  • Verify manifest covers all metrics in both dashboards and both PrometheusRule files

🤖 Generated with Claude Code

Adds a generated metrics-manifest.yaml that lists all metric names
from Grafana dashboards and PrometheusRule CRDs. Changes to this
file in PRs signal that the customer-facing metrics glossary in
unionai-docs may need updating.

- scripts/extract-metrics.py: extracts metrics from dashboard JSON
  and PrometheusRule templates
- make generate-metrics-manifest: regeneration target
- CLAUDE.md: instructions for maintaining the glossary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@aviator-app
Copy link
Copy Markdown
Contributor

aviator-app Bot commented Apr 29, 2026

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue label.


See the real-time status of this PR on the Aviator webapp.
Use the Aviator Chrome Extension to see the status of your PR within GitHub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants