Skip to content

[monitor-api] fix: keep validation metrics compatible with Prometheus 2.x - #189

Open
RainieLLM wants to merge 1 commit into
verl-project:mainfrom
RainieLLM:codex/fix-prometheus-client-compatibility
Open

RainieLLM wants to merge 1 commit into
verl-project:mainfrom
RainieLLM:codex/fix-prometheus-client-compatibility

Conversation

@RainieLLM

Copy link
Copy Markdown

What does this PR do?

With prometheus-client 0.22.1, names such as val-aux_openai_gsm8k_reward_mean@1 cause the default Prometheus 2.54.1 scrape to fail. Require prometheus_client>=0.23.1 in both dependency files so the client's content negotiation escapes names for legacy scrapers. The new regression test covers text and OpenMetrics responses and checks that normal metrics and label values remain intact.

Fixes #184.

Checklist Before Starting

  • Checked the open PR list and related PRs. No matching fix found.
  • Used the required PR title format.

Test

  • pytest -q tests/monitor/ut: 90 passed, 1 skipped with both prometheus-client 0.23.1 and 0.26.0 on macOS / Python 3.12.
  • Both new regression cases fail with 0.22.1.
  • A real Prometheus 2.54.1 scrape against RL-Insight's metrics server: down with 0.22.1, up with 0.23.1. After the upgrade, queries returned train_loss=0.5 and the escaped validation metric =0.17.
  • pre-commit run --all-files --show-diff-on-failure and git diff --check passed.

Checklist Before Submitting

  • Read the contribution guide.
  • Ran pre-commit checks.
  • Added regression tests to the existing monitor unit suite, which is already covered by CI.
  • Documentation: no API or configuration changes; the dependency requirement is declared in both installation files.

Codex assisted with the change and validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]Metric names like val-aux/.../mean@1 break the whole Prometheus scrape

1 participant