Skip to content

feat: configurable model pricing source#971

Open
Spherrrical wants to merge 1 commit into
mainfrom
musa/customizable-pricing-source
Open

feat: configurable model pricing source#971
Spherrrical wants to merge 1 commit into
mainfrom
musa/customizable-pricing-source

Conversation

@Spherrrical

Copy link
Copy Markdown
Collaborator

Summary

Makes Plano's model pricing source configurable. Adds models.dev as a built-in cost provider alongside DigitalOcean (each with an optional url override), honored by both brightstaff routing and the planoai obs cost column. DigitalOcean remains the default, and the source is swappable to models.dev or any endpoint exposing a supported pricing structure.

Config

model_metrics_sources:
  - type: cost
    provider: models.dev          # models.dev | digitalocean (default)
    url: https://models.dev/api.json   # optional; sensible default per provider
    refresh_interval: 3600             # optional, seconds
    model_aliases:                     # optional: catalog key -> Plano model name
      openai/gpt-oss-120b: openai/gpt-4o

Changes

  • crates/common/src/configuration.rsModelsDev variant on CostProvider (#[serde(rename = "models.dev")]) + url: Option<String> on CostMetricsConfig.
  • crates/brightstaff/src/router/model_metrics.rs — provider-aware fetch_cost_pricing dispatcher, per-provider default URLs + url override, and fetch_models_dev_pricing (composes provider/model keys to match Plano routing names; sums input+output per-million for ranking).
  • config/plano_config_schema.yamlmodels.dev in the provider enum + optional url.
  • cli/planoai/obs/pricing.py_parse_models_dev_pricing (per-million → per-token, cache_read → cached rate, 0-rate skip) and PricingCatalog.fetch(provider, url) dispatch; DigitalOcean default.
  • cli/planoai/obs_cmd.py — reads the cost source from config with --config / --pricing-provider / --pricing-url overrides; startup log names the active provider.
  • Docs — new "Cost- and latency-aware selection" section in docs/source/guides/llm_router.rst (DigitalOcean default, swappability, supported structures, model_aliases, obs cost column); model_metrics_sources example added to the full reference config (+ regenerated _rendered.yaml).

Test plan

  • cargo test -p common -p brightstaff --lib (incl. new models.dev parser + config deserialize tests)
  • cargo fmt --check and cargo clippy -D warnings clean
  • pytest cli/test/test_obs_pricing.py (incl. new models.dev tests) — 10 passed; Black clean
  • config/validate_plano_config.sh — all configs validate, rendered reference matches
  • Sphinx docs build cleanly with the new section
  • Reviewer: confirm models.dev key matching against your actual routing_preferences model names (use model_aliases for version skew)

Add models.dev as a built-in cost provider alongside DigitalOcean, each
with an optional url override, honored by both brightstaff routing and the
planoai obs cost column. DigitalOcean remains the default.
@Spherrrical Spherrrical changed the title feat: configurable model pricing source (models.dev + DigitalOcean) feat: configurable model pricing source Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant