random_routing.md

Random Routing

Random routing sends a fixed percentage of traffic to a strong model and the rest to a weak model. It does not inspect the prompt.

Use it for A/B tests, benchmark baselines, and gradual traffic ramps. If routing should depend on task difficulty, use LLM Classifier Routing or Stage-Router Routing.

Algorithm

For each request, Switchyard flips a weighted coin:

• strong_probability: 0.0 sends all traffic to weak.
• strong_probability: 0.3 sends about 30% to strong.
• strong_probability: 0.5 is an even split.
• strong_probability: 1.0 sends all traffic to strong.

Each request is independent, so short runs may not match the configured percentage exactly. rng_seed is optional and only useful when you need a repeatable sequence for tests or benchmarks.

Behavior

Random routing is per request. A multi-turn conversation can move between models from turn to turn. If you need conversation-level stickiness, use Sticky Routing where it is supported, or select a direct tier model instead of the routing model.

Enable It

Use serve --config with a profile config:

targets:
  strong:
    model: openai/gpt-4o
    format: openai
    base_url: https://openrouter.ai/api/v1
    api_key: ${OPENROUTER_API_KEY}
  weak:
    model: openai/gpt-4o-mini
    format: openai
    base_url: https://openrouter.ai/api/v1
    api_key: ${OPENROUTER_API_KEY}

profiles:
  ab-test:
    type: random-routing
    strong: strong
    weak: weak
    strong_probability: 0.3

Run it with:

switchyard serve --config routes.yaml --port 4000

The profile id (ab-test) is the model id clients select when they want the weighted split.

For routing-profile YAML used by launchers, use type: random_routing under routes::

defaults:
  api_key: ${OPENROUTER_API_KEY}
  base_url: https://openrouter.ai/api/v1
  format: openai

routes:
  ab-test:
    type: random_routing
    strong:
      model: openai/gpt-4o
    weak:
      model: openai/gpt-4o-mini
    strong_probability: 0.3
    rng_seed: 42
    fallback_target_on_evict: strong