feat(ai): add OpenRouter provider routing support #859
+28
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this is
Following #843 , this adds support for OpenRouter provider selection as described in the issue.
What it does
These changes allow custom models to specify which upstream providers OpenRouter should route requests to via the
openRouterRoutingfield in model definitions. Which model/provider combinations to use is completely up to you.Why it is so incomplete
Limitations
The idea is that for maintaining quality of inference and cache intact, you should stick to the single provider anyway, and routing between them is not desired.
How it works
The only introduced OpenRouter provider payload fields are:
only: list of provider slugs to exclusively useorder: list of provider slugs to try in orderBoth of these should be used in tandem to ensure provider selection. In my testing, sometimes OpenRouter ignores the
onlyparameter alone, but works whenonlyandorderare present.So, the idea is that you:
~/.pi/models.json... and the rest of the infrastructure would be handled by pi. It should correctly identify if you have OpenRouter API key for a model, it should work with /scoped-models and so on.
The only funky part of this code is the detection of a model by
baseUrland notprovider, but it's intentional: it allows to create custom "virtually provided models" with openrouter-bedrock, openrouter-novita-fp8 etc labels in the UI and they all get the routing and API key pickup seamlessly.