feat(api)!: Allow provider connections to be dynamically managed #3892
Conversation
meta-cla
bot
added
the
CLA Signed
raghotham
changed the title
|
@raghotham FYI here's the docs for access control: https://llamastack.github.io/docs/distributions/configuration#access-control |
raghotham
force-pushed
the
provider-apis
branch
from
fcab15e to
13b6f3d
Compare
raghotham
changed the title
raghotham
requested review from
ashwinb,
bbrowning,
ehhuang,
franciscojavierarceo,
hardikjshah,
leseb,
mattf,
reluctantfuturist,
slekkala1,
terrytangyuan and
yanxi0830
as code owners
ashwinb
changed the title
| ... | ||
|
|
||
| @webmethod(route="/admin/providers/{api}/{provider_id}/test", method="POST", level=LLAMA_STACK_API_V1) | ||
| async def test_provider_connection(self, api: str, provider_id: str) -> TestProviderConnectionResponse: |
There was a problem hiding this comment.
would you want this to be GET instead of POST then - which would then mean that the health of each connection has to be periodically checked and saved somewhere?
There was a problem hiding this comment.
@raghotham yes GET would be a bit more natural right...
| """ | ||
| ... | ||
|
|
||
| @webmethod(route="/admin/providers/{api}/{provider_id}", method="PUT", level=LLAMA_STACK_API_V1) |
There was a problem hiding this comment.
should we level these as alpha first?
There was a problem hiding this comment.
fine with making it alpha
| :param api: API namespace this provider implements (e.g., 'inference', 'vector_io'). | ||
| :param provider_id: Unique identifier for this provider instance. | ||
| :param provider_type: Provider type (e.g., 'remote::openai'). | ||
| :param config: Provider configuration (API keys, endpoints, etc.). |
There was a problem hiding this comment.
I haven't looked at the implementation carefully yet, but is there a security aspect to consider here? we have code which loads external modules -- can someone reference external python paths, etc. via this API?
There was a problem hiding this comment.
I'm not sure what our security posture is when there are external providers (can we have inline external providers?). These providers can take arbitrary payloads in their APIs and execute them.
There was a problem hiding this comment.
@raghotham I think an attacker being able to inject code from remote is very different than a server admin deciding to include some external code right?
There was a problem hiding this comment.
true - i was thinking about what guarantees llama stack can provide. Maybe we can just say that the config passed in to the external providers would be typed checked against a config schema defined by the external provider author. Is there a stronger claim we can make?
4 participants
What does this PR do?
Closes #3809
Backward incompatible change: Changes the ambiguous /v1/providers/{provider-id} to mean /v1/providers/{api} and returns a list of provider infos.
Test Plan
Added unit and integration tests