Skip to content

Conversation

mattf
Copy link
Collaborator

@mattf mattf commented Sep 8, 2025

What does this PR do?

update VertexAI inference provider to use openai-python for openai-compat functions

Test Plan

$ VERTEX_AI_PROJECT=... uv run llama stack build --image-type venv --providers inference=remote::vertexai --run
...
$ LLAMA_STACK_CONFIG=http://localhost:8321 uv run --group test pytest -v -ra --text-model vertexai/vertex_ai/gemini-2.5-flash tests/integration/inference/test_openai_completion.py
...

i don't have an account to test this. get_api_key may also need to be updated per https://cloud.google.com/vertex-ai/generative-ai/docs/start/openai

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 8, 2025
@mattf
Copy link
Collaborator Author

mattf commented Sep 8, 2025

@leseb here you go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant