feat(cherry-pick): fixes for 0.3.1 release #3998
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…3924) ## Summary - `preserve_contexts_async_generator` left `PROVIDER_DATA_VAR` (and other context vars) populated after a streaming generator completed on HEAD~1, so the asyncio context for request N+1 started with request N's provider payload. - FastAPI dependencies and middleware execute before `request_provider_data_context` rebinds the header data, meaning auth/logging hooks could observe a prior tenant's credentials or treat them as authenticated. Traces and any background work that inspects the context outside the `with` block leak as well—this is a real security regression, not just a CLI artifact. - The wrapper now restores each tracked `ContextVar` to the value it held before the iteration (falling back to clearing when necessary) after every yield and when the generator terminates, so provider data is wiped while callers that set their own defaults keep them. ## Test Plan - `uv run pytest tests/unit/core/test_provider_data_context.py -q` - `uv run pytest tests/unit/distribution/test_context.py -q` Both suites fail on HEAD~1 and pass with this change.
ashwinb
requested review from
bbrowning,
ehhuang,
franciscojavierarceo,
hardikjshah,
leseb,
mattf,
raghotham,
reluctantfuturist,
slekkala1,
terrytangyuan and
yanxi0830
as code owners
meta-cla
bot
added
the
CLA Signed
We will be updating our release procedure to be more "normal" or "sane". We will - create release branches like normal people - land cherry-picks onto those branches - run releases off of those branches - no more "rc" branch pollution either Given that, this PR cleans things up a bit - Remove `-maint` suffix from release branch patterns in CI workflows - Update branch matching to `release-X.Y.x` format
Regenerate package-lock.json to match package.json dependency version.
Update all CI workflows and actions to use the standardized release branch naming convention 'release-X.Y.x' (e.g., release-0.3.x) instead of matching multiple numeric patterns. Changes: - Update workflow triggers to match 'release-[0-9]+.[0-9]+.x' pattern - Update bash regex in setup-test-environment to match new format - Simplifies branch pattern from multiple variants to single standard
|
For some reason, we are still not installing the correct |
|
Arghhh. |
The LLAMA_STACK_CLIENT_DIR variable was used by the old `llama stack build` command but is no longer used after switching to `llama stack list-deps`. Replace with direct `uv pip install --force-reinstall` when targeting release branches to ensure the matching client version is installed.
|
Yay, CI is green! |
raghotham
approved these changes
Oct 31, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-picks 5 critical fixes from main to the release-0.3.x branch for the v0.3.1 release, plus CI workflow updates.
Note: This recreates the cherry-picks from the closed PR #3991, now targeting the renamed
release-0.3.xbranch (previouslyrelease-0.3.x-maint).Commits
2c56a85 - fix(context): prevent provider data leak between streaming requests (fix(context): prevent provider data leak between streaming requests #3924)
ddd32b1 - fix(inference): enable routing of models with provider_data alone (fix(inference): enable routing of models with provider_data alone #3928)
f7c2973 - fix: Avoid BadRequestError due to invalid max_tokens (fix: Avoid BadRequestError due to invalid max_tokens #3667)
d7f9da6 - fix(responses): sync conversation before yielding terminal events in streaming (fix(responses): sync conversation before yielding terminal events in streaming #3888)
0ffa865 - fix(logging): ensure logs go to stderr, loggers obey levels (fix(logging): ensure logs go to stderr, loggers obey levels #3885)
75b49cb - ci: support release branches and match client branch (ci: support release branches and match client branch #3990)
Adaptations for 0.3.0
llama_stack.core.telemetry.tracing→llama_stack.providers.utils.telemetry.tracingllama_stack.core.telemetry.telemetry→llama_stack.apis.telemetryself.telemetry_enabled→self.telemetry(0.3.0 attribute name)rerank()method that doesn't exist in 0.3.0Testing
All imports verified and tests should pass once CI is set up.