-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net: Add Async Support for ConverseStreamResponse to Avoid Thread Blocking #10159
.Net: Add Async Support for ConverseStreamResponse to Avoid Thread Blocking #10159
Conversation
@shethaadit Do we really need to rely on |
Hi @RogerBarret0, this fix just got available yesterday in latest Nuget. We would need to keep the new version to get this fix. |
Hi @RogerBarret0 / @markwallace-microsoft, can we merge this as it's already approved? |
Hi @shethaadit, seems that the solution currently does not compile, please before posting the PR ensure the solution compiles and the all UT are passing. Thanks. |
…into shethaadit/FixBug9519-New
Motivation and Context
This change replaces the use of
response.Stream.AsEnumerable()
withawait foreach
for async streaming in the Amazon Bedrock SDK. The previous method blocked the calling thread, leading to potential thread exhaustion in high-concurrency scenarios. This update aligns with best practices for asynchronous operations in .NET.Description
await foreach
for streamingConverseStreamResponse
chunks asynchronously.Contribution Checklist
Fixes #9519