Skip to content

Fix LiteLLM subsequent calls issue #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ShyamSathish005
Copy link

Fixes #65

Add retry logic and error handling for LiteLLM model to handle subsequent prompts without causing an Internal Server Error.

  • src/google/adk/models/lite_llm.py

    • Add retry logic in generate_content_async method to handle transient errors.
    • Add error handling for InternalServerError in LiteLLMClient class.
    • Update _call_llm_async method to handle retries and errors.
  • src/google/adk/tests/unittests/models/test_litellm.py

    • Add unit tests for retry logic in generate_content_async method.
    • Add unit tests for error handling in LiteLLMClient class.
    • Add unit tests for handling InternalServerError in _call_llm_async method.

Fixes google#65

Add retry logic and error handling for LiteLLM model to handle subsequent prompts without causing an Internal Server Error.

* **src/google/adk/models/lite_llm.py**
  - Add retry logic in `generate_content_async` method to handle transient errors.
  - Add error handling for `InternalServerError` in `LiteLLMClient` class.
  - Update `_call_llm_async` method to handle retries and errors.

* **src/google/adk/tests/unittests/models/test_litellm.py**
  - Add unit tests for retry logic in `generate_content_async` method.
  - Add unit tests for error handling in `LiteLLMClient` class.
  - Add unit tests for handling `InternalServerError` in `_call_llm_async` method.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Subsequent calls with LiteLLM fails
1 participant