You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add logic to handle error response from upstream aws bedrock or openai, currently it is assuming the response is 200 and fails on response body unmarshalling.
Add tests for error responses from upstream
time=2025-01-19T18:34:38.104Z level=INFO msg="starting external processor" version=8056bb8bcfe484ca6f6007b021ea69448b3073f4
time=2025-01-19T18:34:38.104Z level=INFO msg="loading a new config" path=/etc/ai-gateway/extproc/extproc-config.yaml
time=2025-01-19T18:34:38.104Z level=INFO msg="start watching the config file" path=/etc/ai-gateway/extproc/extproc-config.yaml interval=5s
time=2025-01-19T18:34:42.422Z level=DEBUG msg="request headers processing" request_headers="headers:{key:\":authority\" raw_value:\"localhost:8080\"} headers:{key:\":path\" raw_value:\"/v1/chat/completions\"} headers:{key:\":method\" raw_value:\"POST\"} headers:{key:\":scheme\" raw_value:\"http\"} headers:{key:\"user-agent\" raw_value:\"curl/8.7.1\"} headers:{key:\"accept\" raw_value:\"*/*\"} headers:{key:\"content-type\" raw_value:\"application/json\"} headers:{key:\"x-ai-gateway-llm-backend\" raw_value:\"kserve-llama-backend\"} headers:{key:\"content-length\" raw_value:\"113\"} headers:{key:\"x-forwarded-for\" raw_value:\"10.244.1.40\"} headers:{key:\"x-forwarded-proto\" raw_value:\"http\"} headers:{key:\"x-envoy-external-address\" raw_value:\"127.0.0.1\"} headers:{key:\"x-request-id\" raw_value:\"41605e8c-7ef8-4fbc-b1f9-a8f01740746e\"}"
time=2025-01-19T18:34:42.422Z level=DEBUG msg="request headers processed" response=request_headers:{}
time=2025-01-19T18:34:42.422Z level=DEBUG msg="request body processing" request="request_body:{body:\"{\\\"model\\\": \\\"dsp.llama-3.3-70b-instruct\\\", \\\"prompt\\\": \\\"Write a poem about colors\\\", \\\"stream\\\":false, \\\"max_tokens\\\": 100}\" end_of_stream:true} metadata_context:{}"
time=2025-01-19T18:34:42.422Z level=INFO msg="Processing request" path=/v1/chat/completions model=dsp.llama-3.3-70b-instruct
time=2025-01-19T18:34:42.422Z level=INFO msg="Selected backend" backend=kserve-llama-backend.ai-gateway
time=2025-01-19T18:34:42.422Z level=DEBUG msg="request body processed" response="request_body:{response:{header_mutation:{set_headers:{header:{key:\"x-envoy-ai-gateway-model\" raw_value:\"dsp.llama-3.3-70b-instruct\"}} set_headers:{header:{key:\"x-envoy-ai-gateway-selected-backend\" raw_value:\"kserve-llama-backend.ai-gateway\"}}} clear_route_cache:true}}"
time=2025-01-19T18:34:57.425Z level=DEBUG msg="response headers processing" response_headers="headers:{key:\":status\" raw_value:\"504\"} headers:{key:\"content-length\" raw_value:\"24\"} headers:{key:\"content-type\" raw_value:\"text/plain\"}"
time=2025-01-19T18:34:57.425Z level=ERROR msg="upstream returned non-200 response"
time=2025-01-19T18:34:57.425Z level=DEBUG msg="response headers processed" response=response_headers:{}
time=2025-01-19T18:34:57.426Z level=DEBUG msg="response body processing" request="response_body:{body:\"upstream request timeout\" end_of_stream:true} metadata_context:{}"
time=2025-01-19T18:34:57.426Z level=DEBUG msg="response body processed" response=<nil>
time=2025-01-19T18:34:57.426Z level=ERROR msg="cannot process request" error="cannot process response body: failed to transform response: failed to unmarshal body: invalid character 'u' looking for beginning of value"
The text was updated successfully, but these errors were encountered:
The text was updated successfully, but these errors were encountered: