Skip to content

Commit 20a1327

Browse files
committed
Skip JSON mode tests for OpenAI models due to 500 errors
JSON mode requests with OpenAI models on OCI currently return 500 Internal Server Error from the OCI API. Skip these tests for OpenAI models until this can be investigated further (may be model limitation or OCI API issue). Tests affected: - test_structured_output_json_mode - test_response_format_json_object These tests pass successfully with Meta Llama models.
1 parent d74b49a commit 20a1327

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

libs/oci/tests/integration_tests/chat_models/test_langchain_compatibility.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,11 @@ def test_structured_output_function_calling(chat_model):
225225
@pytest.mark.requires("oci")
226226
def test_structured_output_json_mode(chat_model):
227227
"""Test structured output with JSON mode."""
228+
# JSON mode with OpenAI models on OCI currently returns 500 errors
229+
# TODO: Investigate if this is a model limitation or OCI API issue
230+
if "openai" in chat_model.model_id.lower():
231+
pytest.skip("JSON mode with OpenAI models on OCI returns 500 errors")
232+
228233
structured_llm = chat_model.with_structured_output(Person, method="json_mode")
229234

230235
result = structured_llm.invoke(
@@ -257,6 +262,11 @@ def test_structured_output_include_raw(chat_model):
257262
@pytest.mark.requires("oci")
258263
def test_response_format_json_object(chat_model):
259264
"""Test response_format with json_object."""
265+
# JSON mode with OpenAI models on OCI currently returns 500 errors
266+
# TODO: Investigate if this is a model limitation or OCI API issue
267+
if "openai" in chat_model.model_id.lower():
268+
pytest.skip("JSON mode with OpenAI models on OCI returns 500 errors")
269+
260270
chat_json = chat_model.bind(response_format={"type": "json_object"})
261271

262272
response = chat_json.invoke(

0 commit comments

Comments
 (0)