Skip to content

fix(inference): let embedding requests omit dimensions - #1755

Merged
PsiACE merged 1 commit into
oceanbase:masterfrom
knqiufan:fix/issue-1754-omit-embedding-dimensions
Sep 28, 2026
Merged

PsiACE merged 1 commit into
oceanbase:masterfrom
knqiufan:fix/issue-1754-omit-embedding-dimensions

Conversation

@knqiufan

Copy link
Copy Markdown
Collaborator

Which issue or RFC does this PR close?

Closes #1754.

Rationale for this change

#1386 always sends the configured embedding dimension as the OpenAI-compatible dimensions field. Models that accept the field, such as text-embedding-3, still need that behavior. SiliconFlow BAAI/bge-m3 does not accept the field at all. Its output width is fixed at 1024, and a request that includes dimensions returns HTTP 400 (code 20015, The parameter is invalid). Memory writes then fail while attaching embeddings.

EMBEDDING_DIMENSION cannot be omitted to work around this. The model, profile ID, and dimension must be configured together, and that dimension is the stored vector width. embedding_model_settings also cannot suppress the field, because composition overwrites it with the profile dimension.

What changes are included in this PR?

  • Add embedding_send_dimensions / POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_SEND_DIMENSIONS, default true.
  • When the switch is true, embedding requests still send the profile dimension and still override a dimensions value in model settings. This keeps the fix: pass embedding dimension to provider and surface sqlite-vec probe failures #1386 behavior.
  • When the switch is false, the provider request omits dimensions, including a dimensions entry from model settings. Returned vectors are still validated against EMBEDDING_DIMENSION.
  • Document the switch in the configuration reference, the vector-search workflow, the Pydantic AI inference notes, and .env.example (English and Chinese).
  • Cover the default send path, the omit path on the wire, and environment loading. The default test still asserts that a model-settings dimension is replaced by the profile dimension.

No persisted format, HTTP API, or profile identity change. Existing deployments keep sending dimensions until the new variable is set to false.

Are there any user-facing changes?

Operators of a fixed-dimension model can set:

POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_SEND_DIMENSIONS=false

POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_DIMENSION stays required. For SiliconFlow BAAI/bge-m3, keep it at 1024.

The default remains true, so providers that require dimensions are unchanged.

How was this change tested?

uv run pytest tests/builtin/runtime/test_composition_embedding.py tests/test_server_embedding.py tests/test_inference_endpoints.py -q --tb=short
uv run ruff check src/powercontext/builtin/runtime/config.py src/powercontext/builtin/runtime/composition.py tests/builtin/runtime/test_composition_embedding.py tests/test_server_embedding.py tests/test_inference_endpoints.py
uv run ruff format --check src/powercontext/builtin/runtime/config.py src/powercontext/builtin/runtime/composition.py tests/builtin/runtime/test_composition_embedding.py tests/test_server_embedding.py tests/test_inference_endpoints.py

26 tests passed. Ruff check and format check passed.

The new wire test remembers one entry with embedding_send_dimensions=false and asserts every /v1/embeddings body has no dimensions key while other settings remain. The existing endpoint test still asserts dimensions == 3 when the switch is left at its default.

AI usage statement

Grok 4.7 assisted with the implementation, tests, and this description. The change was limited to the embedding dimensions switch and reviewed against the existing #1386 send behavior.

SiliconFlow BAAI/bge-m3 rejects the dimensions field. Keep the stored
dimension required, and add a switch that still sends it by default.

@PsiACE PsiACE left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@PsiACE
PsiACE merged commit ab43e3a into oceanbase:master Sep 28, 2026
24 checks passed
@PsiACE PsiACE added the 1.2.0 label Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Always sending embedding dimensions rejects SiliconFlow BAAI/bge-m3

2 participants