Upgrade to text-embedding-3-large model as default, with vector storage optimizations

[pamelafox]

Purpose

This pull request changes the default embedding model to text-embedding-3-large, with 3072 dimensions, along with these AI Search vector storage optimizations:

• Truncate dimensions to 1024
• Binary quantization
• Preserve originals in the search index for rescoring
• Don't store the vectors in the search index itself

See this notebook for a demonstration of the effects of those optimizations. Due to the rescoring, the search quality remains high.

This PR introduces a new environment variable AZURE_SEARCH_FIELD_NAME_EMBEDDING so that developers can theoretically have multiple fields in their index, for different embedding sizes/models.

This PR also changes the SKU for all models to GlobalStandard. It's becoming really tricky to find a region for the Standard SKU that works for all the models. Some developers may not be comfortable with GlobalStandard, depending on their regulations, so they can still change the SKU manually as desired.

Fixes #2383

Does this introduce a breaking change?

When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.

[X] Yes - I am trying to make it backwards compatible, but it's hard! I suspect that developers that recently deployed gpt-4o-mini with Standard sku will get an error, and need to run `azd env set` to change the deployment name or sku name.
[ ] No

Does this require changes to learn.microsoft.com docs?

This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.

[ ] Yes
[X] No

Type of change

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

Code quality checklist

See CONTRIBUTING.md for more details.

• The current tests all pass (python -m pytest).
• I added tests that prove my fix is effective or that my feature works
• I ran python -m pytest --cov to verify 100% coverage of added lines
• I ran python -m mypy to check for type errors
• I either used the pre-commit hooks or ran ruff and black manually on my code.

[github-actions]

Check Country Locale in URLs

We have automatically detected added country locale to URLs in your files.
Review and remove country-specific locale from URLs to resolve this issue.

Check the file paths and associated URLs inside them.
For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_features.md	# Link Line Number 1 https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#models-by-deployment-type 197

[Copilot]

Pull Request Overview

This PR upgrades the default embedding model to "text-embedding-3-large" (3072 dimensions) and implements several vector storage optimizations including truncation, binary quantization, and preserving original values for rescoring. It also introduces new environment variables for embedding field names and updates documentation and related code to support the new configuration.

• Updated tests and environment variables for the new embedding model and dimensions.
• Revised documentation to reflect model and deployment changes.
• Refactored search management and approach modules to use dynamic embedding field names.

Reviewed Changes

Copilot reviewed 24 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/conftest.py	Updated mocked model and dimensions for the new embedding model.
docs/gpt4v.md	Changed embedding model reference from ada to text-embedding-3-large.
docs/deploy_features.md	Updated deployment instructions and embedding model references.
docs/deploy_existing.md	Adjusted instructions for existing deployments to use the new model.
azure.yaml	Added new environment variables for embedding field names.
app/backend/prepdocslib/searchmanager.py	Refactored index creation to use dynamic embedding field names and profiles.
app/backend/integratedvectorizerstrategy.py	Passed new search field names into indexer skill configuration.
app/backend/prepdocs.py	Updated embedding field configuration from environment variables.
app/backend/approaches/*	Modified constructors and vector field usages to accept new embedding field.
app/backend/app.py	Integrated new environment variables for embedding field names in client setup.
.github/workflows/azure-dev.yml & .azdo/pipelines/azure-dev.yml	Included new environment variable exports for embedding field names.

Files not reviewed (3)

• app/backend/requirements.txt: Language not supported
• infra/main.bicep: Language not supported
• infra/main.parameters.json: Language not supported

[mattgotteiner]

deal with image embedding in a future pr

[mattgotteiner]

help e2e test

[github-actions]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them.
For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them.
For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them.
For more details, check our Contributing Guide.

File Full Path	Issues
data/Contoso_Electronics_Company_Overview.md	# Link Line Number 1 http://www.contoso.com 48
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/chatreadretrieveread.py 39 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 13 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them.
For more details, check our Contributing Guide.

File Full Path	Issues
README.md	# Link Line Number 1 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview 79 2 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator 79 3 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#owner 79 4 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview 79 5 https://learn.microsoft.com/azure/cognitive-services/openai/concepts/models#model-summary-table-and-region-availability 178 6 https://learn.microsoft.com/azure/developer/python/get-started-app-chat-template?toc=%2Fazure%2Fdeveloper%2Fai%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fai%2Fbreadcrumb%2Ftoc.json&tabs=github-codespaces 273 7 https://learn.microsoft.com/azure/search/search-what-is-azure-search 275 8 https://learn.microsoft.com/azure/cognitive-services/openai/overview 276
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[Copilot]

Pull Request Overview

This pull request upgrades the default embedding model to text-embedding-3-large with 3072 dimensions and applies vector storage optimizations. Key changes include:

• Adding a new environment variable AZURE_SEARCH_FIELD_NAME_EMBEDDING in configuration files.
• Renaming and updating vector field type usage on both frontend and backend to use the new VectorFields enum.
• Propagating changes in embedding-related logic (e.g. API version updates, search index field naming) in backend modules.

Reviewed Changes

Copilot reviewed 104 out of 105 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
azure.yaml	Added AZURE_SEARCH_FIELD_NAME_EMBEDDING to the pipeline environment variables.
app/frontend/src/pages/chat/Chat.tsx	Renamed vector field state from VectorFieldOptions to VectorFields and updated related references.
app/frontend/src/pages/ask/Ask.tsx	Updated import and state management of vector fields.
app/frontend/src/components/VectorSettings/VectorSettings.tsx	Refactored vector field dropdown logic and state handling with the new VectorFields enum.
app/frontend/src/components/Settings/Settings.tsx	Adjusted prop names to match updated vector fields terminology.
app/frontend/src/api/models.ts	Replaced the VectorFieldOptions enum with the new VectorFields enum and updated its key names.
app/backend/* (various files)	Updated backend logic to require and propagate the new embedding field name, updated API versions, etc.
.github/workflows/azure-dev.yml & .azdo/pipelines/azure-dev.yml	Included AZURE_SEARCH_FIELD_NAME_EMBEDDING in the environment variable configuration.

Files not reviewed (1)

• app/backend/requirements.txt: Language not supported

Comments suppressed due to low confidence (2)

app/frontend/src/pages/chat/Chat.tsx:59

• The new variable name 'vectorFields' is more descriptive than the previous name. Please ensure that all component references and related handlers have been updated consistently.

const [vectorFields, setVectorFields] = useState<VectorFields>(VectorFields.TextAndImageEmbeddings);

app/frontend/src/api/models.ts:13

• The updated enum 'VectorFields' with keys 'textEmbeddingOnly', 'imageEmbeddingOnly', and 'textAndImageEmbeddings' should be reviewed for consistent usage across the codebase.

export const enum VectorFields {

[pamelafox]

@mattgotteiner I made changes to how vector fields are handed for gpt-vision as it was previously using the vector field names in the settings, and also, I think it was actually buggy the way it was implemented. Now it's done very similar to retrieval mode, with either text, image, or both, as the options.