feat: add remote embedding providers (Voyage AI, OpenAI-compatible)#65
Open
adam91holt wants to merge 1 commit intotobi:mainfrom
Open
feat: add remote embedding providers (Voyage AI, OpenAI-compatible)#65adam91holt wants to merge 1 commit intotobi:mainfrom
adam91holt wants to merge 1 commit intotobi:mainfrom
Conversation
0b6196d to
15e13f6
Compare
|
This is awesome. It's exactly what I was hoping for. Thanks Adam! @adam91holt |
- QMD_PROVIDER env var to select provider: voyage, openai, or local (default) - Voyage AI support: voyage-4-lite embeddings + rerank-2 reranking - OpenAI-compatible support: works with OpenAI, Ollama, vLLM, LM Studio, etc. - Provider info shown in qmd status output - Add environment variables to --help output
15e13f6 to
18badc5
Compare
|
I'd love to see this merged! Very keen to use qmd with openclaw, but really like working with Voyage AI's embedding models and hence don't want to move across until this is supported. Voyage AI have also released voyage-4-nano which can be run locally so can keep inference/query completely local which feels aligned with the project's intent. Shared embedding space means we have the option for remote, large, models to embed the docs and then smaller local for queries. Although will need another PR to fully enable it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add support for remote embedding APIs as an alternative to local models. This enables:
Changes
New Features
voyage,openai, orlocal(default)qmd statusoutputTechnical Details
RemoteLLMclass implementing theLLMinterfacegetDefaultLLM()factory function for provider selectionFiles Changed
src/remote.ts- New RemoteLLM implementationsrc/remote.test.ts- Test suite for remote providerssrc/llm.ts- Provider selection functionssrc/store.ts- Use getDefaultLLM for embed/reranksrc/qmd.ts- Show provider in statusREADME.md- Documentation for env varsEnvironment Variables
QMD_PROVIDERlocallocal,voyage,openaiVOYAGE_API_KEYVOYAGE_EMBED_MODELvoyage-4-liteVOYAGE_RERANK_MODELrerank-2OPENAI_API_KEYOPENAI_EMBED_MODELtext-embedding-3-smallOPENAI_API_BASEhttps://api.openai.com/v1Testing
All 15 new tests pass with actual Voyage API calls:
Tests skip gracefully when API keys are not set.