Add support for free Azure Search endpoint and GitHub models #8
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for free access to GitHub Models and Azure AI Search services without cost, allowing users to try the application without incurring charges.
- Migrates settings initialization to support conditional vector store instantiation.
- Introduces helper functions for Azure AI Search and GitHub models configuration.
- Updates documentation and environment setup instructions for free endpoint usage.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| app/api/chat/route.ts | Delegates vector store initialization to a new helper function. |
| app/api/chat/engine/settings.ts | Refactors settings initialization using helper functions and updates error messages. |
| app/api/chat/engine/createIndex.ts | Implements retrieval of search endpoint details with basic error logging. |
| app/api/chat/engine/chat.ts | Uses a helper to configure retriever options based on environment tokens. |
| README.md | Updates documentation regarding no-cost application usage. |
| .env.example | Adds GITHUB_TOKEN instructions for local development. |
| ...openAiConfig, | ||
| model: process.env.AZURE_OPENAI_CHAT_DEPLOYMENT, | ||
| }); | ||
| Settings.llm = new OpenAI(createOpenAiParams(openAiConfig, githubToken)); | ||
| console.log({ openAiConfig }); |
There was a problem hiding this comment.
Consider removing or limiting this debug log to avoid exposing sensitive configuration details in production.
| console.log({ openAiConfig }); | |
| if (process.env.NODE_ENV === 'development') { | |
| console.log("OpenAI configuration initialized."); | |
| } |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
This log statement wasn't introduced in this PR, so I’d prefer to keep this PR focused on the current scope. Happy to include the change here if you'd prefer, just let me know.
| console.log(`Your Azure AI Search Endpoint: ${searchServiceEndpoint}; Index Name: ${searchIndexName}`); | ||
| return { endpoint: searchServiceEndpoint, indexName: searchIndexName }; | ||
| } catch (error) { | ||
| console.error("Error while retrieving search service details", error); |
There was a problem hiding this comment.
Instead of solely logging errors, consider throwing an exception or returning an error result to better surface failures in retrieving search service details.
| console.error("Error while retrieving search service details", error); | |
| console.error("Error while retrieving search service details", error); | |
| throw new Error("Failed to retrieve search service details: " + error.message); |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
In case of an error, searchService object would be null and an exception is already being thrown here.
| const createRetrieverOptions = () => { | ||
| return process.env.GITHUB_TOKEN | ||
| ? { mode: "hybrid" as any } | ||
| : { mode: "semantic_hybrid" as any, similarityTopK: 5 }; |
There was a problem hiding this comment.
Using 'as any' bypasses type checking; consider defining proper types or using existing enums to enforce valid retriever modes.
| const createRetrieverOptions = () => { | |
| return process.env.GITHUB_TOKEN | |
| ? { mode: "hybrid" as any } | |
| : { mode: "semantic_hybrid" as any, similarityTopK: 5 }; | |
| type RetrieverMode = "hybrid" | "semantic_hybrid"; | |
| const createRetrieverOptions = (): { mode: RetrieverMode; similarityTopK?: number } => { | |
| return process.env.GITHUB_TOKEN | |
| ? { mode: "hybrid" } | |
| : { mode: "semantic_hybrid", similarityTopK: 5 }; |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
Adding explicit type for VectorStoreQueryMode results in the following import error:
Attempted import error: 'VectorStoreQueryMode' is not exported from 'llamaindex' (imported as 'VectorStoreQueryMode').
Hence kept the type as any. May be in future version when llamaindex version is upgraded then the explicit type can be used.
This PR leverages GitHub tokens to access free GitHub Models endpoint for GPT models and embeddings, as well as free Azure AI Search endpoints for data indexing and retrieval. These changes allow users to explore and use the application without incurring any costs.