[Question]: LLMReranker

[ddealwis09]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Hello, I am having difficulty getting the LLMReranker setup, below is my code, can I am getting an error when I run the query engine. The error I am getting is "InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>" but the query engine works perfectly if I remove the reranker.

# cache index
 @st.cache_resource
 def load_index():
       index = load_vectorstore_from_storage(path="./storage")
       return index

llm_completion = AzureOpenAI(deployment_name="gpt-35-turbo1", 
                                model_name="text-davinci-003")
    
 #Azure embeddings
embedding_llm = LangchainEmbedding(
                    OpenAIEmbeddings(
                        model="text-embedding-ada-002",
                        deployment="text-embedding-ada-002",
                    ),
                    embed_batch_size=1,
                )
service_context = ServiceContext.from_defaults(llm=llm_completion, 
                                                    embed_model=embedding_llm) 

reranker = LLMRerank(choice_batch_size=5, top_n=3, service_context=service_context)

query_engine = index.as_query_engine(vector_store_query_mode="mmr", 
                                                        vector_store_kwargs={"mmr_threshold": mmr_threshold},
                                                        similarity_top_k=8,
                                                        node_postprocessors=[reranker],  
                                                        response_mode = "tree_summarize",
                                                        text_qa_template=text_qa_template, 
                                                        refine_template=refine_template,
                                                        )

[dosubot[bot]]

🤖

Hello,

Thank you for reaching out. The error message you're encountering, "InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>", is indicating that the 'engine' or 'deployment_id' parameter is not provided when initializing the OpenAI class in the LlamaIndex framework.

In your code, you're initializing the AzureOpenAI class with deployment_name and model_name parameters. However, the AzureOpenAI class requires either an 'engine' or 'deployment_id' parameter.

Here's how you should initialize the AzureOpenAI class:

llm_completion = AzureOpenAI(deployment_id="gpt-35-turbo1", 
                                model_name="text-davinci-003")

In this case, deployment_id is the ID of your deployment on Azure. Please replace "gpt-35-turbo1" with your actual deployment ID.

Also, please note that the LLMReranker needs to be initialized with a LLMPredictor instance. You can do this by modifying your LLMReranker initialization as follows:

from llama_index import LLMPredictor

llm_predictor = LLMPredictor(llm=llm_completion)

reranker = LLMReranker(llm_predictor=llm_predictor, choice_batch_size=5, top_n=3)

Then, you can use reranker in your index.as_query_engine call.

I hope this helps! If you have any other questions, feel free to ask.

Best,
Dosu

Sources

• InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter
• LLAMA-Index Extracting Metadata: nodeparser
• llama_index/llms/azure_openai.py
• tests/agent/openai/test_openai_agent.py
• tests/indices/postprocessor/test_llm_rerank.py
• llama_index/query_engine/retriever_query_engine.py
• llama_index/indices/postprocessor/llm_rerank.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ddealwis09]

I am getting an error now:
TypeError: LLMRerank.init() got an unexpected keyword argument 'llm_predictor'

In your reply you used LLMReranker instead of LLMRerank, I assume that was a mistake?

[logan-markewich]

@ddealwis09 yea the bot was not correct lol

Can you try also setting a global service context? Azure is very annoying to work with...

from llama_index import set_global_service_context

set_global_service_context(service_context)