[Question]:

[Meenakshi2604]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

While using the custom prompts of LlamaIndex, I get " KeyError: 'context_str' ".
I used the template:
template = (
"We have provided context information below. \n"
"---------------------\n"
"{context_str}"
"\n---------------------\n"
"Given this information, please answer the question: {query_str}\n"
)
qa_template = PromptTemplate(template)

What is meant by context_str? Even after defining the context_str, why is the KeyError generated?

[dosubot[bot]]

🤖

Hello,

The "KeyError: 'context_str'" error you're encountering is likely due to the 'context_str' key not being provided when the template is being used. In your code, you've defined a template that expects 'context_str' and 'query_str' as inputs, but it seems like these are not being provided when the template is used.

To fix this issue, you need to ensure that 'context_str' and 'query_str' are provided when the template is used. Here is an example of how you can do this:

context_str = "Some context information"
query_str = "Who is Joe Biden?"

print(
    index.as_query_engine(
        text_qa_template=text_qa_template, refine_template=refine_template
    ).query(query_str, context_str=context_str)
)

In this example, 'context_str' and 'query_str' are provided as arguments to the 'query' method. This should prevent the "KeyError: 'context_str'" error from occurring. You can refer to the chat_prompts.ipynb example for more details.

As for the role of 'context_str', it is used in various PromptTemplates in LlamaIndex. For instance, in the 'QuestionAnswerPrompt', 'context_str' is used to provide the context for answering a question specified by 'query_str'. Similarly, in the 'ChoiceSelectPrompt', 'context_str' is used to provide the context for selecting from a list of choices. In the 'TableContextPrompt' and 'RefineTableContextPrompt', 'context_str' is used to provide unstructured text context for generating or refining a table context. Therefore, 'context_str' is a crucial variable in these prompts, providing the necessary context for various operations. You can refer to the context.py and prompts.py files for more details.

I hope this helps! If you have any other questions, feel free to ask.

Best,
Dosu

Sources

• How to get the context_str from the index query?
• Defining custom prompt not working as expected
• [Demo] Build Tree Index with a custom Summary Prompt, directly retrieve answer from root node
• docs/examples/customization/prompts/chat_prompts.ipynb
• llama_index/chat_engine/context.py
• llama_index/prompts/prompts.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Meenakshi2604]

🤖

Hello,

The "KeyError: 'context_str'" error you're encountering is likely due to the 'context_str' key not being provided when the template is being used. In your code, you've defined a template that expects 'context_str' and 'query_str' as inputs, but it seems like these are not being provided when the template is used.

To fix this issue, you need to ensure that 'context_str' and 'query_str' are provided when the template is used. Here is an example of how you can do this:

context_str = "Some context information"
query_str = "Who is Joe Biden?"

print(
    index.as_query_engine(
        text_qa_template=text_qa_template, refine_template=refine_template
    ).query(query_str, context_str=context_str)
)

In this example, 'context_str' and 'query_str' are provided as arguments to the 'query' method. This should prevent the "KeyError: 'context_str'" error from occurring. You can refer to the chat_prompts.ipynb example for more details.

As for the role of 'context_str', it is used in various PromptTemplates in LlamaIndex. For instance, in the 'QuestionAnswerPrompt', 'context_str' is used to provide the context for answering a question specified by 'query_str'. Similarly, in the 'ChoiceSelectPrompt', 'context_str' is used to provide the context for selecting from a list of choices. In the 'TableContextPrompt' and 'RefineTableContextPrompt', 'context_str' is used to provide unstructured text context for generating or refining a table context. Therefore, 'context_str' is a crucial variable in these prompts, providing the necessary context for various operations. You can refer to the context.py and prompts.py files for more details.

I hope this helps! If you have any other questions, feel free to ask.

Best, Dosu

Sources

• How to get the context_str from the index query?
• Defining custom prompt not working as expected
• [Demo] Build Tree Index with a custom Summary Prompt, directly retrieve answer from root node
• docs/examples/customization/prompts/chat_prompts.ipynb
• llama_index/chat_engine/context.py
• llama_index/prompts/prompts.py

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Thank you for the response. I do have another doubt. I am using chat_engine of llama_index instead of query_engine. Will the custom prompt work in that case?

[dosubot[bot]]

🤖

Hello,

The 'context_str' and 'query_str' are placeholders in the LlamaIndex's PromptTemplates that are replaced with the actual context and query when you call the format() or format_messages() methods on a PromptTemplate instance.

The 'context_str' provides the necessary context information for the Language Models (LLMs) to generate appropriate responses. It is used in the index construction and query engine execution stages of the LlamaIndex pipeline.

To prevent the "KeyError: 'context_str'" error when using custom prompts in LlamaIndex, you need to ensure that the format string of your custom prompt includes the '{context_str}' and '{query_str}' parameters.

Here's an example of how to define a custom prompt with these parameters:

from llama_index.prompts import PromptTemplate

template = (
    "We have provided context information below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given this information, please answer the question: {query_str}\n"
)
qa_template = PromptTemplate(template)

# you can create text prompt (for completion API)
prompt = qa_template.format(context_str='your context here', query_str='your query here')
# or easily convert to message prompts (for chat API)
messages = qa_template.format_messages(context_str='your context here', query_str='your query here')

You can pass your custom prompts into the pipeline at the right place depending on the operation you want to modify. For example, to modify the prompts used in index construction, you can do:

index = TreeIndex(nodes, summary_template=<custom_prompt>)

Or to modify the prompts used in query engine execution, you can do:

query_engine = index.as_query_engine(
    text_qa_template=<custom_qa_prompt>,
    refine_template=<custom_refine_prompt>
)

Remember to replace <custom_prompt>, <custom_qa_prompt>, and <custom_refine_prompt> with your actual custom prompts.

After defining these custom prompts, you can use them in an index query as shown in the shared context:

print(
    index.as_query_engine(
        text_qa_template=text_qa_template, refine_template=refine_template
    ).query("Who is Joe Biden?")
)

This will use your custom prompts when querying the index.

For more details, you can refer to the following documentation and code files:

• docs/core_modules/model_modules/prompts.md
• docs/examples/customization/prompts/completion_prompts.ipynb
• docs/examples/customization/prompts/chat_prompts.ipynb
• llama_index/prompts/default_prompts.py
• llama_index/prompts/prompts.py

I hope this helps! Let me know if you have any other questions.

Sources

• How to get the context_str from the index query?
• Defining custom prompt not working as expected
• Custom qa prompt is not working
• docs/core_modules/model_modules/prompts.md
• docs/examples/customization/prompts/completion_prompts.ipynb
• docs/examples/customization/prompts/chat_prompts.ipynb
• llama_index/prompts/default_prompts.py
• llama_index/prompts/prompts.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.