[FEATURE]: Add possibility to truncate retrieved context #446
Description
Description
For me the retrieved context can be larger than the LLM context window, which makes me get an context window exceeded error.
It would be nice to pass the model context window somewhere or a limit of some sort to truncate the context at. You dont want to truncate at the context window size, because you also need space for answer generation.
Additional Info
No response