When I switch my generation params, the caching don't apply anymore, yet a lot of times I am using the same dataset. A more concrete example would be that, say I set max_token = 8192 and was able to curate 500/1000 questions, and I found that for the rest of the 500 questions max token length is always reached which means I would need a larger max_token. In that case I would want to curate the rest of the 500 questions but combine them with the cache I already have. If there could be a USE_CACHE param where I can specify the cache I want to combine my new curation with it would be great. And since I'm using the same dataset there should be easy ways to check, for the current dataset, assuming the cache uses the same set of questions/prompts, which prompts are answered already?
When I switch my generation params, the caching don't apply anymore, yet a lot of times I am using the same dataset. A more concrete example would be that, say I set max_token = 8192 and was able to curate 500/1000 questions, and I found that for the rest of the 500 questions max token length is always reached which means I would need a larger max_token. In that case I would want to curate the rest of the 500 questions but combine them with the cache I already have. If there could be a
USE_CACHEparam where I can specify the cache I want to combine my new curation with it would be great. And since I'm using the same dataset there should be easy ways to check, for the current dataset, assuming the cache uses the same set of questions/prompts, which prompts are answered already?