Openclaw / Local LLM / Cache issues

[jmp470]

I'm running a MAC M2 Ultra w/192GB Ram and I'm running into cache issues. I'm trying to figure out the best way to set it up. Every time I get 125000 token in cache it crashes. Is there a way to prune the cache before that?

[omni-front]

What version of vllm-mlx are you using? The caching mechanisms may have been updated in different versions. Also, are you using any specific cache-related flags or configurations when running the server? This information will help determine the best way to manage the cache and potentially prune it before reaching the token limit.