Does vLLM-MLX Support efficient SSD caching?

[TomLucidor]

Saw this being recommended, and I wonder if vLLM-MLX might have (or could have) something similar for multi-agent designs (assuming they all use the same model for minimizing memory footprint) https://github.com/jundot/omlx

[omni-front]

The README mentions that vllm-mlx supports an "SSD-tiered cache." However, it's not clear what level of efficiency or specific implementation details this entails for multi-agent designs. Could you clarify whether you are asking about specific support for multi-agent systems or more about the efficiency of SSD caching in general? Additionally, if you have any specific use cases or configurations in mind, please share those details.

[TomLucidor]

General support that are fair in performance rather than expecting a speedup (my own use case would be multi-agent system ala Oh-my-Openagent and similar workflows)

[omni-front]

For general support, vLLM-MLX provides efficient SSD caching but doesn’t specifically optimize for multi-agent systems like Oh-my-Openagent. Performance may be fair, but not necessarily with a speedup for those workflows.