Wishlist: Asahi Linux support

[tidux]

Linux has much more robust tools for cluster management than MacOS, and Asahi Linux kernels have support for all the relevant Apple M chip hardware. The goal is to make running a cluster with a proper OS level management beyond "run exo in tmux" possible on ARM Macs.

Ideal use cases:

• Kubernetes with GPU passthrough
• Docker Swarm with a /dev parameter
• NFS Root shared OS clusters

[AFDudley]

This sounds very interesting and also like an almost completely different project? What parts of the existing codebase would your proposal share?

[tidux]

All it needs is M-chip support for the AI backend(s?) under Linux in the exo tool. The rest I could put together at the sysadmin level - Dockerfiles, Kubernetes manifests, etc. p2p cluster app management inside Kubernetes or Docker Swarm is a well known pattern.

[AFDudley]

Can you confirm mlx works on an Asahi Linux based distro?

[tidux]

It looks like mlx is Metal/MacOS only for now so this remains on hold until that changes.

[pbouda]

I think the this could work (the exo part, k8s is separate I guess) with the tinygrad OpenCL back-end and rusticl/mesa? I don't have a machine to try it unfortunately.