Support for IPEX/Intel GPUs

[hanthor]

The proper bits for supporting Intel GPUs for running ollama and others exists here: https://github.com/intel/ipex-llm/tree/main/docker/llm

Just need someone with the expertise to add the right stuff to support it

[ericcurtin]

@cgruver had joy with this container image, not sure if it's the same type of acceleration:

https://github.com/containers/ramalama/blob/main/container-images/intel-gpu/Containerfile

we would need someone from the community with the hardware to test, expertise, etc. to open a PR.

[cgruver]

@hanthor

ramalama --gpu --imagequay.io/cgruver0/llama-cpp-intel-gpu:latest run granite

[cgruver]

@hanthor I've tested this on Intel 155H with Arc GPU. It's actually pretty snappy.

also... JFYI, that particular image is using OpenCL. I have a refactor on the way that will use Level Zero. It's a bit faster than OpenCL.

[hanthor]

@cgruver had joy with this container image, not sure if it's the same type of acceleration:

https://github.com/containers/ramalama/blob/main/container-images/intel-gpu/Containerfile

we would need someone from the community with the hardware to test, expertise, etc. to open a PR.

This worked great! I'm wondering about these NPUs now... maybe running Whisper on it? still haven't come up with a decent application for these NPUs