fix ollama gpu acceleration #373924
fix ollama gpu acceleration #373924
Conversation
| @@ -199,6 +199,13 @@ goBuild { | |||
| make ${dist_cmd} -j $NIX_BUILD_CORES | |||
| ''; | |||
|
|
|||
| postInstall = lib.optionalString stdenv.hostPlatform.isLinux '' | |||
There was a problem hiding this comment.
@GaetanLepage just wondering, why was this removed in the first place? Want to make sure I'm not overlooking anything by adding it back
There was a problem hiding this comment.
No I think I removed it because of some conflicts. I thought that it was not necessary anymore.
Thanks for fixing this.
nix-owners
bot
requested review from
abysssol,
elohmeier,
RoyDubnium and
dit7ya
| @@ -199,6 +199,13 @@ goBuild { | |||
| make ${dist_cmd} -j $NIX_BUILD_CORES | |||
| ''; | |||
|
|
|||
| postInstall = lib.optionalString stdenv.hostPlatform.isLinux '' | |||
There was a problem hiding this comment.
It may be a good idea to replace the condition stdenv.hostPlatform.isLinux with (enableRocm || enableCuda).
The replacement:
- Means what it says: if GPU acceleration is enabled, copy GPU related libraries. Currently the copying is always attempted when building on Linux, even on a default, CPU only build (which shouldn't need any GPU libraries to be copied).
- Won't activate for CPU builds, which is (probably) good (I'm not entirely certain, though). This script was removed because build errors were encountered when the
cpwas executed without any arguments to copy (there were no libraries to copy). I think that this probably happened because the copy was executed on CPU builds, and there aren't any files to copy on CPU builds (though, strangely, that would imply that there used to be GPU libraries to copy on CPU builds, since this script used to work).
I'm not actually sure whether or not the CPU build does have any GPU libraries to copy, or whether my theory is at all accurate in the first place, so further testing is probably needed. See also my other comment on the topic.
|
Is this compatible with the changes in #373234? |
|
Fixed a bug where Ollama is no longer using GPU acceleration. Tested working with an Nvidia card. I am unsure if this also affected AMD cards
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.