Guys, adding
**acceleration = "cuda";**
has turned into a bit of an inside joke fr me. The resulting error is why I almost gave up on NixOS when I first picked it up a few weeks ago, namely I get the ... has an unfree license (‘CUDA EULA’), refusing to evaluate. whith a prescription for solutions.
I have set the env variable
$env.NIXPKGS_ALLOW_UNFREE
1
added to my flake
packages.config.allowUnfree = true;
It is my understanding that the allowUnfree property I'm setting is not scoped correctly, but I'm having a tough time figuring out the correct scope.
Found a thread on stackoverflow that probably has the answer but I can't follow it. (square peg/round hole).
My biggest issue is the dissonance between calling a nix file a configuration and what it actually is (IMO) a program. I have yet to crack the language.
I'm going to start here. With some luck I'll be able to fix this by the end of the week :)
Shivaraj B H has marked this topic as unresolved.
A message was moved here from #nixify-llm > ✔ Issues with ollama flake by Shivaraj B H.
@Ayrton Mercado I have to document this, you can use this for now:
# Inside perSystem
{
imports = [
"${inputs.nixpkgs}/nixos/modules/misc/nixpkgs.nix"
];
nixpkgs = {
# Required for CUDA
config.allowUnfree = true;
};
}
ollama-flake exports a nixpkgs
module that would handle this, but since that’s not the case after upstreaming to services-flake
, it wasn’t working out of the box for you.
Hold on, there seems to a bug in services-flake
that I just fixed: https://github.com/juspay/services-flake/commit/8145ba10cb02dc0a843bba371fc2d42cea7fd226
This commit demonstrates how to do it in the current services-flake’s example/llm
: https://github.com/shivaraj-bh/services-flake/commit/bf3bf65a920dc6e3b4eb749ec3809fdeaf4b9360
Also documented it: https://github.com/juspay/services-flake/commit/fb6de01eb6e7d2ac1c540fd399b9c0f447ce0ab7
Shivaraj B H said:
Also documented it: https://github.com/juspay/services-flake/commit/fb6de01eb6e7d2ac1c540fd399b9c0f447ce0ab7
Confirmed on my end, the flake compiles
Last updated: Oct 12 2024 at 22:33 UTC