jaym/ollama-nix
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
As of 12 Nov 2023, ollama in nixpkgs fails to utilize CUDA-enabled devices. This repo contains a nix flake that defines: - ollama package with CUDA support - NixOS module that may be helpful if you plan on using ollama as a system-wide LLM runner To simply run the CUDA-enabled ollama binary, make sure you have nvidia-smi available in the PATH and run the following command: nix run github:havaker/ollama-nix