SageAttention fork for Windows wheels and easy installation
This repo makes it easy to build SageAttention for multiple Python, PyTorch, and CUDA versions, then distribute the wheels to other people.
The latest wheels support GTX 16xx, RTX 20xx/30xx/40xx/50xx, A100, H100, AGX Orin (sm75/80/86/87/89/90/120). There are also reports that SageAttention works with B200 (sm100) and DGX Spark (sm121), but I did not bundle these kernels in the wheels, and you need to build from source.
Update: If you're using ComfyUI, you no longer need to install my SageAttention wheel. You can use the SageAttention in comfy-kitchen.
- Know how to use pip to install packages in the correct Python environment, see https://github.com/woct0rdho/triton-windows
- Install triton-windows
- Install a wheel on the release page: https://github.com/woct0rdho/SageAttention/releases
- Unlike triton-windows, you need to manually choose a wheel in the GitHub release page for SageAttention
- Choose the wheel for your PyTorch version. For example, 'torch2.7.0' in the filename
- The recent wheels use libtorch stable ABI and have
torch2.10.0andhigherin the filenames, so they support all versions of PyTorch >= 2.10
- The recent wheels use libtorch stable ABI and have
- No need to worry about the CUDA minor version (12.8/12.9 ...). It can be different from yours, because SageAttention does not yet use any breaking API
- But there is a difference between CUDA major version 12 and 13
- No need to worry about the Python minor version (3.10/3.11 ...). The recent wheels use Python stable ABI (also known as ABI3) and have
cp310-abi3in the filenames, so they support all versions of Python >= 3.10
If you see any error, please open an issue at https://github.com/woct0rdho/SageAttention/issues
We've simplified the installation by a lot. There is no need to install Visual Studio or CUDA toolkit to use Triton and SageAttention (unless you want to step into the world of building from source).
Before using SageAttention in larger projects like ComfyUI, please run test_sageattn.py to test if SageAttention itself works.
To use SageAttention in ComfyUI, you just need to add --use-sage-attention when starting ComfyUI.
Some models such as Wan and Qwen-Image may produce black or noise output when SageAttention is used, because some intermediate values overflow SageAttention's quantization. In this case, you may use the PatchSageAttent
6746
ionKJ node in KJNodes, and choose sageattn_qk_int8_pv_fp16_cuda, which is the least likely to overflow.
You may also adjust pv_accum_dtype. pv_accum_dtype="fp16+fp32" (or "fp32+fp16" in some interfaces) is faster than pv_accum_dtype="fp32", and pv_accum_dtype="fp16" is even faster, but more likely to cause black/noise/degraded output.
(This is for developers)
If you need to build and run SageAttention on your own machine:
- Install Visual Studio (MSVC and Windows SDK), and CUDA toolkit
- Clone this repo
- Checkout
abi3_stablebranch if you want ABI3 and libtorch stable ABI, which supports PyTorch >= 2.10 - Checkout
abi3branch if you want ABI3, which supports PyTorch >= 2.4 - There is no Python API difference between
main/abi3/abi3_stablebranches, but I recommendabi3_stablewhenever possible, because it's more compatible with other PyTorch features such astorch.compileand multi-GPU
- Checkout
- Install the dependencies in
pyproject.toml, including the desired torch version such astorch 2.10.0+cu128 - Run
python setup.py install --verboseto install directly, orpython setup.py bdist_wheel --verboseto build a wheel- This avoids the environment checks of pip. There are warnings like
Please avoid running ``setup.py`` directly, but currently it's the simplest way to deal with the exact version of torch
- This avoids the environment checks of pip. There are warnings like
- The wheels are built using the workflow
- It's tricky to specify both torch (with index URL at download.pytorch.org ) and pybind11 (not in that index URL) in an isolated build environment. The easiest way I could think of is to use simpleindex
- CUDA kernels for sm80/89/90 are bundled in the wheels, and also sm120 for CUDA >= 12.8
- For Turing GPUs (GTX 16xx, RTX 20xx), SageAttention 2 runs Triton kernels, which are the same as SageAttention 1. If you want to help improve the CUDA kernels for Turing, you may see https://github.com/Ph0rk0z/SageAttention2/tree/updates
- Volta GPUs (V100) are not supported because they do not have int8 tensor core
- The wheels do not use CXX11 ABI
- We cannot publish the wheels to PyPI, because PyPI does not support multiple PyTorch/CUDA variants for the same version of SageAttention. People are working on this, see https://astral.sh/blog/introducing-pyx and https://wheelnext.dev/proposals/pep817_wheel_variant_support/
- The recent wheels are bitwise reproducible given the same MSVC/WinSDK/PyTorch/CUDA/cibuildwheel versions. They're built with
/experimental:deterministic,/Brepro,/pathmap - The recent wheels are built with attestation. Given the .whl file, you can query the attestation from GitHub's server and see the workflow's commit hash and the link to job metadata. The checked out code's commit hash is kept in the job name, even after the build log expires