MP-Sparse-Attn provides Triton kernels for Diagonal-Tiled Mixed-Precision Attention, targeting efficient low-bit MXFP inference for Transformer models. It combines tile-level mixed-precision computation and kernel fusion to accelerate attention on modern GPUs.
TaQuants protects you from model collapse.
Official implementation of P4Q: Learning to Prompt for Quantization in Low-Bit CLIP (ACM TOMM).
Deploying logic gate networks on FPGAs for faster Edge AI Inference[Implementation of Differentiable Logic Gate Networks (NeurIPS 2024)]
Low-bit LLM inference engine in Rust + CUDA. Converts models to a 4-bit .wstone format (3.75x smaller) and runs them on consumer GPUs. Built for memory-bandwidth-bound decode on Turing, where bytes per weight — not TOPS — sets token throughput.
RWKV-7 FP8 quantized inference - 6.4x decode speedup on Blackwell GPUs with <0.3% accuracy loss. Full FP8 E4M3 weight quantization with fused Triton kernels.
TurboQuant KV-cache compression (2/3/4-bit WHT + PolarQuant) and TQ weight quantization for llama.cpp, kept in sync with upstream.
experimenting with binary brain(C++ framework for training Binary Neural Networks)
BGEMM-CUDA is a CUDA-based low-bit GEMM kernel library for efficient neural network inference. It implements optimized binary and ternary matrix multiplication primitives, including binary-weight and ternary-activation computation, with PyTorch extension support for model-level integration.
[MobiSys 2026] On-device ultra-low-bit LLM inference with LUT.
llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).
Add a description, image, and links to the low-bit-quantization topic page so that developers can more easily learn about it.
To associate your repository with the low-bit-quantization topic, visit your repo's landing page and select "manage topics."