GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Attention + MTP speculative decode
-
Updated
Jun 19, 2026 - Shell
#
sm120
Here are 32 public repositories matching this topic...
An LLM server for a single RTX 5090, built for agent workloads: tool calls, long conversations, reasoning, and many requests at once. Consistently faster than llama.cpp on the same models, with the numbers in the repo. Written end to end by Claude Code.
cpp cuda quantization inference-engine mixture-of-experts blackwell llm safetensors llama-cpp vllm local-llm llm-inference qwen gguf fp4 rtx-5090 sm120 nvfp4 rtx-pro-6000 gated-deltanet
-
Updated
Aug 25, 2026 - Cuda
NVFP4 inference on Blackwell GeForce (RTX 5090/5080/5070 Ti/RTX PRO 6000) — SM120 patches for vLLM + FlashInfer + CUTLASS. 175 tok/s on Qwen3.6-35B MoE.
gpu-computing quantization cutlass gemm geforce blackwell vllm llm-inference flashinfer rtx-5090 sm120 nvfp4
-
Updated
Apr 27, 2026 - Python
Reproducible SGLang recipe + public prebuilt image (ghcr.io) for DeepSeek-V4-Flash-0731 on 4x RTX PRO 6000 Blackwell (SM120): TP4/DP4/EP4, 1M ctx, benchmarks, and the DSPARK draft-depth corruption boundary
blackwell llm-inference speculative-decoding deepseek sglang flashinfer sm120 rtx-pro-6000 deepseek-v4
-
Updated
Aug 9, 2026 - Dockerfile
Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode.
quantization gemma blackwell fp8 vllm speculative-decoding sm120 nvfp4 abliterated gemma-4 modelopt tensor-parallel
-
Updated
Jun 7, 2026 - Python
Lna-Lab production pipeline: GGUF -> modelopt-format NVFP4 + working MTP head for vLLM on RTX PRO 6000 Blackwell (SM120). Stages 2 (NVFP4) and 3 (MTP graft) are Lna-Lab originals; stage 1 (GGUF->bf16) reuses li-yifei/gguf-to-nvfp4.
-
Updated
Apr 27, 2026 - Python
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
rust ffi cuda inference moe quantization mamba state-space-models deltanet blackwell engram llm qwen speculative-decoding sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Jul 28, 2026 - Rust
Optimized vLLM deployment for NVIDIA Blackwell (RTX 5090) on Linux Kernel 6.14. Resolves SM_120 kernel incompatibilities, P2P deadlocks, and memory fragmentation for high-performance LLM inference.
-
Updated
Jan 17, 2026 - Dockerfile
The only deployed sparse FP4 GEMM on SM120: beats CUTLASS 80b on every shape, wins end-to-end request latency in 81 of 112 serving regimes vs dense NVFP4.
cuda sparse quantization gpu-kernels mixed-precision tensor-cores blackwell vllm llm-inference fp4 sm120 nvfp4 2-4-sparsity
-
Updated
Aug 4, 2026 - Python
Drop-in exact bf16 flash-attention for CUDA with a deterministic backward, tuned for Blackwell (sm_120 / RTX 5090).
deep-learning gpu cuda inference pytorch transformer triton attention memory-efficient attention-mechanism deterministic memory-optimization bfloat16 tensor-cores blackwell kv-cache llm-inference flash-attention rtx-5090 sm120
-
Updated
Jul 8, 2026 - Python
Production-grade FlashAttention FP8 e4m3 forward kernel for NVIDIA Blackwell consumer GPUs (sm_120a, e.g. RTX PRO 6000). 647–652 TFLOPS at hd=128, sl=8192. Multi-kernel dispatcher, C library with Go and Python bindings
cuda transformer attention gpu-kernels tensor-cores blackwell fp8 flash-attention fp8e4m3 sm120 rtx-pro-6000 sm-120a
-
Updated
Aug 3, 2026 - Cuda
Build the GPU inference stack from source, repeatably: for Blackwell sm_120 on CUDA 13.x and Python 3.14. Machine-readable build state, generated patches, and an abductive-triage skill for build failures.
gpu cuda inference pytorch nvidia blackwell build-from-source llama-cpp vllm agent-skills sm120 python-314 cuda-13 abductive-triage
-
Updated
Aug 5, 2026 - Shell
Serve an abliterated Gemma-4-12B at high speeds on Blackwell GPUs without NVLink using vLLM, FP8 quantization, and MTP speculative decoding.
quantization gemma blackwell fp8 vllm speculative-decoding sm120 nvfp4 abliterated modelopt tensor-parallel
-
Updated
Aug 25, 2026 - Python
GLM-5.2-504B NVFP4 at 250K context on 4x RTX PRO 6000 Blackwell (sm_120) using STOCK vLLM — no fork, no Docker, no CUDA 13.2. One ~126-line patch. Documents the 3 upstream bugs that block you, with exact error strings and fixes.
-
Updated
Jul 16, 2026 - Python
Downstream llama.cpp TurboQuant CUDA fork with adaptive KV layout selection for long-context inference on consumer Blackwell GPUs.
cuda inference moe quantization blackwell kv-cache long-context llama-cpp local-llm rtx-5080 sm120 turboquant
-
Updated
May 1, 2026 - C++
Run DeepSeek-V4-Flash-0731 on 4× RTX PRO 6000 with patched SGLang, fixing DSPARK crashes and draft-depth corruption for production serving.
react language terminal ai tui text-editor developer-tools ink r1 vision-language-pretraining prompt-caching llm-inference ai-coding coding-agent sm120
-
Updated
Aug 25, 2026 - Dockerfile
Improve this page
Add a description, image, and links to the sm120 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sm120 topic, visit your repo's landing page and select "manage topics."