Stars
DeepSeek-V3/R1 inference performance simulator
Virtual whiteboard for sketching hand-drawn like diagrams
A Datacenter Scale Distributed Inference Serving Framework
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
DeepEP: an efficient expert-parallel communication library
Fully open reproduction of DeepSeek-R1
Puzzles for learning Triton, play it with minimal environment configuration!
Development repository for the Triton language and compiler
A simple pip-installable Python tool to generate your own HTML citation world map from your Google Scholar ID.
[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
A fast communication-overlapping library for tensor/expert parallelism on GPUs.
You like pytorch? You like micrograd? You love tinygrad! ❤️
Byted PyTorch Distributed for Hyperscale Training of LLMs and RLs
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…
A retargetable MLIR-based machine learning compiler and runtime toolkit.
Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
Standalone Flash Attention v2 kernel without libtorch dependency
Fast and memory-efficient exact attention
Making large AI models cheaper, faster and more accessible
Specialized Parallel Linear Algebra, providing distributed GEMM functionality for specific matrix distributions with optional GPU acceleration.
SLATE is a distributed, GPU-accelerated, dense linear algebra library targetting current and upcoming high-performance computing (HPC) systems. It is developed as part of the U.S. Department of Ene…