Build software better, together

AtomicBot-ai / atomic-llama-cpp-turboquant

llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

Updated Aug 6, 2026
C++

czg1225 / DMax

Star

DMax: Aggressive Parallel Decoding for dLLMs

acceleration efficiency large-language-models parallel-decoding diffusion-language-models multi-token-prediction

Updated Jul 5, 2026
Python

Indras-Mirror / llama.cpp-turboq-mtp

Star

Fused TBQ4 Flash Attention + MTP + Shared Tensors + Qwen35 SWA Hybrid for llama.cpp — 82+ tok/s, lossless 4.25 bpv KV cache, SWA-bounded deep-context decode (w/ long-range recall) on RTX 4090

cuda quantization mtp kv-cache fwht llama-cpp flash-attention qwen speculative-decoding rtx-4090 multi-token-prediction turboquant tbq4 tensor-sharing

Updated Aug 25, 2026
C++

0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark

Star

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.

benchmark nvidia fp8 llm-serving llama-cpp vllm llm-inference qwen speculative-decoding qwen3 prefix-caching gb10 multi-token-prediction dgx-spark dflash2

Updated Aug 19, 2026
Python

JaydenTeoh / beyond-next-token-prediction

Star

Curated collection of research on the limitations of next-token prediction and methods that go beyond it.

machine-learning transformers sequence-to-sequence language-model next-token-prediction multi-token-prediction

Updated Jul 10, 2026

KaiFelixBennett / hermes-claude-code-local

Star

Run Hermes Agent + Claude Code locally on llama.cpp — zero API costs. A 4h / 7M-token session that would have cost $94 on Claude Opus 4.7

telegram-bot wsl2 autonomous-agent ai-agent llama-cpp local-llm llm-inference gguf speculative-decoding litellm private-ai self-hosted-ai claude-code agentic-coding multi-token-prediction hermes-agent qwen3-6-27b hip-rocm

Updated Jun 8, 2026
Shell

hanxiao / Qwen3.6-35B-A3B-MTP-L4

Star

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Docker image. Decode-optimized to ~91–99 tok/s (min ~91 chat, max ~99 math), lossless (full GPU residency + ECC-off).

gcp moe quantization inference-optimization mixture-of-experts large-language-models llm llama-cpp gpu-inference local-llm qwen gguf speculative-decoding qwen3 multi-token-prediction nvidia-l4

Updated Aug 24, 2026
Shell

atandra2000 / DeepSeek-v3-Lite

Star

Faithful from-scratch reimplementation of DeepSeek-V3 (MLA + aux-loss-free MoE + MTP + speculative decoding), ~412M params, Chinchilla-optimal 8.4B-token training on a single A100 80GB.

pytorch transformer moe mla a100 llm deepseek-v3 multi-token-prediction

Updated Aug 20, 2026
Python

anuj0456 / OpenArch

Star

PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more) — written from scratch for readability and learning, based on Sebastian Raschka's LLM Architecture Gallery.

python deep-learning architecture transformers pytorch language-models attention-mechanism gemma mixture-of-experts llms qwen deepseek llama3 multi-token-prediction

Updated Jul 14, 2026
Python

mmiovski / deepseek-reimplementation

Star

A controlled 180-run study of DeepSeek-inspired MLA, sparse MoE routing, V3-style load balancing, and multi-token prediction under constrained compute.

deep-learning reproducible-research transformers pytorch language-models mixture-of-experts model-efficiency deepseek sparse-models multi-token-prediction

Updated Jul 29, 2026
Python

LeonelSalvo / frontier-llm-techniques-2026-Q1

Star

From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

optimizer pytorch muon mtp from-scratch bpe multi-token-prediction llm-from-scratch

Updated Jun 23, 2026
Python

shreyansh26 / qwen-spec-decode-benchmarking

Star

Some benchmarks on running speculative decoding with Qwen-3.6-35B-A3B with EAGLE3, DFlash and MTP.

speculative-decoding sglang multi-token-prediction eagle3 specforge dflash

Updated Aug 8, 2026
Python

avifenesh / hqmtp

Sponsor

Star

MTP draft-head research harness for Qwen3.5-9B. Verdict: function cuts (pruning, low-rank, distillation) pay a 10-19pt off-distribution tax fidelity cuts do not — the zero-training trimmed-vocab NVFP4 recipe wins at 1.8-2.7x end-to-end in memra. Negative-result ledgers included.

pytorch quantization knowledge-distillation llm-inference speculative-decoding multi-token-prediction

Updated Aug 20, 2026
Python

thc1006 / qwen3.8-speculative-decoding-rtx3090

Star

Controlled benchmark of llama.cpp speculative decoding for Qwen3.8-27B on one RTX 3090. Built-in MTP at n-max 2 gives +59.8% decode and 37% less energy per request; DFlash2 (PR #27342) reaches +51.9%. Includes a verification-step cost model, memory-bandwidth vs core-clock response, and byte-level losslessness data.

Updated Aug 25, 2026
Python

stix26 / DeepSeek-V3

Star

Fork of DeepSeek-V3 with MTP speculative-decoding improvements, a cache-aware mask fix, CPU test suite, and docs cleanup.

inference moe mtp mixture-of-experts llm speculative-decoding deepseek deepseek-v3 multi-token-prediction

Updated Aug 8, 2026
Python

dakshjain-1616 / MTP-Evaluation-llama.cpp-vs-Baseline

Star

Benchmarks Multi-Token Prediction in llama.cpp against standard autoregressive baselines. Measures tokens per second, perplexity delta, and speculative acceptance rate across model sizes.

benchmark performance cpp inference language-model mtp llm llama-cpp speculative-decoding multi-token-prediction

Updated May 12, 2026
Python

Orbital-ML-Lab / qwen-fp8-mtp-sidecar-for-mtplx

Star

Recover the native MTP predictor missing from the 8-bit MLX Qwen3.8-27B-Uncensored package, build a BF16 sidecar, and reproduce a 15.59 → 48.75 tok/s controlled M4 Max result with MTPLX.

mtp mlx model-conversion apple-silicon bf16 fp8 safetensors local-llm qwen speculative-decoding qwen3 multi-token-prediction m4-max mtplx qwen3-8-27b

Updated Aug 19, 2026
Python

iprajax / gemma4-mtp

Star

Multi-Token Prediction benchmarks for Gemma 4 on Apple Silicon — LiteRT-LM, transformers, and llama.cpp at batch=1 on a MacBook M4 Pro. ~2× speedup reproducible in one specific runtime.

macos benchmark metal transformers gemma mlx edge-ai on-device-ai apple-silicon llama-cpp speculative-decoding multi-token-prediction litert-lm gemma-4 on-edge-llm

Updated May 7, 2026
HTML

aliuyar1234 / proberoute

Star

Research code for ProbeRoute, a probe-initialized sparse routing method for frozen-backbone multi-token prediction

machine-learning transformers pytorch language-models efficient-llm multi-token-prediction sparse-routing

Updated Apr 18, 2026
Python

Intuition-Labs-LLC / memtp

Star

Multi-Ensemble Memory-Elastic Token Prediction — a numpy implementation of DeepSeek's DSpark speculative decoding + the MEMTP elastic-ensemble extension, on AMD Strix Halo (gfx1151).

amd numpy mtp rocm kv-cache llm-inference speculative-decoding deepseek strix-halo multi-token-prediction gfx1151 dspark

Updated Jun 29, 2026
Python

Navigation Menu

multi-token-prediction

Here are 26 public repositories matching this topic...

AtomicBot-ai / atomic-llama-cpp-turboquant

czg1225 / DMax

Indras-Mirror / llama.cpp-turboq-mtp

0xBakeer / Qwen3.8-27B-FP8-on-a-single-DGX-Spark

JaydenTeoh / beyond-next-token-prediction

KaiFelixBennett / hermes-claude-code-local

hanxiao / Qwen3.6-35B-A3B-MTP-L4

atandra2000 / DeepSeek-v3-Lite

anuj0456 / OpenArch

mmiovski / deepseek-reimplementation

LeonelSalvo / frontier-llm-techniques-2026-Q1

shreyansh26 / qwen-spec-decode-benchmarking

avifenesh / hqmtp

thc1006 / qwen3.8-speculative-decoding-rtx3090

stix26 / DeepSeek-V3

dakshjain-1616 / MTP-Evaluation-llama.cpp-vs-Baseline

Orbital-ML-Lab / qwen-fp8-mtp-sidecar-for-mtplx

iprajax / gemma4-mtp

aliuyar1234 / proberoute

Intuition-Labs-LLC / memtp

Improve this page

Add this topic to your repo