Benchmarks for small (~4B) tool-calling open-weight LLMs on commodity x86 CPU — standard vs TurboQuant KV-cache compression, measuring tool-calling accuracy, throughput, and memory.
-
Updated
May 22, 2026 - Python
#
cpu-inference
Here are 220 public repositories matching this topic...
A FastAPI server for querying Google's Gemma Translate AI models for translations
docker google cuda translate gemma ai-api fastapi cpu-inference gpu-inference translategemma gemmatranslate
-
Updated
Apr 26, 2026 - Python
Local-first CLI, Docker, and Tauri workspace for hardware-aware Kimi K3 inference on consumer systems.
-
Updated
Aug 21, 2026 - C
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
python nlp machine-learning natural-language-processing cpu deep-learning transformers llama language-models faiss sentence-transformers cpu-inference large-language-models llm chatgpt langchain document-qa open-source-llm c-transformers llama-2
-
Updated
Nov 6, 2023 - Python
Summarize, explain, fact-check, or translate any text, URL, or file. No GPU. No cloud. One command
cli search-engine privacy translation summarization fact-checking eli5 cpu-inference llm llama-cpp local-llm local-ai
-
Updated
Jun 15, 2026 - Python
⚡ LLM Gateway is a zero-config local AI gateway. Run any GGUF model on your CPU with one command to instantly spin up an OpenAI-compatible API gateway. The ultimate lightweight LLM gateway—no Python, no Docker, no GPU required.
golang ai api-gateway gateway self-hosted developer-tools zero-config huggingface go-tool openai-api chatbot-backend cpu-inference llm generative-ai llama-cpp local-llm gguf ai-gateway open-source-ai llm-gateway
-
Updated
Feb 21, 2026 - Go
Lightweight web UI for llama.cpp with dynamic model switching, chat history & markdown support. No GPU required. Perfect for local AI development.
javascript sqlite self-hosted developer-tools web-interface python-flask chat-ui ai-assistant conversation-history model-switching cpu-inference performance-optimized llama-cpp local-ai llm-frontend open-source-ai offline-ai private-ai markdown-chat
-
Updated
Dec 10, 2025 - Shell
Open source lightweight Sanskrit language model optimized for CPU, edge devices, and offline inference.
text-to-speech offline edge tts sanskrit edge-extension sanskrit-language sanskrit-grammar fastapi cpu-inference indicf5
-
Updated
Jul 9, 2026 - Python
Compress PyTorch models for edge devices — CPU-only, no GPU, no retraining. One function call.
python iot machine-learning deep-learning pytorch evolutionary-algorithm nas nsga2 model-compression neural-architecture-search onnx edge-ml cpu-inference embedded-ml
-
Updated
Jul 28, 2026 - Python
Dockerized RAG system optimized for CPU inference. No GPU required.
-
Updated
Jul 28, 2026 - Python
A local model router: a tiny always-on model classifies each request and hot-swaps a specialist in via Ollama only when one is needed. Runs entirely on CPU.
-
Updated
Aug 15, 2026 - Python
CPU-only FastAPI server for Typhoon ASR Realtime
speech-recognition speech-to-text thai nemo thai-nlp typhoon asr fastapi cpu-inference fastconformer
-
Updated
Jun 10, 2026 - Python
Real-time facial emotion detection using DeepFace + OpenCV — CPU-only, no GPU required. Frame-skip optimisation for live webcam inference.
-
Updated
Jul 19, 2026 - Python
Un sistema RAG per chattare con documenti locali usando Foundry e modelli LLM su CPU
-
Updated
Oct 27, 2025 - Python
Deepixel develops real-time human understanding algorithms from a single RGB camera, with full in-house capability spanning data acquisition, annotation, model training, and platform-optimized deployment—ensuring robust, efficient, and production-ready vision systems.
python opencv computer-vision deep-learning wasm ios-app android-app landmark-detection face-tracking face-landmarks head-pose-estimation tensorflow-lite virtual-try-on on-device-ai cpu-inference monocular-vision real-time-inference deepixel
-
Updated
Jun 11, 2026
llama-swap Docker image bundling an ik_llama.cpp server alongside mainline llama.cpp (amd64)
-
Updated
Jul 6, 2026 - Dockerfile
Pocket TTS Setup for Windows. Easy to Use.
-
Updated
Jun 11, 2026 - Python
CPU-only local coding agents with OpenCode, llama.cpp and interchangeable GGUF models — tested on Intel Ivy Bridge.
-
Updated
Aug 22, 2026 - Python
CPU-efficient face age and gender models and training pipeline
computer-vision deep-learning pytorch gender-classification face-analysis age-estimation onnx face-attributes mobilenetv3 onnxruntime model-distillation cpu-inference apache-2-0 fastface
-
Updated
Aug 3, 2026 - Python
End-to-end receipt extraction pipeline using LayoutLMv3 + EasyOCR with a Streamlit Human-in-the-Loop review interface. Runs entirely on CPU.
sqlite fastapi huggingface streamlit easyocr document-intelligence token-classification cpu-inference layoutlmv3 receipt-extraction
-
Updated
Aug 6, 2026 - Python
Improve this page
Add a description, image, and links to the cpu-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the cpu-inference topic, visit your repo's landing page and select "manage topics."