Run, manage, and serve on-device AI models from the terminal. One binary, multi-modal: LLM chat, VLM image understanding, speech-to-text, text-to-speech, voice activity detection, and a full voice pipeline — all running locally on the RunAnywhere C++ core.
$ rcli models download qwen3
pulling qwen3-0.6b ▕████████████▏ 100% 639 MB/639 MB 32 MB/s
$ rcli llm generate --model qwen3 "Reply with exactly: RCLI WORKS" --reasoning off
RCLI WORKS
$ rcli tts synthesize "RunAnywhere runs models on device." --output hello.wav
$ rcli stt transcribe hello.wav
$ rcli serve qwen3 # OpenAI-compatible API on :8080Homebrew (macOS Apple Silicon or Linux x86_64):
brew install runanywhereai/tap/rcliInstall script (macOS Apple Silicon or Linux x86_64):
curl -fsSL https://raw.githubusercontent.com/RunanywhereAI/RCLI/main/install.sh | shPowerShell (Windows x86_64):
irm https://raw.githubusercontent.com/RunanywhereAI/RCLI/main/install.ps1 | iexThe Windows installer verifies the release checksum, installs rcli.exe and pinned ONNX Runtime/Sherpa DLLs under %LOCALAPPDATA%\Programs\rcli\bin, and adds that directory to the user PATH.
| Platform | Engines | Notes |
|---|---|---|
| macOS Apple Silicon | llama.cpp, MLX, Sherpa-ONNX, ONNX Runtime, CoreML | Signed/notarized DMG in GitHub Releases |
| Linux x86_64 | llama.cpp, Sherpa-ONNX, ONNX Runtime | |
| Windows x86_64 | llama.cpp, Sherpa-ONNX, ONNX Runtime | rcli serve is macOS/Linux-only |
MLX is Apple Silicon only. Commands that require an unavailable engine return a clear unsupported-backend error.
From source: see Building from source.
The command surface follows the SDK public API spec: one namespace per modality, the spec's verb under it, and option names that match the spec's option fields (--max-output-tokens, --top-p, --reasoning, --speed, --guidance-scale).
| Command | Description |
|---|---|
rcli llm generate [prompt] |
Complete a prompt and print the result |
rcli llm stream [prompt] |
Complete a prompt, printing tokens as they arrive |
rcli vlm generate --image f.png [prompt] |
Answer a prompt about an image |
rcli stt transcribe a.wav |
Transcribe an audio file (default: whisper-tiny) |
rcli tts synthesize "…" -o o.wav |
Write spoken audio to a WAV file (default: Piper Lessac) |
rcli vad detect a.wav |
Report speech segments with timestamps (default: silero) |
rcli embed [text] |
Turn text into embedding vectors |
rcli rerank <query> -m <model> -d "…" |
Score documents against a query, best first |
rcli image generate -p "…" -o o.png |
Render an image from a prompt (Core ML, Apple only) |
rcli diarize a.wav -m <model> |
Label who spoke when |
rcli segment image.ppm -m <model> |
Label every pixel of an image by class |
rcli rag query <question> |
Answer a question over --doc / --file documents |
rcli voice a.wav [-o reply.wav] |
Hold one spoken turn: STT → LLM → TTS |
rcli models list |
List models, downloaded ones by default (--all for the catalog) |
rcli models get <model> |
Show one model's registry entry |
rcli models register <url> |
Add a model from a URL or hf.co ref |
rcli models download <model> |
Fetch a model, resuming a partial download |
rcli models delete <model> |
Remove a model's files and registration (-f skips the prompt) |
rcli models load <model> |
Load a model now instead of on first use |
rcli models unload [category] |
Free loaded models, all of them by default |
rcli models state |
Report resident models and disk usage |
rcli lora {apply,remove,list,catalog} |
Attach LoRA adapters to a language model |
rcli serve [model] |
OpenAI-compatible HTTP server (/v1/chat/completions, /v1/models, /health) |
rcli bench [model] |
Benchmark downloaded LLM/STT/TTS/VLM models |
rcli telemetry {emit,blast} |
Drive the control-plane telemetry pipeline (no model needed) |
rcli backends |
Registered inference backends per primitive |
rcli info (doctor) / rcli version |
Environment and version info |
rcli auth login |
Authenticated control-plane login (production) |
Generation, transcription and synthesis load what they need: name a model with --model and rcli downloads it if it is missing, then loads it before the first token. rcli models load is for paying that cost when you choose to.
The shorter spellings are the same commands, not separate ones:
| Alias | Namespaced form |
|---|---|
rcli run <model> [prompt], rcli chat <model> |
rcli llm stream (REPL when no prompt is given) |
rcli list, rcli ls |
rcli models list |
rcli show <model> |
rcli models get |
rcli pull <model> |
rcli models download |
rcli rm, rcli remove |
rcli models delete |
rcli stt --input a.wav |
rcli stt transcribe a.wav |
rcli tts --text "…" |
rcli tts synthesize "…" |
rcli vad --input a.wav |
rcli vad detect a.wav |
rcli run --image f.png |
rcli vlm generate --image f.png |
Older flag spellings keep working next to the spec names: --max-tokens for --max-output-tokens, --temp for --temperature, --system for --system-prompt, --no-think for --reasoning off, --negative for --negative-prompt, --guidance for --guidance-scale, --min-duration for --minimum-duration-ms, --merge-gap for --merge-gap-ms.
Global flags: --json (one machine-readable document on stdout), --home <dir>, -v/--verbose, -q/--quiet, --no-progress, plus the control-plane trio --environment <development|production>, --base-url <url>, and --api-key <key> (see docs/RELEASING.md).
Exit codes: 0 ok · 1 runtime error · 2 usage error · 130 cancelled.
Launch with no prompt when stdin is a TTY:
rcli chat qwen3Features line editing and history (~/.local/state/runanywhere/history; disable with RUNANYWHERE_NOHISTORY=1).
Slash commands: /set system <text>, /set temperature <f>, /set max-output-tokens <n>, /show, /bye (or Ctrl-D). One Ctrl-C cancels the current generation.
Thinking models (qwen3 family): thought tokens stream dimmed to stderr, answers to stdout. --hide-thinking keeps the thoughts off your terminal while the model still thinks; --reasoning off stops it thinking at all.
rcli models list --all shows the built-in catalog — Qwen3, Llama 3.2, SmolLM2, SmolVLM2, Whisper, Piper voices, Silero VAD, MiniLM embeddings, and more. Short aliases work everywhere: qwen3, whisper-tiny, piper, smolvlm2, …
Fetch models outside the catalog:
rcli models download hf.co/Qwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf
rcli models download https://example.com/model.ggufrcli models register <url> does the registration step alone, when you want the entry now and the bytes later.
URL registrations persist under <home>/RunAnywhere/Registry/.
One knob: the RunAnywhere home (--home, $RUNANYWHERE_HOME, default ~/.local/share/runanywhere).
~/.local/share/runanywhere/Models/{LlamaCpp,Sherpa,ONNX,...}/<model-id>/…
~/.local/share/runanywhere/Registry/ # persisted URL registrations
~/.config/runanywhere/secure/ # secure store (0600 files)
~/.local/state/runanywhere/history # REPL history
Models pulled by rcli are shared with other RunAnywhere desktop apps using the same home directory.
This repo does not compile the SDK. Stage a C++ desktop kit first (from
runanywhere-sdks: cmake --preset cpp-desktop-macos-arm64 then
--target package-cpp-desktop), then:
cmake -B build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_PREFIX_PATH=/path/to/cpp-desktop-macos-arm64
cmake --build build -j "$(sysctl -n hw.logicalcpu)"
./build/rcli-cxx version
./build/rcli-cxx backendsOn Apple Silicon the shipping binary is the Swift MLX host:
scripts/build-mlx.shThat writes build/rcli. CMake's binary is named rcli-cxx so a later
cmake --build cannot overwrite it.
The kit version is pinned in cmake/sdk-pin.cmake. Pointing RCLI_SDK_DIR at
SDK source is a configure error.
See CONTRIBUTING.md and docs/RELEASING.md.
ctest --test-dir build --output-on-failure
bash scripts/smoke.sh ./build/rcli-cxxrcli is a consumer of the rac_* C ABI via find_package(RunAnywhere).
Commands are thin wrappers: parse → bootstrap → one commons call → render.
Inference, catalog, download, and lifecycle live in the kit, not here.
serveis LLM-only and single-model.- REPL turns are independent (no conversation memory yet).
rcli voicewith thinking models may speak reasoning text — use a non-thinking LLM (--llm lfm2) until voice-agent thinking control lands.- macOS x86_64, Linux ARM64, and Windows ARM64 binaries are not published yet.
- Spec verbs with no standalone command yet:
tts speak(commons synthesizes to a buffer and has no playback path), andrag open/rag ingest(RAG indexes are in-memory per process).rcli rag queryandrcli rag searchopen, ingest, then ask or retrieve in one invocation instead. VLMGenerationOptionswas deleted;vlm generate(andrun --image) now share the exactLLMGenerationOptionsthe text path uses, so--seed,--frequency-penalty, and--presence-penaltyall apply to VLM generation too.rcli lora importwas removed:idl/lora_options.protodeletedLoraAdapterImportRequest/Resultoutright, and commons permanently stubsrac_lora_adapter_import_prototoRAC_ERROR_NOT_IMPLEMENTED. No replacement verb exists in this namespace yet.models loadtakes--engineand--categoryonly.ModelLoadRequestcarries no context length, thread count or GPU switch;rcli servehas--context,--threadsand--gpu-layersfor the server it runs.vad detectexposes--activation-threshold. The spec'sminSpeechMs,minSilenceMsandprefixPaddingMsare stream-level knobs that the per-framerac_vad_component_processcall does not accept.stt transcribehas no--translate-to-english:rac_stt_options_thas no field for it.
- Documentation: docs.runanywhere.ai
- Discord: discord.gg/N359FBbDVd
- Email: founders@runanywhere.ai
See the repository LICENSE.