Tags: basnijholt/agent-cli
Tags
Fix quadratic Wyoming audio buffering (#634) * fix: accumulate Wyoming audio chunks in place * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci: preserve Ruff lint baseline after 0.16 * ci: pin Ruff hook to project baseline --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
fix(whisper/mlx): release Metal buffer cache after each transcription… … to prevent unbounded memory growth under --ttl 0 (#620) * fix(whisper/mlx): release Metal buffer cache after each transcription Under `--ttl 0` the MLX Whisper worker subprocess is never recycled, and mlx-whisper never releases MLX's reusable Metal buffer cache between requests. The cache ratchets up to the largest working set ever seen and gets pushed to swap; on a 12-day-old deploy the worker reached a 17.1 GB footprint (vmmap: IOAccelerator 16.9 GB resident / 12.6 GB swapped) for a 2.9 GB model, degrading a ~2.0s transcription to ~5.2s. Call `mx.clear_cache()` after each `mlx_whisper.transcribe()`. It frees only the reusable buffer cache, not live arrays, so the loaded model weights stay resident and inference stays hot (no cold reload). Validated on the real large-v3 model (100 requests, cycling clip lengths): cache peak 878.8 MB -> 0.0 MB, at a cost of 7.6 ms/req (~0.3%). Adds tools/repro_cache_leak.py and a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(whisper/mlx): clear Metal cache in finally + validate repro request count Address Greptile review on PR #620: - P1: move mx.clear_cache() into a finally block so a transcription that raises still releases the Metal buffers it allocated. The worker subprocess is long-lived under --ttl 0, so without this a failed request would retain cache and reintroduce swap-driven growth. Adds a regression test covering the failure path. - P2: reject non-positive --requests in tools/repro_cache_leak.py with a clear argparse error instead of an IndexError on empty results. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(whisper/mlx): pin mlx>=0.24 for clear_cache; address review The top-level mx.clear_cache() added in the previous commit does not exist before mlx 0.24 -- 0.21.1, 0.22.1 and 0.23.2 only expose the deprecated mx.metal.clear_cache(). mlx-whisper only requires mlx>=0.11, so the resolver was free to land on a version without it. Because the call sits in a finally block, the resulting AttributeError would replace the successful transcription result and fail every request. Pin mlx>=0.24 in the mlx-whisper extra. Also from review of the repro harness: - Move tools/repro_cache_leak.py to scripts/, which already exists for this kind of helper (and carries the ruff per-file-ignore); drop the now unnecessary sys.path.insert -- the package is importable under `uv run`. - The comment claimed clips >30s drive the largest working set, but CLIP_LENGTHS topped out at exactly 30.0s, so the multi-window path was never exercised. Add 60s/120s clips and a --clip-seconds override for quick runs. - Report the active_mb trend (first/final/peak/drift) in the summary. Active memory is live arrays: flat active with growing cache means buffer-cache retention, which clear_cache fixes; growing active means something retains live arrays and clear_cache would only be a band-aid. * fix(whisper/mlx): pin mlx>=0.24 and finish the repro-harness review fixes Carries the content for the previous commit, which landed the tools/ -> scripts/ move only. mx.clear_cache() does not exist before mlx 0.24 -- 0.21.1, 0.22.1 and 0.23.2 only expose the deprecated mx.metal.clear_cache(). mlx-whisper only requires mlx>=0.11, so the resolver was free to land on a version without it, and since the call sits in a finally block the resulting AttributeError would replace the successful transcription result and fail every request. Pin mlx>=0.24 in the mlx-whisper extra. Repro harness: - Drop the sys.path.insert; the package is importable under `uv run`. - The comment claimed clips >30s drive the largest working set, but CLIP_LENGTHS topped out at exactly 30.0s, so the multi-window path was never exercised. Add 60s/120s clips plus a --clip-seconds override for quick runs. - Report the active_mb trend (first/final/peak/drift). Active memory is live arrays: flat active with growing cache means buffer-cache retention, which clear_cache fixes; growing active means something retains live arrays and clear_cache would only be a band-aid. The diarization/nemo-whisper requirements churn is unrelated uv-export marker drift that the sync-requirements hook regenerates; it is not caused by the mlx pin. * chore: drop the one-off MLX cache-leak repro harness The script was a throwaway benchmark for validating the clear_cache() fix: it needs Apple Silicon, a real large-v3 download and a multi-hour run to say anything, so it cannot be exercised by CI and would rot. Its results are recorded in the PR description; keep those rather than the code. * build: pin uv in the uv-lock and sync-requirements hooks additional_dependencies: ["uv"] is unpinned, so agent_cli/_requirements/*.txt depended on whichever uv each contributor's pre-commit cache had resolved. uv 0.11.32 exports different markers than 0.11.8 for extras whose closure contains a python-version fork, which is what produced the diarization and nemo-whisper churn in this PR: on a clean main, 0.11.8 regenerates the files byte-for-byte while 0.11.32 rewrites 32 lines. pre-commit.ci skips both hooks, so CI never surfaced the difference. Pin uv==0.11.32 (current latest, and what the regenerated files here were produced with) so the output is a function of the config, not of cache age. Known caveat with 0.11.32's output: in the diarization export it narrows aiohttp (and its attrs/aiosignal/frozenlist/multidict/yarl subtree) to python_full_version < '3.13', while lightning -- which requires fsspec[http], which requires aiohttp -- stays unconditional. That combination looks inconsistent on 3.13+. Accepted deliberately to keep everyone on one uv; revisit if a 3.13 diarization install turns up a missing aiohttp. --------- Co-authored-by: Vraj Shah <vrajshah@Vrajs-Mac-mini.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Bas Nijholt <bas@nijho.lt>
Cut ~1.3s off hotkey-to-microphone latency (#622) * Cut ~1.3s off hotkey-to-microphone latency Pressing the transcription hotkey showed the recording pill almost instantly, but the microphone did not open for another ~1.8s. The dominant cost was `AgentRuntime.loginShellPATH`, which spawned a login+interactive shell (`zsh -lic`) to discover PATH on *every* command spawn. With a real dotfiles setup that is 1.13-1.40s, paid on each hotkey press in user-installed CLI mode. Cache the login shell PATH per runtime instance. It is already resolved during the startup warm-up, so even the first press is free. The cache is invalidated on a forced install so "Install or Update CLI" still picks up a moved binary. The lookup now goes through the injectable `processRunner` so the caching is testable. On the Python side, resolve `__version__` lazily and defer `rich.table` to the version callback, so `importlib.metadata` is no longer imported by every command. `import agent_cli.cli` drops from ~234ms to ~167ms. Measured hotkey-to-first-audio-chunk: ~1.8s -> ~0.50s. * Address PR review: drop no-op rich.table deferral, DRY new tests Deferring `from rich.table import Table` into the version callback saved nothing: `rich.spinner` (via `rich.console` in `core.utils`) and `typer.rich_utils` both import it unconditionally. It also went against the CLAUDE.md rule that rich stays a top-level import. Reverted. The remaining `__version__` laziness is real: the old module-level `version("agent-cli")` call parsed distribution metadata on every command (~33ms, of which ~4ms is reclaimed later by pydantic anyway), so ~29ms net for something only `--version` needs. Extract the duplicated runtime construction in the two new Swift tests into `makeShellPATHProbingRuntime`. * Drop unneeded __all__ and correct the lazy-version comment `__all__` was not needed: the `if TYPE_CHECKING` annotation already tells type checkers about `__version__`, and ruff/mypy pass without it. The comment blamed the ~40ms `importlib.metadata` import, but pydantic imports that module anyway (~4ms). The cost this actually avoids is the `version("agent-cli")` call parsing distribution metadata, ~29ms net. * Only cache successful login shell PATH lookups The first version stored the lookup result in a `String??`, which cached a *failed* probe just as permanently as a successful one. A login shell that failed once — plausible at login, since AgentCLI is a login item — would leave the app on the fallback PATH for the rest of the session, where the old uncached code would have recovered on the next command. Cache only non-nil results, matching how `UserInstalledCLICheckCache` already treats `force`/failure. This also drops the double optional. Add a test for the retry, and let the shared test helper return a `CommandResult` so it can model a failing probe.
Add zellij multiplexer support to agent-cli dev (#590) * Add zellij multiplexer support to agent-cli dev Zellij >= 0.44.0 gained stable tab IDs, initial commands for new tabs, and detached-session control, enabling tmux-parity support: - New Multiplexer ABC (shared session naming, open_in_session, attach_command); Tmux and Zellij both implement it - --multiplexer zellij on dev new/agent: creates or reuses a detached repo-scoped session when outside zellij - --zellij-session flag mirroring --tmux-session - Worktree cleanup closes zellij tabs by matching pane cwd, across all live sessions, skipping the current tab - Legacy write-chars tab opening kept for zellij < 0.44 inside a session - Evidence-backed verification tests (docs/PRs + live probes on 0.44.3) * Address PR review feedback on zellij support - Parse session names from list-sessions with a regex so names containing spaces are preserved (was: first token only) - Fail safe in close_tabs_for_worktree: when inside zellij and the current tab cannot be resolved, skip all tabs in the current session instead of risking closing our own tab - Cache the zellij version check per instance to avoid repeated `zellij --version` subprocess calls * Fix zellij tests on Windows: compare against str(path), not literal POSIX paths
Add cmux support to agent-cli dev (#589) * Add cmux support to agent-cli dev cmux (https://cmux.dev) is a Ghostty-based macOS terminal organized as workspaces with tabs, controlled via a Unix socket through its bundled CLI. agent-cli dev maps repos to cmux workspaces: each repo gets a workspace named after it (created on demand), and each worktree launch opens a new tab inside that workspace, named after the branch, running the agent in the worktree directory. - New Cmux terminal adapter (detect via CMUX_WORKSPACE_ID/CMUX_SURFACE_ID) - Auto-detected when running inside cmux; tmux/zellij still win as innermost - --multiplexer cmux supported on dev new and dev agent (socket-based, works from any terminal) - Success summary shows the cmux workspace; tmux attach hint no longer printed for non-tmux handles - Tests document the cmux CLI evidence (help text + live verification against cmux 0.64.14) * Make cmux detection-only, drop --multiplexer cmux cmux is a terminal, not a multiplexer: it is used when auto-detected (via CMUX_WORKSPACE_ID/CMUX_SURFACE_ID), like other GUI terminals. * Address review feedback and add per-workspace colors for cmux Review fixes: - A failed `send` now closes the idle tab and reports failure instead of returning a handle for a tab where the agent never started - A failed `workspace list` aborts immediately instead of falling through to a doomed (or duplicating) `workspace create` - open_new_tab resolves the main repo root so the generic Terminal interface keeps the one-workspace-per-repo invariant for worktrees New: workspaces get a deterministic color on creation (sha256 of the workspace name mapped onto cmux's 16 named colors), so each repo always gets the same color. Existing workspaces are left untouched. * Fix cmux tests on Windows by computing expected paths from Path/shlex Path('/some/path') stringifies with backslashes on Windows, so the hardcoded POSIX argv strings did not match.
Add live transcription preview (#571) * Add live transcription preview * Update auto-generated docs * Make live preview overlay opt-in * Update auto-generated docs * Document live transcription preview * Address live preview review feedback * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix live preview Swift test expectations * Keep Rich status for inactive live preview console * Cover live preview cancel before final race * Signal live preview stop before cancellation
PreviousNext