OSGym: Scalable OS Infra for Computer Use Agents

It is natural to consider three design options for the state manager: centralized, semi-decentralized, and decentralized, as illustrated in Figure 2. Full centralization introduces a critical performance bottleneck and poses significant risks to robustness. As the number of OS replicas scales into thousands, the centralized manager quickly becomes overwhelmed, leading to increased latency, reduced responsiveness, and an increased risk of single-point failures that can halt the entire system. In the semi-decentralized alternative, inter-group coordination still requires complex communication mechanisms, which may introduce delays and synchronization challenges, limiting scalability. OSGym adopts a fully decentralized design, where each OS replica has its own dedicated state manager. This architecture achieves optimal scalability and robustness, effectively eliminating bottlenecks associated with centralized control. Individual managers handle state transitions, monitor health, and recover autonomously from local failures. This isolation ensures that failures in one replica do not propagate, greatly enhancing system reliability and ease of maintenance.

As we know, running full-fledged OS sandboxes with GUI environment can consume a non-trivial amount of computing resource. We choose to run replicas as Dockers rather than Virtual Machines to save the per-replica resource, and we found that the Docker images provided by OSWorld [34] are a good starting point. Cloud-hosting or self-hosting the OS replicas requires a large number of virtual or physical CPU servers, and the maximizing resource usage can significantly improve horizontal scaling. One option is to spread N replicas into N small servers, and the second option is to group the replicas and host each group on a larger server, as illustrated in Figure 3. Let N be the number of OS replicas, and M be the number of servers, then K = N / M is the group size. We provide a useful insight regarding the different bottlenecks faced by the system under different K:

To help the readers understand, we freeze N and change K and plot the two graphs at the bottom of Figure 3. The bottom-left plot shows that for small K, almost every replica is CPU-overloaded. When changing to a large K, even though the total amount of CPU resource is unchanged, the CPU overload diminishes because different replicas usually have peak CPU usage at not completely overlapping time. Under a large K, the bottleneck is no longer the CPU but the RAM. Scaling RAM is significantly cheaper than scaling CPUs. A 32GB RAM with DDR4 is usually only 10% to 20% of the price of a 16-core CPU. We also provide a more concrete example of the cost in the bottom right plot of Figure 3. Under the current market price of cloud computer on-demand rental, assuming the CPU is Intel Xeon series, and the RAM is DDR4 from Samsung, and we run 128 OS replicas. The daily cost is around 300 USD if K = 1, but only around 30 USD if K = 64. Each replica costs around 30 / 128 = 0.234 USD per day. Typically, 128 replicas can already support a decent academia-scale experiment on agent training, and the cost perfectly fits into academia budget in many cases. We hope that this discovery can help many academia labs unlock at least a part of the scaling potentials in general-purpose computer agent research.

A practical challenge in deploying a large amount of OS replicas is disk provisioning. Each VM requires its own bootable disk image of approximately 24 GB. Naively duplicating the base image for each replica would require terabytes of storage and minutes of provisioning time per VM, making large-scale deployment infeasible.

OSGym addresses this through filesystem-level reflink copy-on-write (CoW). Each per-VM disk image is created instantaneously with cp --reflink=always and shares physical disk blocks with the base image. Blocks are only physically allocated when a VM writes to them. Only blocks modified by each VM are allocated. Combined with KVM hardware virtualization via /dev/kvm, each replica boots from its own CoW copy of the base image with near-zero disk overhead and near-native CPU performance. We will show in the experiment section that our method reduces disk consumption by around 88%, which means we can obtain the full logical disk by using only 12% physical disk.

We provision storage on NVMe drives striped in RAID 0 or LVM, formatted as XFS, the only production-grade filesystem supporting both reflink copy-on-write and high-concurrency I/O. Several XFS performance tunings are applied, including large preallocation (allocsize=1G) to reduce fragmentation from concurrent qcow2 writes, disabled access-time updates (noatime), and an enlarged log buffer (logbsize=256k). To counteract gradual XFS performance degradation under sustained high-concurrency VM I/O, OSGym optionally deploys a systemd timer that reboots idle nodes every 48 hours, keeping the filesystem in peak condition.

Creating and destroying VM instances on-demand is prohibitively slow and fragile: a single failure in the creation-execution-teardown chain can stall an entire training batch. Also, running 128 VMs per node quickly exhausts default kernel resource, causing silent failures under high concurrency.

OSGym addresses these challenges through the following design: Pre-Warmed Runner Pool: Each executor node maintains a fixed-size runner pool (default: 128 runners) initialized at startup. Rather than provisioning VMs on demand, OSGym pre-creates all instances before training begins and recycles them between tasks: upon task completion, each runner is reset and returned to the pool, ready for immediate reassignment. Resource Guard: Before each VM creation, OSGym reads /proc/meminfo and /proc/loadavg to verify that the host can safely accommodate another instance. Creation is blocked if available memory falls below 10% or under 8 GB absolute. To prevent over-provisioning under burst scenarios, OSGym tracks in-flight creations and subtracts their estimated memory footprint (each container is limited to 6 GB) before evaluating available headroom. System Limits Tuning: Running 128 VMs within Docker containers requires explicit kernel parameter tuning. Default system limits, including file descriptors, inotify watches, AIO contexts, and netfilter connection tracking, are far too low for high-concurrency VM workloads and cause silent failures without adjustment. OSGym raises these limits to production-appropriate values (e.g., fs.aio-max-nr from 65,536 to 1,048,576; fs.inotify.max_user_instances from 128 to 8,192). Multi-Layer Retry and Error Recovery: OSGym implements fault tolerance at two levels. At the step level, each action execution is wrapped with a configurable retry policy (default: 10 retries) covering connection errors, timeouts, and runtime operation failures. At the task level, if a runner fails permanently, the task is reassigned to a fresh runner from the pool; leaked tasks that exceed a timeout threshold are automatically reclaimed and their runners returned to circulation. A gateway layer further provides task-affinity routing, background health checks every 10 seconds, and automatic failover when an executor node becomes unreachable. Together, these mechanisms ensure that isolated replica failures do not propagate into training disruptions.

OSGym inherently supports an extensive range of tasks thanks to its deployment of fully operational OS replicas, rather than specialized, constrained sandboxes. Tasks from diverse software domains, such as software engineering (e.g., code debugging, software testing), office applications (e.g., word processing, spreadsheet manipulation), internet browsing, tool-based interactions, file management, and even complex multi-software workflows, can all be naturally supported within the unified OSGym infrastructure. OSGym adopts a unified execution flow comprising four consistent phases: 1) Configure. Setting up necessary software, and preparing the OS environment with customized conditions. 2) Reset. Before executing a task, the OS environment is reset to the initial conditions defined during the configuration, ensuring reproducibility and consistency between runs. 3) Operate. The agent interacts with the OS through actions such as keyboard inputs, mouse movements, clicks, and potentially API-driven tool interactions, driven by observations typically captured through screenshots or additional metadata extracted from the OS. 4) Evaluate. OSGym evaluates outcomes based on predefined criteria or metrics. We give the user full flexibility to customize the evaluation function, and call the evaluation function whenever necessary. We illustrate this section in Figure 4.

OSGym has a high-level centralized data server Python Class that provides an intuitive, single-entry interface to simplify interactions and data handling across numerous parallel OS replicas. The centralized data server manages all internal communications and queuing complexities with state managers, thus abstracting the low-level details away from the end-users. We illustrate the data server in Figure 5. The key features of this centralized data server include: 1) Single Entry Interface: Offers straightforward, batched methods such as reset and step, making the interaction with multiple OS replicas seamless and easy. 2) Asynchronous Operations: The step method supports asynchronous execution, preventing blocking behavior during training or evaluation loops, significantly enhancing overall efficiency. 3) Internal Queuing and Management: Automatically handles task queuing, replica availability checks, and dynamic load balancing, thereby maintaining system stability and scalability. 4) Fault Tolerance and Recovery: Includes built-in error-handling capabilities to quickly recover from replica failures without interrupting overall service availability.

Scalability. A crucial metric to examine the scalability of a system is whether its throughput proportionally increases with the parallelization size. It is not uncommon to see diminishing returns in large-scale systems, where increasing the system size fails to yield proportional gains in throughput due to bottlenecks, resource contention, or system overhead. OSGym, in contrast, demonstrates highly favorable scalability. As shown in the left plot of Figure 6, the system throughput, measured in steps per second, increases nearly linearly with the number of OS replicas. This indicates that OSGym scales efficiently across a wide range of deployment sizes, from tens to thousands of environments.

Further, the middle plot of Figure 6 reveals that the average step latency per replica experiences only a marginal increase as the number of concurrent replicas grows exponentially. This is a strong indication that OSGym’s decentralized management architecture and semi-decentralized orchestration strategy successfully mitigate common scaling pitfalls. It ensures that each environment continues to operate with minimal degradation even under heavy load. Taken together, these results provide solid evidence of OSGym’s strong scalability, making it suitable for both small-scale experimental setups and large-scale training infrastructures. The system maintains high throughput and reliability across different scales, a critical requirement for sustained and efficient training of general-purpose agents in complex operating system environments.

Robustness. In large-scale distributed systems, robustness is critical to ensuring sustained functionality in the presence of inevitable faults. OS replicas can encounter a wide range of stochastic failures due to software bugs, kernel crashes, system misconfigurations, or network issues. If left unhandled, such failures can accumulate and ultimately halt the entire system. To address this, OSGym integrates a decentralized self-recovery mechanism within each OS state manager. When a replica encounters a critical error or becomes unresponsive, its local manager detects the failure, isolates the faulty instance, and autonomously initiates a recovery procedure. As shown in the right plot of Figure 6, even when the system is initialized in a fully crashed state, OSGym is capable of self-restoring all replicas to a healthy condition within a short recovery window. This high degree of robustness is essential for maintaining long-term, uninterrupted agent training and evaluation at scale.

Resource Efficiency. As shown in Table 1, careful selection of server configurations, particularly those with high memory capacity, allows substantial cost savings when running OSGym at scale. By hosting multiple OS replicas on large-RAM servers, we significantly reduce the per-replica cost. For example, using a server with an 88-core Intel E5-2699 CPU and 768 GB DDR4 RAM, the cost per OS replica can be brought down to just 0.23 USD per day. This makes large-scale experimentation with hundreds of replicas financially feasible for academic labs. This cost efficiency makes it practical for both academic and commercial users to pursue research and development of general-purpose computer agents without excessive infrastructure expenses.

We also evaluate the effectiveness of our KVM virtualization with copy-on-write disk management. As explained in Section 3.3, since each OS replica VM requires its own bootable disk, launching a large amount of VM instances on a bare metal machine would consume huge physical disk if there is no optimization. With reflink copy-on-write, we share the unmodified common blocks across the VMs and only provision the modified blocks for each VM instance. The result is a 88% reduction of physical disk usage and 37 times faster disk provisioning speed as shown in Table 2. The logical disk of each instance is not affected, and it also gives near-native CPU performance.

We used OSGym to implement an example pipeline to train computer-use agents. The pipeline includes highly-parallel data generation, supervised finetuning, and reinforcement learning.

Data Generation with OSGym. We first manually prepared 244 task prompts following the style of OSWorld (but not overlapping with OSWorld original tasks), involving multiple software such as LibreOffice Writer / Calc / Impress, Chrome, GIMP, VLC, VS Code, and ThunderBird, spanning office tasks, professional tasks, daily tasks, multi-app workflow tasks, etc. Then we used existing open-source computer-use agents [2, 23] to run on these tasks to generate a large number of demonstration trajectories. Leveraging OSGym’s massive parallelization, we deployed 1024 OS replicas to execute and collect these demonstrations simultaneously, at an average speed of 1420 trajectories per minute. Each trajectory contains 10 to 25 steps of interleaved states, actions, and thoughts (reasoning) before each action. Thanks to the cost-efficient infrastructure provided by OSGym, the entire dataset was generated within minutes and at a total cloud cost of only 43 USD, making it highly accessible for academic-scale research.

Supervised Finetuning. After data generation, we finetuned the Qwen 2.5-VL 32B [3] model on the collected data. Each data sample is structured as a sequence: task instruction $\rightarrow$ screenshot₁ $\rightarrow$ thoughts₁ $\rightarrow$ action₁ $\rightarrow$ screenshot₂ $\rightarrow$ thoughts₂ $\rightarrow$ action₂ $\rightarrow\dots\rightarrow$ screenshot_C $\rightarrow$ thoughts_C $\rightarrow$ action_C. For training, we conditioned the model on the initial task instruction and the history of prior elements (screenshots, thoughts, and actions). We then applied a softmax cross-entropy loss to the model prediction for each subsequent thought and action in the sequence. We trained the model using the Adam [16] optimizer with a learning rate of $10^{-5}$ until it converged, which took approximately half a day on a 8×H100 machine.

Agent Reinforcement Learning with OSGym. We performed reinforcement learning on the finetuned model using a semi-online asynchronous pipeline implemented with OSGym, where data rollouts and model updates are decoupled and run in parallel. This design maximizes resource utilization and training throughput by keeping the OS replicas continuously busy with interactions while the model updates run independently. For each interaction step, actions are predicted by the current model and dispatched to the corresponding OS replicas via OSGym’s batched, asynchronous interface. The resulting experiences are added to a replay buffer, from which the model samples batches for policy and value updates using standard PPO objectives. The model was trained for 200 steps with batch size 64 and learning rate $10^{-6}$ using Adam [16] optimizer.

To evaluate the model trained with OSGym, we ran it on OSWorld-Verified benchmark with each task given a 100-step limit. The model achieves a success rate of 56.3, which is competitive with existing methods given that it uses a 32B parameter base model with no task-specific tuning. The goal of this experiment is not to establish a benchmark result, but to validate that OSGym supports an effective end-to-end training pipeline, from data collection through supervised finetuning to reinforcement learning, and that the resulting model is a functional computer-use agent.