ABot-World-0 Runs Interactive Worlds on One Desktop GPU

Interactive world models face a different constraint from offline video generators: they must respond to user actions quickly, keep visual state coherent over long horizons, and fit on hardware someone could plausibly run at a desk. ABot-World-0 addresses that deployment target directly. The paper presents an action-conditioned video world model that turns keyboard control into continuous world rollout, then pairs the model with data, training, and inference systems built around the same latency and memory budget.

Figure 1 states the operating target: real-time action-conditioned rollout at 720P on a single desktop GPU, with a reported 1.2 s action-to-first-frame latency and peak memory around 19 GiB.

Core Contribution

The main contribution is not just a larger video model. The authors combine a controllable world-model data pipeline, a causal student training recipe, and a streaming inference stack into one system aimed at closed-loop interaction. That matters because long video generation papers often optimize sample quality or duration under offline generation assumptions, while interactive rollout adds a harder distribution problem: each generated chunk becomes the conditioning context for the next user action.

The paper’s answer is to start from a high-quality bidirectional action-conditioned teacher, then convert it into a causal autoregressive student that can run in a few denoising steps. The new piece is LongForcing, a final distillation stage that exposes the student to longer self-generated contexts and uses an extended-horizon teacher for distribution-level correction. The stated goal is to reduce the drift that appears when local prediction errors accumulate over many rollout chunks.

Technical Approach

ABot-World-0 uses raw keyboard actions as the control interface. The visible architecture page shows action sequences packed into frame-aligned action tokens, passed through an action-control adapter, and injected additively at the patch-embedding stage of a pretrained video DiT. The paper also adds reference-character memory for third-person rollouts: canonical character images are encoded with the same VAE, prepended as memory tokens, and assigned negative temporal RoPE indices so the model can retrieve identity information without treating it as part of the generated trajectory.

Figure 4 shows this model path: VAE tokenization, patch embedding, action-token injection, identity memory, and a causal DiT backbone for frame generation.

The training pipeline has three stages. Teacher forcing adapts the bidirectional teacher into a causal student by conditioning on ground-truth history while predicting future chunks. Causal ODE distillation then trains a few-step model to approximate the probability-flow ODE endpoint of the Stage 1 causal model under the same causal context. LongForcing extends supervision to longer student self-rollouts, so the model is trained on the kinds of contexts it will actually see during closed-loop generation rather than only short clean histories.

Figure 3 summarizes that conversion from pretrained video generator to action-conditioned teacher and then to a causal student through teacher forcing, ODE distillation, and LongForcing.

Results and Analysis

The clearest quantitative result is the deployment envelope. Table 1 reports batch size 1, 1280 × 704 resolution, chunk-wise streaming inference, up to 16 FPS, 1.2 s action-to-first-frame latency, and no more than 19.3 GiB peak VRAM on one NVIDIA RTX 5090. The paper attributes that envelope to a full-stack deployment design: a lightweight VAE decoder, memory-aware scheduling, Fast-RoPE, low-bit mixed-precision DiT execution, and context-memory management.

Those numbers support the paper’s central systems claim: ABot-World-0 is engineered for interactive use rather than only offline video sampling. The 1.2 s first-response latency is still not equivalent to game-engine immediacy, but it is within a range where closed-loop visual interaction becomes plausible for research prototypes. The memory number is also important because it keeps the system inside a single high-end consumer GPU rather than requiring a multi-GPU server.

The visual examples and extracted figures show hour-scale and day-scale rollouts, plus out-of-domain scenes and controllable characters. They are useful evidence for qualitative coherence and action consistency, but they should be read cautiously: sampled keyframes can show persistence without proving frame-by-frame physical consistency, control precision, or failure frequency. The abstract also reports evaluation on WorldRoamBench and extended interactive rollouts, with competitive controllability and coherent long-horizon evolution, but the provided pages expose fewer benchmark details than the deployment table.

Limitations

The evidence is strongest for feasibility on the reported hardware configuration and weaker for general claims about open-ended world simulation. The system depends on a large curated data infrastructure spanning games, simulation engines, and internet videos, with 14 deterministic quality checks, VLM assessment, and synchronized action and text annotation. That data engine is part of the result, but it also makes replication harder if the collection process and datasets are not released.

The method also inherits the usual risks of autoregressive video rollout. LongForcing is designed to reduce accumulated drift, not remove it. The paper’s long-horizon keyframes suggest persistence over hours or days, but interactive world models need stronger tests of causal consistency, controllability under adversarial user input, and recovery from compounding errors before they can be treated as reliable simulators rather than controllable video generators.