Verifiable Rewards Improve Native Visual Reasoning Training

Visual generators are usually assessed for fidelity, while visual reasoning requires a sequence to satisfy semantic constraints: take the correct route, merge objects in order, preserve the scene, or reach a target without teleporting. VBVR-Pro recasts those requirements as a controlled training and evaluation environment for image, video, and interleaved generation. The authors supply 300 procedurally generated tasks, deterministic task-specific scorers, and an RL pipeline intended to make intermediate visual states part of the optimization target.

Core Contribution

The paper's central contribution is not another generator architecture. It is a closed-loop task distribution in which a generated visual trajectory can be checked against known rules. This matters because a final frame can look plausible while concealing an invalid process. The supplied scorers therefore assess both endpoint correctness and trajectory properties such as object order, continuity, scene preservation, synchronization, and path following.

The task examples make that distinction concrete. In Greedy Ball Eating, a black ball must absorb colored balls in ascending-size order; its scorer combines final-state and process terms. Key-Door Navigation checks whether an agent reaches a key and then a door while avoiding walls. Move Objects to Targets evaluates two transport paths and gives a synchronization term only when their start and finish times align. These are narrow synthetic settings, but they produce rewards whose intended semantics are inspectable rather than delegated to a general-purpose judge.

Technical Approach

VBVR-Pro supports video, image, and interleaved generators under the same task distribution. The paper argues that the visual sequence is itself a reasoning trajectory: diffusion samples can explore candidate paths or superposed states before converging. Its counterfactual interventions support that interpretation. For a trained VBVR-Pro-SenseNova-U1 model, replacing the input text with a semantic rephrasing retained an overall score of 0.640 versus 0.638 for the original input, whereas removing task-specific text semantics reduced it to 0.317. Removing the input image was far more damaging, dropping the score to 0.064. Removing intermediate images likewise reduced the score to 0.099, compared with 0.530 after removing intermediate text.

For RL, the authors use group-relative optimization with verifiable terminal rewards and a CPS sampler. A one-step-delayed rollout pipeline overlaps reward computation with subsequent rollout generation and policy optimization. The implementation generates training samples at 512 × 512, then upsamples them to 1024 × 1024 for reward evaluation. This design is practical because rule-based scorers can run on CPU workers, unlike VLM judging that competes for GPU memory.

Results and Analysis

The transfer evidence is broad for a synthetic training suite. Fine-tuning VBVR-Pro-Wan2.2-I2V-A14B produced gains reported as high as 20 percentage points on seven external benchmarks. On V-ReasonBench, it reached 38.22 average pass@5, versus 18.21 for Wan2.2-I2V-A14B. On VideoThinkBench's vision-centric split, it scored 52.86 average, compared with 25.71 for Wan2.2-I2V-A14B. BabyVision-Gen remained difficult: the same model reached 12.50 overall accuracy, with 0.00 on visual tracking. The transfer claim is credible for the selected benchmarks, but it should not be read as evidence that procedural tasks solve open-ended visual reasoning.

The RL comparison isolates the reward source under matched sampling. With CPS at $\eta=0.7$, Verifiable-RL achieved 0.548 overall in-domain score, compared with 0.509 for VLM-Judge RL; the corresponding out-of-domain averages were 0.377 and 0.352. Increasing CPS stochasticity from $\eta=0.1$ to $\eta=0.7$ improved the Verifiable-RL overall score from 0.509 to 0.548, while $\eta=0.9$ fell to 0.539. That pattern supports the authors' claim that semantic exploration needs some diversity, but also shows that more sampling noise is not a general answer.

The systems result is equally relevant. On 128 H800 GPUs, 2,200 RL steps took about 5.1 days with the verifiable scorer and 8.3 days with a VLM judge: a 1.63× throughput increase and 38.7% less wall-clock time. The strongest evidence is therefore comparative within this suite: deterministic scoring improves the measured RL target while lowering evaluator cost. Whether those hand-designed rewards cover the ambiguity of real visual tasks remains the central practical question.

Caveats in Practice

The scorers are deliberately task-specific, which gives reproducibility but requires engineering each task's semantics. Several external benchmarks still show low absolute scores, and the reported comparisons center on a particular Wan2.2 video generator, CPS configuration, and seven benchmark suite. The paper also finds different modality trade-offs rather than a universal winner: video is favored for persistent spatiotemporal state tracking, while interleaved generation is presented as a more compute-efficient alternative.