Criticality-Guided Data Collection Reduces Embodied Policy Failures

Task-specific finetuning can collect data randomly even after a pretrained embodied policy has become competent on nominal cases. That can leave a training pool dominated by scenarios the policy already handles while underrepresenting the rare states where it fails. The paper frames this as a data-selection problem: improving adaptation requires finding more informative failures, not simply gathering more nominal rollouts.

The authors introduce Self-Evolving Learning, a loop that learns from a policy’s own execution outcomes and uses that signal to collect more useful finetuning data. Its central component is a state-wise criticality model, $C_\phi$, trained to estimate the probability that a state will lead to future failure. The model then guides importance sampling toward failure-prone scenarios, after which the policy is fine-tuned on the curated data.

Core Contribution

The paper connects failure prediction directly to data collection for embodied-policy adaptation. Rather than treating every collected sample as equally informative, the framework prioritizes states predicted to be critical while using importance weights during training. The authors argue that this increases the information density of the training pool while preserving the intended learning objective.

Figure 1 summarizes the iterative process: a pretrained policy is rolled out, its success and failure outcomes train the criticality model, the model guides sampling toward high-criticality regions, and the policy is fine-tuned on the resulting data. The process repeats as the policy and its failure modes change.

Technical Approach

The criticality model is trained from policy execution outcomes to predict failure probability at the state level. That prediction is used to focus collection on scenarios that are more likely to expose weaknesses in the current policy, instead of relying solely on random data collection.

The paper also describes deployment-time routing based on criticality. Candidate perturbations are scored using the criticality of their resulting states; when the score passes a threshold, the fine-tuned policy handles the decision, while the original baseline is otherwise used. This makes criticality useful both for selecting training data and for deciding when adaptation should be applied.

Results and Analysis

Across quadrupedal locomotion, multi-task manipulation, vision-language-action benchmarks, and a real-robot task, the paper reports failure-rate reductions of 51–67% relative to trained baselines. It also reports 8–25% lower failure rates than the compared vision-language-action models. These results span multiple embodied settings, supporting the paper’s claim that failure-guided data selection can improve adaptation beyond a single simulator or task family.

Figure 3 shows multi-round failure-rate trends for Go2 locomotion and averaged ManiSkill manipulation tasks. In both displayed settings, the criticality-guided method improves over a control that collects data randomly. The figure is consistent with the proposed iterative mechanism: as new failures are identified and targeted, the policy’s reported failure rate declines over successive rounds.

The reported ranges should be interpreted as results for the evaluated tasks and conditions rather than as a guarantee of the same improvement in every deployment. Performance depends on whether the learned criticality signal identifies informative failure-prone states in the target environment.

Limits in Practice

The approach depends on the quality and coverage of the policy’s execution data. If early rollouts do not reveal relevant failures, or if the criticality model is poorly calibrated, the sampling process may focus on less useful regions of the state space. The method also relies on task-specific evaluation and thresholding decisions when used for deployment-time routing. As with other embodied learning systems, transfer to substantially different environments or operating conditions requires validation rather than being assumed from the reported benchmarks.