A 35B Agent Matches Trillion-Parameter Models on Long-Horizon Tasks

Large agent models are usually scaled by adding parameters, then asked to solve tasks that require many turns of search, coding, tool use, and verification. This paper argues for a different axis: keep the deployable model at 35B parameters, but scale the length and heterogeneity of the agent trajectories used for training. The result is Agents-A1, a 35B mixture-of-experts agent trained from Qwen3.5-35B-A3B to behave like a long-horizon system rather than a short-response assistant.

Core Contribution

The central claim is that “agent horizon” can be a practical substitute for raw parameter count on tasks where success depends on planning, tool selection, retrieval, intermediate checking, and final answer verification. The authors build a knowledge-action infrastructure that records external knowledge, actions, observations, and verifier outcomes, then use it to train on trajectories averaging 45K tokens. The full-domain SFT set contains roughly 100K trajectories, with average lengths of 44K for deep research, 48K for coding and engineering, 37K for scientific reasoning and problem solving, 3K for instruction following, and 39K for general agentic tasks.

That design is meaningfully different from a standard instruction-tuning recipe. The paper does not just add more examples; it tries to expose the model to extended action loops where the answer depends on accumulated evidence and delayed consequences.

Technical Approach

Agents-A1 is trained in three stages. First, the authors perform full-domain supervised fine-tuning on broad agent behaviors. The SFT run uses a cross-entropy loss only on response tokens, masks instruction tokens, and trains with a learning rate of $1\times10^{-5}$, cosine warmup, warmup ratio 0.05, batch size 16, one epoch, AdamW, weight decay 0.1, and maximum sequence length 131,072. They also use sample packing to reduce padding while masking attention across packed examples.

Second, the paper trains domain-level teacher models for specialized abilities. The search teacher uses SFT followed by GRPO, with three tools: web search, page reading, and Python code execution. Its RL data contains about 2,000 multi-hop questions; each question is attempted five times and filtered to keep examples that are neither always solved nor never solved. During rollout, the model generates 8 responses per prompt and receives a reward combining answer correctness, search-efficiency penalties, repetition penalties for repeated google_search or read_page calls, and format calibration.

Science, instruction following, long-context learning, and tool calling receive their own teacher-style training. The science teacher separates reasoning-enhanced SFT from tool-augmented SFT, preserving derivation ability while adding search, visiting, coding, and scholar-style interactions. Instruction following uses GRPO in two stages: verifiable rule-based instruction rewards first, then long-context learning rewards that require answer matching against task-specific evidence. Tool-calling RL is framed around complete trajectories, because wrong early tool choices may only become visible many turns later.

The final unification step is multi-teacher domain-routed on-policy distillation with salient vocabulary alignment. The paper’s stated purpose is to transfer heterogeneous teacher behavior into one deployable 35B student without collapsing domain-specific habits.

Results and Analysis

The reported benchmark pattern supports the paper’s main thesis, with an important caveat: the strongest evidence is comparative benchmark performance, not independent reproducibility. Figure 1 summarizes the benchmark profile, including the paper’s headline comparisons against trillion-parameter systems.

Agents-A1 reports leading results over 1T-parameter comparators such as Kimi-K2.6 and DeepSeek-V4-pro on SEAL-0 at 56.4, IFBench at 80.6, HiPhO at 46.4, FrontierScience-Olympiad at 79.0, and MolBench-Bind at 56.8. It remains competitive on SciCode at 44.3, HLE at 47.6, and BrowseComp at 75.5. The interesting point is not that a 35B model wins every benchmark; the claim is narrower and more useful. For tasks dominated by long evidence chains and tool-mediated execution, the training horizon appears to recover part of what practitioners often try to buy with much larger parameter counts.

The training details also make the result plausible. A 131K-token maximum sequence length and 45K-token average SFT trajectories are aligned with the target behavior, while the domain teachers address different failure modes: inefficient search, shallow scientific derivation, brittle instruction compliance, weak in-context grounding, and premature tool stopping. The paper’s design reads less like one new architecture trick and more like a data-and-RL systems recipe for agent behavior.

The main weakness is reproducibility and attribution. Without released artifacts in the paper metadata here, outside groups cannot easily separate the effect of horizon scaling from base-model choice, data quality, teacher specialization, benchmark selection, or verifier design. The comparison to 1T models is useful for deployment intuition, but the cleanest scientific test would require ablations that isolate trajectory length, domain routing, salient vocabulary alignment, and each teacher’s contribution under the same base model and evaluation stack.