Kimi K3 Brings Open Weights Closer to Frontier Agents
A 2.8T-parameter MoE combines Delta Attention, 16-of-896 expert routing, and agentic reinforcement learning to approach leading proprietary systems at lower task cost.
Underlying Paper
Kimi K3: Open Frontier Intelligence
We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.
Open-weight language models increasingly compete on short-form benchmarks, but long-horizon work exposes a different set of requirements: maintaining context, selecting tools, recovering from failed actions, and delivering an artifact that can be checked. Kimi K3 targets that gap with a 2.8T-parameter mixture-of-experts model, 104B activated parameters, native vision, and a 1-million-token context window. The paper’s central argument is that architecture, distributed training, post-training environments, and serving design must be developed together for capable agents.
Core Contribution
Kimi K3 combines Kimi Delta Attention and Attention Residuals to improve information flow over long sequences and depth, then uses Stable LatentMoE to route each token to 16 of 896 experts. The authors report roughly 2.5× better overall scaling efficiency than Kimi K2. The contribution is broader than a new backbone: it joins long-context modeling to a training program built around tool-mediated, verifiable work.
The post-training design explicitly separates reasoning effort into low, medium, high, and max levels. During reinforcement learning, a per-problem token budget controls how much inference-time computation a trajectory may consume. This makes the desired trade-off visible at training time rather than treating longer chains of thought as an unconditional improvement.
Technical Approach
The agentic RL system is designed for tasks whose success can be judged from environment state rather than only from a model-generated answer. Its unified white-box environment composes tool interfaces, system prompts, context management, skills, memories, and subagents into different harnesses. The paper describes persistent rollouts in which sandbox state carries across iterations; partial trajectories can be prioritized for resumption once a fraction of rollouts finishes, reducing stragglers in long-horizon training.
Task generation uses a hierarchical knowledge graph. Agents expand coarse domain nodes into finer concepts, retrieve public material from sampled related nodes, then synthesize a task from those materials. The environments cover web development, personal-assistant workflows modeled on applications such as Gmail and Notion, visual reasoning, web research, and GPU kernel optimization. For the latter, tasks range from single kernels to fused mega-kernels across CUDA, Triton, CuTe DSL, ThunderKittens, and TileLang, with performance measured against a reference implementation.
Deployment is part of the design rather than an afterthought. The authors quantize MoE expert weights to MXFP4 while retaining activations in MXFP8 and higher precision for non-expert components. A draft-model fine-tuning procedure adapts an MTP layer into an EAGLE-3-style speculative draft model. The chat template also preserves tool-call identity and ordering for parallel calls, with typed arguments and separately organized think, response, and tools channels.
Results and Analysis
The reported in-house results support the claim that Kimi K3 is competitive on agentic workloads, though the evidence is not uniformly favorable. On the authors’ harnesses at maximum reasoning effort, it scores 90.0 on Deep Research Bench, ahead of GPT-5.6 Sol at 85.3 and Claude Opus 4.8 at 87.2; it records 76.3 on Swarm Bench versus 73.2 and 72.6, respectively. In coding, it scores 73.7 on Kimi Code Bench 2.0 using the Claude Code harness, below Claude Fable 5’s 76.9 but above GLM-5.2’s 64.2.
The cost analysis is more consequential than an isolated benchmark lead. On BrowseComp, Kimi K3’s best reported score is 91.2% at $2.03 per task, compared with 90.4% for GPT-5.6 Sol at about twice the cost. On Kimi Code Bench 2.0, its 73.7 score is 4.0 points below Claude Fable 5 while costing 38% as much. That is a meaningful operating point for teams willing to accept a small performance gap for a substantially lower per-task price.
Independent evaluations temper the framing. Artificial Analysis assigns Kimi K3 an Intelligence Index of 57.1, behind Claude Fable 5 at 59.9 and GPT-5.6 Sol at 58.9. Vals AI similarly places it second of 39 models at 74.7%, behind 75.1%. The paper therefore makes a credible case for a high-performing open model, not for outright leadership over the strongest closed systems.
Limits in Practice
The broad in-house suite provides useful task coverage, but several comparisons use different harnesses across model families, complicating direct attribution to the model rather than its agent scaffold. Cybersecurity results also show the remaining distance from expert execution: Kimi K3 solves 14 of 36 Tier-2 exploit tasks, or 38.9%, with only 4 from the kernel track. The authors describe their cyber evaluation as a lower bound conditioned on the model version and available task coverage; that caution should extend to the wider agentic claims.
Evidence Box
strongKey Claims
- •Long-context MoE architecture improves scaling efficiency over Kimi K2
- •Agentic RL produces stronger long-horizon coding and research behavior
- •Open model weights offer a lower-cost frontier-agent option
- •Reasoning-effort control improves token-efficiency trade-offs
Key Results
- •90.0 on Deep Research Bench, versus 85.3 for GPT-5.6 Sol and 87.2 for Claude Opus 4.8
- •76.3 on Swarm Bench, versus 73.2 for GPT-5.6 Sol and 72.6 for Claude Opus 4.8
- •91.2% on BrowseComp at $2.03 per task, versus 90.4% for GPT-5.6 Sol at about twice the cost
- •57.1 Artificial Analysis Intelligence Index, versus 59.9 for Claude Fable 5 and 58.9 for GPT-5.6 Sol
Limitations & Caveats
- •Trails Claude Fable 5 and GPT-5.6 Sol on independent Intelligence Index results
- •In-house comparisons sometimes use different harnesses across model families
- •Tier-2 cybersecurity exploit suite solved 14 of 36 tasks, with only 4 kernel-track solutions
- •Cybersecurity evaluation is conditioned on model version and available task coverage
