---
title: "Kimi K3: Open Frontier Intelligence"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.24653v1"
permalink: "https://feed.astralor.com/items/itm_fd1f9b8972934caac5de5dc8"
published: "2026-07-27T16:49:54.000Z"
score: 78
channel: "ai"
tags: ["Kimi-K3","Reinforcement Learning","Moonshot AI","Attention Mechanism","Mixture-of-Experts"]
---

# Kimi K3: Open Frontier Intelligence

## AI commentary

> This paper showcases Kimi K3's breakthroughs in ultra-large MoE models, especially the design of KDA and Attention Residuals. It is valuable for researchers studying LLM architecture and RL post-training, and the open-sourced weights lower the barrier for replication.

## At a glance

Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model developed by Moonshot AI, with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Built on Kimi Delta Attention and Attention Residuals to enhance information flow, combined with Stable LatentMoE that activates 16 of 896 routed experts per token, it achieves approximately 2.5x scaling efficiency improvement over Kimi K2. Post-training involves reinforcement learning across general, agentic, and coding domains with multiple reasoning-effort levels. Evaluations show frontier-level performance in long-horizon coding, agentic, knowledge, reasoning, and vision tasks, trailing only Claude Fable 5 and GPT-5.6 Sol but outperforming other open and proprietary models. Full model weights are released.

## Key points

- 2.8T-parameter MoE with 104B activated parameters and 1M context window
- Innovations: Kimi Delta Attention, Attention Residuals, and Stable LatentMoE
- RL post-training across general, agentic, and coding domains with multiple effort levels
- Open-sourced full weights; performance exceeds most models except top closed-source ones

## Full text

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_fd1f9b8972934caac5de5dc8