---
title: "Kimi K3：开放前沿智能模型（2.8T参数MoE）"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.24653v1"
permalink: "https://feed.astralor.com/items/itm_fd1f9b8972934caac5de5dc8"
published: "2026-07-27T16:49:54.000Z"
score: 78
channel: "ai"
tags: ["Kimi-K3","强化学习","月之暗面","注意力机制","混合专家模型"]
---

# Kimi K3：开放前沿智能模型（2.8T参数MoE）

## AI 点评

> 这篇论文展示了Kimi K3在超大规模MoE模型上的突破，尤其是KDA和Attention Residuals的设计思路值得关注。对研究大模型架构和强化学习后训练的读者非常有价值，开源权重也降低了复现门槛。

## 速览

Kimi K3是由月之暗面推出的2.8万亿参数混合专家模型，激活参数1040亿，拥有原生视觉能力和百万token上下文窗口。基于Kimi Delta Attention和Attention Residuals改善信息流动，结合Stable LatentMoE实现每token激活16/896专家，整体扩展效率较K2提升约2.5倍。后训练在通用、Agent和编程领域使用强化学习，支持多种推理努力水平。评估显示其在长程编程、Agent、知识、推理和视觉任务上达到前沿水平，整体性能仅次于Claude Fable 5和GPT-5.6 Sol，但优于其他开源和专有模型。模型权重已完全开源。

## 关键点

- 2.8T参数MoE架构，激活104B参数，支持百万token上下文
- 引入Kimi Delta Attention、Attention Residuals和Stable LatentMoE三大创新
- 强化学习后训练覆盖通用、Agent和编程，支持多推理努力级别
- 开源全部权重，性能超越多数模型，仅落后于顶尖闭源模型

## 全文

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_fd1f9b8972934caac5de5dc8