On July 28, Moonshot AI announced late last night that it would open source the Kimi K3 model weights, release a technical report, and simultaneously open three key infrastructure (Infra) technologies: MoonEP, FlashKDA, and AgentEnv. This opening covers models, training methods and underlying training systems. Developers can download models for deployment and secondary development. The specific scope of use must comply with the Kimi K3 license.
According to the official introduction, Kimi K3 adopts a hybrid expert (MoE) architecture, with a total parameter scale of 2.8 trillion, native visual understanding capabilities, and support for 1 million Token context windows. Compared with the previous generation Kimi K2.5, the parameter scale of Kimi K3 has been increased by about 3 times. The Dark Side of the Moon stated that with limited computing resources, using technologies such as Kimi Delta Attention, Attention Residuals, and MoonEP, its model scale efficiency increased by approximately 2.5 times.
Also released simultaneously with the model weights is the Kimi K3 technical report. According to public content, the report disclosed many technical details of model training and architecture design, including KDA and Gated MLA hybrid attention structure, Stable LatentMoE expert routing mechanism, MoonViT-V2 visual encoder training solution, and post-model training and evaluation methods. Among them, the model adopts a MoE design with 896 routing experts and 16 experts activated by each token, and covers the training and evaluation of multiple capabilities such as general reasoning, agent and programming agent.
In addition to the model itself, Dark Side of the Moon also announced the open source of key infrastructure technologies that support Kimi K3 training. Among them, MoonEP is a set of high-performance expert parallel communication libraries for large-scale MoE models; FlashKDA is Kimi Delta For the high-performance computing operator corresponding to Attention, official data shows that its Prefill speed on NVIDIA H20 is 1.72 to 2.22 times higher than the flash-linear-attention baseline; AgentEnv is an Agent sandbox system jointly developed by Dark Side of the Moon and KVCache.ai, which can support capabilities such as fast snapshots, recovery, and Fork, and is used in large-scale Agent training environments.
The Dark Side of the Moon stated that this opening aims to promote the development of an open weighted model ecosystem. It hopes to lower the threshold of model deployment and development by opening models, training methods and underlying infrastructure, and provide more reusable technical resources for the AI community and related research.