Moonshot AI Unveils Kimi K3: A 2.8 Trillion Parameter Open MoE Model with 1M Context

Moonshot AI Redefines Open AI with Kimi K3

At a glance, Moonshot AI has officially launched Kimi K3, a groundbreaking large language model that sets new benchmarks for open AI development. Boasting an impressive 2.8 trillion parameters and a native vision capability, Kimi K3 distinguishes itself with an extraordinary 1-million-token context window. Moonshot AI proudly positions it as the world’s first open 3-trillion-class model, pushing the boundaries of what’s possible in AI.

Understanding Kimi K3: A New Breed of AI

Meanwhile, At its core, Kimi K3 is a sparse Mixture-of-Experts (MoE) model, meticulously engineered with two significant architectural advancements: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). These innovations fundamentally alter how information flows through the model, optimizing both sequence length processing and model depth. K3 is specifically designed to excel in demanding applications such as long-horizon coding, complex knowledge work, and intricate reasoning tasks.

For nearly a year, Moonshot’s Kimi models have consistently defined the upper limits of open-model sizes. With K3, they’ve solidified this leadership, introducing a model with 2.8 trillion parameters that represents a significant leap forward in scale and capability within the open-source community.

The Innovative Architecture Underneath

Kimi K3’s impressive performance is rooted in its sophisticated underlying architecture:

Kimi Delta Attention (KDA)

KDA is a hybrid linear attention mechanism that dramatically improves efficiency. Moonshot AI reports that KDA enables up to 6.3 times faster decoding when handling million-token contexts, a critical advantage for processing vast amounts of information quickly.

Attention Residuals (AttnRes)

For example, Working in tandem with KDA, AttnRes focuses on optimizing model depth. Instead of uniformly accumulating representations, AttnRes selectively retrieves them, leading to roughly 25% higher training efficiency with less than 2% additional cost. This innovative approach allows the model to learn more effectively without a proportional increase in computational overhead.

Leveraging Sparsity for Efficiency

Sparsity is another key component of K3’s design. Utilizing Stable LatentMoE, the model efficiently activates only 16 out of 896 experts. Managing such high sparsity presents unique challenges in routing and optimization, which Moonshot AI addresses with:

• Quantile Balancing: This method directly derives expert allocation from router-score quantiles, eliminating the need for heuristic updates and sensitive hyperparameters.
• Per-Head Muon: An extension of Muon, this optimizes attention heads independently, enhancing flexibility and performance.
• SiTU and Gated MLA: These components further refine activation control and attention selectivity, respectively.

That said, These structural changes, combined with refined training and data recipes, contribute to an approximate 2.5 times better overall scaling efficiency compared to its predecessor, Kimi K2.

Performance Benchmarks: Where K3 Shines

While Moonshot AI acknowledges that K3’s overall performance still trails the most powerful proprietary models like Claude Fable 5 and GPT 5.6 Sol, its evaluation suite shows consistent outperformance against other tested open models. K3’s scores, achieved with reasoning effort set to maximum, highlight its strengths in specific domains:

• Leading Performance: Kimi K3 demonstrated superior results in Program Bench, SWE Marathon, BrowseComp, Automation Bench, and OmniDocBench.
• Areas for Growth: It currently trails Fable 5 on FrontierSWE and HLE-Full, and GPT 5.6 Sol on DeepSWE.

Interestingly, Notably some comparisons include caveats, such as Fable 5’s fallback to Opus 4.8 for refused requests and K3’s performance on BrowseComp benefiting from context compaction.

Practical Applications and Use Cases

Kimi K3’s advanced capabilities open up a wide range of practical applications:

• Repo-scale Engineering: Facilitates long coding sessions with minimal human intervention, leveraging the Kimi Code environment.
• Vision in the Loop: Enables seamless iteration between code and live screenshots, utilizing its native vision capabilities.
• Research Reproduction: Capable of processing vast amounts of information, exemplified by handling 20+ papers and 3,000+ lines of Python for complex research.
• Deep Research Reports: Supports extensive data gathering, such as a 42-year ASIC study involving 2.8k+ fetches and 11k+ pages.
• Document Parsing: Achieves high accuracy, as indicated by an OmniDocBench score of 91.1, with structured output.

However, Notably, Kimi K3 employs a single native multimodal architecture that seamlessly handles text, images, and video, making it incredibly versatile for diverse tasks.

Accessing Kimi K3 and Pricing Details

Kimi K3 is readily available for use on Kimi.com, Kimi Work, Kimi Code, and through its API. Developers can access the API using the standard OpenAI SDK, simply by pointing to a Moonshot base URL.

Meanwhile, Key considerations for API usage include setting reasoning_effort=”max” and noting that parameters like temperature, top_p, and n are fixed. The max_completion_tokens defaults to 131,072 and can reach up to 1,048,576.

Moonshot AI has implemented a flat pricing structure, independent of context length:

• Cache-hit input: $0.30 per Megatoken (MTok)
• Cache-miss input: $3.00 per MTok
• Output: $15.00 per MTok

In practical terms, The cache-hit rate is a crucial factor for cost efficiency, and Moonshot AI reports an impressive above 90% cache-hit rate in typical coding workloads.

Expert Perspective

A practical read on Kimi K3 starts with kimi. That is where the earliest effects are likely to show up if this development keeps building.

What happens next will come down to adoption speed, policy response, and execution quality. That combination could make Kimi K3 a meaningful reference point across model.

For decision-makers, the useful lens is not the headline alone but how moonshot changes priorities once organizations have to respond.

Frequently Asked Questions

Why is Kimi K3 important?

Moonshot AI Redefines Open AI with Kimi K3 At a glance, Moonshot AI has officially launched Kimi K3, a groundbreaking large language model that sets new benchmarks for open AI development.

What impact could Kimi K3 have?

Boasting an impressive 2.8 trillion parameters and a native vision capability, Kimi K3 distinguishes itself with an extraordinary 1-million-token context window.

What should readers watch next with Kimi K3?

Moonshot AI proudly positions it as the world’s first open 3-trillion-class model, pushing the boundaries of what’s possible in AI.

How does this relate to kimi?

It connects because the article frames kimi as one of the clearest areas where the topic may be felt in practice.

Key Takeaways

• Kimi K3 is Moonshot AI’s new 2.8 trillion-parameter open MoE model, activating 16 of 896 experts.
• Innovative architecture, including Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), along with refined training, yields approximately 2.5x better scaling efficiency than Kimi K2.
• It leads in benchmarks like Program Bench, SWE Marathon, BrowseComp, Automation Bench, and OmniDocBench, while trailing top proprietary models in some areas.
• Accessible via an OpenAI-SDK compatible API, with a 1-million-token context window and competitive pricing ($0.30/$3.00/$15.00 per MTok for cache-hit input/cache-miss input/output).
• Supports diverse use cases from repo-scale engineering to deep research and multimodal document parsing.

Source: https://www.marktechpost.com/2026/07/16/moonshot-ai-releases-kimi-k3-a-2-8-trillion-parameter-open-moe-model-with-kimi-delta-attention-and-1m-context/