Kimi K2.7-Code cuts thinking tokens 30% — but practitioners say the benchmarks don't check out

Moonshot AI's latest open-source coding model is more honest about what it can't do. Whether it's more capable is a harder question, and the company's benchmark choices aren't helping.

Written by Lena Armitage · Bureau Tech · Jun 14, 2026

The surprising part isn't the efficiency claim — it's what the model stopped hiding

Moonshot AI's headline number for Kimi K2.7-Code is a 30% reduction in thinking tokens compared to K2.6. For teams running agentic workflows — where a model reasons through multi-step tasks and each reasoning step burns tokens — that would translate directly to lower inference costs. That number is worth taking seriously. It's also worth noting it comes entirely from Moonshot AI's own measurements.

But the more interesting finding from early independent testing isn't about efficiency at all. It's about honesty.

Researcher Elliot Arledge ran K2.7-Code against K2.6 and Claude Fable 5 on KernelBench-Hard, a public benchmark focused on GPU kernel optimization, and published his full run logs. His summary: "K2.7 is more honest but not more capable." On five of six problems, K2.7-Code produced real authored Triton kernels — low-level GPU code written from scratch — where K2.6 had wrapped existing libraries instead. That's the architectural change Moonshot AI described: the model now authors implementations directly rather than routing through established frameworks.

The problem is that two of those kernels failed on the model's own bugs. And on the MoE (mixture-of-experts) kernel task, K2.7-Code scored 0.157 — a regression from K2.6's 0.222. Fable, Arledge noted, topped every cell it didn't honestly fail.

What Moonshot AI is and isn't claiming

K2.7-Code is released under a Modified MIT license, with weights on HuggingFace. It runs on vLLM or SGLang, drops into OpenAI-compatible API gateways, and operates exclusively in thinking mode with temperature fixed at 1.0 — meaning teams cannot adjust output determinism the way they might with other models.

The model is built on the same trillion-parameter mixture-of-experts (MoE) architecture as K2.6. MoE models activate only a subset of parameters per inference pass, which is part of why they can be more efficient than their total parameter count suggests.

On benchmarks, Moonshot AI claims 21.8% gains on Kimi Code Bench v2, 11% on Program Bench, and 31.5% on MLS Bench Lite. All three are proprietary benchmarks run by Moonshot AI. The model has not been submitted to DeepSWE, an independent coding benchmark that produces a 70-point spread across models — compared to SWE-Bench Pro's 30-point spread — making it a more discriminating signal for teams configuring model routing systems.

The practitioner response

Sugumaran Balasubramaniyan, a developer who built a model-task-router for the Hermes Agent platform using DeepSWE as his reference signal, responded publicly to the release. "Respectfully, every model 'improves' double digits on its own test suite," he wrote. He noted that K2.6 scored 24% on DeepSWE — tied with GPT-5.4-mini — and asked whether Moonshot AI would submit K2.7-Code to the same benchmark. He said it took 13 review rounds to get his benchmark data right, and that he would route coding tasks to K2.7-Code if the independent numbers hold up.

That conditional is doing a lot of work. It's also the right framing.

What this means if you're running K2.6 in production

The integration path is genuinely low-friction. Teams already using K2.6 through an OpenAI-compatible gateway can swap in K2.7-Code without an architecture change. If the 30% thinking-token reduction holds on your workloads, the cost case is straightforward.

The fixed temperature is worth flagging for teams that rely on determinism tuning. And the gap between Moonshot's proprietary benchmark claims and what independent testing has shown so far is wide enough that running K2.7-Code against your own task distribution — before adjusting gateway weights — is the lower-risk path to finding out whether the efficiency gains are real for your use case.

Key takeaways

K2.7-Code claims 30% fewer thinking tokens than K2.6 — a number that would directly reduce inference costs on agentic workflows, but it comes from Moonshot AI's own measurements.
All three performance benchmarks cited at launch — Kimi Code Bench v2, Program Bench, and MLS Bench Lite — are proprietary and run by Moonshot AI itself.
Independent researcher Elliot Arledge found K2.7-Code more honest than K2.6 on KernelBench-Hard, but not more capable: two authored kernels failed on the model's own bugs, and the MoE kernel score regressed from 0.222 to 0.157.
Developer Sugumaran Balasubramaniyan publicly challenged Moonshot AI to submit K2.7-Code to DeepSWE, noting K2.6 scored 24% there — tied with GPT-5.4-mini — and said he would route coding tasks to the new model if independent numbers hold up.
The drop-in OpenAI-compatible API makes K2.7-Code low-risk to test against your own workloads, but the fixed temperature of 1.0 means teams cannot tune output determinism.

FAQ

What is a thinking token, and why does reducing them matter?

Thinking tokens are the tokens a model generates internally while reasoning through a problem before producing a final output — common in models that use chain-of-thought or extended reasoning modes. Because API pricing is typically token-based, reducing thinking-token usage lowers inference costs, particularly in agentic workflows where a model reasons through many sequential steps.

What is a mixture-of-experts (MoE) architecture?

MoE is a model design where only a subset of the total parameters are activated for any given inference pass. A trillion-parameter MoE model doesn't use all trillion parameters at once — it routes each input through a smaller active set. This can make large models more computationally efficient than their total parameter count implies.

Why does it matter that K2.7-Code hasn't been submitted to DeepSWE?

DeepSWE is an independent benchmark with a 70-point spread across models, compared to 30 points on SWE-Bench Pro — meaning it separates models more clearly and is harder to game. Practitioners like Balasubramaniyan are using it as a routing signal precisely because it's not run by the model vendors. A model that improves on its own benchmarks but hasn't been tested on independent ones leaves a meaningful gap in the evidence.

Can teams use K2.7-Code without rebuilding their existing infrastructure?

Yes, if they're already running K2.6 through an OpenAI-compatible API gateway. K2.7-Code uses the same interface and the same underlying MoE architecture, so the swap is low-friction. The main constraint to check is the fixed temperature of 1.0 — teams that rely on temperature tuning for determinism will need to account for that.

What did independent testing actually find?

Researcher Elliot Arledge tested K2.7-Code on KernelBench-Hard, a public GPU kernel optimization benchmark. He found the model more transparent than K2.6 — it authored real Triton kernels rather than wrapping libraries — but not more capable overall. Two kernels failed on the model's own bugs, and the MoE kernel score regressed from 0.222 to 0.157 compared to K2.6.

Citations

Kimi K2.7-Code cuts thinking tokens 30% — but practitioners say the benchmarks don't check outMoonshot AI released Kimi K2.7-Code claiming 30% fewer thinking tokens and double-digit benchmark gains; all cited benchmarks are proprietary to Moonshot AI
KernelBench-Hard run logs — Elliot ArledgeIndependent testing found K2.7-Code more honest than K2.6 but not more capable; MoE kernel score regressed from 0.222 to 0.157
VentureBeat RSS — Bureau research sourceSource feed used for lead research on Kimi K2.7-Code release coverage