Le Chonk Returns: Mistral Large 4, European Open Weights, and the Harness Problem

1. Guest Background

This episode of Matthew Berman / Forward Future is hosted by Matthew Berman and centers on Mistral Large 4, the model he repeatedly calls Le Chonk. The “guest” in the indexed evidence is therefore not a person giving an interview. It is the AI model being discussed, and the supported role is exactly that: an AI model under analysis.

The organization behind the subject is Mistral AI, described in the evidence context as a European AI company building open-weight and open-source frontier models. Le Chonk’s relevant background is a product and technical profile rather than a biography: it is presented as a trillion-parameter, open-weight, open-source, natively multimodal mixture-of-experts model with 49 billion active parameters. Its claimed areas of emphasis include instruction following, reasoning, agentic workflows, coding, multimodal understanding, and cybersecurity benchmarks.

Berman’s episode is not a neutral spec-sheet reading. He frames the release as a test of European AI sovereignty, global open-model competition, and the practical gap between a powerful model and a usable developer product. The episode covers architecture, pricing, benchmark placement, the fact that the model was still in public preview, the promised future weight release, and Berman’s own attempt to use the model in OpenCode for a Rubik's cube coding task.

2. What the Episode Covers

Berman begins with a clear thesis: Mistral has produced a frontier model built from scratch in Europe, not a post-trained derivative on top of a Chinese open-source model. That opening shapes the whole episode. Le Chonk is not treated merely as another model release; it is treated as evidence that Europe may be re-entering a competitive field where Berman sees the United States as strongest in closed frontier models and China as strongest in open models.

He then explains the model’s basic technical shape. Mistral Large 4 is presented as a trillion-parameter open-weight, open-source model, but its mixture-of-experts architecture means only a subset of parameters is active for any given request. The active-parameter number Berman cites is 49 billion. He sees that ratio as important because it suggests a way to combine large total capacity with a more manageable inference footprint. The model is also described as natively multimodal and as combining instruction, reasoning, and agentic capabilities in one system.

The episode quickly moves from enthusiasm to market comparison. Berman reports API pricing of $1.36 per million input tokens and $4.18 per million output tokens, then compares that to a world in which GPT 6.1 Sol and Claude Opus 5.5 are, in his telling, very strong, fast, and inexpensive. His point is not that open weights have no value. It is that the old reasons for reaching for open source, especially price and speed, are less automatic when closed frontier labs are also cutting cost and improving latency. Open models then have to justify themselves through control, privacy, self-hosting, zero data retention, and configurable guardrails.

The benchmark section forms the evidence-heavy middle of the episode. Berman discusses DeepSweep 1.1, Terminal Bench, Automation Bench, Artificial Analysis Cyber Index, CyberGym, and other comparisons. He sees a large improvement from Mistral Medium 3.5 and places Mistral Large 4 preview in the competitive tier of open models, but he repeatedly keeps the reference set honest: Chinese open models remain serious competitors, and U.S. closed frontier models remain a different level of productized power. The final part of the episode is hands-on: he connects a Mistral API key to OpenCode and asks the model to build a Rubik's cube simulation, exposing problems with thinking tokens, context limits, harness integration, and the final visual result.

3. Core Views: Reasoning, Examples, and Limits

Berman’s first major view is that Le Chonk matters because it is a European-built foundation model, not simply because it is another capable model. He says it was conceived, designed, created, trained, and served for inference from Europe. He also says it was trained from scratch on 3,800 Nvidia Grace Blackwell GPUs in Mistral’s own European data center, with the public preview served on the same infrastructure. The reasoning is geopolitical and technical at the same time: sovereignty here means control over model creation, compute, and serving, not just a European company name on a launch post. For Berman, that matters because the frontier has largely been framed as U.S. closed models versus Chinese open models.

His second view is that the model is genuinely competitive, but the evidence does not support a simple “best open model” story across the board. On DeepSweep 1.1, Berman reports Mistral Large 4 preview at 62, behind Kimi K3 with KimiCode CLI at 68 on the open-source chart he discusses. On Terminal Bench, he says Mistral is again second, while GLM 5.3 comes in at a dominant 40 and Kimi K3 at 22. On Automation Bench, Mistral Large 4 preview is reported at 59.9, close to GLM 5.3 at 62.2 and ahead of Kimi K3 at 58.3. The supported interpretation is significant progress into the first tier of open models, especially relative to prior Mistral results, not universal dominance over Chinese competitors.

The strongest single-domain claim in the episode is cybersecurity. Berman says Le Chonk is not merely good for an open-weight model but competitive with all models, including closed-source systems. He cites the Artificial Analysis Cyber Index framing that ML4 ranks among the top five globally in an independent evaluation of how well AI models find and fix security flaws in real software, and leads open-weight models developed outside China by a wide margin. On CyberGym, he reports a score of 82 and says Mistral Large 4 is number one among the open-source models shown, including the Chinese models on that chart. But he also gives an important limitation: that CyberGym chart is missing Fable, Astra, and other closed-source models from OpenAI and Anthropic, so the result cannot be generalized into “best cybersecurity model overall.”

The fourth view is that open weights do not automatically create a usable product. Berman’s OpenCode attempt is the clearest example. He plugs in a Mistral API key, the model emits extensive thinking or chain-of-thought output, the context window fills up, and the run stops without useful feedback. He says he is fairly technical and still had to debug API-key setup, context-window size, and thinking settings. That leads to a practical distinction: model capability, API access, and product experience are separate layers. Claude Code, Codex, Grok, and Cursor are not just model names in his comparison; they stand for the kind of integrated experience where users do not need to manage those details manually.

The fifth view is a reassessment of open-source business value. Berman argues that if closed frontier labs are now offering strong, fast, cheaper models, open models cannot rely on low price and speed alone. Their value moves toward ownership, privacy, zero data retention, self-hosting, and the ability to add or remove guardrails. The tradeoff is audience size. In his view, open source works best today for organizations with engineering expertise, scale, and internal knowledge, especially if they can move some tasks to a slightly less capable model they fully control. That is a strong enterprise use case, but it is not the same as being plug-and-play for a broad audience.

The sixth view comes from the Rubik's cube test and is more nuanced than a simple model failure. With high thinking effort enabled, Berman says the run burned through about 32,000 tokens and stopped. After he raised the context limit and turned off thinking, the result improved: the cube looked good and could rotate. But scramble initially did not animate, later side rotations worked while colors disappeared, and autosolve appeared to proceed before colors reset at the end. Berman calls the result a failure, yet says he blames the interaction between model and harness more than the model itself. That is the episode’s most transferable uncertainty: benchmark quality may not predict real agentic workflow success when the harness is poorly matched to the model.

4. Learning and Application

The most useful lesson for model selection is to separate model ability, deployment control, and product experience. Le Chonk may be a serious open-weight model worth testing, but Berman’s hands-on example shows that connecting an API key to a generic agentic harness is not the same as having a stable coding assistant. A team evaluating it should begin with a narrow proof of concept: define the task type, set context budgets, choose thinking settings deliberately, log failures, and compare end-to-end task completion rather than only benchmark rank.

For organizations whose main requirements are privacy, data-retention control, deployment location, and guardrail ownership, a model like Le Chonk becomes more attractive. Berman’s supported use case is not the casual user who wants everything to work immediately; it is the company with technical staff, high token volume, and enough internal knowledge to offload some work from OpenAI or Anthropic to a slightly less capable but more controllable model. The condition is that the organization must be able to operate the surrounding system: inference, evaluation, context management, task routing, quality measurement, and fallback behavior.

If the goal is simply cheapest or easiest, the episode warns against assuming that open source is automatically the answer. Berman’s comparison to GPT 6.1 Sol and Claude Opus 5.5 suggests that closed frontier labs have compressed the open-model advantage on price and speed while retaining stronger productized experiences. A real procurement or integration decision should therefore include task success rate, human debugging time, context length, token speed, maintenance burden, and failure recovery cost. The API price per million tokens is only one line item.

Cybersecurity is the domain where Le Chonk deserves special attention, but with boundaries. The episode supports treating it as a candidate for vulnerability discovery, repair suggestions, secure-code review, and controlled security automation experiments. It does not support treating benchmark placement as a universal guarantee. Because Berman highlights strong CyberGym and Artificial Analysis Cyber Index evidence while also noting missing closed models on one chart, the prudent application is local retesting: run it on the organization’s own repositories, known vulnerability classes, and controlled red-team scenarios before assigning high-trust security work.

For developer-tool builders, the clearest application is to invest in the harness, not only the model picker. The Rubik's cube example failed through context handling, thinking-token behavior, animation correctness, and visual state issues. If a team wants to integrate Le Chonk into an internal coding agent, it should treat model configuration, context-window detection, tool-call protocols, retries, test execution, and visual or unit validation as one product surface. Merely swapping a model name in an existing harness may leave most of the model’s capability stranded.

For analysts and readers, the episode offers a durable checklist for future open-model launches. Ask whether the model is trained from scratch, whether weights are actually released or merely promised, whether public preview differs from vetted-partner access, what the benchmark reference set includes, and whether a real user can operate the model without manual intervention. On that framework, Berman’s conclusion is positive but not romantic: Le Chonk is a meaningful European open-weight advance, but its price, context window, ease of use, and agentic harness fit remain practical constraints.

Source

• Original episode: Mistral is BACK! (Le Chonk)