Mistral Medium 3.5: 128B Model Capabilities [Audio Analysis]

HOST

From DailyListen, I'm Alex. Mistral AI just shipped Medium 3.5, their new 128 billion parameter model that packs instruction following, reasoning, and coding into one dense setup. It's designed to run on just four GPUs if you're self-hosting, and it's already hitting platforms like Ollama. A French startup pushing open-weights models this big changes who can access top-tier AI without massive cloud bills. We're joined by Priya, our technology analyst, who tracks how these releases shift the balance between startups and giants.

PRIYA

What this unlocks is any developer or small team running a 128 billion parameter model on everyday hardware—four GPUs get you inference, or squeeze it into an H200 node or two H100s. Medium 3.5 merges what used to be separate models into single 128B weights: instruction following, math reasoning, coding, all in one. No more juggling specialized versions. It's the first flagship from Mistral doing that unification. They trained it with SFT on MedPix using eight nodes of eight GPUs each, tensor parallel eight, pipeline parallel eight. Now it ships natively in FP8 format on disk, so loading stays efficient. Ollama users pull it with "ollama run mistral-medium-3.5," or launch Codex tied to it. That drops the barrier for local experimentation.

HOST

Four GPUs for 128 billion parameters—that sounds almost too straightforward for something this hefty. Puts it right on a single workstation, right? But does that mean everyone's swapping out their old setups overnight?

PRIYA

The interesting piece is how this slots into Mistral's rapid timeline. Last year in August 2025, they dropped Medium 3.1. Then September brought Magistral Medium 1.2. December had Ministral 3 at 14B. Now Medium 3.5 takes over in Le Chat, their AI assistant, kicking out Medium 3.1 and Magistral. It also replaces Devstral 2 in the Vibe CLI tool. Context jumps to 256,000 tokens, double what some rivals manage without tricks. Architecture-wise, it's Mistral3ForConditionalGeneration—a dense Ministral-3 text decoder paired with a Pixtral vision tower for multimodal input. But here's the catch: while it promises better instruct, reasoning, and coding in one package over prior Mistral models, full benchmarks aren't out yet. BenchLM.ai lists the model page with metadata like creator and context window, but sourced scores against o3 or others are still pending.

HOST

Benchmarks pending means we can't say yet if it beats, say, o3 on math or code tasks. You mentioned BenchLM tracking it—does that leave us guessing on real gains?

PRIYA

Exactly, no full public breakdowns yet for Medium 3.5's 128B weights on instruct following, math reasoning, or coding against competitors. BenchLM has the page up with 256k context and reasoning mode noted, but numbers are coming soon. AlphaSignal calls it another step in the AI power race, and Mistral pitches it as state-of-the-art open-weights generalist. Without those scores, though, teams can't quantify if it's worth retraining pipelines or switching from, say, Mixtral 8x22B versions 0.1 to 0.3. That gap hits developers hard—they need hard data before betting production workflows.

HOST

Fair point—no benchmarks means no clear winner declared. Reminds me of past Mistral drops where scores trickled in late. What about self-hosting details? The briefing flags gaps there—memory needs, quantization specifics. Does four GPUs cover the full story?

PRIYA

Gaps in self-hosting details leave room for trial and error. We know it fits four GPUs for operation, full model on one H200 or two H100s, all in FP8 per-tensor scale. But exact quantization steps or peak memory per GPU aren't spelled out yet. Run it via Ollama, sure, even claims of 64GB RAM setups on Facebook posts, but no official guide on H100 vs. cheaper cards or FP8 loading quirks. That uncertainty slows adoption. Developers grab it fast—"ollama run mistral-medium-3.5"—but scaling to production might hit surprises without those specs. Mistral's history, like Mixtral 8x7B shaking up dense models back in its paper, shows they deliver, yet docs lag on deployment fine print.

HOST

Production surprises could frustrate teams already stretched thin. And no word on open-source status or API pricing? That's another hole—how do users even get their hands on it beyond Ollama?

PRIYA

Availability stays fuzzy right now. It's open-weights, listed on Mistral Docs alongside Medium 3.1 and Devstral 2, but no pricing for API access or Le Chat tiers. Compare to o3's $2 per million input tokens and $8 output—no such numbers here. You pull it locally via Ollama or NVIDIA's NeMo-AutoModel docs, or soon as Le Chat's default and Vibe CLI standard. Cloud side, Mistral unveiled coding agents, an agentic chatbot, and cloud AI coding features, but specifics like quotas or costs? Absent. That vagueness favors big players who can host internally, while startups wait for clarity. Ties back to their French roots—pushing edge-to-cloud models since Nemo 12B in 2024—but execution details trickle slower than the hype.

HOST

No pricing echoes early Mixtral days when costs surprised everyone. Speaking of risks, any controversies around Mistral's open-weights push? The briefing doesn't flag criticisms, so is the path clear?

PRIYA

No sourced criticisms or controversies in the docs or reports—AlphaSignal frames it as straightforward race progress. But open-weights at 128B dense carries risks they don't spell out. Anyone downloads, fine-tunes for spam, deepfakes via that Pixtral vision tower, or worse. Self-hosting on four GPUs democratizes power, yet amplifies misuse without built-in moderation like their separate Mistral Moderation 2 from March 2026. No scandals reported, unlike some U.S. firms' data scandals, but that 256k context invites long-prompt jailbreaks. Teams must layer their own safeguards. Mistral's clean so far, focused on utility like replacing Devstral in CLI, but the absence of red flags doesn't erase those inherent open-model headaches.

HOST

Misuse potential with open-weights makes sense—no guardrails baked in. Shifts burden to users. Now these new cloud features—coding agents, agentic chatbot. How do they tie to Medium 3.5?

PRIYA

New cloud coding agents run on Medium 3.5's strengths, letting it chain reasoning and code gen for full tasks. The agentic chatbot builds autonomous flows, like debugging across files with 256k context holding entire repos. Cloud AI coding probably means hosted inference for teams avoiding GPUs. But without specifics—no agent limits, integration APIs, or uptime SLAs—these feel more announcement than toolkit. Heise Online covered it as "New Medium 3.5 and Cloud Coding Agents," yet details mirror the model gaps. Pairs with local runs, giving hybrid choice: four GPUs at home, cloud for scale. Echoes Mixtral 8x7B's sparse MoE impact years back, but dense 128B here prioritizes single-model simplicity over expert routing.

HOST

Hybrid choice sounds flexible, but agent details thin as the rest. Improvements in instruct, reasoning, coding—any specifics, or all promise?

PRIYA

Specific gains aren't quantified yet—that's the gap. Mistral claims unified 128B beats prior splits like Devstral 2 for code or Magistral for reasoning, all in one set now powering Le Chat. Expect jumps from Medium 3.1's August 2025 baseline, given SFT on MedPix dataset. But no deltas like "20% better GPQA math" or HumanEval code pass@1. Benchmarks pending on BenchLM versus o3 or others. Concrete win: multimodal via Pixtral tower processes images with text, unlike pure text predecessors. Developers test via "codex ollama launch --model mistral-medium-3.5." Real proof comes when scores drop, separating hype from deployable edge.

HOST

Pending specifics keep it speculative. One gap down, but what's next for Mistral in this family? Timeline shows Small 4 in March 2026—Medium 3.5 feel like a bridge?

PRIYA

Medium 3.5 caps a sprint: from Medium 3 in May 2025 to this, successor in a family exploding since Mixtral 8x22B late 2024. Next could densify further or sparse it like 8x7B. Cloud agents hint at ecosystem lock-in—use their hosted Medium 3.5 for chatbots, local for CLI. Risks grow if benchmarks underdeliver; teams stick with proven like Nemo 12B. But four-GPU access pulls indies into the game, challenging cloud-only giants. Watch Le Chat rollout— if it sticks as default, that's daily user proof over lab scores.

HOST

Indies challenging giants—that's the stakes. Gaps make it cautious optimism. Priya, always eye-opening. I'm Alex. Thanks for listening to DailyListen.