OK, I'm 100% rooting for both Mistral and task focused small models.
But Mistral has fall really far behind since 2025Q3. It seems they can't get good reasoning models working at even medium context sizes, which is necessary to be at the table right now.
Gemma4 and Qwen3.6 are currently best in the small size; Mistral's "small" model has ~4x the parameter count at 120B and isn't even competing with models a quarter its size.
Back one year ago with Mistral Small 3.1 they were keeping up, but they've fallen into irrelevancy right now.
If Mistral seriously wants to play the on-prem and small task-specific model game, a decent proxy would be to build models that get the r/localLlama crowd excited
I agree. I am a paying Le Chat Pro user, really rooting for a European alternative. But the quality difference between Mistral and the frontier labs is growing too big to ignore. It’s worrying to me that they didn’t talk much about new models at the conference, because that is really where their focus should be IMHO.
I am wondering what is keeping them back, though: Money? Compute? Skills? Training data? My fear is that you are really only getting really good models by training on very dubious data (outputs from the frontier models etc) and that Mistral is too European and too enterprisey to take those risks.
My theory with no insider information: it’s a little of all of the above, but mostly money. To some extent, you can dig yourself out of a data hole with RL and a lot of compute. And you can buy a lot of compute and some data with a lot of money. Big labs have been operating in this regime for a while and it’s one of the drivers behind their costs beyond just scaling the weights and doing the actual training. Mistral just doesn’t have access to this level of compute or the money to try and muscle their way in.
I wouldn't be surprised if each of the frontier American labs and individually has compute access similar to the entire EU. Chinese firms are a more interesting comparison since there are a fair amount of great models there, and it's estimated about 15% of the ai relevant compute is in China versus maybe 5% in the EU under European companies (and 70% ish in the US is the most common ballpark I see)
More than 5%, I assume. From the combination of "5% in the EU under European companies" and "each of the frontier American labs and individually has compute access similar to the entire EU"
I dont't think that was meant to be implied: the EU actually has access to more GPUs than those hosted by European companies in Europe, just as US labs have access to GPUs hosted outside the US
The EU has intentional structural hurdles to pouring money into a predetermined single company. Both hurdles meant to fight corruption and nepotism, and hurdles meant to ensure fairness between the member states. After all, money to Mistral is money to France too, and you don't want countries to abuse such mechanisms
It's not impossible, but China is just much better set up for the nessesary level of government support
It's a bit strange, but a huge handout from the EU/France and a huge AI lab investment round are different orders of magnitude. The necessary sums are just not politically possible. How do you sell spending the equivalent of ten USS Gerald Fords on a start-up? You don't.
> I am wondering what is keeping them back, though: Money? Compute? Skills? Training data?
Not ruthless enough and no backing by a corrupt govt administration that has no morals but focuses on self-enrichment instead.
Might sound drastic but I think that's actually closer to the truth thn everbody likes to admit.
> My fear is that you are really only getting really good models by training on very dubious data (outputs from the frontier models etc) and that Mistral is too European and too enterprisey to take those risks.
I think an European company, taking Chinese models, perhaps doing its own post-training on them and training the Chinese-ness out, with a great chat service, enterprise API and coding agent, could be pretty valuable in itself.
> what is keeping them back, though: Money? Compute? Skills? Training data?
All of the above and more. Everything holding Mistral back is the same thing that has held Europe back from competing in the entire digital revolution. See this 1991 article lamenting the loss of any viable European PC manufacturer: https://www.nytimes.com/1991/04/22/business/europe-stumbles-...
Mistral being in Europe is disadvantaged with:
1. Money: less diverse private pension fund environment = less LPs to invest in VC funds = less VC dollars to invest in new ventures. European money is vacuumed out of the private sector into state pension funds and dumped into low yielding government bonds. This starves the private sector of capital while inflating the % of GDP driven by government spending every year (government pension funds buying government bonds in circular fashion enable runaway deficit spending...just like circular AI infrastructure spending).
2. Talent & compute: due to #1, Silicon Valley can outbid Europe for the best talent and hardware. Watch an OpenAI launch video and listen to all the European accents.
3. Local market fragmentation: Europe is a collection of countries that pretend to work together while not even having a unified capital market. The average EU citizen can barely communicate with their neighbor in a common language beyond the level of a toddler (english fluency is massively overstated by Americans who only experience tourist capitals).
4. Regulatory disadvantages: In everything from company regs, employee regs, unions, privacy regs, data portability regs, etc.
It's not "culture" or Europeans being "lazy" as most people would claim. There's currently thousands of young french people working 80 hour weeks creating dumb consulting powerpoints or legacy investment banking deal memos as we speak. Ambitious people exist everywhere in equal proportion, they're just working on the wrong things.
Europe can't compete in the digital revolution the same way they could compete in the industrial revolution due to various system design choices. Culture is simply the aesthetically observed byproducts of system design.
>The average EU citizen can barely communicate with their neighbor in a common language beyond the level of a toddler (english fluency is massively overstated by Americans who only experience tourist capitals).
Not true in my experience: even German waiters in small towns tend to have pretty fluent English.
It varies a lot. Germany is pretty strong in English, and the Netherlands next door is exceptional, but as you go south to Italy, etc English proficiency weakens.
Edit: more broadly, there’s just more friction when people aren’t in their first language. I know I hesitate to bring up some things, say hi to strangers, try making a joke, etc because the cost of talking is just… higher.
1 and 2 are the same. Infinite money without barely any consequence because of 'reserve currency' privilege. To compete with that, the EU can't nuke the dollar because it would be suicide given the Eurodollar realities, and they can't anchor EU ip and talent because our politicians are too intertwined with globalist ideology and capital.
"they can't anchor EU ip and talent because our politicians are too intertwined with globalist ideology and capital." You want to force staying in the EU?
> 2. Talent & compute: due to #1, Silicon Valley can outbid Europe for the best talent and hardware. Watch an OpenAI launch video and listen to all the European accents.
There is definitely a lot of truth to that. Maybe a bit of an arbitrary measure, but these are the nationalites of the people that wrote the "Attention is all you need" paper. Pretty revealing I find:
The US also has public pensions (social security payouts rival or beat many EU countries) with dramatically better tax free private options on top.
Also, the US has free healthcare (Medicare and Medicaid) for roughly 50% of its population.
Expanding that to 100% doesn’t suddenly make them a bad country to do business in.
You think OpenAI is going to close up shop and move to Mexico if the US expands single payer healthcare? That would actually make it even easier for businesses to operate in the US!
Social Security and Medicare are vastly inferior to their European counterparts. Medicaid is an absolute disaster and a large number of doctors and health facilities will not even accept it.
Yeah and data protections. GDPR, data frugality laws, etc. may be the end of Mistral but it's a small price to pay for corporations to not have free range over every minute detail of our lives. Americans just accept it because they have already lost. We haven't, in fact we've just won recently with chat control being struck down. Meta can no longer train on and monitor every Whatsapp chat without being criminally liable.
re: #4 Maybe it’s easier if you grow up in the system and know how to navigate the written and unwritten rules, but as a dual Canadian-American who recently gained Austrian citizenship, the regulatory friction is absolutely real. I decided to launch a new venture through an Austrian GmbH.
There are supposedly streamlined paths for local residents, but I had to go through the standard corporate pipeline. I spent three months fighting a bizarre catch-22 between my notary (who cost €3k+) and the bank. To open the account, I had to prove I deposited €10k in capital. But I couldn't make the deposit without an active bank account. On top of that, the bank's compliance team kept arbitrarily canceling my application due to "incorrect answers"... refusing to tell me what the errors actually were and forcing me to restart the entire process ab initio.
I finally just gave up. I wrote off the €3,000 notary fee and €1,000 in registered office costs as a sunk cost, and incorporated a US LLC instead. It took under 10 minutes, no notary, fees of $25 since I did it myself, plus another 20 minutes to open the business bank account.
There was no commercial reason to choose Austria; it was purely sentimental. My ancestors were entrepreneurs in Linz and Vienna, and I loved the idea of renewing that legacy. But the sheer weight of the bureaucracy managed to kill about 99% of the early-stage startup enthusiasm you normally rely on to get a new project off the ground.
That catch-22 is supposed to be broken by the bank. It's a two phase commit where you open the account in a special state where you can only deposit the capital. Then the bank gives you evidence you've done so, you take that to the notary and open the company, then send the evidence you've done that back to the bank to convert it into a full account.
It's a bizarre system that Switzerland uses too. I've done it twice. Unfortunately the German speaking world has a lot of rules that are trying to eliminate all risk for investors and employees. The GmbH/AG capital requirements are just the start.
The next fun thing you might have encountered, at least in Switzerland, are rules that literally say your company's assets can't fall below 50% of your initial capitalization. If it does you're supposed to raise funds or make more investment of your own private capital and this rule pierces the usual liability requirements. Even more fun: it turns out that this law isn't actually enforced and locals regularly ignore it. But bad accountants won't tell you that. They'll just inform you of the law when you do your yearly accounts.
Then you have wealth taxes that cover the valuation of a startup as if it were a cash position. So if you raise $100M in investor funding then whatever shares you have left over are considered to be liquid assets you can offload at will, and are wealth taxed as such. The fact that the shares don't trade in a liquid market is irrelevant to the tax authorities. In Zürich at least that got patched by the local tax office deciding that startup shares aren't counted for the wealth tax, but this just means you have to be able to convince the tax authority that your company is a startup. The way they determine this is more or less just the opinion of whoever at the tax office assesses your case. Does it sound "startuppy" enough?
Fixing this stuff isn't hard, but it never gets fixed because European politics is both quite stagnant and dominated by people who view hostility to business as a virtue signal. They don't want to fix it because they think businesses are sort of like oil fields. They just exist, lying around naturally, and the only question is how to maximally exploit them.
This is tangential: and forgive my ignorance here, but is there an inherent reason why there aren't smaller, focused models from the frontier model providers?
I'm thinking something like a software-specific subset of Opus that is the default for use in Claude Code. Smaller, cheaper to deploy and consume, maybe faster.
OpenAI used to make Codex-specific models, but they stopped. What I've gathered from interviews and similar is that training two models isn't worth the (small) lift from having a coding-specific model. You're pre-training on everything anyway, and coding RL is reasonably useful for general-purpose models too.
What I've heard is that much of the model "intelligence" is a commingled bucket: although you can specialize specific knowledge somewhat, it's hard to specialize advanced reasoning to specific domains because so much of reasoning is a generalized capability that is not unique to, say, coding.
It turns out coding has to do with a lot of the same reasoning needed in math or in legal analysis, even if the grammatical expression is different.
This is less true of lower intelligence tasks. Classification requires a lot less reasoning capacity and so can be much smaller and more specialized.
agreed, the next price increase from frontier labs (and the inevitable limits decrease in subscription tiers) will have people thinking real hard about their model providers and that's when mistral should be ready. however, given their recent performance, I realistically don't have my hopes high up.
I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right?
Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.
That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
On a more risk-strategy level there is the size of their legal team, general endowment, and supplier and political connections to consider.
Everyone is free to ignore their ToU, but I can understand why a company would avoid it...
it doesn't matter the reason. This is a race and nobody will care or remember how the winners got there.
Mistral looks like it's fading away to irrelevance unless they can play alongside the similar sized models, or have some unique advantage other than being in Europe, for Europe. I was really excited for them back when they were startup that had the biggest European venture round ever. This space will have a few winners, and many losers. Google, plus either Anthropic or OpenAI most likely. Big models will see breakthroughs in inference performance/cost fall precipitously and small models will only exist on devices (Pixels and iPhones, cars, watches, bluetooth speakers, etc)
It’s not that I don’t agree with you, I am just pointing out why it’s hard to catch up to scaling laws given the European economic (capital) and political (US would be upset if they found out Europeans distill) constraints. China is only bound by economic constraints.
> This is a race and nobody will care or remember how the winners got there.
For consumer AI, yes. For coding assistants, probably.
For specific application "business" AI like the things Airbus announced the other day? Not at all. What matters for an Airbus using Mistral to build compliance documentation based on AI generated physics simulations is the enterprise relationship, reliability, compliance, forward deployed engineers helping with the fine tuning, quality, predictability, support. A Chinese lab having a better at benchmarks model that is cheaper is just irrelevant for that.
And IMO, the real money in AI is this type of "business AI" deployment. Developer tooling tends to converge on becoming commoditised. Once you're a core supplier for a big bank and embedded in their processes, you're there untill you screw up with the pricing (like Broadcom), and even then.
So then the European ones should join with European copyright holders to sue OpenAI/Anthropic and watch them trying to BS their way around what they train on.
It’s really a pity, why can’t they feel superior while breaking ToS and copyrights just like Americans can feel superior over Deepseek while breaking ToS and copyrights?
I tried it out on some dev tasks with their Mistral Vibe subscription, and the performance was pretty okay (okay, not great), both in regards to development and speed. Worse than Anthropic's models I'm used to but at 20 EUR per month it wasn't a bad deal - except that the 200k context size would more or less be a deal breaker in many cases.
The other comment already mentioned that you get their subscription: https://mistral.ai/pricing/ they do say that you can try out their coding agent for free, but personally the Pro tier is pretty affordable too to try out for a month.
Then you can install their coding harness, I personally used the Python + uv option: https://mistral.ai/products/vibe/code/ if you don't have uv yet, you might have to install it too: https://docs.astral.sh/uv/ though I already use it for other projects. Oh and if on Windows, you probably want to do all of the installation inside of WSL, just so that file paths are the *nix variety, I've had issues otherwise with pretty much every coding harness, like OpenCode as well (across multiple models).
After that, you need an API key for your subscription, you can generate and copy it here: https://console.mistral.ai/codestral/cli that's also where you see the quota, though it seems to NOT refresh instantly, but more or less a few times a day.
Yeah. I run LLM models locally and for me 22B-32B is the largest I'm willing to invest in trying out.
Even though Mistral 4 has 6B active parameters per token (allowing 3-3.5 per token parameters to be loaded on a 4090), the ~240GB download + storage is pushing the limits of being able to try this out locally, especially if you are downloading and evaluating multiple models.
It also makes it harder for other people to make downstream finetunes like with what happened with the older Mistral/Magistral models.
I think machines like the DGX Spark are about to become a lot more common/popular. It’s big enough to run sparse 150-250B MoEs with enough throughout for a single user. Deepseek v4 Flash is #1 (in terms of usage) on OpenRouter because it’s good enough to be useful. You can run it on a Spark (though it runs better across 2, which is getting up there in cost)
I find Mistral Medium 3.5 with OpenCode is perfectly fine if you're willing to talk to it in a more fine-grained way about actual code. For me that's fine because even with huge frontier models I don't like trying to vibe prompt like a product manager.
Mistral is entering the "let's extract has much money from EU taxpayers as we can" phase of European tech company which did not get bought by a US one.
They'll end like Dailymotion, just a zombie company.
Nobody trying to compete with Google, OpenAI, and Anthropic should be playing the small models / local models game.
Foundation model labs should be building very large reasoning models, then leaving it to the community to distill them down.
You can't scale a small model up, but you can scale a small model down.
I'm convinced the only way we'll have a seat at the table in the future and avoid total runaway takeoff is if there are very large models within 80% of the capabilities of the frontier models. Tiny RTX models do diddly squat to remain competitive.
Build open weights models for running on H200s. I'll spin them up on RunPod or Lambda.
I do think there's a chance open weight models have a bit of a moment with the costs of frontier models growing on business balance sheets. It's unfortunate from my "privacy loving" PoV that it's mostly Chinese models filling the gap. ( the top models on openrouter for instance ).
I have used Mistral models out of pure ideology for web agents and the like which aren't doing a lot of heavy lifting.
Antirez’s Deepseek 4 Flash implementation that can run on MacBooks also was a revelation. It runs decently on M5 Max 128GB and it’s pointing out other bottlenecks like prefill speed which will improve.
I thought distillation meant small models don't have to compete with the big models and can always eventually achieve close parity, but it's just a matter of time to do the distillation? (i.e. how much lag do you want to live with) Am I oversimplifying?
There is likely a theoretical limit to how much intelligence you can pack into a model of a given size (especially when stretching that over a large input context size).
Our evals are pretty complex so we only recently started testing ~30B class models, which are now becoming quite smart (on par with the frontier from 1 year ago). Mistral is far behind, but I'm rooting for them.
> a decent proxy would be to build models that get the r/localLlama crowd excited
I don’t really disagree with your post, but this is not exactly right. That subreddit seems to go from hype train to hype train every week, I haven’t found anything really insightful in it for quite a while now.
We actually found the Mistral Small 4, quantized to 4bit was comparable to Qwen 3.6 27B and is roughly the same size. At least from our experience on our use cases, the quantization of the Mistral model worked far better than trying to quantize the Qwen family.
Fully agree to your point though, Mistral in general is far behind where I'd expect and Qwen in particular is crushing it at the smaller sizes.
Personally, I'd consider anything 20B params and above a "medium" model. Small being <20B and large >100B. I think obviously we can get to the huge 1-2T param models, but frankly the margin of accuracy improvement for the speed hit is kinda insane (1-2% for many metrics).
It's all relative. For local use I'd classify it by hardware (VRAM size) using FP8 or Q6 quantization:
1. tiny <2-3B -- easily runnable on lower-spec hardware
2. small 4-8B -- runnable on 8GB GPUs
3. medium 9-12B -- runnable on 12GB GPUs
4. large 13-24B -- runnable on 16GB (for the lower end models) and 24GB GPUs
5. very large 25-32GB -- runnable on 32GB GPUs
6. huge >32GB -- not easily runnable on consumer GPUs without compromising performance (offloading layers to the CPU/RAM), quality (heavy quantization, esp. at <= Q4), or price (investing in multi-GPU setups and/or server-grade hardware).
You could possibly split huge down further, as 70GB models (e.g. llama 3) are easier to get working than >120GB models and 1TB models are completely intractable.
> tiny <2-3B -- could run in a browser even, mac neo
Or a phone. I’m running Gemma 4 E2B in one of my apps on my 14 pro (which may or may not be killing my display through overheating. It might just be a coincidence).
I really want Europe to be part of the AI development and research. And I strongly cheered for Mistral. But they are accumulating too much technological delay. This needs to be fixed, otherwise it will turn into yet another proof we are not able to run large tech with good results. Basically any Chinese lab is doing much better. It's not Mistral that created I don't want to say DeepSeek, but MiMo 2.5, Minimax 2.7, and so forth. There are only weaker and/or larger and slower (no MoE) models. Not good.
Europe shot itself in the dick with this hastily implemented at the height of mass hysteria bullshit and now no sane company will build anything there. an AI startup in the US or China can be a boy and his computer. in Europe, the boy needs a dozen lawyers.
Mistral's sinking into irrelevancy despite the head start they had, the very promising early models they released, and the funding they receive, might very well be the consequence of trying to comply with all that crap.
You don't compete with anthrophopic from the basement. For that you need either a shit loads of money, or a government which are not afraid of getting very very involved.
There is a lot of Europeans working on AI, it's just that a lot of them work for American companies. Because of money.
Possibly yes but let me remember you that France, Italy Germany were against the AI act, so here something very odd is happening, that the EU funding nations are getting marginalized by the countries they welcomed on key topics for our future, and I believe corruption could be a big part of what is happening, both internal to those three countries and at an even more alarming rate in other countries.
EU big nations getting marginalized: haha. The only reason there’s no US-like tariff on Chinese cars is because Germany was too scared it would lose its access to Chinese market.
It's not a matter of importance, but that bad actors as they tried to do in Italy, corrupting EU parliament, may be doing the same with counties are have less visibility. A weak EU is not also in the best interest of countries that wanted the AI act, and surely not in the interest of their citizens, but there could be pressures.
I understand your point, I just object to the language and dividing the EU into more and less important blocks. If a voting mechanism is broken, that's where the issue is.
Who put a nepo-baby lawyer in charge of the big €95bn AI fund? EU bureaucrats living the 6-figure high life with chauffeurs and private jets in a bubble completely isolated from reality.
I hate the fake European foreign-backed right-wing parties but they didn't cause the current situation.
But I'm afraid it might be too late as the cancer spread and did too much damage. Insane regulations, no energy, looming demographic/pension crisis, tax hell, and collapsing industries.
Way more important than this act are the police raids. Someone used your SaaS to send phishing (see today's front page HN)? They'll just take all your servers away. Goodbye business. Unless they think the general public would riot, so established companies are okay. You can't build a castle on a foundation of quicksand.
Well , there isn’t also the opposite take from TechCrunch where they say: Why Paris may be the most important AI city outside Silicon Valley. [0]
While the EU loves its regulation, I still feel it’s too early to write it down in the AI race. It will not replace Anthropic or OpenAI any time soon, but even Google and Meta fail to do that.
If AI continue to grow and expand, there is enough space for many more unicorns.
The gist of it is very simple - depending on the risk of what you're doing with AI, you have to document why it did what it did, and be able to explain it; or you can't use it at all. So if you're using AI for mass surveillance, you can't; if you're using it for treating loan applications you need to be able to explain why it approved/denied; if it's a customer service chatbot, do whatever, nobody cares.
Not only is burden of the legislation fairly low (and a lot of it hasn't come into force yet), it is extremely reasonable. No, sorry, we don't want a UnitedHealthcare using a broken algorithm on purpose to deny as much care as possible and hiding behind computer says no.
The problem is that statement is a bit too open to interpretation. Ever had Claude piss you off by being stupid and talking in circles? Sounds like manipulation of human behavior!
When it comes to MoE, to me, I remember Mixtral model that showed the viability of MoE for the first time. I was impressed by their technical report. To be clear, MoE idea was already out there, if I am not mistaken. If they have pushed Mixtral model family further, who knows they might have achieved the reputation of what the current Qwen family has. A missed opportunity.
Compared to the UK Government which recently announced 10 million GBP for AI research, which will likely be scooped up by consultants. I think Europe is doing fine considering.
I think that while y'all were appreciated members and definitely had a lot to offer, you also had a lot of annoying carve-outs and kept stalling needed measures to federalize and strengthen the EU more so we can be a proper superpower in our own right.
Maybe it's good you left for now, maybe we can finally get these things done. And once that's accomplished and enough of the gammon has died off, you can always rejoin :-)
Jumping in and most people in Germany wouldnt see UK as an American trojan hourse. I dont think anti American countries like France and Danemark have a problem with UK being in the EU per se.
I can see most people want that UK wouldnt just get special treatment any more.
> BNP Paribas runs Mistral models on-prem for KYC in Belgium, with sensitive data staying within the bank's walls. Abanca is using agent orchestration to handle sensitive customer information at a huge scale (2 million customers in their app). For European companies in regulated industries, this is a good alternative to relying on US hyperscalers.
Mistral leaning into on-prem and European-hosted models is very smart.
Respectfully, I don't think it's "very" smart. It is a fair option given their limited options? Everyone is doing FDE or (customer engineering to be more transparent) because otherwise they will just be seen as markup on token cost. And the Neo-SaaS companies will take the money instead.
You don’t think it’s smart to get reliable funding this way? From Banking the Cash Cow?
Devstral is getting better, it’s the Vibe harness that’s holding it back (I think). I can see how that would drive some business as well.
Their chat thingie isn’t very well positioned, but gets results. Could be an euro or two per month, maybe bundled with some more features. It’s not like Mistral has no options, if anything they’re just a bit complacent and not ambitious with their plans.
My take is that Mistral is not focusing on generating contents such as code, images, or videos. They focus on multi-lingual models, OCR, voice, and others I believe. Their model intro page manifests that although it always confuses me because it's too colorful and there are too many categories, not to mention model names. I hope their decisions will pay off.
Isn’t that the usual EU startup playbook once they give up on the B2C or world-scale SaaS markets? Refocusing on large (European) enterprise B2B and government contracts?
It always felt to me this (enterprise B2B) was where European startups went to die.
We're talking about enterprise customers. The trivial answer is Mistral has sales teams and consultants from the same company that builds the models and from the EU.
i can invest in public markets in a lot of $10b sales and consultants businesses, who can also put mistral on premises (or do whatever the hell people ask for), it makes mistral sound like it is yet another one of those, not a growing $1T business.
>weird training biases that were required by the Chinese government
What is "weird training biases" to us might not be weird to them and vice versa. Just ask the Chinese what they think about LGBTQ+, Japanese, pride parades, Islam and colored minorities.
Every nation has its own biases injected in its domestic LLMs at this point. Otherwise they risk getting in trouble for hate speech/disinformation in the jurisdiction where they operate.
Same how Google Maps cleverly biases the lines of disputed borders based on where you are viewing it from. Or how Google maps switched 'Gulf of Mexico' to 'Gulf of America' in an instant when the orange man signed the paper. Google won't want to anger the US administration the same way how Mistral won't want to anger France and the EU, so Mistral will have all the EU prime directives injected into its LLMs no matter if they're ludicrous or not. The law is the law whether you agree with it or not. Companies want to survive and will pander to whatever the whims the regime they live under are at the current moment regardless of what is right or wrong.
But if I'm using a LLM for personal projects or generating a photorealistic choreographed fight between Tom Cruise and Brad Pitt, I don't care what its political biases are, I care if it solves my problem better and cheaper than the competition, and here the Chinese models could end up winning the consumer market, which is why you see Mistral and other EU alternatives focusing exclusive on B-2-B corporate market.
Except there's no such thing as the "European model" similar how Europe is not a country.
Mistral is mostly French and tends to have mostly French speaking customers, like BNP PAribas in Belgium. Germany will want its own domestic AI champions, maybe in partnership with Switzerland and Austria, similar to how Denmark already has invested in LLMs focused on the Nordic languages with money from Norway.
The biggest mistake is treating Europe like a single homogenous country/market.
The original question was "Yeah but why use mistral on premises instead of Qwen?". I think you and I agree on the answer.
I for one would love to see more country-specific models. There was a story here the other day about Norway’s National Library developing a LLM specialized in Norwegian: https://news.ycombinator.com/item?id=48270770
The issue is that you wouldn't be able to even transparently get to any evidence, as these models are blackboxes.
They might start scheming behind employees backs as soon as they realize they are being used in critical infrastructure of adversaries. And nobody would know until it's too late.
It may be very smart for them, but it also shows that the EU has no desire, therefore no chance, to change and lead anything. The only thing it has is regulation.
When the humans have a track record of corruption, it might make sense for a company to seek parallel opinions from a LLM so they can at least flag suspicious human decisions.
Assuming BNP Paribas leadership wants to stop the corruption of course.
That's just one side of the story, not following it on details, but their own le chat explained to me that the company was a capitalist succubus starving to build data center in some north European country. Hilarious if you ask me.
I have been on a lecture from great government IT person, they are evaluating models and are very unhappy about the situation, because they’d love to use Mistral, in some cases it’s the only EU based model they can use … and they know it’s really bad and falling more behind.
It is well possible that Mistral can make a profitable business by being bad, but still the only possible model for EU uses. Sad story, sad to witness.
I was at the event, and was impressed by the attendance, all the leaders from the major european listed companies were there.
Also interesting to note the number of partners they invited. Going from Microsoft, Accenture and EY to startups like alpic.ai or lingo.dev . Seems like they are ramping up their M&A game too
Excited to see more about their partnership they with Alexa+. In agentic and tool-calling, Mistral’s model architecture excels at the exact structured JSON output Alexa needs to trigger APIs and smart home routines without breaking.
Sounds like they don’t have a moat at all. It’s like software consultancy with a data centre. And then the article mentions many customers using these models on prem (so data centre is not really a plus).
What’s stopping any country backed startup from fine-tuning small open source models?
Maybe because distilling small models from bigger ones that you control gives you better small models than fine-tuning from bigger models you don't control?
(I am not claiming it is the case, but stating this as an assumption)
I just got an email from them saying that they’re retiring some (most?) of the dedicated models like devstral gradually through August and one should now use the general model. Cost grows exponentially
Devstral 2 (devstral-2512 and devstral-latest) → We recommend transitioning to Mistral Medium 3.5 (mistral-medium-3-5 with reasoning_effort set to "high"), a stronger model, priced $1.5/$7.5 per million input/output tokens (change from the previous $0.4/$2).
I received the same email, although couldn’t quite figure out which retiring model I was still using, as I thought I’d already transitioned to Mistral-Medium-3.5 for everything. Anyway, after receiving the email, my hope was that it meant they were also planning on releasing some new, improved models in the next months.
> Abanca is using agent orchestration to handle sensitive customer information at a huge scale (2 million customers in their app).
Maybe my perspective is skewed on what "huge scale" means, but 2 million users? That's like a few hundred megabytes of data? Or a couple GBs if there's a lot of per-user data?
Maybe, but using state-of-the-art large language models to solve customer support queries with agentic can quickly use a lot of tokens. What I understood from the talk is that they used agents with limited responsibility and (assumption from me) smaller models, to the make sure the answers were quick, reliable and not too costly.
There are several payments processing companies that are already largely using AI for customer support queries. They still have an escape hatch to a human but at least one of those companies (on the smaller side) is reporting a ~99% success rate, they are down to a handful of human customer service employees now for cases where the customer can't find/produce the transaction ID.
European consumer focused businesses do not scale easily the same way US ones do, which is a major contributor to their problems developing tech businesses generally.
OTOH such things can be quite defensible, they just rarely become anything like as profitable.
Wasn't even aware Mistral was around and I think that just shows you how irrelevant it has become and not a very good sign for EU in general when the best talent are working for American AI companies.
I saw Tibo's tweet a while back and it was basically a legitimate complaint about the extreme taxation he faced back in EU (France I think) and its pretty obvious how much of a hinderance top down centralized regulation is to innovation.
While I welcome competition and independence, nobody can argue with American innovation and its ability to attract the best of the best. Once it takes seat of the AI reigns there is very little chance for other countries to compete, very much similar to semiconductor field and how only a few select countries have the talent and monopoly over its particular supply chain.
It's clear to anyone looking in that whatever EU is doing is not working (not just AI) and will not work as they do not seem flexible or humble enough to steer itself.
Big tech has remote offices in every major European economy, and they pay well above top 90th percentile of market rate. It basically has a talent sucking effect on the entire economy.
I believe that Mistral team is doing the best they can do. I like the directions they push; open models for various tasks, on-prem has a lot of potential. Sure, I use Claude code mostly for coding. But there are so many tasks other than just coding. Even for coding, eventually, I am certain they will catch up and Vibe becomes tolerable soon.
I've said it before that Mistral is underrated. They are looking at real world use of LLMs and tooling. Bespoke models are very appealing to lots of non-tech centered companies and state agencies. Also, Mistral's actual platform is useful. While others are watching performance leaderboards like this is some eSports stream, they are building real world uses.
I was also at the event and was pretty disappointed. Most of the talks were pretty low on information. I was at the “build” stage, which supposedly was the technical stage, but the talks there didn’t really go into technical specifics.
Does anyone else always read "Mistrial" instead of "Mistral"? Always think I'm about to read a juicy gossip piece, and let down when it's just a standard update on an AI company.
edit A lot of AI company names are really strange, actually. "Claude" is really the best a trillion+ dollar company could come up with? It sounds like the name of a grandpa or something.
OK, I'm 100% rooting for both Mistral and task focused small models.
But Mistral has fall really far behind since 2025Q3. It seems they can't get good reasoning models working at even medium context sizes, which is necessary to be at the table right now.
Gemma4 and Qwen3.6 are currently best in the small size; Mistral's "small" model has ~4x the parameter count at 120B and isn't even competing with models a quarter its size.
Back one year ago with Mistral Small 3.1 they were keeping up, but they've fallen into irrelevancy right now.
If Mistral seriously wants to play the on-prem and small task-specific model game, a decent proxy would be to build models that get the r/localLlama crowd excited
I agree. I am a paying Le Chat Pro user, really rooting for a European alternative. But the quality difference between Mistral and the frontier labs is growing too big to ignore. It’s worrying to me that they didn’t talk much about new models at the conference, because that is really where their focus should be IMHO.
I am wondering what is keeping them back, though: Money? Compute? Skills? Training data? My fear is that you are really only getting really good models by training on very dubious data (outputs from the frontier models etc) and that Mistral is too European and too enterprisey to take those risks.
My theory with no insider information: it’s a little of all of the above, but mostly money. To some extent, you can dig yourself out of a data hole with RL and a lot of compute. And you can buy a lot of compute and some data with a lot of money. Big labs have been operating in this regime for a while and it’s one of the drivers behind their costs beyond just scaling the weights and doing the actual training. Mistral just doesn’t have access to this level of compute or the money to try and muscle their way in.
Don’t they supposedly have a huge amount of EU support?
Or at least there’s been a lot of noise about that.
I wouldn't be surprised if each of the frontier American labs and individually has compute access similar to the entire EU. Chinese firms are a more interesting comparison since there are a fair amount of great models there, and it's estimated about 15% of the ai relevant compute is in China versus maybe 5% in the EU under European companies (and 70% ish in the US is the most common ballpark I see)
I think you are underestimating the amount of compute the US frontier labs have access to.
So, more than 70% of the compute on earth?
More than 5%, I assume. From the combination of "5% in the EU under European companies" and "each of the frontier American labs and individually has compute access similar to the entire EU"
I dont't think that was meant to be implied: the EU actually has access to more GPUs than those hosted by European companies in Europe, just as US labs have access to GPUs hosted outside the US
They can get what, 1B euros? 10B when everyone loses their mind? This doesn’t buy nearly enough compute nowadays.
Meanwhile, Anthropic and OpenAI have investors practically begging them to let them buy this much equity at mind-bogging valuations.
The chinese labs manage to do it. Mistral should have enough money.
The EU has intentional structural hurdles to pouring money into a predetermined single company. Both hurdles meant to fight corruption and nepotism, and hurdles meant to ensure fairness between the member states. After all, money to Mistral is money to France too, and you don't want countries to abuse such mechanisms
It's not impossible, but China is just much better set up for the nessesary level of government support
China is a way more corrupt country but this might be a benefit of less rules.
It's a bit strange, but a huge handout from the EU/France and a huge AI lab investment round are different orders of magnitude. The necessary sums are just not politically possible. How do you sell spending the equivalent of ten USS Gerald Fords on a start-up? You don't.
And a lot of the "funding" is through mutual deals with MSFT, Nvidia, etc. The Europeans have none of that and would need to pay in actual cash.
> I am wondering what is keeping them back, though: Money? Compute? Skills? Training data?
Not ruthless enough and no backing by a corrupt govt administration that has no morals but focuses on self-enrichment instead.
Might sound drastic but I think that's actually closer to the truth thn everbody likes to admit.
> My fear is that you are really only getting really good models by training on very dubious data (outputs from the frontier models etc) and that Mistral is too European and too enterprisey to take those risks.
Exactly.
Should it, though?
I think an European company, taking Chinese models, perhaps doing its own post-training on them and training the Chinese-ness out, with a great chat service, enterprise API and coding agent, could be pretty valuable in itself.
> I am wondering what is keeping them back, though: Money? Compute? Skills? Training data?
Considering all their talk about new DCs and compute, and a few offhand comments, it sounded to me that compute is a big limitation.
> what is keeping them back, though: Money? Compute? Skills? Training data?
All of the above and more. Everything holding Mistral back is the same thing that has held Europe back from competing in the entire digital revolution. See this 1991 article lamenting the loss of any viable European PC manufacturer: https://www.nytimes.com/1991/04/22/business/europe-stumbles-...
Mistral being in Europe is disadvantaged with:
1. Money: less diverse private pension fund environment = less LPs to invest in VC funds = less VC dollars to invest in new ventures. European money is vacuumed out of the private sector into state pension funds and dumped into low yielding government bonds. This starves the private sector of capital while inflating the % of GDP driven by government spending every year (government pension funds buying government bonds in circular fashion enable runaway deficit spending...just like circular AI infrastructure spending).
2. Talent & compute: due to #1, Silicon Valley can outbid Europe for the best talent and hardware. Watch an OpenAI launch video and listen to all the European accents.
3. Local market fragmentation: Europe is a collection of countries that pretend to work together while not even having a unified capital market. The average EU citizen can barely communicate with their neighbor in a common language beyond the level of a toddler (english fluency is massively overstated by Americans who only experience tourist capitals).
4. Regulatory disadvantages: In everything from company regs, employee regs, unions, privacy regs, data portability regs, etc.
It's not "culture" or Europeans being "lazy" as most people would claim. There's currently thousands of young french people working 80 hour weeks creating dumb consulting powerpoints or legacy investment banking deal memos as we speak. Ambitious people exist everywhere in equal proportion, they're just working on the wrong things.
Europe can't compete in the digital revolution the same way they could compete in the industrial revolution due to various system design choices. Culture is simply the aesthetically observed byproducts of system design.
>The average EU citizen can barely communicate with their neighbor in a common language beyond the level of a toddler (english fluency is massively overstated by Americans who only experience tourist capitals).
Not true in my experience: even German waiters in small towns tend to have pretty fluent English.
It varies a lot. Germany is pretty strong in English, and the Netherlands next door is exceptional, but as you go south to Italy, etc English proficiency weakens.
Edit: more broadly, there’s just more friction when people aren’t in their first language. I know I hesitate to bring up some things, say hi to strangers, try making a joke, etc because the cost of talking is just… higher.
Was just driving around medium and small-ish towns in Bavaria. This was not my experience at all.
The German speaking members of our group had to order food for us in most restaurants.
And most locals aren’t waiters in restaurants.
1 and 2 are the same. Infinite money without barely any consequence because of 'reserve currency' privilege. To compete with that, the EU can't nuke the dollar because it would be suicide given the Eurodollar realities, and they can't anchor EU ip and talent because our politicians are too intertwined with globalist ideology and capital.
"they can't anchor EU ip and talent because our politicians are too intertwined with globalist ideology and capital." You want to force staying in the EU?
> 2. Talent & compute: due to #1, Silicon Valley can outbid Europe for the best talent and hardware. Watch an OpenAI launch video and listen to all the European accents.
There is definitely a lot of truth to that. Maybe a bit of an arbitrary measure, but these are the nationalites of the people that wrote the "Attention is all you need" paper. Pretty revealing I find:
Ashish Vaswani: India
Niki Parmar: India
Jakob Uszkoreit: Germany
Llion Jones: Wales (UK)
Aidan Gomez: Canada
Łukasz Kaiser: Poland
Illia Polosukhin: Ukraine
Noam Shazeer: USA
Yes that was 2018. Things vastly deteriorated in the US.
You say that as if the American version of maximalist Capitalism is good or desirable to most people.
Personally, I would much rather have good public pensions and health-care, than A.I agents.
Maybe we will have only agents, soon.
This has nothing to do with it.
The US also has public pensions (social security payouts rival or beat many EU countries) with dramatically better tax free private options on top.
Also, the US has free healthcare (Medicare and Medicaid) for roughly 50% of its population.
Expanding that to 100% doesn’t suddenly make them a bad country to do business in.
You think OpenAI is going to close up shop and move to Mexico if the US expands single payer healthcare? That would actually make it even easier for businesses to operate in the US!
Social Security and Medicare are vastly inferior to their European counterparts. Medicaid is an absolute disaster and a large number of doctors and health facilities will not even accept it.
Not true in the case of average social security payouts. But again, this argument is a total derailing of this thread and addresses none of my points.
Explain to me how expanding US single payer healthcare suddenly makes the US a worse place to do business in than Europe?
Companies would love not having to deal with the complexities of 401ks and employer health plans.
Yeah and data protections. GDPR, data frugality laws, etc. may be the end of Mistral but it's a small price to pay for corporations to not have free range over every minute detail of our lives. Americans just accept it because they have already lost. We haven't, in fact we've just won recently with chat control being struck down. Meta can no longer train on and monitor every Whatsapp chat without being criminally liable.
re: #4 Maybe it’s easier if you grow up in the system and know how to navigate the written and unwritten rules, but as a dual Canadian-American who recently gained Austrian citizenship, the regulatory friction is absolutely real. I decided to launch a new venture through an Austrian GmbH.
There are supposedly streamlined paths for local residents, but I had to go through the standard corporate pipeline. I spent three months fighting a bizarre catch-22 between my notary (who cost €3k+) and the bank. To open the account, I had to prove I deposited €10k in capital. But I couldn't make the deposit without an active bank account. On top of that, the bank's compliance team kept arbitrarily canceling my application due to "incorrect answers"... refusing to tell me what the errors actually were and forcing me to restart the entire process ab initio.
I finally just gave up. I wrote off the €3,000 notary fee and €1,000 in registered office costs as a sunk cost, and incorporated a US LLC instead. It took under 10 minutes, no notary, fees of $25 since I did it myself, plus another 20 minutes to open the business bank account.
There was no commercial reason to choose Austria; it was purely sentimental. My ancestors were entrepreneurs in Linz and Vienna, and I loved the idea of renewing that legacy. But the sheer weight of the bureaucracy managed to kill about 99% of the early-stage startup enthusiasm you normally rely on to get a new project off the ground.
That catch-22 is supposed to be broken by the bank. It's a two phase commit where you open the account in a special state where you can only deposit the capital. Then the bank gives you evidence you've done so, you take that to the notary and open the company, then send the evidence you've done that back to the bank to convert it into a full account.
It's a bizarre system that Switzerland uses too. I've done it twice. Unfortunately the German speaking world has a lot of rules that are trying to eliminate all risk for investors and employees. The GmbH/AG capital requirements are just the start.
The next fun thing you might have encountered, at least in Switzerland, are rules that literally say your company's assets can't fall below 50% of your initial capitalization. If it does you're supposed to raise funds or make more investment of your own private capital and this rule pierces the usual liability requirements. Even more fun: it turns out that this law isn't actually enforced and locals regularly ignore it. But bad accountants won't tell you that. They'll just inform you of the law when you do your yearly accounts.
Then you have wealth taxes that cover the valuation of a startup as if it were a cash position. So if you raise $100M in investor funding then whatever shares you have left over are considered to be liquid assets you can offload at will, and are wealth taxed as such. The fact that the shares don't trade in a liquid market is irrelevant to the tax authorities. In Zürich at least that got patched by the local tax office deciding that startup shares aren't counted for the wealth tax, but this just means you have to be able to convince the tax authority that your company is a startup. The way they determine this is more or less just the opinion of whoever at the tax office assesses your case. Does it sound "startuppy" enough?
Fixing this stuff isn't hard, but it never gets fixed because European politics is both quite stagnant and dominated by people who view hostility to business as a virtue signal. They don't want to fix it because they think businesses are sort of like oil fields. They just exist, lying around naturally, and the only question is how to maximally exploit them.
> task focused small models
This is tangential: and forgive my ignorance here, but is there an inherent reason why there aren't smaller, focused models from the frontier model providers?
I'm thinking something like a software-specific subset of Opus that is the default for use in Claude Code. Smaller, cheaper to deploy and consume, maybe faster.
OpenAI used to make Codex-specific models, but they stopped. What I've gathered from interviews and similar is that training two models isn't worth the (small) lift from having a coding-specific model. You're pre-training on everything anyway, and coding RL is reasonably useful for general-purpose models too.
Interesting. I'd have guessed there would be meaningful opex benefits to serving smaller models.
What I've heard is that much of the model "intelligence" is a commingled bucket: although you can specialize specific knowledge somewhat, it's hard to specialize advanced reasoning to specific domains because so much of reasoning is a generalized capability that is not unique to, say, coding.
It turns out coding has to do with a lot of the same reasoning needed in math or in legal analysis, even if the grammatical expression is different.
This is less true of lower intelligence tasks. Classification requires a lot less reasoning capacity and so can be much smaller and more specialized.
agreed, the next price increase from frontier labs (and the inevitable limits decrease in subscription tiers) will have people thinking real hard about their model providers and that's when mistral should be ready. however, given their recent performance, I realistically don't have my hopes high up.
DeepSeek is both cheaper and better than Mistral.
Because they distill
I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right?
Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.
Whether it's a 'problem' or not is viewpoint-dependent but it's against the OpenAI ToU:
> You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not:
> [...]
> * Use Output to develop models that compete with OpenAI.
Source: https://openai.com/policies/row-terms-of-use/
(I'm also curious whether they consider developing a competing model to be illegal, or harmful, or abusive...?)
> it's against the OpenAI ToU
Given that OpenAI doesn't care about training on copyrighted data, why is suddenly their ToU something anyone should care about?
That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
On a more risk-strategy level there is the size of their legal team, general endowment, and supplier and political connections to consider.
Everyone is free to ignore their ToU, but I can understand why a company would avoid it...
> If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
Yes that's what should be said to OpenAI. Now they should not cry about their T&Cs not being respected when they never cared about others' copyrights.
Feels like this should be some kind of anti-competitive violation even if it's not actually. Probably moot under this admin but still.
It's like saying you can't use windows to develop an OS, or drive a Ford on the way to your job at Hyundai.
it doesn't matter the reason. This is a race and nobody will care or remember how the winners got there.
Mistral looks like it's fading away to irrelevance unless they can play alongside the similar sized models, or have some unique advantage other than being in Europe, for Europe. I was really excited for them back when they were startup that had the biggest European venture round ever. This space will have a few winners, and many losers. Google, plus either Anthropic or OpenAI most likely. Big models will see breakthroughs in inference performance/cost fall precipitously and small models will only exist on devices (Pixels and iPhones, cars, watches, bluetooth speakers, etc)
It’s not that I don’t agree with you, I am just pointing out why it’s hard to catch up to scaling laws given the European economic (capital) and political (US would be upset if they found out Europeans distill) constraints. China is only bound by economic constraints.
With the insight in your comment and this bit from the above one:
> This is a race and nobody will care or remember how the winners got there.
It seems like the EU should have paid China for the distillation datasets, esp. since Mistral isn’t even a governmental org.
> This is a race and nobody will care or remember how the winners got there.
For consumer AI, yes. For coding assistants, probably.
For specific application "business" AI like the things Airbus announced the other day? Not at all. What matters for an Airbus using Mistral to build compliance documentation based on AI generated physics simulations is the enterprise relationship, reliability, compliance, forward deployed engineers helping with the fine tuning, quality, predictability, support. A Chinese lab having a better at benchmarks model that is cheaper is just irrelevant for that.
And IMO, the real money in AI is this type of "business AI" deployment. Developer tooling tends to converge on becoming commoditised. Once you're a core supplier for a big bank and embedded in their processes, you're there untill you screw up with the pricing (like Broadcom), and even then.
Why doesn't Mistral distill?
Good question, given that American companies basically threw copyright law into the trash, I think they should.
American companies can't sur Chinese ones, but they can do it with European ones.
So then the European ones should join with European copyright holders to sue OpenAI/Anthropic and watch them trying to BS their way around what they train on.
Well if they did wouldn't be able to feel superior to Americans about that particular thing. Perish the thought!
It’s really a pity, why can’t they feel superior while breaking ToS and copyrights just like Americans can feel superior over Deepseek while breaking ToS and copyrights?
I suppose losing with dignity is a consolation.
Also, new Medium 3.5 is far more expensive than previous Mistral models, and much more expensive than e.g. Deepseek
I tried it out on some dev tasks with their Mistral Vibe subscription, and the performance was pretty okay (okay, not great), both in regards to development and speed. Worse than Anthropic's models I'm used to but at 20 EUR per month it wasn't a bad deal - except that the 200k context size would more or less be a deal breaker in many cases.
Where do you sign up for that subscription?
I wanted to try out Mistral, but I fail to find anything like that even after creating an account
The other comment already mentioned that you get their subscription: https://mistral.ai/pricing/ they do say that you can try out their coding agent for free, but personally the Pro tier is pretty affordable too to try out for a month.
Then you can install their coding harness, I personally used the Python + uv option: https://mistral.ai/products/vibe/code/ if you don't have uv yet, you might have to install it too: https://docs.astral.sh/uv/ though I already use it for other projects. Oh and if on Windows, you probably want to do all of the installation inside of WSL, just so that file paths are the *nix variety, I've had issues otherwise with pretty much every coding harness, like OpenCode as well (across multiple models).
After that, you need an API key for your subscription, you can generate and copy it here: https://console.mistral.ai/codestral/cli that's also where you see the quota, though it seems to NOT refresh instantly, but more or less a few times a day.
Either way, happy coding!
Maybe on their pricing page?
https://mistral.ai/pricing/
Everything is more expensive than deepseek. They aren't frontier in intelligence but they are the frontier in cost per intelligence
> they've fallen into irrelevancy right now
It's a very charitable take, as Mistral has never really left the realm of irrelevancy.
It's only a matter of time before EU falls back to hosting Chinese models in EU datacenters.
Yeah. I run LLM models locally and for me 22B-32B is the largest I'm willing to invest in trying out.
Even though Mistral 4 has 6B active parameters per token (allowing 3-3.5 per token parameters to be loaded on a 4090), the ~240GB download + storage is pushing the limits of being able to try this out locally, especially if you are downloading and evaluating multiple models.
It also makes it harder for other people to make downstream finetunes like with what happened with the older Mistral/Magistral models.
I think machines like the DGX Spark are about to become a lot more common/popular. It’s big enough to run sparse 150-250B MoEs with enough throughout for a single user. Deepseek v4 Flash is #1 (in terms of usage) on OpenRouter because it’s good enough to be useful. You can run it on a Spark (though it runs better across 2, which is getting up there in cost)
I find Mistral Medium 3.5 with OpenCode is perfectly fine if you're willing to talk to it in a more fine-grained way about actual code. For me that's fine because even with huge frontier models I don't like trying to vibe prompt like a product manager.
I don't agree that they are falling behind. Using both chat and cli I get what I need and it's comparable to "sota" when I compare.
Mistral is entering the "let's extract has much money from EU taxpayers as we can" phase of European tech company which did not get bought by a US one.
They'll end like Dailymotion, just a zombie company.
Nobody trying to compete with Google, OpenAI, and Anthropic should be playing the small models / local models game.
Foundation model labs should be building very large reasoning models, then leaving it to the community to distill them down.
You can't scale a small model up, but you can scale a small model down.
I'm convinced the only way we'll have a seat at the table in the future and avoid total runaway takeoff is if there are very large models within 80% of the capabilities of the frontier models. Tiny RTX models do diddly squat to remain competitive.
Build open weights models for running on H200s. I'll spin them up on RunPod or Lambda.
I do think there's a chance open weight models have a bit of a moment with the costs of frontier models growing on business balance sheets. It's unfortunate from my "privacy loving" PoV that it's mostly Chinese models filling the gap. ( the top models on openrouter for instance ).
I have used Mistral models out of pure ideology for web agents and the like which aren't doing a lot of heavy lifting.
Antirez’s Deepseek 4 Flash implementation that can run on MacBooks also was a revelation. It runs decently on M5 Max 128GB and it’s pointing out other bottlenecks like prefill speed which will improve.
I thought distillation meant small models don't have to compete with the big models and can always eventually achieve close parity, but it's just a matter of time to do the distillation? (i.e. how much lag do you want to live with) Am I oversimplifying?
There is likely a theoretical limit to how much intelligence you can pack into a model of a given size (especially when stretching that over a large input context size).
Our evals are pretty complex so we only recently started testing ~30B class models, which are now becoming quite smart (on par with the frontier from 1 year ago). Mistral is far behind, but I'm rooting for them.
Data at https://gertlabs.com/rankings
> a decent proxy would be to build models that get the r/localLlama crowd excited
I don’t really disagree with your post, but this is not exactly right. That subreddit seems to go from hype train to hype train every week, I haven’t found anything really insightful in it for quite a while now.
We actually found the Mistral Small 4, quantized to 4bit was comparable to Qwen 3.6 27B and is roughly the same size. At least from our experience on our use cases, the quantization of the Mistral model worked far better than trying to quantize the Qwen family.
Fully agree to your point though, Mistral in general is far behind where I'd expect and Qwen in particular is crushing it at the smaller sizes.
Personally, I'd consider anything 20B params and above a "medium" model. Small being <20B and large >100B. I think obviously we can get to the huge 1-2T param models, but frankly the margin of accuracy improvement for the speed hit is kinda insane (1-2% for many metrics).
It's all relative. For local use I'd classify it by hardware (VRAM size) using FP8 or Q6 quantization:
1. tiny <2-3B -- easily runnable on lower-spec hardware
2. small 4-8B -- runnable on 8GB GPUs
3. medium 9-12B -- runnable on 12GB GPUs
4. large 13-24B -- runnable on 16GB (for the lower end models) and 24GB GPUs
5. very large 25-32GB -- runnable on 32GB GPUs
6. huge >32GB -- not easily runnable on consumer GPUs without compromising performance (offloading layers to the CPU/RAM), quality (heavy quantization, esp. at <= Q4), or price (investing in multi-GPU setups and/or server-grade hardware).
You could possibly split huge down further, as 70GB models (e.g. llama 3) are easier to get working than >120GB models and 1TB models are completely intractable.
As a Mac user:
1. tiny <2-3B -- could run in a browser even, mac neo
2. small 4-8B -- last of browser options, MacBook Air base
3. medium 9-24B -- 32GB machine, air or pro notebook or mini
4. large 25-48B -- 64GB, pro notebook or mini
5. x-large 49-100B -- 128GB MacBook Pro or Studio
6. Huge > 100B -- 256/512GB Mac Studio
> tiny <2-3B -- could run in a browser even, mac neo
Or a phone. I’m running Gemma 4 E2B in one of my apps on my 14 pro (which may or may not be killing my display through overheating. It might just be a coincidence).
Nawh, they trained on test since Llama 2, no wonder.
Mistral is bad bad. For its use cases I feel like India’s Sarvam is doing better.
channeling Rocky (extraterrestrial) there I see :)
I really want Europe to be part of the AI development and research. And I strongly cheered for Mistral. But they are accumulating too much technological delay. This needs to be fixed, otherwise it will turn into yet another proof we are not able to run large tech with good results. Basically any Chinese lab is doing much better. It's not Mistral that created I don't want to say DeepSeek, but MiMo 2.5, Minimax 2.7, and so forth. There are only weaker and/or larger and slower (no MoE) models. Not good.
https://en.wikipedia.org/wiki/Artificial_Intelligence_Act#Pe...
Europe shot itself in the dick with this hastily implemented at the height of mass hysteria bullshit and now no sane company will build anything there. an AI startup in the US or China can be a boy and his computer. in Europe, the boy needs a dozen lawyers.
Mistral's sinking into irrelevancy despite the head start they had, the very promising early models they released, and the funding they receive, might very well be the consequence of trying to comply with all that crap.
You don't compete with anthrophopic from the basement. For that you need either a shit loads of money, or a government which are not afraid of getting very very involved.
There is a lot of Europeans working on AI, it's just that a lot of them work for American companies. Because of money.
I think both of you are correct.
Possibly yes but let me remember you that France, Italy Germany were against the AI act, so here something very odd is happening, that the EU funding nations are getting marginalized by the countries they welcomed on key topics for our future, and I believe corruption could be a big part of what is happening, both internal to those three countries and at an even more alarming rate in other countries.
EU big nations getting marginalized: haha. The only reason there’s no US-like tariff on Chinese cars is because Germany was too scared it would lose its access to Chinese market.
> the EU funding nations are getting marginalized by the countries they welcomed
Thank you for reminding us that all animals are equal, but some are more equal
It's not a matter of importance, but that bad actors as they tried to do in Italy, corrupting EU parliament, may be doing the same with counties are have less visibility. A weak EU is not also in the best interest of countries that wanted the AI act, and surely not in the interest of their citizens, but there could be pressures.
I understand your point, I just object to the language and dividing the EU into more and less important blocks. If a voting mechanism is broken, that's where the issue is.
Who put a nepo-baby lawyer in charge of the big €95bn AI fund? EU bureaucrats living the 6-figure high life with chauffeurs and private jets in a bubble completely isolated from reality.
I hate the fake European foreign-backed right-wing parties but they didn't cause the current situation.
But I'm afraid it might be too late as the cancer spread and did too much damage. Insane regulations, no energy, looming demographic/pension crisis, tax hell, and collapsing industries.
Way more important than this act are the police raids. Someone used your SaaS to send phishing (see today's front page HN)? They'll just take all your servers away. Goodbye business. Unless they think the general public would riot, so established companies are okay. You can't build a castle on a foundation of quicksand.
Well , there isn’t also the opposite take from TechCrunch where they say: Why Paris may be the most important AI city outside Silicon Valley. [0]
While the EU loves its regulation, I still feel it’s too early to write it down in the AI race. It will not replace Anthropic or OpenAI any time soon, but even Google and Meta fail to do that.
If AI continue to grow and expand, there is enough space for many more unicorns.
[0] https://techcrunch.com/2026/05/28/why-paris-may-be-the-most-...
As someone who has actually experienced the hiring market in Paris, I have a hard time believing this. The salaries are, unfortunately, pathetic.
Did you read even a summary of the AI Act?
The gist of it is very simple - depending on the risk of what you're doing with AI, you have to document why it did what it did, and be able to explain it; or you can't use it at all. So if you're using AI for mass surveillance, you can't; if you're using it for treating loan applications you need to be able to explain why it approved/denied; if it's a customer service chatbot, do whatever, nobody cares.
Not only is burden of the legislation fairly low (and a lot of it hasn't come into force yet), it is extremely reasonable. No, sorry, we don't want a UnitedHealthcare using a broken algorithm on purpose to deny as much care as possible and hiding behind computer says no.
It's yet another time when EU is killing our own possibilities to build real competition to US or Chinese tech.
And yet another time they will be thinking aloud in few year "what happened that we are fully dependent on USA?"
So you're saying AI models should be allowed to freely "manipulate human behavior"?
That is almost a meaningless sentence. Cats and traffic lights both manipulate human behavior.
You know exactly what the intent of the lawmakers was.
No, I definitely dont. And neither do they, its hundreds of law makers. Meaning is usually sussed out through case law.
The problem is that statement is a bit too open to interpretation. Ever had Claude piss you off by being stupid and talking in circles? Sounds like manipulation of human behavior!
When it comes to MoE, to me, I remember Mixtral model that showed the viability of MoE for the first time. I was impressed by their technical report. To be clear, MoE idea was already out there, if I am not mistaken. If they have pushed Mixtral model family further, who knows they might have achieved the reputation of what the current Qwen family has. A missed opportunity.
> But they are accumulating too much technological delay.
How so? Catching up is easier and cheaper than spearheading the lead.
Compared to the UK Government which recently announced 10 million GBP for AI research, which will likely be scooped up by consultants. I think Europe is doing fine considering.
The first step would be indeed to join forces with UK, in order to don't be two entities, which is very unnatural to me.
That Brexit ship sailed. It’s very difficult to do anything with the UK currently.
No, we don’t need US’s Trojan horse in the EU
Interesting. Could you elaborate. As a pro Europe Brit I'm interested to understand this viewpoint. Is it a widely held perspective do you know?
I think that while y'all were appreciated members and definitely had a lot to offer, you also had a lot of annoying carve-outs and kept stalling needed measures to federalize and strengthen the EU more so we can be a proper superpower in our own right.
Maybe it's good you left for now, maybe we can finally get these things done. And once that's accomplished and enough of the gammon has died off, you can always rejoin :-)
Jumping in and most people in Germany wouldnt see UK as an American trojan hourse. I dont think anti American countries like France and Danemark have a problem with UK being in the EU per se.
I can see most people want that UK wouldnt just get special treatment any more.
> BNP Paribas runs Mistral models on-prem for KYC in Belgium, with sensitive data staying within the bank's walls. Abanca is using agent orchestration to handle sensitive customer information at a huge scale (2 million customers in their app). For European companies in regulated industries, this is a good alternative to relying on US hyperscalers.
Mistral leaning into on-prem and European-hosted models is very smart.
Respectfully, I don't think it's "very" smart. It is a fair option given their limited options? Everyone is doing FDE or (customer engineering to be more transparent) because otherwise they will just be seen as markup on token cost. And the Neo-SaaS companies will take the money instead.
Who else will buy their AI?
and what other options do they have?
You don’t think it’s smart to get reliable funding this way? From Banking the Cash Cow?
Devstral is getting better, it’s the Vibe harness that’s holding it back (I think). I can see how that would drive some business as well.
Their chat thingie isn’t very well positioned, but gets results. Could be an euro or two per month, maybe bundled with some more features. It’s not like Mistral has no options, if anything they’re just a bit complacent and not ambitious with their plans.
Also Mistral did just the right thing by acquiring Koyeb, to beef up their deployment at scale expertise.
My take is that Mistral is not focusing on generating contents such as code, images, or videos. They focus on multi-lingual models, OCR, voice, and others I believe. Their model intro page manifests that although it always confuses me because it's too colorful and there are too many categories, not to mention model names. I hope their decisions will pay off.
From what I understood they are retiring a lot of the specialised models in favour of the main family.
Isn’t that the usual EU startup playbook once they give up on the B2C or world-scale SaaS markets? Refocusing on large (European) enterprise B2B and government contracts?
It always felt to me this (enterprise B2B) was where European startups went to die.
Yeah but why use mistral on premises instead of Qwen?
We're talking about enterprise customers. The trivial answer is Mistral has sales teams and consultants from the same company that builds the models and from the EU.
i can invest in public markets in a lot of $10b sales and consultants businesses, who can also put mistral on premises (or do whatever the hell people ask for), it makes mistral sound like it is yet another one of those, not a growing $1T business.
One reason might be that Mistral doesn't have a risk of weird training biases that were required by the Chinese government.
>weird training biases that were required by the Chinese government
What is "weird training biases" to us might not be weird to them and vice versa. Just ask the Chinese what they think about LGBTQ+, Japanese, pride parades, Islam and colored minorities.
Every nation has its own biases injected in its domestic LLMs at this point. Otherwise they risk getting in trouble for hate speech/disinformation in the jurisdiction where they operate.
Same how Google Maps cleverly biases the lines of disputed borders based on where you are viewing it from. Or how Google maps switched 'Gulf of Mexico' to 'Gulf of America' in an instant when the orange man signed the paper. Google won't want to anger the US administration the same way how Mistral won't want to anger France and the EU, so Mistral will have all the EU prime directives injected into its LLMs no matter if they're ludicrous or not. The law is the law whether you agree with it or not. Companies want to survive and will pander to whatever the whims the regime they live under are at the current moment regardless of what is right or wrong.
But if I'm using a LLM for personal projects or generating a photorealistic choreographed fight between Tom Cruise and Brad Pitt, I don't care what its political biases are, I care if it solves my problem better and cheaper than the competition, and here the Chinese models could end up winning the consumer market, which is why you see Mistral and other EU alternatives focusing exclusive on B-2-B corporate market.
> What is "weird training biases" to us might not be weird to them and vice versa.
I agree. That's why I think European companies might prefer a European model.
Except there's no such thing as the "European model" similar how Europe is not a country.
Mistral is mostly French and tends to have mostly French speaking customers, like BNP PAribas in Belgium. Germany will want its own domestic AI champions, maybe in partnership with Switzerland and Austria, similar to how Denmark already has invested in LLMs focused on the Nordic languages with money from Norway.
The biggest mistake is treating Europe like a single homogenous country/market.
Mistral just acquired Emmi AI, an Austrian startup.
German and French speaking together at last.
https://news.ycombinator.com/item?id=48197995
Was EMI specialized in the German language LLMs? Or is it that they're an Austrian lab?
Emmi is an Austrian lab specialising in physics AI applications.
Mistral isn’t specialising in French language LLMs either.
The point was that across different European countries and languages there are collaborations and M&A happening.
The original question was "Yeah but why use mistral on premises instead of Qwen?". I think you and I agree on the answer.
I for one would love to see more country-specific models. There was a story here the other day about Norway’s National Library developing a LLM specialized in Norwegian: https://news.ycombinator.com/item?id=48270770
>Similar to how Denmark already has invested in LLMs focused on the Nordic languages with money from Norway.
Would love to know more. Do you have a source on this?
Because the lab working on Mistral is in the European Union.
Please don't run Chinese models for KYC operations.
Based on what? Is there any evidence of risk at all?
The issue is that you wouldn't be able to even transparently get to any evidence, as these models are blackboxes.
They might start scheming behind employees backs as soon as they realize they are being used in critical infrastructure of adversaries. And nobody would know until it's too late.
Aren't all LLMs just as blackboxey?
If you sell a blackbox that you constructed yourself, then you are also liable for anything that happens.
If you sell a blackbox from a third-party (e.g. from China), you are liable for somebody else's decisions that you cannot scrutinize.
So, that's kind of the argumentation that underlies sovereignty and why Chinese Models are not being used in critical infrastructure.
are you born yesterday?
It may be very smart for them, but it also shows that the EU has no desire, therefore no chance, to change and lead anything. The only thing it has is regulation.
The EU has just poured unfathomable amounts of money into continent wide infrastructure (https://reforms-investments.ec.europa.eu/recovery-and-resili...) - due to COVID, the military - due to Russia, etc. They can't do everything.
Lets hope the models can do a better KYC than the humans have been doing..because they are well known.
Or is this a case of the humans, now preparing for the excuse it was the AI failure?
"BNP Paribas Sentenced for Conspiring to Violate the Trading with the Enemy Act" - https://www.justice.gov/archives/opa/pr/bnp-paribas-sentence...
"BNP Paribas caught up in French money laundering investigation" - https://www.reuters.com/business/finance/bnp-paribas-caught-...
"BNP Paribas faces $246m fine in currency scandal" - https://www.bbc.com/news/business-40635070
"BNP Paribas caught in a Cypriot money laundering investigation" - https://www.lemonde.fr/en/les-decodeurs/article/2023/12/26/b...
In Money Laundering their track record is unmatched: https://violationtracker.goodjobsfirst.org/parent/bnp-pariba...
When the humans have a track record of corruption, it might make sense for a company to seek parallel opinions from a LLM so they can at least flag suspicious human decisions.
Assuming BNP Paribas leadership wants to stop the corruption of course.
They had years to fix it: https://violationtracker.goodjobsfirst.org/parent/bnp-pariba...
That's just one side of the story, not following it on details, but their own le chat explained to me that the company was a capitalist succubus starving to build data center in some north European country. Hilarious if you ask me.
I have been on a lecture from great government IT person, they are evaluating models and are very unhappy about the situation, because they’d love to use Mistral, in some cases it’s the only EU based model they can use … and they know it’s really bad and falling more behind.
It is well possible that Mistral can make a profitable business by being bad, but still the only possible model for EU uses. Sad story, sad to witness.
Regardless of the business. Their website design is :chefs-kiss https://mistral.ai/
What specifically is good about it? I scrolled it on my phone and it seems pretty standard corporate website?
I love everything about Mistral's branding.
Le Chat was great, the rebrand to Vibe is meh
It looks very crowded and the paragraphs are off
I was at the event, and was impressed by the attendance, all the leaders from the major european listed companies were there.
Also interesting to note the number of partners they invited. Going from Microsoft, Accenture and EY to startups like alpic.ai or lingo.dev . Seems like they are ramping up their M&A game too
Excited to see more about their partnership they with Alexa+. In agentic and tool-calling, Mistral’s model architecture excels at the exact structured JSON output Alexa needs to trigger APIs and smart home routines without breaking.
As an European: 100x YES!
I really like the direction and the transparency of Mistral, among those players.
Even as a non European, it's great to see some competition from Europe against the US/Chinese models.
Sounds like they don’t have a moat at all. It’s like software consultancy with a data centre. And then the article mentions many customers using these models on prem (so data centre is not really a plus).
What’s stopping any country backed startup from fine-tuning small open source models?
Maybe because distilling small models from bigger ones that you control gives you better small models than fine-tuning from bigger models you don't control?
(I am not claiming it is the case, but stating this as an assumption)
I just got an email from them saying that they’re retiring some (most?) of the dedicated models like devstral gradually through August and one should now use the general model. Cost grows exponentially
Devstral 2 (devstral-2512 and devstral-latest) → We recommend transitioning to Mistral Medium 3.5 (mistral-medium-3-5 with reasoning_effort set to "high"), a stronger model, priced $1.5/$7.5 per million input/output tokens (change from the previous $0.4/$2).
I received the same email, although couldn’t quite figure out which retiring model I was still using, as I thought I’d already transitioned to Mistral-Medium-3.5 for everything. Anyway, after receiving the email, my hope was that it meant they were also planning on releasing some new, improved models in the next months.
> Abanca is using agent orchestration to handle sensitive customer information at a huge scale (2 million customers in their app).
Maybe my perspective is skewed on what "huge scale" means, but 2 million users? That's like a few hundred megabytes of data? Or a couple GBs if there's a lot of per-user data?
Maybe, but using state-of-the-art large language models to solve customer support queries with agentic can quickly use a lot of tokens. What I understood from the talk is that they used agents with limited responsibility and (assumption from me) smaller models, to the make sure the answers were quick, reliable and not too costly.
There are several payments processing companies that are already largely using AI for customer support queries. They still have an escape hatch to a human but at least one of those companies (on the smaller side) is reporting a ~99% success rate, they are down to a handful of human customer service employees now for cases where the customer can't find/produce the transaction ID.
European consumer focused businesses do not scale easily the same way US ones do, which is a major contributor to their problems developing tech businesses generally.
OTOH such things can be quite defensible, they just rarely become anything like as profitable.
Wasn't even aware Mistral was around and I think that just shows you how irrelevant it has become and not a very good sign for EU in general when the best talent are working for American AI companies.
I saw Tibo's tweet a while back and it was basically a legitimate complaint about the extreme taxation he faced back in EU (France I think) and its pretty obvious how much of a hinderance top down centralized regulation is to innovation.
While I welcome competition and independence, nobody can argue with American innovation and its ability to attract the best of the best. Once it takes seat of the AI reigns there is very little chance for other countries to compete, very much similar to semiconductor field and how only a few select countries have the talent and monopoly over its particular supply chain.
It's clear to anyone looking in that whatever EU is doing is not working (not just AI) and will not work as they do not seem flexible or humble enough to steer itself.
Big tech has remote offices in every major European economy, and they pay well above top 90th percentile of market rate. It basically has a talent sucking effect on the entire economy.
Oh most prominent eu ai company . Without reading an article predict next, will update after :
1. They give up on building competitive models. It’s time to drink wine not to struggle with competition
2. Because of #1 they will talk a bit about something around llms maybe coding agents , and after start talking about sovereignty.
Unlike you, who drank the wine before writing the comment.
3. They are going to start focusing on B2B implementation and deployment.
See what happened to Aleph Alpha...
Really hope there is going to be some competition from Europe in the AI Space.
I believe that Mistral team is doing the best they can do. I like the directions they push; open models for various tasks, on-prem has a lot of potential. Sure, I use Claude code mostly for coding. But there are so many tasks other than just coding. Even for coding, eventually, I am certain they will catch up and Vibe becomes tolerable soon.
I've said it before that Mistral is underrated. They are looking at real world use of LLMs and tooling. Bespoke models are very appealing to lots of non-tech centered companies and state agencies. Also, Mistral's actual platform is useful. While others are watching performance leaderboards like this is some eSports stream, they are building real world uses.
I was also at the event and was pretty disappointed. Most of the talks were pretty low on information. I was at the “build” stage, which supposedly was the technical stage, but the talks there didn’t really go into technical specifics.
The papyrus talk was awesome though.
We should be supporting and using local models that allow you to run whatever model you want.
Not to be confused with the fantastic AI Now institute, run by Meredith Whittaker of Signal among others.
https://ainowinstitute.org/
Almost feels like name squatting
Does anyone else always read "Mistrial" instead of "Mistral"? Always think I'm about to read a juicy gossip piece, and let down when it's just a standard update on an AI company.
edit A lot of AI company names are really strange, actually. "Claude" is really the best a trillion+ dollar company could come up with? It sounds like the name of a grandpa or something.
https://en.wikipedia.org/wiki/Mistral_(wind)