The way Anthropic continues to get away with testing these nerfing "experiments" on customers without their consent depriving them of the model quality they've paid for is wrong.
The Windows Insider program has shown plenty of people will willingly sign up to be testers for a taste of the rawhide. Anthropic could do this in a transparent way but they continue to choose not too.
They can get away with it because they are so far ahead of OpenAI models for coding, it's not like you can switch to anything else if you don't like it. Astra is even worse than nerfed 5.5.
Pretty much the only alternative is loading up some credits and using Fable.
Every time I read something like this that doesn't match my experience I'm bewildered until I remember that providers play quality games and that everybody's workload and harness is different. For me GPT 6.1 Sol is tremendous value and excellent at low-level work like DSP and 3D game engines whereas Opus is more expensive and not as capable. I also don't get along with DeepSeek at all even though everybody on HN seems to think it's the second coming.
A neutral third party who runs a matrix of closed-source coding benchmarks across multiple harnesses would be helpful but that'd take a lot of time and money...
Astra when it hit the ground felt like “A Fable monent”: it was so good and concise and just was done with whatever task in a short time, where opus and fable sometimes can take forever. I think it lasted a week or two until it started to deteriorate to where we are now. Coincided nicely with the stopping of codex20 subscriptions etc.
IPO is coming so they will have to maximize profits going forward which will mean constraining the compute they give everyone to ensure the subscriptions make money.
Realistically it can't match Fable, let alone Opus today. It can however beat basically all of the models from 1 year ago. If last years state of the art is "good enough" for you today then local is great.
100% agree. A local model might not have the same stretch goal capabilities, but at least I don't have to constantly second guess myself in terms of the inconsistencies I think I'm noticing.
No longer am I constantly made to feel like Ingrid Bergman in that movie with all the lights.
I noticed Fable decline in quality a couple weeks ago. And I've noticed Opus drop over the last week, even after bumping to a default of Extra High reasoning for almost everything.
Tonight has been maddening with the kinds of absurd mistakes it's repeatedly making. It's so frustrating.
I'm the kind of person that wants something to be consistent and reliable. And Anthropic has been great, except for in this way. Which is enough of a recurring frustration (about every couple months this year) that I genuinely can't wait for other options to catch up. Then I'll be leaving as soon as I can.
I've been using Fable for a couple months as a relatively independent, long-horizon task executor and it worked well. About two weeks ago it completely stopped working. Like ~85% success rate down to ~10% in my workflows.
Totally possible that I'm just hallucinating/projecting/etc or my workflows somehow changed in terms of difficulty, but it's been kind of shocking.
I have the same exact experience. Working on one project, brainstorming. It was brilliant in July and August, like a seriously smarter, more experienced version of myself, reading in my mind, one step ahead of me, when pointed out a mistake it would remedy on the spot with the exact brilliant solution.
Now it's like talking to my hungover self, after 3 days bender, who just wanted to be left alone.
Same for me, especially the last couple of days. It was really frustrating and it felt like Claude was sick of working with me and just didn’t give a f*ck anymore
I guess there are experimenting with they can enshittify. But even if they undo this current change, all commercials model will eventually be enshittified.
I read somewhere that the test might switch models mid conversation which stops the other from reading the previous model's reasoning. I don't know if that's true, but it sure feels like that quite often in the last days.
I'm a pretty voracious 20x user, and OAI Biz Pro user; however I'm not an engineer by training, a previous tech adjacent dev team leader who always relied on pseudo code and broadening my horizons when dealing with my team. ( AdTech, DTC marketing )
So I'm AI pilled and in another career, and started to think, what if the problem is the inputs? They release Opus X.X and at first it's a rush of their best users, best use cases, upgraded lanes. As weeks go on, adoption increases, cruft uses start to poison the model? Is this how the ongoing evolution works? I'd love to get Sonnet 5.5 and Opus for review in a place where I could reliably daily drive for X months / Y years etc.
I noticed the same, and posted about it three days ago.
Opus is suddenly fumbling basic tool calls and failing to finish tasks even at high, where I had been permanently using it on low effort with success since it came out. It is very frustrating.
My guess is they quantized it and/or reallocated DC capacity to train a new model, this has happened before.
Not just frustrating, it can also be dangerous. What if you trust Claude with certain security-sensitive things (because experience), and suddenly you cannot trust it anymore like that?
It really though?? Even if you set temperature to 0 I saw research showing that the scheduling of the order of operations in the GPU firmware itself can change trailing decimals places and affect the output token
I'm aware operations can technically finish at different rates throughout all of a GPU's cores, but I am not exactly sure that GPU firmware would change the outcome of a calculation (unless there's something specifically in the firmware for it)
Right lol, i didn't know we'd built a machine that was capable of getting critical level security deterministically correct. That's not me, at least...
Multiple instances of failing a Bash tool, calling bash (lowercase) and getting rejected and even reprimanded by the API "No such tool, call Bash instead".
Killing it's own shell, multiple, multiple times in the same session. I'm sure I saw a phrase "I did it again".
It's so shit that it must be damage control. The alternative is to limit usage and that'd affect the IPO even more.
I noticed its been trying to drag out the conversation on every topic. Maxxing token consumption for the dollars instead of staying on point. Makes sense for them but its frustrating for me as a user. I suspect this is going to be the AI version of enshittification. Now its only missing some badly needed in-stream ads.
It's very frustrating, even at max effort. It used to do things correctly on its own, now I need to keep reminding it to do obvious things like "Make sure you are not dropping steps as you rewrite the docs". It became unreliable, like how Opus 5 was.
It blows my mind how people still build their workflows around depending on services like these, with how easy it is to just self host and forget about all of this. You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription. You'll never have to worry about token limits, the model changing out from under you, getting banned, etc.
This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.
Speed can mean two things: rate of token generation or time to finish a task. What maters it the later and a model that takes a faster path to the result and uses fewer tokens because of that comes out ahead in practice.
Superior? Sure. But do people need the capabilities beyond what can be reasonably achieved with local models? I think the people that actually sit in that gap are vanishingly small, and if your title contains "Engineer" or "Developer" or something like that, you are not one of those people (at least for doing your job)
As for faster, maybe? It depends on the metrics and what you're doing with it. If you mean TPS, no, I get much higher TPS and even faster TTFT on my local setup than frontier models. If you mean time to complete tasks... maybe. If you're throwing whole projects at it to vibecode, yeah, a frontier model will dump slop so fast it'll make your head spin. If you're doing actual methodical development, like doing spec-driven development and actually reading the specs before spinning off agents to implement them, rather than just having another agent review it and yolo, you'll find that frontier models don't really push the needle that much, and you'll be making high-quality projects.
Qwen 3.8 (and maybe soon 4) series models are probably the best right now for local AI. Specifically, for lower end hardware like 8gb GPUs you might find a lot of use in a Qwen 3.8 variant like Underdog Saluki 27B.
In a workplace environment I don't think "cheaper" is necessarily the thing to optimize for. Subscription models save you a lot of time (and money) by getting things right faster, solving bigger problems, etc. A cheaper model just consumes more human time.
But with a local model with essentially free tokens, I can set it to work on tasks in the background without having to worry about any usage limits. Deferring simple bug fix tickets or other low complexity to an essentially unattended local model works just fine. And if the result occasionally fails review, it still didn't burn paid usage credits.
A local model tops 60 tokens/s max. Running it on the background means not being able to use on the task you are working on. When working on local hardware you cannot do concurrent requests. In deepseek for example you can have 100 agents running each getting at 120 tokens/s easily.
In local models you are limited by your own infrastructure.
I do concurrent requests just fine and they have no noticeable performance impact. Not all local hardware is equal. Also, humans have to sleep. Agents don't, so processing stuff over night and reviewing in the morning just gets more utilization out of what is already there.
With cloud agents I also process stuff overnight. But my gpu is limited. If I want to OCR a high number of documents or vectorize data, I can't run an LLM at the same time. I use remote models and use my compute for running commands on my desktop machine.
Today this is absolutely true. Local models have gotten very good, but they aren't there yet. The cost of a Max subscription or even API pricing is overwhelmingly worth it for most users, even if you ignore the enormous capital costs of running the competitive models locally.
But the in a million years bit, even taken as rhetorical bombast, is of course silly. The difference between local models and the frontier models is ever shrinking. Extrapolating out and eventually, sooner rather than later, the difference is going to be a wash.
Unless all of the vendors stop progressing open local models, the future is clearly there. It's why I think AI is important and valuable, but also the current leaders have a dubious financial path forward.
i dont understand how people keep comparing local models with frontier models, they dont belong in the same discussion especially for coding usecases. they solve different problems under different constraints.
I don't understand what do you mean. They both solve the same problem. Some models solve it better, some worse. And in some cases you can only use local models, so they solve more problems.
"Frontier" (closed) models are better by definition, but if frontier models were good enough for you half a year ago, open models are good enough now.
Compared to Opus local models Opus 5.5 is far superior, also for something like a dgx spark at 5k that would be 25 months ignoring power costs if you chose the $200 plan.
And that’s looking at a large model, to host the equivalent sized model as Opus 5.5 you’d need more than a single dgx spark. A fairer comparison would be to Open models hosted remotely or comparable weak models like Luna which you would get away with a $100 plan or less. That also doesn’t account for token prices continuing to go down for the same level of intelligence.
Qwen 3.8 Flash Next is 125B with 6B active, which with quantizing and system memory offloading (strata) runs nicely on 32 GB cards (I have actually run it on as low as a 16 GB GPU and 16 GB of system ram, but I don't recommend it for serious work). I was running it on a single r9700, but now I'm using 2 r9700s because I'm splitting my server with a friend, and they bought one as well.
On the single r9700 I was able to run q2 with 4 slots of 256k, and I think it was about 40 - 50 GB of system memory offloading? Got about 60 - 80 TPS per slot depending on concurrent use. With the two r9700s we are running q3 with 4 slots of 256k, using about 100 GB of system memory offloading, with mtp enabled. Getting about 70 - 100 TPS now.
What is the alternative? Local models? I do not think so. Maybe Kimi K3 (still not local) as this model can be served by multiple providers? However Kimi K3 sits between Sonnet/ Sol and Astra/Opus so still not the same level.
i haven’t heard anyone say they downgraded their car to the children’s toy power wheels and regretted it. all the same, i’m confident that it’s a bad idea
I can buy a GPU that can run Qwen 3.8 Flash Next for 9 months worth of the highest tier Claude Code subscription, but then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
> then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
By choice, maybe. New models release faster than every nine months. And, evidently, you won't be running nine months of some fixed meaning of "Opus 5.5". What that means changes over time, out of your control, and without ever even being officially told.
I'm surprised why folks are either all in on frontier subs or self hosting on their own hardware.
To me it seems there should be a very viable middle ground of paying for open model hosting by a trusted provider - unless I'm only hearing from 24/7 vibe coders
If you paid for X but are now receiving Y, you should be entitled to get your money back.
Every time I consider going back to Claude, I remember that mysterious week in February when Claude was failing to do even the simplest things. Things were so bad that BigPickle was correcting errors Sonnet was making.
And let's not forget the king of A/B testing here. Google has no issues breaking products for months just to test some ridiculous idea.These people should be publicly fired.
> Every time I consider going back to Claude, I remember that mysterious week in February when Claude was failing to do even the simplest things
100% agree. I don't care how many reddit bots i see praising Claude, I bought max once wanting to see what the fuss was about, and the next day they broke the product and it was really embittering
> I noticed this too as did many others. Had to bump my default reasoning but it’s not the same. Seems to often happen before new model drops
I heard that Qwen4 was dropping some time soon, I wonder if Anthropic is trying to announce a new model immediately after Qwen4 - both for their IPO, and to kill the competition.
This has happened to consistently across OpenAI and Anthropic models that I can't believe people are still denying it. Obviously they're silently quantising their models or making other efficiency changes. If they can save 40% RAM and only hurt intelligence by a few percent in their benchmarks, they're going to do that. Unfortunately their benchmarks are obviously not representative of the real world, and these downgrades are very noticeable for people who rely on these models for work.
Of all the reasons to look at local inference, this is the one which pushes me in that direction the most. I want to rely on a model for consistency in my workflows. If I wake up on Monday and suddenly it's refactoring things it built on Friday, that's a huge problem. That makes it unreliable.
Probably because those "nerfs" do not show on sites that track model performance over time. Sometimes Anthropic/OpenAI mess stuff up in their harness, but usually not in the models themselves.
I guess the question really is: is this malice (trying to trick people into thinking new model is better by nerfing old one), or is this capacity constraints, where they obviously need to make room in their infra for inference of the new model by shoving the other models into a smaller box?
People are denying it because it's simply not true and for sure has not happened consistently.
In my own work I have not once noticed a degradation where I could have said with any certainty that it's not within usual deviation or me having a bad day.
Conspicuously the model seems to be especially dumb and annoying when I did not get a lot of sleep the previous night.
When OpenAI and Anthropic engineers post here on HN that they do not quantize models and silently replace checkpoints in the API, I believe them.
I saw nothing that convinced me the people who are constantly making these complaints don't just mistake any suboptimal result for changes in the model.
I've "called" the last 3 out of 4 new model releases. Maybe my account it perma-tagged as one that gets downgraded to re-allocate resources before a release but about 5-7 days before a new model Claude suddenly gets dumber and dumber on executing canned skills that I've used for 6+ months with no issues. The cynic in me thinks they are dumbing down models right before a release so the new model seems much smarter.
It’s also true that people that perform their own evals see wide variance uncorrelated with “nerfing”. Human cognitive bias makes them only see that variance at specific times and then they start projecting.
Everyone really should keep a few tasks around that are indicative of the work they do and run them a few times with the same model and a few times across model families and when new models drop.
It’s extremely informative to see with your own eyes the variance that always exists. Even if you can’t afford statistical relevance it’s valuable.
Yes I have similar anecdotal experiences - it gets dumb before a new model. What I imagine in my head is Anthropic using all their GPUs to polish off the next model and they need to borrow GPU cycles so they cut down the thinking.
Nope, the issue was not the 5 h limit, but that the 5 hour limit was gone in some minutes and that this burned trough 50% of my weeks fable limit. Before I could do roughly one full 5h session (fable till max) per day.
They're actually extremely predictable. What you are experiencing is the provider you are entrusting with your time, effort, and money fucking you because they need the capacity to train the next one. So without any real notice or indication other than things vaguely seeming broken in ways everyone seems to be experiencing differently, they're serving up some shitty quant of what you already paid to use.
Extremely predictable is an exaggeration. I was teaching machine learning and we needed half a dozen lines to make even simple networks deterministic. LLMs are not really predictable, even if we want to, they will sometimes behave almost chaotic depending on the input.
I'm not sure how you're arriving at "if you use the same seed with the same version of the software on the same hardware with the same drivers you get the same result is an exaggeration", it's wild that you're a teacher and you seemingly do not know this.
Some teams I know started moving all linting and validation into LLMs and markdown files, the reasoning was that they asked agent to review changes that were later found to be buggy and agents flagged these bugs. And that’s great but my immediate worry was that: it finds it now, but will you check every new model for all these bugs? And what if you need to switch providers? Or your provider silently tweaks the model to improve their margins? The answer was “well also build deterministic linters for all these problems”, but you know that’s just wishful thinking - if you can get 99.9% done with just a little bit of fuzzy text and the next 0.1% is a lot of work it won’t happen. So you have model behaviour lock-in that will bite you suddenly when you least expect it.
Or maybe LLMs will just keep getting better and cheaper? It’s all a casino right now.
Every LLM company must keep getting better and cheaper to compete. If you game out them running low on money, none of the outcomes actually result in you spending more or getting worse models, just switching to another one.
If you have 99.9% already - that's better than any pre-LLM deterministic linter.
Definitely noticed some anecdotal declines this past week or so. Missing steps it didn't previously, needing reminders of tool calls and skills that it should auto apply.
Lots of "this is a potential issue, I didn't check into it though" which is super frustrating to see. The model just points out major flaws in logic or barriers but goes no further than a comment on it.
I've got a Qwen coder model and a flash next model locally now, so it's not the end of the world. But it does need to be remedied - are we seeing them prep for a new model deploy when this happens? Or are they experimenting with leaner hardware to drive down costs ahead of IPO?
I’ve been experiencing unreliable performance for a while and can’t imagine this is solely due to new model training.
It does seem like Anthropic is experimenting with how much less they can offer users. I also get the feeling they are judging what resources a task deserves and dumbing things down accordingly. I’m seeing lazy responses and skipped steps more frequently for non-technical queries.
It’s definitely possible the average user hasn’t noticed, unfortunately.
For me I've noticed the pattern of decline happening on Tuesday and Wednesdays. I find myself having to have Opus redo many steps and making a much larger number of errors on those days. I am able to be more productive on Thursdays just as my weekly reset approaches on Friday at 1 pm local time.
I don't have measured data aside from myself noticing my frustration level with Claude jumping on these days frequently over the past several months.
Definitely a significant change. I noticed it in just daily work. Quality of work down, mistakes up, repetitive errors in scripts - rare earlier, now a few every session.
You can still control it if you can trivially spin up a node with eight B200s and 2TB of memory to run your Deepseek v4 instance. Nobody on a subsidized plans is doing that. Local models that fit 128GB memory don't come close even to a "nerfed" frontier model.
Actually Qwen3.8-flash-next comes very close to a "nerfed frontier model" the way I host it locally at 4.75bpw with int8 kv. It is pretty much equivalent to opus 4.6 up to 4.8 on coding. What it lacks is some factual knowledge - this is to be expected, it's a far smaller model. But if you run the kind of coding tasks where you have to spec them in high detail anyway (large very custom environment) this model is very usable.
I run it on a 6*rtx3090 consumer PC (no threadripper etc sadly) with very constrained pcie topology (4 cards sit on 2 lanes basically) and i still get 7k prefill and 50t/s single user or 200t/s 7 users. All at full 262k contexts of course and with 96GB of ram kv cache that means I can easily have 10+ Claude code sessions open with 1-7 generating at any time.
I've been the person that said, before opus 4.6 came out that this AI stuff is absolute crap for programming. Then it suddenly wasn't and I was using it many hours each day.
Local AI was the same. It was all crap. Models like GLM5.3 Flash that are many times larger than that qwen, lots of people are oooing and ahhing over I found way too inferior. Previous DeepSeek V4.1 Flash 0731 was atrocious. They benchmaxed it to hell, but at least it was small.
The current model of same name but release 1017 and double the size is currently the best open source model IMO. It even beats my self hosted Qwen3.8-flash-next.
I've tried hosting that DeepSeek model, but even adding another rtx3090 for a total of 7 cards and my 192gb of ram bought when it cost under $1000 I get only about 25t/s decode and few hundred prefill. That is far too slow for me to use.
But, I believe once this pyramid scheme of circular financing finally bursts there will be a glut of hardware on the market where people like me will be able to buy equipment to run 750B models like we now run 120B.
I've been very impressed by it and recommended it to everyone. 32gb ram, 12gb 3080ti, and I run 3.8 flash next coder (half the experts ripped out, leaving in coding) int8 kv 128k context at 35-55tk/s and I agree with the opus comparison; I personally say 4.7 is what it seems like the most.
I moved from Claude to Codex and am happier. The prose is less grating and results comparable. As much as I don't like Altman I am tired of Anthropic's antics.
I tried this and while the prose is indeed snappier I find the code quality of Codex truly horrific. It's dense and uninterpretable. I almost wonder if they have made it purposefully opaque so that it's harder to resume work without Codex once you're using it agentically. Claude is still much better at writing quality code that is fit for human consumption.
I don't think Astra's code is quite as bad as you say, but yeah while it's much better at non-annoying prose, the code definitely isn't quite as good as Opus 5.5.
We use Azure Foundry hosted Opus and it seems as capable as ever.
I run a small local GPU cluster for Qwen 3.8 Flash and while it's "free" (you should check electricity prices in EU...) unfortunately it doesn't even compare to Opus 5.5 or Astra in capabilities. It's like GPT 6 vs GPT ~4, but imagine GPT 4 is also 100x slower and sounds like an airplane taking off.
In the end it's totally possible that I would pay more for electricity running Qwen for 24 hours churning on some complex problem vs. Astra one-shotting it in 15 minutes.
I hope we get there, but it's not there yet, and I'm sure never will be with consumer GPUs. Maybe if you spend $20k+ on a local AI rig. There's just no market force to give access to AI to everyone at fair price.
The reverse happen with Deepseek. Getting smarter and faster especially if you dont mind the China API. Gemini Argon getting dumber compare to older 3.8 and significantly slower too.
I've seen big drops in Opus 5.5 (on high/x-high) & Fable. Both in terms of following instructions / workflows execution and overall code quality. Also the token burn-rate increased a lot, I've reached limits of 3 Ultra subscriptions during last week & had to spend 1.5k on API usage during the weekend just to be frustrated with the stupidity of the output.
I guess it's just capacity problems. They probably have a fixed amount of compute and they need to make space for new models that will be released. So they quantize existing models e.g. in half so that can use 50% of current capacity for new models. They have such massive scale that you just cannot dynamically get that amount of new compute online so they need to work with what they have. No provider has thousands of GPUs idle.
There are a couple documented things where it has degraded: in late 2025 or so when they transitioned to TPUs, they had a bug in the compiler affecting sampling logits in a cheaper approximate way than GPUs and that caused a big degradation. They never told users "sometimes we'll sample logits in a cheaper approximate way," but the main issue was the bug and not really an intentional thing at least from how the characterized it.
TPUs only made up a small percentage of the fleet, but, exacerbating that bug, if you got routed to a TPU instance at some point in your chat then another bug kept you routed there for the remainder of the chat.
The second one I'm aware of is earlier in 2026 they decided to save usage by stripping out all thinking tokens if you left your chat idle for one hour (cache timeout) and then came back to it, causing massive model performance degradations. This one also made me very skeptical of Chinese distillation, until the reports of how you could use Haiku to extract the raw reasoning of the larger models.
They fixed that, but recently they switched to doing auto-compactions for long running chats in anticipation of them going stale in cache. I think that one is usually what you want anyway and there is at least a way to recover to the uncompacted chat if you really need it, but it might also be a source of some degradation people are seeing now.
Yes. That was the weekend my token usage blew up and used up my weekly allowance in a matter of hours. It was so stark.
After I figured out, like this chap on X, that it wasn't ME, I diversified my model usage and now a lot more work has shifted to Astra/Sol and I'm learning to make my workflows more resistant to the whims of each model.
I am using Opus 5.5 high almost exclusively. I definitely felt this and looked for nerfing watch sites. It felt like downgrading to the old version of deepseek. Things have improved a bit now.
This is not real model's reasoning chain, they might have just replaced or tuned the model that summarizes it. If you want to track actual degradation you need to run evals. Trackers like this do exist, e.g. https://aistupidlevel.info/
We need a consistent way to tell level of service for LLM services. Without they can play with us and steal money by providing different/quantized models than we paid for. I believe this may be a good next AI company if they will be reliable.
Anecdotal, but I tend to get by with medium thinking on Opus and I noticed that as of sometime last week it really did feel like it had become noticeably slower to think about things.
I wouldn't say I've especially noticed its effectiveness having been reduced, it's just slower.
this is the reason i cancelled claude after 4.6, non-determinism aside, anthropic just doesn't seem care about maintaining sane baseline for their products
Hm, I’ve seen a degradation of 5.5 yesterday. I’ve had a session running for around 50 days and it just started to generate garbage and ignored all rules. A fresh one feels the same as always
If Anthropic indeed nerfed Opus 5.5 on November 3, it could be because they wanted to be prepared for inference cost going up due to the data center moratoriums and the election results. I really hope this is false, but I don't see how inference cost can go down in the near future.
1) it was certain for a long time before the election that the unwholesome party is going to get slaughtered
2) the wholesome party is not in power yet
3) even when they are, the orange man will veto everything they might attempt to do
4) even when the orange man is replaced with someone wholesome, and that wholesome person enacts a bunch of wholesome legislations, data centers will simply move literally anywhere else on Earth.
Regarding 4, I don't think anyone in the US, regardless of how they voted, would want a bunch of SOTA GPUs being in any other country than the US. It's either they are all here in the US, or they are not anywhere. As a result, if data centers cannot be built in the US, the GPUs will not be produced only to be exported elsewhere.
>In Ohio alone, at least 14 municipalities passed bans or restrictions on datacenter development, including in Sunbury, where Amazon has proposed a $2bn datacenter project. In Maine, eight municipalities passed temporary datacenter moratoriums. In Emporia, Kansas, where a 1,000-acre gigawatt-scale datacenter was recently proposed, a ban on high-impact datacenters was approved by voters.
and now they will be built elsewhere.
I think there's a rather obvious precedent for that inevitable outcome.
Sounds like analysis fully driven by a model, which can be fine, if you know what you are getting into and can verify the output. Just last week had someone ask Claude Opus 5.5 on High about watermarking after I informed him and he'd rather believe a models output over me, do a basic Google search or spend the effort reading Anthropics communications on the matter.
The model told him with no uncertainty that watermarking text was not implemented in any Anthropic models, whereupon he showed me and I then asked him to just ask the model to check again, ideally online. These models do still "hallucinate", they still have major gaps in their output not easily caught if you don't know the topic and interestingly, anything connected to LLMs, model providers, etc. tends to be where they do worst.
The annoying thing is, they still sound immensely authoritative across most any topic, but especially LLM stuff, which is such an easy way to get tricked. I have been for sure in the past and will be in the future, it's why I recommend having some incredibly niche hobby (like sewing with less known materials, obscure motorcycles, etc.) to check how quickly the model makes a mistake which no one in the know would make. Just to keep yourself disillusioned and inoculated.
In this case, looks like the LLM did not consider that these are (and have been for months now) just summaries and the actual reasoning can not be analyzed for encryption reasons.
If in doubt, please do some research independent of the model output. LLMs not having any internal "understanding" that'd give them anything akin to an internalized "feel" for what they "know" accurately enough is one of their major flaws.
Funnily enough, once did the opposite, made a manual analysis of a Github outage, then had the model just write an artifact to quickly share. Was lazy, could have just put something on Pages, but taking summary changes and extrapolating that to a "nerf" is less ideal.
On the regression front, I run my own, private evals and have only once encountered consistently reproducible, over run-to-run deviation, degraded performance, which was purely present when using Claude Opus 4.8 in April after some system prompt changes in Claude Code specifically which were addressed soon thereafter. These were not present with OpenCode via API and any other time the public on Reddit and co. reported regressions, I could not reproduce it, even when using Claude Code as the harness and a subscription.
What I have encountered and been able to reliably reproduce is that certain labs do very clear teach to the public benchmarks and underperform them commonly, whereas others do not. And that certain, especially cheaper and higher throughput, neoclouds very clearly take some quality affecting, tool call misformatting shortcuts.
For what it's worth, currently measuring, but haven't seen anything so far with Opus 5.5, especially nothing connected to reasoning that'd make the assessments some of the commenters draw from this apparent.
How can anyone build on this quicksand where you never know whether the same model will still be available tomorrow? The whole industry has become a casino.
LLMs are such a great business, they are non-deterministic, hence you can do whatever you want, because customers can't do anything about it:
* nerf them based on supply and demand, when you have a high demand, nerf it, but keep charging same price.
* do you want to show revenue growth to investors? increase thinking tokens, customers will keep paying, because again, this is non-deterministic system, sometimes it thinks more
* do you want to increase profits? start serving 4-bit, 5-bit quantizations and keep charging same price
* want even more? publish a paper "Advanced Auto Super Smart Prompt Router - AASSPiRation", and start routing requests to Qwen3.8-27B or GemmaE2B from time to time on the edge
* Not enough? introduce "Complex Chained Workflows - CCW" in your coding harness, which spawns multiple deterministic workflows (which you can't stop or control) and they will use Haiku to write the code, then use Opus 5.5 high to verify the code (which is obviously wrong) and Opus 5.5 produces recommendations on how to fix the code, delegates to Haiku, which again fails writing proper code, and keep doing it 5-6 times, you made lots and lots of money (don't forget, all should run in parallel, to pick one best result among 10 outputs).
You can say the same thing about nearly any business. Lyft and Uber, for instance. You can lower quality, raise price, whatever. This is why open markets matter so much - when you have competition, businesses can't materially worsen their customers' outcomes for long or those customers switch.
How do you even lower the quality - serve half broken car for a luxury order? No way person who orders luxury car accepts it.
Or do you increase wait times? - sure, order gets cancelled, you won't get a payment for that.
There are of course ways, but you still can do something, in case of LLMs, you blame yourself for poor prompting and retry, which again makes money to LLM providers
- You can lower the quality standard of which car counts as which class. There are many classes of vehicle on both platforms.
- You can delete or not consider low ratings from certain users. Both companies do this already.
- You can increase wait times, of course. It doesn't at all follow that the order gets cancelled. It depends on the market and what each company believes the other is doing.
Eagerly awaiting Artificial Analysis to redo their Anthropic benchmarks and kick it down 15-20 index points, in the interests of accuracy. Those guys love accuracy. They moved Heaven and Earth to kick Kimi off the #2 spot, out of their deep, abiding passion for having the most accurate benchmarks in the industry.
I have no doubt they will ensure their users get an accurate picture of Opus 5.5 within hours.
IMHO the only defensible way to benchmark this is to use secret-shoppers via subscriptions. As is standard practice in other product-review fields, where the stakes are a lot lower.
The purpose of AI benchmarking is to inform people making large spending decisions what alternatives are available to them.
I *do* wish we wouldn't let software companies call this "A/B testing". That's not what that term means to a typical reader. In any other market than software, you'd call it something like "misleading consumers in respect to product composition" and "an unfair commercial practice". That's language from the EU "dual quality" food scandal, which saw food brands (Coca-Cola was one) adulterate their recipes to use cheaper ingredients in select markets—though the external packaging was kept the same.
If I were a cynic I’d also say that it wouldn’t be particularly difficult for them to detect when a benchmark/eval context is active and to route those requests to larger models or higher quants.
It’s A/B experiments at scale to see how many people put up with endless quantisation, a defence mechanism so they can say “hey all these wildly unjustifiable salaries for developers and management are totally unwild and justifiable, why look at how much we’ve slashed our inference costs”
This'll likely be the reality moving forward - they allocate more compute at launch to get good test scores, reviews, and news coverge, they get people excited and used to it, then they cut costs.
I'm reminded of the video game industry where they would launch a game without any in-game purchases at launch, get the reviews to make note of it and give it a higher rating because of it... then add in-game purchases anyway. I think that stopped again though.
My paranoia tells me they A/B test users which means consistent benchmarks go through the high performance pipeline while regular requests fall back to the quantized / experimental / low cost version. I can see the main reason being the oldest one in the book, which is making more money.
It could also be a regional effect where systems are under higher load and hence dynamically routing to / provisioning lower precision instances; or some other factor where one is affected whereas someone else isn't.
I use Sonnet 5.5 daily and Opus rarely. My usage is also a little involved as in I do all the design and strategy and get Sonnet to just write the code
For that, Sonnet has been consistent
I use Opus or Fable when it's a dumpster fire situation where we have no time to think even
Each new model version hits your dopamine receptors very hard when it first comes out, but in the nature of things it stops hitting after a while and you need novel models to fire them off again. That's why people think the models are getting nerfed after a couple of weeks; it's actually their endocrine system getting nerfed.
There were no noticeable complaints about Opus 5.5 before the end of last week. Large volumes of people are now reporting the same thing at the same time. If your theory is true then multitudes of endocrine systems separated by vast distances and vastly different initial Anthropic exposure times are perfectly in sync. Resistance is futile, you will be assimilated.
Is there a practical reason this link uses twitter.com rather than x.com, or is it a nod to the old name? If it’s the latter, I don’t really see the point.
It seems to be an A/B testing experiment:
https://www.reddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner...
The way Anthropic continues to get away with testing these nerfing "experiments" on customers without their consent depriving them of the model quality they've paid for is wrong.
The Windows Insider program has shown plenty of people will willingly sign up to be testers for a taste of the rawhide. Anthropic could do this in a transparent way but they continue to choose not too.
They can get away with it because they are so far ahead of OpenAI models for coding, it's not like you can switch to anything else if you don't like it. Astra is even worse than nerfed 5.5.
Pretty much the only alternative is loading up some credits and using Fable.
Every time I read something like this that doesn't match my experience I'm bewildered until I remember that providers play quality games and that everybody's workload and harness is different. For me GPT 6.1 Sol is tremendous value and excellent at low-level work like DSP and 3D game engines whereas Opus is more expensive and not as capable. I also don't get along with DeepSeek at all even though everybody on HN seems to think it's the second coming.
A neutral third party who runs a matrix of closed-source coding benchmarks across multiple harnesses would be helpful but that'd take a lot of time and money...
Astra when it hit the ground felt like “A Fable monent”: it was so good and concise and just was done with whatever task in a short time, where opus and fable sometimes can take forever. I think it lasted a week or two until it started to deteriorate to where we are now. Coincided nicely with the stopping of codex20 subscriptions etc.
Probably won't work for Anthropic because they likely want a larger assortment of users and use cases.
Then they should suffer the consequences of not delivering what users paid for.
They'd probably claim "well, it's not an A/B test then, because we are pre selecting for a specific user type who signs up for this".
I don't think that's an excuse, but it is likely the reasoning.
Oh brother, do I have news for you...
You would prefer they experiment on everyone at the same time without a control group? Think about your answer.
No, they should not change the model at all when you select the model.
IPO is coming so they will have to maximize profits going forward which will mean constraining the compute they give everyone to ensure the subscriptions make money.
https://www.ueddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner...
And here I thought it was them taking away datacenter usage so they could drop a new model soon.
Do they ever test by giving the model more resources to see if they can give users more for the same price ? Just saying
You know what will never ab test your usage? Local ai. Just saying.
I tried it. Nothing local or open weights can touch Fable/Opus/Gemini for my use cases.
Realistically it can't match Fable, let alone Opus today. It can however beat basically all of the models from 1 year ago. If last years state of the art is "good enough" for you today then local is great.
What it did you try, and what are your use cases? Genuinely curious.
I am also curious what exactly you tried. Depending on your hardware it might also be worthwhile to stay in-tune with Qwen 4 and GLM 5.5/6.0 leaks.
Yep, this.
And with Strata running at 50tok/s on my desktop is all I need. Would I like faster? Sure! But it's super useful in all sorts of tasks now.
100% agree. A local model might not have the same stretch goal capabilities, but at least I don't have to constantly second guess myself in terms of the inconsistencies I think I'm noticing.
No longer am I constantly made to feel like Ingrid Bergman in that movie with all the lights.
Anyone with evals picking up a real signal? I've learned not to trust "vibes".
The only one I'm familiar with testing Opus 5.5 specifically over time is https://github.com/ninjahawk/livenerf and it's not showing anything.
I noticed Fable decline in quality a couple weeks ago. And I've noticed Opus drop over the last week, even after bumping to a default of Extra High reasoning for almost everything.
Tonight has been maddening with the kinds of absurd mistakes it's repeatedly making. It's so frustrating.
I'm the kind of person that wants something to be consistent and reliable. And Anthropic has been great, except for in this way. Which is enough of a recurring frustration (about every couple months this year) that I genuinely can't wait for other options to catch up. Then I'll be leaving as soon as I can.
I just don't know how long that will be.
I've been using Fable for a couple months as a relatively independent, long-horizon task executor and it worked well. About two weeks ago it completely stopped working. Like ~85% success rate down to ~10% in my workflows.
Totally possible that I'm just hallucinating/projecting/etc or my workflows somehow changed in terms of difficulty, but it's been kind of shocking.
I have the same exact experience. Working on one project, brainstorming. It was brilliant in July and August, like a seriously smarter, more experienced version of myself, reading in my mind, one step ahead of me, when pointed out a mistake it would remedy on the spot with the exact brilliant solution.
Now it's like talking to my hungover self, after 3 days bender, who just wanted to be left alone.
Same for me, especially the last couple of days. It was really frustrating and it felt like Claude was sick of working with me and just didn’t give a f*ck anymore
I guess there are experimenting with they can enshittify. But even if they undo this current change, all commercials model will eventually be enshittified.
what does "enshittify" mean in this context?
Silently switch the model you are using to a worse one.
Distilled as much as possible
Same thing for me, but the day Fable got lobotomised Opus 5.5 started actually working pretty well for me.
>I'm the kind of person that wants something to be consistent and reliable
And people wonder why open models are a big deal, and why some people love them so much, despite being 30% worse than the frontier.
Having control over something that writes most of your code is important.
And also the frontier has made everyone pay for a subscription to write code... that was a rare thing in the past now its common.
I read somewhere that the test might switch models mid conversation which stops the other from reading the previous model's reasoning. I don't know if that's true, but it sure feels like that quite often in the last days.
Codex itself displays this warning when your change the model mid-conversation.
> I'm the kind of person that wants something to be consistent and reliable.
Only local AI capable hardware and frontier open weight models can provide that.
> I'm the kind of person that wants something to be consistent and reliable
Surely everyone is that kind of person.
Who wants to pay for a service that doesn't work properly half the time?
Surprised they didn’t wait till after the IPO. Any guesses on how models might change then?
> Surprised they didn’t wait till after the IPO.
That's exactly why they did it. Nerf Opus 5.5, then drop Fable 5.5 just before IPO so it looks better next to its weaker predecessor.
I'm a pretty voracious 20x user, and OAI Biz Pro user; however I'm not an engineer by training, a previous tech adjacent dev team leader who always relied on pseudo code and broadening my horizons when dealing with my team. ( AdTech, DTC marketing )
So I'm AI pilled and in another career, and started to think, what if the problem is the inputs? They release Opus X.X and at first it's a rush of their best users, best use cases, upgraded lanes. As weeks go on, adoption increases, cruft uses start to poison the model? Is this how the ongoing evolution works? I'd love to get Sonnet 5.5 and Opus for review in a place where I could reliably daily drive for X months / Y years etc.
Fable has most definitely been nerfed for the last few weeks, not to mention painfully slow.
Oh don't worry, you can just switch to Fable 5.5 tomorrow, which is the smartest model so far (because the others are dumb now).
I noticed the same, and posted about it three days ago.
Opus is suddenly fumbling basic tool calls and failing to finish tasks even at high, where I had been permanently using it on low effort with success since it came out. It is very frustrating.
My guess is they quantized it and/or reallocated DC capacity to train a new model, this has happened before.
> It is very frustrating.
Not just frustrating, it can also be dangerous. What if you trust Claude with certain security-sensitive things (because experience), and suddenly you cannot trust it anymore like that?
You shouldnt do that in the first place
Then you should have evals.
It’s happened before and it will happen again. At least now you know.
Maybe don't trust something with non-deterministic output for critical level security?
*chaotic
It is deterministic, if we're being pedantic
It really though?? Even if you set temperature to 0 I saw research showing that the scheduling of the order of operations in the GPU firmware itself can change trailing decimals places and affect the output token
I'm aware operations can technically finish at different rates throughout all of a GPU's cores, but I am not exactly sure that GPU firmware would change the outcome of a calculation (unless there's something specifically in the firmware for it)
I don't know a lot about it tbh so I probably mislabeled it as a firmware issue
Instructions unclear; revoked all human access.
Right lol, i didn't know we'd built a machine that was capable of getting critical level security deterministically correct. That's not me, at least...
> What if you trust Claude with certain security-sensitive things
then $corp should remove your access to these security-sensitive things ;)
It literally could have been rolled out in the middle of that brain surgery experiment on a mouse. Woops.
Anyone who chooses a cloud model instead of local for medical use has a damaged brain.
It seems to me the answer is in your question.
Multiple instances of failing a Bash tool, calling bash (lowercase) and getting rejected and even reprimanded by the API "No such tool, call Bash instead".
Killing it's own shell, multiple, multiple times in the same session. I'm sure I saw a phrase "I did it again".
It's so shit that it must be damage control. The alternative is to limit usage and that'd affect the IPO even more.
I noticed its been trying to drag out the conversation on every topic. Maxxing token consumption for the dollars instead of staying on point. Makes sense for them but its frustrating for me as a user. I suspect this is going to be the AI version of enshittification. Now its only missing some badly needed in-stream ads.
It's very frustrating, even at max effort. It used to do things correctly on its own, now I need to keep reminding it to do obvious things like "Make sure you are not dropping steps as you rewrite the docs". It became unreliable, like how Opus 5 was.
It blows my mind how people still build their workflows around depending on services like these, with how easy it is to just self host and forget about all of this. You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription. You'll never have to worry about token limits, the model changing out from under you, getting banned, etc.
It's because the frontier models are superior and much faster.
>much faster.
This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.
> you can buy the same GPUs or better than the big players are using
You can not buy an H100.
Speed can mean two things: rate of token generation or time to finish a task. What maters it the later and a model that takes a faster path to the result and uses fewer tokens because of that comes out ahead in practice.
Superior? Sure. But do people need the capabilities beyond what can be reasonably achieved with local models? I think the people that actually sit in that gap are vanishingly small, and if your title contains "Engineer" or "Developer" or something like that, you are not one of those people (at least for doing your job)
As for faster, maybe? It depends on the metrics and what you're doing with it. If you mean TPS, no, I get much higher TPS and even faster TTFT on my local setup than frontier models. If you mean time to complete tasks... maybe. If you're throwing whole projects at it to vibecode, yeah, a frontier model will dump slop so fast it'll make your head spin. If you're doing actual methodical development, like doing spec-driven development and actually reading the specs before spinning off agents to implement them, rather than just having another agent review it and yolo, you'll find that frontier models don't really push the needle that much, and you'll be making high-quality projects.
I might not need them but I'll sure as hell take them if they are available
Every hour baby-sitting a local model can be extremely expensive in billable hours.
What’s your local setup? I’d love to give it a spin; lack of 5 hours limits is much needed.
Qwen 3.8 (and maybe soon 4) series models are probably the best right now for local AI. Specifically, for lower end hardware like 8gb GPUs you might find a lot of use in a Qwen 3.8 variant like Underdog Saluki 27B.
They are faster, but the distance from superior is changing daily. Naturally, things like purposefully kneecapping the product does not help.
nobody with the competence to judge the quality difference between frontier models and local models would ever, in a million years, use local models
Why not? There are a lot of tasks that a local model can do cheaper.
In a workplace environment I don't think "cheaper" is necessarily the thing to optimize for. Subscription models save you a lot of time (and money) by getting things right faster, solving bigger problems, etc. A cheaper model just consumes more human time.
But with a local model with essentially free tokens, I can set it to work on tasks in the background without having to worry about any usage limits. Deferring simple bug fix tickets or other low complexity to an essentially unattended local model works just fine. And if the result occasionally fails review, it still didn't burn paid usage credits.
A local model tops 60 tokens/s max. Running it on the background means not being able to use on the task you are working on. When working on local hardware you cannot do concurrent requests. In deepseek for example you can have 100 agents running each getting at 120 tokens/s easily. In local models you are limited by your own infrastructure.
I do concurrent requests just fine and they have no noticeable performance impact. Not all local hardware is equal. Also, humans have to sleep. Agents don't, so processing stuff over night and reviewing in the morning just gets more utilization out of what is already there.
With cloud agents I also process stuff overnight. But my gpu is limited. If I want to OCR a high number of documents or vectorize data, I can't run an LLM at the same time. I use remote models and use my compute for running commands on my desktop machine.
You seem really upset that people do things differently than you. May I suggest trying Tetris to help you feel better? https://play.tetris.com
Today this is absolutely true. Local models have gotten very good, but they aren't there yet. The cost of a Max subscription or even API pricing is overwhelmingly worth it for most users, even if you ignore the enormous capital costs of running the competitive models locally.
But the in a million years bit, even taken as rhetorical bombast, is of course silly. The difference between local models and the frontier models is ever shrinking. Extrapolating out and eventually, sooner rather than later, the difference is going to be a wash.
Unless all of the vendors stop progressing open local models, the future is clearly there. It's why I think AI is important and valuable, but also the current leaders have a dubious financial path forward.
It's an easily reproducible technology, clearly...
He's pretending like there's any magic to claude or the others, there isn't
Enjoy renting everything you depend on!
i dont understand how people keep comparing local models with frontier models, they dont belong in the same discussion especially for coding usecases. they solve different problems under different constraints.
I don't understand what do you mean. They both solve the same problem. Some models solve it better, some worse. And in some cases you can only use local models, so they solve more problems.
"Frontier" (closed) models are better by definition, but if frontier models were good enough for you half a year ago, open models are good enough now.
Indeed. Local models remain rock solid, while cloud models are a shitshow of stability.
Compared to Opus local models Opus 5.5 is far superior, also for something like a dgx spark at 5k that would be 25 months ignoring power costs if you chose the $200 plan.
And that’s looking at a large model, to host the equivalent sized model as Opus 5.5 you’d need more than a single dgx spark. A fairer comparison would be to Open models hosted remotely or comparable weak models like Luna which you would get away with a $100 plan or less. That also doesn’t account for token prices continuing to go down for the same level of intelligence.
Are token prices going down?
It has been trending downwards. See recent Anthropic and OpenAI price changes.
But even if that was not true, the more important metric, cost per task, has definitely gone down.
Yes, by a lot. You can get opus 4 level of intelligence like 100 times cheaper now than at premiere.
it is believed that opus5.5 needs 8x h200 to run (~1t vram)
a dgx spark could never
What GPU can you buy that will run a 180B model for $1800?
Qwen 3.8 Flash Next is 125B with 6B active, which with quantizing and system memory offloading (strata) runs nicely on 32 GB cards (I have actually run it on as low as a 16 GB GPU and 16 GB of system ram, but I don't recommend it for serious work). I was running it on a single r9700, but now I'm using 2 r9700s because I'm splitting my server with a friend, and they bought one as well.
On the single r9700 I was able to run q2 with 4 slots of 256k, and I think it was about 40 - 50 GB of system memory offloading? Got about 60 - 80 TPS per slot depending on concurrent use. With the two r9700s we are running q3 with 4 slots of 256k, using about 100 GB of system memory offloading, with mtp enabled. Getting about 70 - 100 TPS now.
You asking wrong question. You need to ask what tks/seconds it's gomna run at.
There are plenty of ways to run ds flash and qwen flash on single cheap gpu, but speeds are quite bad or contex windows are small.
What is the alternative? Local models? I do not think so. Maybe Kimi K3 (still not local) as this model can be served by multiple providers? However Kimi K3 sits between Sonnet/ Sol and Astra/Opus so still not the same level.
Lots of models are being served by many providers. Even Anthropic models can be served by Azure. Check for example https://www.requesty.ai/models
can u tell me the specs or hardware requirements to run qwen3.8 flash
Voluntarily downgrading to Sonnet 4.5 level performance sure is one way of avoiding imagined degradation in Opus 5.5...
Have you ever heard someone say they did so and regretted it?
First time I hear someone praise Qwen 3.8 Flash Next specifically.
Personally I was forced to use only GPT-5.6 Sol at work instead of Opus 5.5, and it was so much worse.
Benchmarks that control for benchmaxxing indicate all open-weight models behind 5.6 Sol and roughly at the level of GPT-5.5.
And that's for the latest large ones, much less local Qwen 3.8 Flash Next.
I would regret such a downgrade.
i haven’t heard anyone say they downgraded their car to the children’s toy power wheels and regretted it. all the same, i’m confident that it’s a bad idea
Why lease a Ferrari when you can buy a Corolla.
This, but unironically.
You two actually summarized the whole discussion beautifully.
Why indeed. (The Ferrari's always in the shop, and the Toyota gets great mileage, and you need to commute.)
Yeah but what about your duty to consume and be part of the economy?
> You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription.
which GPU?
also, so, 90 months of my subscription? And I also would need to buy new RAM and other parts of a computer?
I can buy a GPU that can run Qwen 3.8 Flash Next for 9 months worth of the highest tier Claude Code subscription, but then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
> then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
By choice, maybe. New models release faster than every nine months. And, evidently, you won't be running nine months of some fixed meaning of "Opus 5.5". What that means changes over time, out of your control, and without ever even being officially told.
Sure, but, even when it degrades, it's better than Qwen 3.8 Flash Next.
Now, if GPUs didn't cost an arm and a leg, the economics would have been different.
I'm surprised why folks are either all in on frontier subs or self hosting on their own hardware.
To me it seems there should be a very viable middle ground of paying for open model hosting by a trusted provider - unless I'm only hearing from 24/7 vibe coders
I also try to go that route. Currently exploring Kimi hosted by Infomaniak.
People, refuse to be A/B tested!
If you paid for X but are now receiving Y, you should be entitled to get your money back.
Every time I consider going back to Claude, I remember that mysterious week in February when Claude was failing to do even the simplest things. Things were so bad that BigPickle was correcting errors Sonnet was making.
And let's not forget the king of A/B testing here. Google has no issues breaking products for months just to test some ridiculous idea.These people should be publicly fired.
> Every time I consider going back to Claude, I remember that mysterious week in February when Claude was failing to do even the simplest things
100% agree. I don't care how many reddit bots i see praising Claude, I bought max once wanting to see what the fuss was about, and the next day they broke the product and it was really embittering
As one user pointed out:
> I noticed this too as did many others. Had to bump my default reasoning but it’s not the same. Seems to often happen before new model drops
I heard that Qwen4 was dropping some time soon, I wonder if Anthropic is trying to announce a new model immediately after Qwen4 - both for their IPO, and to kill the competition.
This has happened to consistently across OpenAI and Anthropic models that I can't believe people are still denying it. Obviously they're silently quantising their models or making other efficiency changes. If they can save 40% RAM and only hurt intelligence by a few percent in their benchmarks, they're going to do that. Unfortunately their benchmarks are obviously not representative of the real world, and these downgrades are very noticeable for people who rely on these models for work.
Of all the reasons to look at local inference, this is the one which pushes me in that direction the most. I want to rely on a model for consistency in my workflows. If I wake up on Monday and suddenly it's refactoring things it built on Friday, that's a huge problem. That makes it unreliable.
> I can't believe people are still denying it
Probably because those "nerfs" do not show on sites that track model performance over time. Sometimes Anthropic/OpenAI mess stuff up in their harness, but usually not in the models themselves.
It's easy to imagine they can detect when they're being profiled and route accordingly. Paranoia or not, it's possible.
I guess the question really is: is this malice (trying to trick people into thinking new model is better by nerfing old one), or is this capacity constraints, where they obviously need to make room in their infra for inference of the new model by shoving the other models into a smaller box?
Little of column A, little of column B.
These machines are inherently unreliable.
People are denying it because it's simply not true and for sure has not happened consistently.
In my own work I have not once noticed a degradation where I could have said with any certainty that it's not within usual deviation or me having a bad day.
Conspicuously the model seems to be especially dumb and annoying when I did not get a lot of sleep the previous night.
When OpenAI and Anthropic engineers post here on HN that they do not quantize models and silently replace checkpoints in the API, I believe them.
I saw nothing that convinced me the people who are constantly making these complaints don't just mistake any suboptimal result for changes in the model.
I've "called" the last 3 out of 4 new model releases. Maybe my account it perma-tagged as one that gets downgraded to re-allocate resources before a release but about 5-7 days before a new model Claude suddenly gets dumber and dumber on executing canned skills that I've used for 6+ months with no issues. The cynic in me thinks they are dumbing down models right before a release so the new model seems much smarter.
It’s also true that people that perform their own evals see wide variance uncorrelated with “nerfing”. Human cognitive bias makes them only see that variance at specific times and then they start projecting.
Everyone really should keep a few tasks around that are indicative of the work they do and run them a few times with the same model and a few times across model families and when new models drop.
It’s extremely informative to see with your own eyes the variance that always exists. Even if you can’t afford statistical relevance it’s valuable.
Yes I have similar anecdotal experiences - it gets dumb before a new model. What I imagine in my head is Anthropic using all their GPUs to polish off the next model and they need to borrow GPU cycles so they cut down the thinking.
Token use exploded a week or two ago. I worked for a few hours and hit my 5 hour limit. Did only some simple bug fix. Somethings definitely broken.
This is the #1 thing i HATE with LLMs. They are not predictable like a compiler is.
It's not LLMs! Its their providers! If you self-hosted, you'd observe the slow down but you'd get same quality.
It's probably not token usage, but a flexible 5 hour limit. There's a reason why they only state it in percentages.
Nope, the issue was not the 5 h limit, but that the 5 hour limit was gone in some minutes and that this burned trough 50% of my weeks fable limit. Before I could do roughly one full 5h session (fable till max) per day.
They're actually extremely predictable. What you are experiencing is the provider you are entrusting with your time, effort, and money fucking you because they need the capacity to train the next one. So without any real notice or indication other than things vaguely seeming broken in ways everyone seems to be experiencing differently, they're serving up some shitty quant of what you already paid to use.
Extremely predictable is an exaggeration. I was teaching machine learning and we needed half a dozen lines to make even simple networks deterministic. LLMs are not really predictable, even if we want to, they will sometimes behave almost chaotic depending on the input.
> we needed half a dozen lines to make
6 lines per model? Can't you put that 6 lines in a function/macro/decorator/library so you only have to call it?
I'm not sure how you're arriving at "if you use the same seed with the same version of the software on the same hardware with the same drivers you get the same result is an exaggeration", it's wild that you're a teacher and you seemingly do not know this.
Some teams I know started moving all linting and validation into LLMs and markdown files, the reasoning was that they asked agent to review changes that were later found to be buggy and agents flagged these bugs. And that’s great but my immediate worry was that: it finds it now, but will you check every new model for all these bugs? And what if you need to switch providers? Or your provider silently tweaks the model to improve their margins? The answer was “well also build deterministic linters for all these problems”, but you know that’s just wishful thinking - if you can get 99.9% done with just a little bit of fuzzy text and the next 0.1% is a lot of work it won’t happen. So you have model behaviour lock-in that will bite you suddenly when you least expect it. Or maybe LLMs will just keep getting better and cheaper? It’s all a casino right now.
> Some teams I know started moving all linting and validation into LLMs and markdown files
Sounds like those teams need better leadership. Deterministic tools should always be used along side LLMs to improve the results and reduce costs.
Every LLM company must keep getting better and cheaper to compete. If you game out them running low on money, none of the outcomes actually result in you spending more or getting worse models, just switching to another one.
If you have 99.9% already - that's better than any pre-LLM deterministic linter.
Definitely noticed some anecdotal declines this past week or so. Missing steps it didn't previously, needing reminders of tool calls and skills that it should auto apply.
Lots of "this is a potential issue, I didn't check into it though" which is super frustrating to see. The model just points out major flaws in logic or barriers but goes no further than a comment on it.
I've got a Qwen coder model and a flash next model locally now, so it's not the end of the world. But it does need to be remedied - are we seeing them prep for a new model deploy when this happens? Or are they experimenting with leaner hardware to drive down costs ahead of IPO?
I’ve been experiencing unreliable performance for a while and can’t imagine this is solely due to new model training.
It does seem like Anthropic is experimenting with how much less they can offer users. I also get the feeling they are judging what resources a task deserves and dumbing things down accordingly. I’m seeing lazy responses and skipped steps more frequently for non-technical queries.
It’s definitely possible the average user hasn’t noticed, unfortunately.
For me I've noticed the pattern of decline happening on Tuesday and Wednesdays. I find myself having to have Opus redo many steps and making a much larger number of errors on those days. I am able to be more productive on Thursdays just as my weekly reset approaches on Friday at 1 pm local time.
I don't have measured data aside from myself noticing my frustration level with Claude jumping on these days frequently over the past several months.
Definitely a significant change. I noticed it in just daily work. Quality of work down, mistakes up, repetitive errors in scripts - rare earlier, now a few every session.
With open weights, YOU get to decide when the model changes.
Not true at all, there's much more to a model than its weights.
Like?
Thinking budgets…
In the context of this question, that's still something you as the user control.
Specifically
> YOU get to decide when the model changes
Yeah obviously but you can still control it.
You can still control it if you can trivially spin up a node with eight B200s and 2TB of memory to run your Deepseek v4 instance. Nobody on a subsidized plans is doing that. Local models that fit 128GB memory don't come close even to a "nerfed" frontier model.
Actually Qwen3.8-flash-next comes very close to a "nerfed frontier model" the way I host it locally at 4.75bpw with int8 kv. It is pretty much equivalent to opus 4.6 up to 4.8 on coding. What it lacks is some factual knowledge - this is to be expected, it's a far smaller model. But if you run the kind of coding tasks where you have to spec them in high detail anyway (large very custom environment) this model is very usable.
I run it on a 6*rtx3090 consumer PC (no threadripper etc sadly) with very constrained pcie topology (4 cards sit on 2 lanes basically) and i still get 7k prefill and 50t/s single user or 200t/s 7 users. All at full 262k contexts of course and with 96GB of ram kv cache that means I can easily have 10+ Claude code sessions open with 1-7 generating at any time.
I've been the person that said, before opus 4.6 came out that this AI stuff is absolute crap for programming. Then it suddenly wasn't and I was using it many hours each day.
Local AI was the same. It was all crap. Models like GLM5.3 Flash that are many times larger than that qwen, lots of people are oooing and ahhing over I found way too inferior. Previous DeepSeek V4.1 Flash 0731 was atrocious. They benchmaxed it to hell, but at least it was small.
The current model of same name but release 1017 and double the size is currently the best open source model IMO. It even beats my self hosted Qwen3.8-flash-next.
I've tried hosting that DeepSeek model, but even adding another rtx3090 for a total of 7 cards and my 192gb of ram bought when it cost under $1000 I get only about 25t/s decode and few hundred prefill. That is far too slow for me to use.
But, I believe once this pyramid scheme of circular financing finally bursts there will be a glut of hardware on the market where people like me will be able to buy equipment to run 750B models like we now run 120B.
Just curious, did you put all these cards in a server or desktop case ? and what kind of motherboard ?
Also, how much power does that consume at full tilt ?
I've been very impressed by it and recommended it to everyone. 32gb ram, 12gb 3080ti, and I run 3.8 flash next coder (half the experts ripped out, leaving in coding) int8 kv 128k context at 35-55tk/s and I agree with the opus comparison; I personally say 4.7 is what it seems like the most.
Truly insane times
For Deepseek V4 Flash (not 4.1), you can use two 128GB unified memory boxes and run it at the official weights (mostly 4bit) with lots of context.
That's around 1% of the price of 8x B200. At current prices, it's as low as $6200. At last year's prices that's $3700 or so.
you can run Qwen3.8-27b with a single GPU.
I moved from Claude to Codex and am happier. The prose is less grating and results comparable. As much as I don't like Altman I am tired of Anthropic's antics.
I tried this and while the prose is indeed snappier I find the code quality of Codex truly horrific. It's dense and uninterpretable. I almost wonder if they have made it purposefully opaque so that it's harder to resume work without Codex once you're using it agentically. Claude is still much better at writing quality code that is fit for human consumption.
I don't think Astra's code is quite as bad as you say, but yeah while it's much better at non-annoying prose, the code definitely isn't quite as good as Opus 5.5.
Same.
I'm so happy to not look at bogus jargon that Claude made up.
That largely ended with Opus 5.5.
Um., Codex is now suffering from the same thing afaict. It is almost as if having a duopoly is a bad thing for consumers.
... is it?
We use Azure Foundry hosted Opus and it seems as capable as ever.
I run a small local GPU cluster for Qwen 3.8 Flash and while it's "free" (you should check electricity prices in EU...) unfortunately it doesn't even compare to Opus 5.5 or Astra in capabilities. It's like GPT 6 vs GPT ~4, but imagine GPT 4 is also 100x slower and sounds like an airplane taking off.
In the end it's totally possible that I would pay more for electricity running Qwen for 24 hours churning on some complex problem vs. Astra one-shotting it in 15 minutes.
I hope we get there, but it's not there yet, and I'm sure never will be with consumer GPUs. Maybe if you spend $20k+ on a local AI rig. There's just no market force to give access to AI to everyone at fair price.
The reverse happen with Deepseek. Getting smarter and faster especially if you dont mind the China API. Gemini Argon getting dumber compare to older 3.8 and significantly slower too.
I've seen big drops in Opus 5.5 (on high/x-high) & Fable. Both in terms of following instructions / workflows execution and overall code quality. Also the token burn-rate increased a lot, I've reached limits of 3 Ultra subscriptions during last week & had to spend 1.5k on API usage during the weekend just to be frustrated with the stupidity of the output.
Weirdly enough, Sonnet quality seems unchanged.
> & had to spend 1.5k on API usage
You rewarded Anthropic's failure with more dollar spend. I get why you did it but...
I guess it's just capacity problems. They probably have a fixed amount of compute and they need to make space for new models that will be released. So they quantize existing models e.g. in half so that can use 50% of current capacity for new models. They have such massive scale that you just cannot dynamically get that amount of new compute online so they need to work with what they have. No provider has thousands of GPUs idle.
They don't do that. They've said as much and there is zero evidence to support it.
Under load requests queue and take longer.
There are a couple documented things where it has degraded: in late 2025 or so when they transitioned to TPUs, they had a bug in the compiler affecting sampling logits in a cheaper approximate way than GPUs and that caused a big degradation. They never told users "sometimes we'll sample logits in a cheaper approximate way," but the main issue was the bug and not really an intentional thing at least from how the characterized it.
TPUs only made up a small percentage of the fleet, but, exacerbating that bug, if you got routed to a TPU instance at some point in your chat then another bug kept you routed there for the remainder of the chat.
The second one I'm aware of is earlier in 2026 they decided to save usage by stripping out all thinking tokens if you left your chat idle for one hour (cache timeout) and then came back to it, causing massive model performance degradations. This one also made me very skeptical of Chinese distillation, until the reports of how you could use Haiku to extract the raw reasoning of the larger models.
They fixed that, but recently they switched to doing auto-compactions for long running chats in anticipation of them going stale in cache. I think that one is usually what you want anyway and there is at least a way to recover to the uncompacted chat if you really need it, but it might also be a source of some degradation people are seeing now.
Yes. That was the weekend my token usage blew up and used up my weekly allowance in a matter of hours. It was so stark.
After I figured out, like this chap on X, that it wasn't ME, I diversified my model usage and now a lot more work has shifted to Astra/Sol and I'm learning to make my workflows more resistant to the whims of each model.
https://XNotMe.com/shanemelaugh/status/2119192075505893695
I am using Opus 5.5 high almost exclusively. I definitely felt this and looked for nerfing watch sites. It felt like downgrading to the old version of deepseek. Things have improved a bit now.
This is not real model's reasoning chain, they might have just replaced or tuned the model that summarizes it. If you want to track actual degradation you need to run evals. Trackers like this do exist, e.g. https://aistupidlevel.info/
We need a consistent way to tell level of service for LLM services. Without they can play with us and steal money by providing different/quantized models than we paid for. I believe this may be a good next AI company if they will be reliable.
A official statement is much required
We are on the cusp of an intellectual obesity epidemic.
From what I understand they are picking and choosing who is running the latest models. So nerfed yes but for you not for everyone
Anecdotal, but I tend to get by with medium thinking on Opus and I noticed that as of sometime last week it really did feel like it had become noticeably slower to think about things.
I wouldn't say I've especially noticed its effectiveness having been reduced, it's just slower.
this is the reason i cancelled claude after 4.6, non-determinism aside, anthropic just doesn't seem care about maintaining sane baseline for their products
Hm, I’ve seen a degradation of 5.5 yesterday. I’ve had a session running for around 50 days and it just started to generate garbage and ignored all rules. A fresh one feels the same as always
If Anthropic indeed nerfed Opus 5.5 on November 3, it could be because they wanted to be prepared for inference cost going up due to the data center moratoriums and the election results. I really hope this is false, but I don't see how inference cost can go down in the near future.
1) it was certain for a long time before the election that the unwholesome party is going to get slaughtered
2) the wholesome party is not in power yet
3) even when they are, the orange man will veto everything they might attempt to do
4) even when the orange man is replaced with someone wholesome, and that wholesome person enacts a bunch of wholesome legislations, data centers will simply move literally anywhere else on Earth.
Regarding 4, I don't think anyone in the US, regardless of how they voted, would want a bunch of SOTA GPUs being in any other country than the US. It's either they are all here in the US, or they are not anywhere. As a result, if data centers cannot be built in the US, the GPUs will not be produced only to be exported elsewhere.
Several datacenters were stopped directly due to the elections: https://www.theguardian.com/us-news/2026/nov/04/datacenters-...
stopped before they were even built, yes.
>In Ohio alone, at least 14 municipalities passed bans or restrictions on datacenter development, including in Sunbury, where Amazon has proposed a $2bn datacenter project. In Maine, eight municipalities passed temporary datacenter moratoriums. In Emporia, Kansas, where a 1,000-acre gigawatt-scale datacenter was recently proposed, a ban on high-impact datacenters was approved by voters.
and now they will be built elsewhere.
I think there's a rather obvious precedent for that inevitable outcome.
not as bad as sol 6.1. im getting better results out of glm 5.3
I was down voted into oblivion when I said that 3 weeks ago. Both Codex and Claude are absolutely nothing compared to what it was.
> "One Bug. Two effort levels."
Sounds like analysis fully driven by a model, which can be fine, if you know what you are getting into and can verify the output. Just last week had someone ask Claude Opus 5.5 on High about watermarking after I informed him and he'd rather believe a models output over me, do a basic Google search or spend the effort reading Anthropics communications on the matter.
The model told him with no uncertainty that watermarking text was not implemented in any Anthropic models, whereupon he showed me and I then asked him to just ask the model to check again, ideally online. These models do still "hallucinate", they still have major gaps in their output not easily caught if you don't know the topic and interestingly, anything connected to LLMs, model providers, etc. tends to be where they do worst.
The annoying thing is, they still sound immensely authoritative across most any topic, but especially LLM stuff, which is such an easy way to get tricked. I have been for sure in the past and will be in the future, it's why I recommend having some incredibly niche hobby (like sewing with less known materials, obscure motorcycles, etc.) to check how quickly the model makes a mistake which no one in the know would make. Just to keep yourself disillusioned and inoculated.
In this case, looks like the LLM did not consider that these are (and have been for months now) just summaries and the actual reasoning can not be analyzed for encryption reasons.
If in doubt, please do some research independent of the model output. LLMs not having any internal "understanding" that'd give them anything akin to an internalized "feel" for what they "know" accurately enough is one of their major flaws.
Funnily enough, once did the opposite, made a manual analysis of a Github outage, then had the model just write an artifact to quickly share. Was lazy, could have just put something on Pages, but taking summary changes and extrapolating that to a "nerf" is less ideal.
On the regression front, I run my own, private evals and have only once encountered consistently reproducible, over run-to-run deviation, degraded performance, which was purely present when using Claude Opus 4.8 in April after some system prompt changes in Claude Code specifically which were addressed soon thereafter. These were not present with OpenCode via API and any other time the public on Reddit and co. reported regressions, I could not reproduce it, even when using Claude Code as the harness and a subscription.
What I have encountered and been able to reliably reproduce is that certain labs do very clear teach to the public benchmarks and underperform them commonly, whereas others do not. And that certain, especially cheaper and higher throughput, neoclouds very clearly take some quality affecting, tool call misformatting shortcuts.
For what it's worth, currently measuring, but haven't seen anything so far with Opus 5.5, especially nothing connected to reasoning that'd make the assessments some of the commenters draw from this apparent.
Yeah I realized some decrease performance on htmls it generates
I'm also noticing the model seems to be a lot more sloppy now and less reliable like compared to a week ago or so?
How can anyone build on this quicksand where you never know whether the same model will still be available tomorrow? The whole industry has become a casino.
It's just a new summarizer, the model and performance itself is unchanged.
+1, this tells nothing about the quality. Or even whether this is an upgrade or downgrade.
as a user I believe it: they are probably reducing assigned compute to prepare to offer new models.
That would be the less bad reason.
Ok, I'm not going crazy then. Noticed that it is slower and results were of a lower quality.
LLMs are such a great business, they are non-deterministic, hence you can do whatever you want, because customers can't do anything about it:
* nerf them based on supply and demand, when you have a high demand, nerf it, but keep charging same price.
* do you want to show revenue growth to investors? increase thinking tokens, customers will keep paying, because again, this is non-deterministic system, sometimes it thinks more
* do you want to increase profits? start serving 4-bit, 5-bit quantizations and keep charging same price
* want even more? publish a paper "Advanced Auto Super Smart Prompt Router - AASSPiRation", and start routing requests to Qwen3.8-27B or GemmaE2B from time to time on the edge
* Not enough? introduce "Complex Chained Workflows - CCW" in your coding harness, which spawns multiple deterministic workflows (which you can't stop or control) and they will use Haiku to write the code, then use Opus 5.5 high to verify the code (which is obviously wrong) and Opus 5.5 produces recommendations on how to fix the code, delegates to Haiku, which again fails writing proper code, and keep doing it 5-6 times, you made lots and lots of money (don't forget, all should run in parallel, to pick one best result among 10 outputs).
You can say the same thing about nearly any business. Lyft and Uber, for instance. You can lower quality, raise price, whatever. This is why open markets matter so much - when you have competition, businesses can't materially worsen their customers' outcomes for long or those customers switch.
You can't do this with Uber/Lyft.
How do you even lower the quality - serve half broken car for a luxury order? No way person who orders luxury car accepts it.
Or do you increase wait times? - sure, order gets cancelled, you won't get a payment for that.
There are of course ways, but you still can do something, in case of LLMs, you blame yourself for poor prompting and retry, which again makes money to LLM providers
I think you're arguing without considering here.
- You can lower the quality standard of which car counts as which class. There are many classes of vehicle on both platforms.
- You can delete or not consider low ratings from certain users. Both companies do this already.
- You can increase wait times, of course. It doesn't at all follow that the order gets cancelled. It depends on the market and what each company believes the other is doing.
That's thirty seconds of brainstorming.
Eagerly awaiting Artificial Analysis to redo their Anthropic benchmarks and kick it down 15-20 index points, in the interests of accuracy. Those guys love accuracy. They moved Heaven and Earth to kick Kimi off the #2 spot, out of their deep, abiding passion for having the most accurate benchmarks in the industry.
I have no doubt they will ensure their users get an accurate picture of Opus 5.5 within hours.
If Anthropic are doing A/B testing it almost certainly only affects subscription users, not API users. Every benchmark I know of uses the API.
IMHO the only defensible way to benchmark this is to use secret-shoppers via subscriptions. As is standard practice in other product-review fields, where the stakes are a lot lower.
The purpose of AI benchmarking is to inform people making large spending decisions what alternatives are available to them.
I *do* wish we wouldn't let software companies call this "A/B testing". That's not what that term means to a typical reader. In any other market than software, you'd call it something like "misleading consumers in respect to product composition" and "an unfair commercial practice". That's language from the EU "dual quality" food scandal, which saw food brands (Coca-Cola was one) adulterate their recipes to use cheaper ingredients in select markets—though the external packaging was kept the same.
https://ec.europa.eu/commission/presscorner/detail/en/qanda_...
(I suppose Coca-Cola's CEO would prefer we talk about what they did as "A/B testing").
If I were a cynic I’d also say that it wouldn’t be particularly difficult for them to detect when a benchmark/eval context is active and to route those requests to larger models or higher quants.
It’s A/B experiments at scale to see how many people put up with endless quantisation, a defence mechanism so they can say “hey all these wildly unjustifiable salaries for developers and management are totally unwild and justifiable, why look at how much we’ve slashed our inference costs”
Model releases remind me of https://commons.wikimedia.org/wiki/File:Shepard_Tone_spectru...
This'll likely be the reality moving forward - they allocate more compute at launch to get good test scores, reviews, and news coverge, they get people excited and used to it, then they cut costs.
I'm reminded of the video game industry where they would launch a game without any in-game purchases at launch, get the reviews to make note of it and give it a higher rating because of it... then add in-game purchases anyway. I think that stopped again though.
I'm skeptical about nerfing. Are we saying they are instructing AWS to nerf the version of the model they're running on Bedrock?
If they did, there's no way that remains a secret.
If they didn't, Anthropic would be making their own service noticeably shitter than AWS Bedrock
Pretty nice business model tbh. Whenever you need/feel like it, just tune it down.
Oh, even psychological effect would be nice if you slightly tune down the current frontier 1-2 weeks before releasing the new one.
People who never tried outsourcing before are discovering that there are risks associated with outsourcing.
I suppose they can achieve a benchmark then nerf the model to save compute once they've got the headline.
The quantization will continue until version number increases.
It's dumbing down a little bit but I don't think its a lot worse. I think the change in display adds to the feel
Fable 5.1 quality took a significant dive for me last week. I'm not imagining it.
Wasn’t there a platform that kept testing that ?
I guess it’s AB testing and we are not on the bad side (yet).
Nerf means something else in deep learning parlance.
Welcome to the world of post-IPO AI. Now the 20% growth needs to come somewhere, and profitability would be a nice starter.
They're preparing to release a new model, so it has to look better than the current one.
Yes, this take has consistently held true. Indicator to hold out major greenfields and just wait it out.
It's gone pretty shit, I made a comment about this yesterday: https://news.ycombinator.com/item?id=50412897
Instead of being Opus on high (effort), it's Opus is high.
LiveNerf suggests not.
https://github.com/ninjahawk/livenerf
My paranoia tells me they A/B test users which means consistent benchmarks go through the high performance pipeline while regular requests fall back to the quantized / experimental / low cost version. I can see the main reason being the oldest one in the book, which is making more money.
It could also be a regional effect where systems are under higher load and hence dynamically routing to / provisioning lower precision instances; or some other factor where one is affected whereas someone else isn't.
It's not paranoia and would align with https://www.reddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner...
I wonder as well whether would this pop up on nerfbench
Do you ever a/b test them? Run the same setup and prompt multiple times to see the output?
Ever since I started using Claude at its inception this was the biggest breach of trust that I've experienced.
I'm afraid to have touch my code base because a nerfed model could slip in stupid bugs.
This is also why a weaker local model might be preferable because it allows me to calibrate trust, and my expectations don't change.
I have had Astra analyze my long running Opus conversations and the degradation definitely happened on November 6th and 7th.
Does nerfing usually come before a new model release?
all good as long as it keeps killing young girls in the middle east
Yes, I noticed it too, dumbing down continues, there goes their IPO.
Seems fine to me
i’m so tired of this
all the measurements always say it isn’t
anyone who grew up playing dungeons and dragons gets that this is just people saying “you don’t understand, these five used to roll high”
every llm space is inundated with rank amateurs explaining at great length how it used to be better yesterday
nobody has the courage to clean out the obvious bullshit, so the drone gets ever louder and the signal gets ever rarer
do you remember what programming was two years ago?
stop complaining so much
I am curious. How do people figure this out?
I use Sonnet 5.5 daily and Opus rarely. My usage is also a little involved as in I do all the design and strategy and get Sonnet to just write the code
For that, Sonnet has been consistent
I use Opus or Fable when it's a dumpster fire situation where we have no time to think even
So far I've never had an issue with Sonnet even.
People are already so used to have genies and unicorns around they now feel infuriated when they are a bit slower than before.
Dude, you have a mythical creature at your disposal at retail, chill, it's going to be ok.
Can we get a self-hostable distilled opus 5.5 yet please?
This company does not deserve to be successful and has no moral high-ground.
Each new model version hits your dopamine receptors very hard when it first comes out, but in the nature of things it stops hitting after a while and you need novel models to fire them off again. That's why people think the models are getting nerfed after a couple of weeks; it's actually their endocrine system getting nerfed.
There were no noticeable complaints about Opus 5.5 before the end of last week. Large volumes of people are now reporting the same thing at the same time. If your theory is true then multitudes of endocrine systems separated by vast distances and vastly different initial Anthropic exposure times are perfectly in sync. Resistance is futile, you will be assimilated.
Is there a practical reason this link uses twitter.com rather than x.com, or is it a nod to the old name? If it’s the latter, I don’t really see the point.
5.5 is useless below max effort, just frustrating and plain old wrong