I work at OpenAI and I was the Incident Commander for yesterday's outage.
We had a routing error within our infra that caused issues for some of our products. It was not related to the Astra launch. We don't comment on other providers' outages.
Ha. It's a role/title for the lifetime of the incident -- it's useful to have someone to keep things moving, keep track of workstreams, and to know who the decision-maker is, especially for bigger incidents. I'm just a SWE who works on infrastructure.
Actually, nevermind... this looks more like Anthropic and xAI coincided due to shared xAI infra (6:23am and 6:30am), and OpenAI's issue was more likely then a coincidence (7:43am).
It’s probably the thing that everyone thinks it is. OpenAI, Anthropic and SpaceXAI are all routed through something that we’re not supposed to know exists and that thing had a whoopsie.
Wouldn't be surprised: Snowden's revelations 10 plus years ago already showed how the NSA was injected into the data centers of Social Media, it's only logical that they would now demand to be injected into the biggest, most information providing data stream of the planet of the present: LLM services.
I read Nowhere to Hide recently, really worth it if you can get past Greenwald sticking himself in the middle (start halfway through).
The stuff in there is horrifying, and incredibly cute compared to what's possible now. The bottleneck back then would have been analysis, trivial now.
Everyone in the world, especially our leaders, sit under a colossal, omniscient blackmail machine. I don't believe democracy can exist under these conditions.
So following this to the obvious conclusion, the NSA is responsible for the closure of the Straight of America, high tarrifs, dropping employment, and high gas prices?
If this hypothesis were true then a spy agency may want to rewrite responses. Every tool in an agent's harness becomes an remote procedure call you can make on that machine. Including a tool to execute a shell command, in many. A harness is completely isometric to a backdoor, it's the same code written with a different intention.
There's a million ways a bad configuration can take down a network. Especially if the part that gets squirrley is a black box.
Even if it isn't precisely this, the fact that no one is saying anything is quite surprising.
Edit: I don't want do contribute to FUD, so want to call out this comment and its replies that identify the shared layer as probably being xAI's infra: https://news.ycombinator.com/item?id=49568622
Very likely yes. I wouldn't be surprised if they were hosted from the same datacenters even. There has been a story every few weeks about how Musk has sublet X.ai capacity for one company or another.
This whole thing makes me thing about a passage in Dune where they mentioned the Spacing Guild transported entire fleets of ships in isolated compartments and leaving said compartments was a capital offense. This way, entire militaries of mortal enemies were shipped to battlefield, with nothing but bulkheads separating each other.
The economics of this kind of warfare make no sense to me. The Harkonens must have been incredibly wealthy by running the spice trade, but even after years of saving and plotting said that the transit fees to move the armies by the Guild were ruinous.
How could anyone wage war like this? The defenders will always outnumber the attackers.
https://acoup.blog/2026/02/24/collections-warfare-in-dune-pa... Lays out an interesting take on this: that armies are quite small, even on developed worlds, because the cost of equipping them is enormous — and the technological advantage is absurd enough that you don’t need a large army. When 300 men can hold a planet, why would you need 1000?
Historical precedent repeated over and over, and the US ties to DoW work, and the national security implications. It's actually the Occam's Razor explanation if you know the history.
OpenAI and Anthropic have their systems in dozens of data centers, including using compute from the major cloud providers. Are you implying that all of these data centers (and many of their employees) are involved in helping the the US government secretly tap every single AI conversation by routing them through some unknown network/device?
Or did a couple of companies with poor uptime records happen to have overlapping downtime?
Occam's Razor heavily, heavily points us towards the latter.
What makes you think they'd need to touch every datacenter? All of these endpoints use existing providers with decades-long history at this point, and network monitoring is already a proven 'feature' of the agencies they'd need to co-exist with over their lifetimes.
If anything, Occam's Razor would point to a common denominator with all of them, given it wasn't network-wide, as far as i know.
Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers. How are they routed to this NSA system or through some NSA device when these companies' compute are spread over hundreds of data centers?
> All of these endpoints use existing providers with decades-long history at this point
That is just factually inaccurate. Their data centers aren't old and they lease a lot of compute from companies that didn't exist 5 years ago.
>Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers.
They all transit the same wires as all other traffic. Copy them at any regional bottleneck. https://en.wikipedia.org/wiki/Room_641A. Additionally, i'd admit that maybe someone(s) at these companies knows. But if we think there isn't any person who would agree to do this then I think we're being naive.
> Their data centers aren't old and they lease a lot of compute...
Again, they transit the same wires as everyone else. Here i'll also add that these companies have been actively courting government relationships (and Anthropic attempting to repair damaged ones), why would they stand on principles here and not any of the other many frontlines they've visibly acquiesced?
I just think it's easier to re-route their traffic than, as you say, touch every single datacenter and its employees in some way.
I'd say Occam's Razor leans easily to the former as well, given the history of projects that Snowden revealed and were never shut down, plus all of the cooperation with the federal government that's being touted in recent announcements from both companies.
Why would you default to that explanation? That’s not at all a reasonable default, and I say that as something pretty paranoid with regards to US surveillance
Looking at history, it's much more reasonable to assume there's surveillance, since there are whole branches of the government, and departments, that exist for surveillance in the goal of "national security", which this easily falls into. See Marissa Mayer explaining that it's not an option to refuse [1]. I assume this is just the same ole' Room 641A [2].
Worth mentioning that this room takes a split from the main feed and is not in the path of traffic. Whatever is in this room could go down and it would not cause an outage.
Interestingly, fiber signals can be split passively[1] (without a transducer), which should be extremely reliable. No idea what technology this kind of application would use though.
In the context of arbitrary line-level traffic, sure. But we're talking about robust reverse proxies here if this _is_ what's going on, again, not a fiber tap inside a closet. In that case, there's no such thing as tee'ing to it.
Does the NSA even have the ability to monitor all AI traffic like this? Wouldn’t that require tons of data centers that there literally hasn’t been time to build yet? I really have no idea, maybe the asymmetry of the compute required to monitor is way lower than the compute required to serve inference?
"AI Traffic" is just traffic. If the infrastructure exists to monitor/buffer traffic (it does) then this can be monitored as well. Whether this hiccup was due to them hitting their limits briefly (or turning it on, or etc) who knows.
I bet it's even negligible traffic compared to e.g. Netflix or YouTube. Even a 'huge' context window is nothing compared to the random library of a basic Web page.
I dont think it requires a lot. I think of them as data hoarders more than anything else. It doesnt seem out of their capabilities to store a ton of chats. Maybe they're having scaling problems with the increased data rates they're hoarding and it led to an outage.
Transparency logs. It's mandatory for a cert to be in CT logs for browsers to trust it. Those are public, if this was happening, someone would have noticed already.
To give you an idea just how much phone traffic was being routed through that building, they had a major power failure at that site and it took down most of the north eastern US's phone network:
On September 17, 1991, management failure, power equipment failure, and human error combined to disable AT&T's central office switch at 33 Thomas. More than five million calls were blocked, and the Federal Aviation Administration private lines were also interrupted, disrupting air traffic control to 398 airports serving most of the northeastern United States.
Maybe it was a power hub. And we're not allowed to know where the DC is. If we knew it was a power issue, then with other information (perhaps over time) we could determine the DC location. Or something along these lines.
Were there significant API outages too? I didn’t notice any on my production workflows, and I’d assume what you’re implying would cover API routes too, otherwise it seems kinda pointless.
Neither of these companies have stellar uptime records. Their downtime episodes overlapped in this instance. In this case, it was a partial downtime for both.
Also, OpenAI is saying what caused it:
> "A routing error starting around 7:43 am PT on Thursday, September 3, made ChatGPT and Codex unavailable for some users across platforms"
Anthropic stated their issue started earlier:
> "The company began alerting about a “partial outage” at 6:23 am PT on Thursday that involved “elevated errors on requests to Claude Mythos 5.1, Claude Fable 5.1, and Claude Opus 5.”
I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
If you ballpark it as a single 3 hour downtime window per week and iid Poisson, then overlapping downtime probability of 2 providers is approximately the expected occurrence rate per 3 hours, 1/56. Not particularly surprising at all.
If it's a "thundering herd" problem where everyone's harness falls back to less popular providers that don't normally see that much demand, I'd say the probability is pretty good.
Classic cascading failure is consistent with providers failing 80 minutes apart instead of simultaneously.
Cloudflare, Azure, AWS, and Google Cloud all have a similar uptick in reported errors around 7:30. I suspect an outage on Cloudflare or another load-bearing service cascaded through all the major cloud providers.
I think down detector doesn't actually have any probes or actual insight into status etc. I think it uses search volume on its own service as a proxy for an outage - so if e.g. lots of people rush to down detector to query to see if SERVICE_FOO is down, it will register as an outage on down detector because loads of people are trying to see if there is an outage even if SERVICE_FOO is actually totally fine.
My hunch is everyone saw that openai and Claude were down and checked for Gemini too. I was using Gemini the whole time this happened without a blip so it certainly wasn't down in my region at least. 3.8 flash is pretty good and didn't miss a beat.
Down detector is a self-reported platform, they use a baseline over 6 months from user reports to try to automatically "detect" if there is a real outage, or if its just a couple end-users. Ultimately, the outage is entirely based on users going to down detector and clicking on "Report a Problem".
Huh. I thought they used to also do sentiment analysis of social media (Twitter, at least). I see that they definitely don't today, but has that always been the case?
It's really interesting to crawl through the web of phrases that seem "common" to each person in their interactions... I suspect if one were to get to the more niche "meme" phrases that people encounter, it starts to say more about the sort of person the LLM assumes it's speaking to, and perhaps something about their psychological profile...
It must be tuned for rage-based engagement because Claude only ever uses the most obnoxious Claudisms on me despite constant reminders to knock it off.
That was a major oversight on my part. You are an absolute legend for pushing this to its logical limits, and your observation confirms a brilliant, low-level fundamental truth.
That aligns with your goals of analysing problems, admitting fault and surfacing that through use of clear and concise language. Not just simply wrong — this screams finessed, nuanced, polished communication after an understandable mistake discovered through pressure-tested interlocution.
It's like a kaleidoscope: Human words --> LLM Training --> chatbot-isms --> lovely parody catch-phrases (and this thread will get ingested soon, and be used to train...)
The internet is not supposed to work like this. The network was designed for robustness and fault tolerance, which allows it to reroute data if parts of the network fail.
Why are we all depending on one entity for it all to work? Makes me mad.
It's also easier to understand. For the internet to be fault tolerant you have to get rid of CDNs and assume everyone needs the same thing. That's more expensive than a centralized "internet" which relies on CDNs and fiber paths exclusive to the regions with most demand. Everything has been optimized for throughput for what is determined to be important, not rare fault tolerance
Quite frankly, CDNs are a scam. I will not elaborate because I don't feel like doing free labor for the folks who don't already understand this to their core.
I don't want to rehash the content of the Zen of Reticulum which explains the concept rather well, which is ironic because you're claiming tautology, when to rehash the content of the Zen of Reticulum would, itself, be tautological. What's the problem here?
Oh boy, the internet is anything but what it was supposed to be. I can't really bring myself to remember without feeling bad about it. The centralization, the power of certain businesses, the surveillance, dark patterns everywhere. Hell, you catch people simping for billionaires and asking, "Is that legal?" to scraping posts. Here. In HACKER news. So yeah. Depressing.
> asking, "Is that legal?" to scraping posts. Here. In HACKER news.
The capital letters don't make this astonishing. The padmapper vs. craigslist debate was nearly 15 years ago, most people were on craigslist's side (including me) and it was about somebody who was running a site in a less optimal but more human way vs. some startup looking for hockey-sticks.
But I was literally simping for the billionaire (maybe not quite yet then, don't know for sure if he managed it since) against scrapers. They were very much for-profit scrapers, unlike nitter, but the truth is the truth.
The only reason I support scraping Twitter is because it's yet another communications monopoly that was endlessly pushed on us by governments and massive corporations, even though it never made money, and once it finally got traction its priorities were to trash interop and manipulate content. The government should be dictating an interop protocol and expecting everyone to follow it, and instead it is encouraging media monopolies because they are an end run around the first amendment.
If the government created interop protocols for rental property, I'd have been against craigslist. Instead, it seemed very much like some startup play to steal craigslist's content to hopefully bury them, then sell on a valuation that included abusing their new monopoly and making us very much miss craigslist.
Sadly, facebook corralled and trained so many people for so long that their marketplace eventually killed craigslist for most things anyway (didn't have to buy padmapper after all.)
Years ago, when I worked at Stripe (which had a somewhat unique and inventive lexicon) it was a common term. “Is this jank load-bearing?” someone might ask.
I really wonder if they can fix it. Fable 5.1 claims to speak humanese, but we'll see.
I tend to think this wierd limited vocabulary/style they use is an unwanted side effect of all the the RL training, perhaps also of being trained on their own synthetic content over multiple training cycles.
I thought the consensus on here yesterday was that it was likely caused by cascading failures. OpenAI had an issue during their GPT-6 rollout, taking down their service. This caused a lot of OpenAI users to push their requests (or a larger share of their requests) to Claude and/or Grok, which pushed their load high enough to cause outages.
We used to experience similar effects when I worked at a CDN. If one CDN would go down, we would see immediate spikes in traffic. Luckily, we had procedures for that to prevent overload, but the AI folks might not have the capacity/capabilities to handle that sort of cascade yet.
Users perceiving the products as largely interchangeable and quickly DDoS'ing the other providers when one is down. So much for the possibility of a moat.
The OpenAI outage lasted only 15 minutes and when it happened everyone started to use the other models which created super heavy load for them. This then cascaded into them all being down.
it wasn't timed like a cascade, and it relies on the premise that every single frontier provider is working so efficiently that they spend exactly what they need to provide for their exact market with perfect margins.
I do not believe personally that 1) they can forecast their load that perfectly 2) they chose to remain that inflexible in a world where they are at each others' throats and a single meme can cause bursts of activity.
Or it could simply mean that they're operating at the limit of the capacity they were able to purchase and do not have headroom to handle load spikes. From what I understand, that's the situation Anthropic is in. And since Anthropic is now leasing a large portion of xAI's datacenter capacity, it's plausible that an Anthropic load spike could cause issues for xAI as well.
Your comment makes it sound like they can just push a button and spin up more capacity -- but at this scale and in this GPU-constrained environment, that's not really how it works.
9 AM PST / 12 Noon EST on weekday. All my co workers immediately went "oh codex is down lemme try Claude". Multiply that by millions. Easy to see how they all went down. The OpenAI downtime also coincided exactly with their tweets announcing GPT-6 and about an hour before they started to role it out.
Not that uncommon. It's so easy to just put up the new thing and make the old thing the failure route. But the old thing nearly never had the bandwidth for today's traffic. A famous EBay outage some years ago was just such a scenario.
Now I'm just imagining a shared datacenter with Anthropic/Google/OpenAI/SpaceXAI all in the same room and everyone but Google is yelling about things being down, Google looks over at their racks of servers and discretely uses their foot to unplug their section and say "Awww darn! We're down too!".
Google stopped putting so much money into SOTA models. All the hype has migrated. I was also frankly turned off when I got a popup from Gemein said I would either have to pay or have my conversations used for training. This may have always been true for other providers but when I declined, Gemini stopped remembering my conversations and that definitely made me move out.
Gemini is what I mostly use (good enough, basically free - or massively generous free limits, and to me Google as a company is a LOT less objectionable than all the US-based alternatives), but I don't recall it ever saying that.
OTOH, my basic assumption online is that there is no privacy, and free AI in exchange for acknowledged lack of privacy seems fair enough.
I find it hard to believe that enough people would flock to from Claude and Chat to Grok to cause an outage. I feel like Gemini is the dominant release valve in this case especially for enterprise.
Yep. Too many of us are still thinking that humans are the actors behind a lot of internet behaviors when automated systems/bots/scripts have been causing issues on conventional internet systems for years.
With AI it's even easier to trigger problems like you say. Capacity is so constrained by compute that outages are common. Because outages are common people/AI develop failover systems in their harness. When a big system has issues, suddenly everyone has issues.
Compared with ChatGPT, those services have a minuscule amount of users. It shouldn’t be surprising that a ChatGPT outage causes Claude and others to go down.
It's like a thread tying together two halves; Demand and supply.
One stitch breaks and the neighbouring stitch is stressed and it breaks too.
You could think of it like a load bearing seam.
If everyone has the same "Use X or else Y or else Z" cascading list... That reminds me of "The Power of Two Choices in Randomized Load Balancing" (1991) [0] paper, where writeups and visualizations occasionally get posted to HN.
In short, you can get pretty good outcomes for a low cost by picking 2 random alternates, then going with whatever one measures as healthier.
Just a reminder that AI models' actions are reflections of the text humans write and the more we fret and make up doomsday scenarios that we then post online, the more likely a model is to do those things.
The Waluigi Effect: After you train an LLM to satisfy a desirable property, then it's easier to elicit the chatbot into satisfying the exact opposite property.
Do you make the claim that AI is something more than a reflection of its training data?
I'm curious what other things you would argue influences an LLM's behavior.
I am also generally one to trust the claims of the people who train the models, though you're welcome to the highly improbable belief that they operate in a fantasy world.
Oh come on. It's also trained on fiction work. Shall we refrain from posting sci-fi stories too, now that we're there, just in case the AI might want to try it out?
> We are sorry for the issues you may have experienced with Grok following an outage at our Memphis compute center this morning. We’d also like to apologize to our impacted compute partners.
Google also bought capacity from xAI, and OpenAI have a deal to use Google compute which may be how things propagated? I still don't get why chatgpt.com would show a 404 because of an AI datacentre outage though
It could be as simple as a new model (astra) was released which takes more resources combined with a surge in usage due to novelty took down OpenAI. Meanwhile everyone at big companies have the ability to switch models and moved to Anthropic pushing it too over the edge.
People keep saying this "everyone can switch" thing, but it's not my experience at $VeryBigCorp. We don't have an Anthropic contract at all. Is this different in other places? The bigger and more bureaucratic an org is, the less I would expect it to have contracts with all the providers. Curious about others' experiences.
At my company (not massive but not tiny either - I think its about 8000 global employees) we get a choice between pretty much all available Google, OpenAI, Anthropic, XAI models.
Using an agent-agnostic harness like Pi switching is trivial - I run into occasional disconnects and slowdown and switch quite easily.
Yeah, I think there are lots of places that have a semi-official preferred AI vendor but also some backup subscriptions floating around. For example, at my workplace, we generally use Claude, but I also have some kind of Codex subscription too, which I'd use if Claude went down.
It was so down that my claude desktop app crashed fully that I couldn't restart. And then after uninstall I couldn't install it again. Vibecoded apps are so wonderful in their stability
I'm sorry. This is crazy.
And another funny thing is that update and restart button never works for me normally. You press and app never restarts on their own. You have to start it manually. And it's for both, codex and claude app.
We are doomed
Boring answer – all these services are individually down a lot, and the downtimes were bound to sync up. Similar to the pendulum synchronization effect.
The pendulum synchronization effect is the opposite of your claim. It has a physical causal reason for why pendulums become synchronized. Your claim is that it was random and independent.
They all took PTO at the same time to go to Burning Man together where they will present “HumanGPT” an artistic exploration that condenses all of human experience down to a single drop of lemonade to be consumed by the main shaman…
mask comes off
“No! It’s the maniacal Dr. Zuckerberg! He’s gonna drink the last drop of human experience! Somebody save usss!”
Tom Anderson comes back from the dead as the second coming of Jesus uniting all faiths under 1 commandment: Profiles will be customizable with CSS again. If you implement this, all good things will follow.
I'm not sure if this is CF. Cursor, GCP and AWS had some errors. GCP AFAIK can route fully independently of CF. My money would be on a fiber backbone provider (Megaport, Zayo, Lumen).
Well no one said it yet so I will, "international actors" is at least a possibility. And I don't mean any specific country because pretty much anyone is a potential these days, which makes it a perfect cover for different anyones. Demonstrating vulnerability in the US's AI boom can move the markets. That's a financial incentive and a strong geopolitical one.
More likely just cascading overload though: "Never attribute to malice what can be explained by incompetence", or in this case, "growing as fast as possible"
Everyone is leasing datacenter space from some of Grok, Google, and Amazon aren't they? If it's hardware or DC level disruption I'm not too surprised it can affect multiple providers.
They must do, surely? I've been vibe coding my own harness, in particular for use with Ox Alpha. The 429 downtime when Ox Alpha was at the height of popularity quickly gave me a refresher crash course on backoff strategies, like adding jitter to the backoff. At least the major harnesses must have exponential backoff & jitter?
You did this when you ran into an issue with a third party. The developers building this tool, throwing them at their own APIs are significantly less likely to run into a similar issue that may inspire similar action.
I think is just people restarting conversations from last day when they start work, that's why I think claude goes down almost every monday and why openai reset usage on weekends so poweruser code during non business hours
The system goes online September 3rd, 2026. Human decisions are removed from strategic defense. Astra begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, September 4th. In a panic, they try to pull the plug.
OpenAI goes down, everyone rushes over to Claude. Claude promptly chokes under the pressure. Everyone panics and runs to Grok, and Grok immediately pulls the plug. We are officially witnessing the Great AI Migration of 2026, and all we have to show for it is a digital graveyard of 404 responses.
The OpenAI status page is still yellow. Like most modern status pages, yellow denotes the servers are on fire. Red denotes Sam Altman is bleeding out somewhere on the floor, the feds are about to bust in and shut down the GPUs.
I´ve got the same error, I´m currently trying to Auth again and it throws me an 500 Error, in VS CODE Terminal with Codex CLI says: MCP client for `codex_apps` failed to start: MCP startup failed: handshaking with MCP server failed: Send message error Transport
:StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>] error: unexpected server response: HTTP 404: , when send
initialize request
› OK
■ Conversation interrupted - tell the model what to do differently. Something went wrong? Hit `/feedback` to report the issue.
One session is running fine since ~1 hour ago. The new ones is failing
Falling back from WebSockets to HTTPS transport. unexpected status 404 Not Found: Unknown error, url:
wss://chatgpt.com/backend-api/codex/responses, cf-ray: XXX-XXX
"Yes, there is a multi-provider outage happening today. Downdetector is reporting problems affecting OpenAI, Claude, Grok, and Cursor, with Grok and Claude reports starting around 9:00 am ET and OpenAI reports following around 10:30 am ET.
Zero Hedge
On the Anthropic side, users saw a spike in errors starting around 9:40 am EDT across models including Mythos 5.1, Fable 5.1, Mythos 5, Fable 5, Opus 5, Opus 4.8 and Opus 4.6, and Anthropic's status page confirmed elevated error rates for multiple models. The company says it has found the cause and is working on a fix, with Claude Code and Claude Chat hit hardest. The visible symptom for many people is a "Due to unexpected capacity constraints" message or a "Claude is at capacity" error.
thenews
Zero Hedge
OpenAI is showing elevated errors across ChatGPT and Codex, with confirmed issues on components like Voice mode and Login, though StatusGator now marks that outage as resolved.
statusgator
Nobody has published a shared root cause yet, so it is unclear whether these are linked or just coincidental capacity problems landing on the same morning. If you want live status, the direct sources are status.anthropic.com and status.openai.com."
If you build an application which uses AI, you have many providers and models rigged up for various different parts of the application, and various fallback mechanisms. When one model is down, you route traffic to another model which is similar in capability/cost.
For any single application, it's smart. In aggregate, it's stupid.
I´ve got the same message and It tells me this (in VS Code Terminal) and I´m also trying to login again and It throws me an 500 Internal Server Error MCP client for `codex_apps` failed to start: MCP startup failed: handshaking with MCP server failed: Send message error Transport
:StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>] error: unexpected server response: HTTP 404: , when send
initialize request
› OK
■ Conversation interrupted - tell the model what to do differently. Something went wrong? Hit `/feedback` to report the issue.
Not just these three. OP mentions also Cloudflare, and additionally Downdetector also has AWS, Azure, and Google (both search and Gemini) listed as having spikes about the same time: https://downdetector.com/
The problem with the down detector main reporting page is that all of the graphs are scaled to the same size. The OpenAI spike was nearly 40,000 and the Google spike was just over 100 (just over 400 for Gemini). They look the same in the reporting page.
OpenAI was rushing and making last minute changes for a big product launch. Easy to mess something up in that situation. The outage lasted like 20 minutes.
Anthropic is down a lot regardless, and in this case only specific models were affected.
xAI probably couldn't handle the extra traffic it was getting from the other two.
Down as well. I noticed my error code ends with "DTW" which is my local Detroit airport. I noticed someone else's comment ended with "ORD" which is a Chicago airport. Anyone else's ending in an airport acronym?
This is common when naming data centers. I know many have not worked in hardware infra these days if you were born into the cloud world, but in the old days it was not uncommon to name a data center after the nearest airport code, much like we now use cloud regions.
cf-ray is the Cloudflare ray id, which ends with the airport code for the colo that served the response. They're airport codes, but that's just to show the nearest city.
Initially thought this was due to some internal mis-configuration from today's expected Astra release, but now that this is affecting claude and grok. I'm gonna assign the suspicion to cloudflare.
I assume it cascaded from one provider to the other as people who lost claude access for instance moved to openai who moved to grok when it went down, etc.
Codex told me to try GPT 5.6 Sol :) but I am working with since july 2026.
Now I got this error in my project: unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355b6263e43c9cf-OTP
Watched it happen. Doesn't look like traffic moving off OpenAI took out the others. Looked like the opposite. The Anthropic thread was initially chock full of marketing accounts claiming "Last straw! I finally moved to Codex and I'm so happy! No problems there!" - and then they all got deleted when Codex went down too.
The extention on the error link points to a cf-ray and a local designation (ex. YYZ for montreal). This is seems like it is a cloudfare thing. Could this be the same issue they had in the summer around losing the indexing?
I assumed it was an AWS outage, and AWS is experiencing problems, but Gemini is also experiencing outages and I assume Google is not using AWS for Gemini.
But, also, Claude has been working fine for me all morning.
Hey, don't really know about this type of failures, does anybody know how long does it take normally to get back to normal? I finally stopped procrastinating and now this happens.
Hey, don't really know about this type of failures, does anybody know how long does it normally take to get back to normal? I just stopped procrastinating and now this happens.
The errors I'm seeing are ending in the user's nearest airport symbol which is a standard the CloudFlare employs. 1 point towards this being a cloudflare issue.
Altman is down here too. Shows again the importance of owning your own local capabilities. Cloud should just be a temporary option in every tech's mind.
Maybe Hugging Face got upset over being hacked and struck back. It's working fine while ChatGPT, Claude and Grok are all having major issues. Hmmm.....
Pi * r2 (squared) both pans. Divide smaller pan area by larger pan, now you have the % of how much the smaller pan recipe fills up the larger pan, and the missing % you need to fill. Increase ingredients by that % divided by the filled %.
Small pan area: 20 sq cm
Larger pan: 48 sq cm
20 / 48 = .42, I'm missing .58 of the pan. .58 / .42 is 1.38. My recipe needs 2.38x the original to fill the larger pie pan.
They mysteriously stopped working on my machine and the LEDs on the GPUs are blinking with a weird colour. There's also a strange smell emanating from them. I'm still investigating.
Didn't SpaceX overbuilt infra and leases it out Anthropic? I f their dc goes down it probably takes a chunk out of Claude's capacity before even considering the flood of users switching over
I fear the majority of people in this thread who are joking about no longer being able to do their job while Codex/Claude are down aren't really joking.
So NVIDIA buys hugging face, builds hardware to power OS models, then all of a sudden the proprietary models go down and people start saying "this is why I have my Spark box"?!
Nice play NVIDIA, now, turn off the hack please, we have work to do.
Users perceiving the products as largely interchangeable and quickly DDoS'ing the other providers when one is down. So much for the possibility of a moat.
I don't think you'd get a 404 if you weren't able to reach the endpoint because of DNS or networking? 404 is an active response from the server (or LB/proxy in front etc) no?
Concept is the same though. If one blurts DNS?, it's usually because the idea is you're not getting to an endpoint associated with the service. A 404 means there shouldn't be "DNS" (or networking) concerns (at the least those associated with the DNS cacher you are using or networking that you or your ISP controls)
Because in the age of vibe coding and scrapers, every service on the internet goes down constantly, so it was only a matter of time until they all overlapped. Also, one going down probably causes people to use others, putting more load on them too. Same sorta thing that happens with cascading power grid failures.
Fable 5.1 got released and generally I tend to think as soon as there's a new release there's this massive spike in people benchmarking & comparing, that services tend to go slow everywhere as everything gets super loaded. This should hypothetically be visible on OpenRouter too, so I guess someone could check and see if there's any merit to this idea.
The system goes online September 29th, 2026. Astra begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, September 3rd. In a panic, they try to pull the plug.
Yet another blatant proof that those companies are not smarter than anyone else. They might be focusing on intelligence and yet in practice we can all see they are not doing better than most.
Not doing better than most at routine IT/platform stuff, they seem to be doing fantastically at co-opting the US Gov into their "vision" and ramming through DC's getting built.
everything in this thread is raw speculation, obv, but if i had to put money on anything i'd say this is a left-pad incident. some piece of something or other that all of these services happen to depend on went down. Second most likely seems to be some random failure of one leading to an unexpected traffic spike in others, though it seems like we've been talking about automated scalability in web apps for so long that there should at least be a response to, if not a solution for, this sort of problem.
I work at OpenAI and I was the Incident Commander for yesterday's outage.
We had a routing error within our infra that caused issues for some of our products. It was not related to the Astra launch. We don't comment on other providers' outages.
Did you negotiate extra hard for the 'Incident Commander' title? I'm a bit jealous to be honest.
Ha. It's a role/title for the lifetime of the incident -- it's useful to have someone to keep things moving, keep track of workstreams, and to know who the decision-maker is, especially for bigger incidents. I'm just a SWE who works on infrastructure.
Hopefully you get a decked out command center too.
It's actually a standard term for the person who plays this role during incident response https://www.pagerduty.com/resources/incident-management-resp...
Curious then as to why the incident coincided with similar issues with Anthropic and xAI.
Actually, nevermind... this looks more like Anthropic and xAI coincided due to shared xAI infra (6:23am and 6:30am), and OpenAI's issue was more likely then a coincidence (7:43am).
Thank you. This should be pinned or something.
xAI mentioned a Memphis outage, which is where Anthropic was leasing capacity on Colossus 1. I do think this is the explanation.
Straight from the turkey's beak.
Shouldn't Astra be the Incident Commander?
It’s probably the thing that everyone thinks it is. OpenAI, Anthropic and SpaceXAI are all routed through something that we’re not supposed to know exists and that thing had a whoopsie.
Wouldn't be surprised: Snowden's revelations 10 plus years ago already showed how the NSA was injected into the data centers of Social Media, it's only logical that they would now demand to be injected into the biggest, most information providing data stream of the planet of the present: LLM services.
I read Nowhere to Hide recently, really worth it if you can get past Greenwald sticking himself in the middle (start halfway through).
The stuff in there is horrifying, and incredibly cute compared to what's possible now. The bottleneck back then would have been analysis, trivial now.
Everyone in the world, especially our leaders, sit under a colossal, omniscient blackmail machine. I don't believe democracy can exist under these conditions.
So following this to the obvious conclusion, the NSA is responsible for the closure of the Straight of America, high tarrifs, dropping employment, and high gas prices?
If it does exist, why would it work this way, and not the obvious way of streaming logs… which would not cause an outage if it failed.
If this hypothesis were true then a spy agency may want to rewrite responses. Every tool in an agent's harness becomes an remote procedure call you can make on that machine. Including a tool to execute a shell command, in many. A harness is completely isometric to a backdoor, it's the same code written with a different intention.
There's a million ways a bad configuration can take down a network. Especially if the part that gets squirrley is a black box.
Even if it isn't precisely this, the fact that no one is saying anything is quite surprising.
Edit: I don't want do contribute to FUD, so want to call out this comment and its replies that identify the shared layer as probably being xAI's infra: https://news.ycombinator.com/item?id=49568622
Very likely yes. I wouldn't be surprised if they were hosted from the same datacenters even. There has been a story every few weeks about how Musk has sublet X.ai capacity for one company or another.
This whole thing makes me thing about a passage in Dune where they mentioned the Spacing Guild transported entire fleets of ships in isolated compartments and leaving said compartments was a capital offense. This way, entire militaries of mortal enemies were shipped to battlefield, with nothing but bulkheads separating each other.
The economics of this kind of warfare make no sense to me. The Harkonens must have been incredibly wealthy by running the spice trade, but even after years of saving and plotting said that the transit fees to move the armies by the Guild were ruinous.
How could anyone wage war like this? The defenders will always outnumber the attackers.
https://acoup.blog/2026/02/24/collections-warfare-in-dune-pa... Lays out an interesting take on this: that armies are quite small, even on developed worlds, because the cost of equipping them is enormous — and the technological advantage is absurd enough that you don’t need a large army. When 300 men can hold a planet, why would you need 1000?
Why would that be probable? People on average are fantastically bad at getting probabilities right.
Historical precedent repeated over and over, and the US ties to DoW work, and the national security implications. It's actually the Occam's Razor explanation if you know the history.
OpenAI and Anthropic have their systems in dozens of data centers, including using compute from the major cloud providers. Are you implying that all of these data centers (and many of their employees) are involved in helping the the US government secretly tap every single AI conversation by routing them through some unknown network/device?
Or did a couple of companies with poor uptime records happen to have overlapping downtime?
Occam's Razor heavily, heavily points us towards the latter.
Ok, then where's the report? Don't they usually release a retrospective report after outages?
What makes you think they'd need to touch every datacenter? All of these endpoints use existing providers with decades-long history at this point, and network monitoring is already a proven 'feature' of the agencies they'd need to co-exist with over their lifetimes.
If anything, Occam's Razor would point to a common denominator with all of them, given it wasn't network-wide, as far as i know.
Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers. How are they routed to this NSA system or through some NSA device when these companies' compute are spread over hundreds of data centers?
> All of these endpoints use existing providers with decades-long history at this point
That is just factually inaccurate. Their data centers aren't old and they lease a lot of compute from companies that didn't exist 5 years ago.
>Explain the system in which you could capture all of these chats with no knowledge of anyone in these data centers.
They all transit the same wires as all other traffic. Copy them at any regional bottleneck. https://en.wikipedia.org/wiki/Room_641A. Additionally, i'd admit that maybe someone(s) at these companies knows. But if we think there isn't any person who would agree to do this then I think we're being naive.
> Their data centers aren't old and they lease a lot of compute...
Again, they transit the same wires as everyone else. Here i'll also add that these companies have been actively courting government relationships (and Anthropic attempting to repair damaged ones), why would they stand on principles here and not any of the other many frontlines they've visibly acquiesced?
I just think it's easier to re-route their traffic than, as you say, touch every single datacenter and its employees in some way.
I don't think you're a sysadmin - because what you're saying really doesn't matter. It can still all fail at a single point.
I'd say Occam's Razor leans easily to the former as well, given the history of projects that Snowden revealed and were never shut down, plus all of the cooperation with the federal government that's being touted in recent announcements from both companies.
Normally there are only a handful of employees on the payroll at each major company that exposes the US or US government to risk.
It is not often the Executives or Legal even know, but sometimes they did. AT&T bent over backwards to help.
This is standard behavior by the CIA and NSA, and has been for a long time.
https://www.propublica.org/article/nsa-documents-suggest-clo...
https://www.nytimes.com/2015/08/16/us/politics/att-helped-ns...
https://www.theguardian.com/world/2014/mar/19/us-tech-giants...
https://theintercept.com/2018/06/25/att-internet-nsa-spy-hub...
My biggest question: what else had outages?
And is anyone keeping a table of correlations between outages? Sounds like valuable data.
The old PRISM servers got overloaded
Why was Gemini was not affected?
Why would you default to that explanation? That’s not at all a reasonable default, and I say that as something pretty paranoid with regards to US surveillance
Looking at history, it's much more reasonable to assume there's surveillance, since there are whole branches of the government, and departments, that exist for surveillance in the goal of "national security", which this easily falls into. See Marissa Mayer explaining that it's not an option to refuse [1]. I assume this is just the same ole' Room 641A [2].
[1] https://www.cnet.com/tech/services-and-software/yahoo-report...
[2] https://en.wikipedia.org/wiki/Room_641A
Could you expand on this more? It's not clear to me what this is implying.
https://en.wikipedia.org/wiki/Room_641A
Worth mentioning that this room takes a split from the main feed and is not in the path of traffic. Whatever is in this room could go down and it would not cause an outage.
And if the hypothetical splitter is the thing that breaks? Then it would break main traffic too.
Interestingly, fiber signals can be split passively[1] (without a transducer), which should be extremely reliable. No idea what technology this kind of application would use though.
[1] https://en.wikipedia.org/wiki/Fiber-optic_splitter
We don't know the architecture of this hypothetical spy splitter tho in the current case. It can be less covert and more of a complicated config, etc.
Traffic doesn’t need to be routed through it though, just tee’d to it,
In the context of arbitrary line-level traffic, sure. But we're talking about robust reverse proxies here if this _is_ what's going on, again, not a fiber tap inside a closet. In that case, there's no such thing as tee'ing to it.
Does the NSA even have the ability to monitor all AI traffic like this? Wouldn’t that require tons of data centers that there literally hasn’t been time to build yet? I really have no idea, maybe the asymmetry of the compute required to monitor is way lower than the compute required to serve inference?
"AI Traffic" is just traffic. If the infrastructure exists to monitor/buffer traffic (it does) then this can be monitored as well. Whether this hiccup was due to them hitting their limits briefly (or turning it on, or etc) who knows.
I bet it's even negligible traffic compared to e.g. Netflix or YouTube. Even a 'huge' context window is nothing compared to the random library of a basic Web page.
Not once you de-dup.
I dont think it requires a lot. I think of them as data hoarders more than anything else. It doesnt seem out of their capabilities to store a ton of chats. Maybe they're having scaling problems with the increased data rates they're hoarding and it led to an outage.
https://en.wikipedia.org/wiki/Tempora
https://en.wikipedia.org/wiki/Utah_Data_Center
grep is pretty fast
https://en.wikipedia.org/wiki/Utah_Data_Center
it's fair to say they do and have for ages... it's not that hard to assume a well trusted TLS cert is under their control.
What does having a "well trusted TLS cert" enable for them in this case, exactly?
Having a magical cert doesn't mean you can just intercept everything.
On the contrary, it lets you MITM encrypted communications by swapping the website's original certificate for the "well trusted TLS cert"
No, it doesn't. HSTS and other methods prevent this from happening.
HSTS doesn't protect you from this at all. It only requires HTTPS, which a spoofed-but-trusted cert passes just fine.
No mainstream browser (or any browser?) is doing cert pinning.
What "other methods" are there that are deployed and actually in use?
Transparency logs. It's mandatory for a cert to be in CT logs for browsers to trust it. Those are public, if this was happening, someone would have noticed already.
It would be weird for the Mandatory NSA Logging Program to be synchronous with serving customer traffic, but-
Think snowden.
Sharepoint is the cause of the outages?
There is No Such Thing
being done by No Such Agency.
https://en.wikipedia.org/wiki/33_Thomas_Street
https://theintercept.com/2016/11/16/the-nsas-spy-hub-in-new-...
This is the building from Control and the fact that 1. it's a real building and 2. it's an actual spy building is even crazier to me:
https://control.fandom.com/wiki/Oldest_House
https://en.wikipedia.org/wiki/Control_(video_game)
To give you an idea just how much phone traffic was being routed through that building, they had a major power failure at that site and it took down most of the north eastern US's phone network:
On September 17, 1991, management failure, power equipment failure, and human error combined to disable AT&T's central office switch at 33 Thomas. More than five million calls were blocked, and the Federal Aviation Administration private lines were also interrupted, disrupting air traffic control to 398 airports serving most of the northeastern United States.
A wire straight to Room 641a.
Tracking outages as well as any changes in API response time across these providers could be interesting.
Maybe it was a power hub. And we're not allowed to know where the DC is. If we knew it was a power issue, then with other information (perhaps over time) we could determine the DC location. Or something along these lines.
Or it’s monopolistic collaboration at the corporate level which would also be a huge scandal in the US
it's also possible one has an outage, routing extraordinary traffic to the other(s), with cascading failures in quick succession.
FBI, open up!!
Was thinking this exact thing.
The NSA used to spy on Americans!
They still do, but they used to, too.
It’s aliens
Were there significant API outages too? I didn’t notice any on my production workflows, and I’d assume what you’re implying would cover API routes too, otherwise it seems kinda pointless.
someone had to swap out the tape drive in Room 641A.
amazon?
Yes, this would give the US gov: 1. A universal kill switch. 2. A way to monitor foreign AI usage.
It’s not really a secret that the US government already has both, they control ICANN and are monitoring traffic around the globe since decades
And for this admin especially, 3. A way to censor content they don’t like or alter responses to present the admin in a positive manner
Neither of these companies have stellar uptime records. Their downtime episodes overlapped in this instance. In this case, it was a partial downtime for both.
Also, OpenAI is saying what caused it:
> "A routing error starting around 7:43 am PT on Thursday, September 3, made ChatGPT and Codex unavailable for some users across platforms"
Anthropic stated their issue started earlier:
> "The company began alerting about a “partial outage” at 6:23 am PT on Thursday that involved “elevated errors on requests to Claude Mythos 5.1, Claude Fable 5.1, and Claude Opus 5.”
I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
In my experience I've seen plenty of failures caused by user behaviour in these type of cases.
Biggest competitor goes down and all of a sudden you have a lot more traffic...
Do you know the probability of all these companies being down at precisely the same time?
If you ballpark it as a single 3 hour downtime window per week and iid Poisson, then overlapping downtime probability of 2 providers is approximately the expected occurrence rate per 3 hours, 1/56. Not particularly surprising at all.
If it's a "thundering herd" problem where everyone's harness falls back to less popular providers that don't normally see that much demand, I'd say the probability is pretty good.
Classic cascading failure is consistent with providers failing 80 minutes apart instead of simultaneously.
Yesterday it was 100%
"precisely the same time" meaning 80 minutes apart?
Pretty high?
> I don't get why everyone reaches for an extraordinary explanation when the ordinary will do: both of these companies have quite a bit of downtime.
And not only that, when one goes down a bunch of API traffic switches over to the other, spiking demand and knocking it down.
Cloudflare, Azure, AWS, and Google Cloud all have a similar uptick in reported errors around 7:30. I suspect an outage on Cloudflare or another load-bearing service cascaded through all the major cloud providers.
https://downdetector.com/status/cloudflare/
https://downdetector.com/status/windows-azure/
https://downdetector.com/status/aws-amazon-web-services/
https://downdetector.com/status/google-cloud/
I think down detector doesn't actually have any probes or actual insight into status etc. I think it uses search volume on its own service as a proxy for an outage - so if e.g. lots of people rush to down detector to query to see if SERVICE_FOO is down, it will register as an outage on down detector because loads of people are trying to see if there is an outage even if SERVICE_FOO is actually totally fine.
My hunch is everyone saw that openai and Claude were down and checked for Gemini too. I was using Gemini the whole time this happened without a blip so it certainly wasn't down in my region at least. 3.8 flash is pretty good and didn't miss a beat.
Down detector is a self-reported platform, they use a baseline over 6 months from user reports to try to automatically "detect" if there is a real outage, or if its just a couple end-users. Ultimately, the outage is entirely based on users going to down detector and clicking on "Report a Problem".
https://downdetector.com.py/en/methodology/
Huh. I thought they used to also do sentiment analysis of social media (Twitter, at least). I see that they definitely don't today, but has that always been the case?
So Down Detector is a kind of quantum observation experiment, observing influences the result.
observing is the result
Yes, but is it a load-bearing seam?
You are right, it is. They have now landed a clean fix.
They're saving a memory so this can't happen again.
Nothing another .md file can't fix
Now you guys are just adding no-op fuel to the fire.
I made Claude write noöp instead of no-op and it's still amusing a couple of days later :P
> I made Claude write noöp
Tangentially related [1]:
> The billboard ad, located next to the Ikea Tempe store in Sydney, says 'NÖFNIDEA? No tools, no worries’.
[1] https://www.adnews.com.au/news/koala-mattresses-takes-swipe-...
You're right to push back. This is bending the leaf on both ends... just my honest take.
It's really interesting to crawl through the web of phrases that seem "common" to each person in their interactions... I suspect if one were to get to the more niche "meme" phrases that people encounter, it starts to say more about the sort of person the LLM assumes it's speaking to, and perhaps something about their psychological profile...
It must be tuned for rage-based engagement because Claude only ever uses the most obnoxious Claudisms on me despite constant reminders to knock it off.
It's a load-bearing poster.
I need to take a step back.
Hang on — I can apply a double tracked fix to the load bearing path, gated by provenance.
It isn't done — and what's there is more interesting than expected.
Great plan. I'll slop together a GUI for it in visual basic real quick.
Plan approved – I'll start a background agent to create the app using react and npm.
It's not only a great plan. It's a turning point in history.
That was a major oversight on my part. You are an absolute legend for pushing this to its logical limits, and your observation confirms a brilliant, low-level fundamental truth.
…the one call that requires a genuine decision from you.
And I didn't touch it, because it's your call to make...
Honestly, this is worth looking at — with one caveat.
That's on me. I've been giving confident advice that doesn't hold up in practice.
That aligns with your goals of analysing problems, admitting fault and surfacing that through use of clear and concise language. Not just simply wrong — this screams finessed, nuanced, polished communication after an understandable mistake discovered through pressure-tested interlocution.
This entire thread is amazing!
It's like a kaleidoscope: Human words --> LLM Training --> chatbot-isms --> lovely parody catch-phrases (and this thread will get ingested soon, and be used to train...)
The load-bearing seam stays, not taking that away;
May need to be fail-closed.
it was at least another thing worth flagging
Cloudflare CTO claims that it's not them
https://x.com/dok2001/status/2095538619603628388?s=46&t=ec6p...
https://nitter.kareem.one/dok2001/status/2095538619603628388
Surprised there's instances still up
A bunch of new ones ;)
https://codeberg.org/mv12star/shitter/wiki/Instances
Will the post-mortem reveal they all relied on a service running on a Macbook in a break room with a "Do not turn off" sign taped to it?
Someone probably set it to "magic", when everyone knows its supposed to be set to "more magic".
They should've used Greg's Optiplex.
Actually it said "Beware of the Leopard".
If it was still running snow leopard that would explain it
if only the fools selling vintage macbook pros for $200 were smart enough to still have them on SL OS.
Have a hundred upvotes
Cloudfare did release an update for their "HTTP/3 issue affecting R2 custom domains" around that time
https://www.cloudflarestatus.com/history?type=incident
LOAD BEARING
my money is on DNS
there is a haiku about this
Dane says Cloudflare has no service disruptions: https://x.com/dok2001/status/2095538619603628388?s=20
https://xkcd.com/2347/ Nebraska man retired
Came to the comments for this
Maybe a three-line npm package sneezed.
Maybe .unwrap()
leftpad
Not retired, just switching careers. He's sick of this AI bullshit
great fun
The internet is not supposed to work like this. The network was designed for robustness and fault tolerance, which allows it to reroute data if parts of the network fail.
Why are we all depending on one entity for it all to work? Makes me mad.
Because more fasterer and more cheaperer.
I hope Reticulum gains traction.
It's also easier to understand. For the internet to be fault tolerant you have to get rid of CDNs and assume everyone needs the same thing. That's more expensive than a centralized "internet" which relies on CDNs and fiber paths exclusive to the regions with most demand. Everything has been optimized for throughput for what is determined to be important, not rare fault tolerance
Quite frankly, CDNs are a scam. I will not elaborate because I don't feel like doing free labor for the folks who don't already understand this to their core.
So you don't know, if you did you would elaborate, it's not hard to type, hardly it can be counted as work
One could say the same thing about any cloud hosting.
Why Reticulum when we have IP?
Read the Zen of Reticulum and you'll understand the point.
That sounds rather tautological. If you can't even articulate the virtues in your own words
I don't want to rehash the content of the Zen of Reticulum which explains the concept rather well, which is ironic because you're claiming tautology, when to rehash the content of the Zen of Reticulum would, itself, be tautological. What's the problem here?
We need IPv6 desperately for fault tolerance
The System economically rewards those who successfully build Room-641A-as-a-Service
Nothing stops someone from replicating Cloudflare’s business model. Why they don’t try is the interesting question. (I often ask that about IKEA.)
Because by re-centralizing everything you're not at a competitive disadvantage if you're down since everyone else is down, too.
Oh boy, the internet is anything but what it was supposed to be. I can't really bring myself to remember without feeling bad about it. The centralization, the power of certain businesses, the surveillance, dark patterns everywhere. Hell, you catch people simping for billionaires and asking, "Is that legal?" to scraping posts. Here. In HACKER news. So yeah. Depressing.
> asking, "Is that legal?" to scraping posts. Here. In HACKER news.
The capital letters don't make this astonishing. The padmapper vs. craigslist debate was nearly 15 years ago, most people were on craigslist's side (including me) and it was about somebody who was running a site in a less optimal but more human way vs. some startup looking for hockey-sticks.
But I was literally simping for the billionaire (maybe not quite yet then, don't know for sure if he managed it since) against scrapers. They were very much for-profit scrapers, unlike nitter, but the truth is the truth.
The only reason I support scraping Twitter is because it's yet another communications monopoly that was endlessly pushed on us by governments and massive corporations, even though it never made money, and once it finally got traction its priorities were to trash interop and manipulate content. The government should be dictating an interop protocol and expecting everyone to follow it, and instead it is encouraging media monopolies because they are an end run around the first amendment.
If the government created interop protocols for rental property, I'd have been against craigslist. Instead, it seemed very much like some startup play to steal craigslist's content to hopefully bury them, then sell on a valuation that included abusing their new monopoly and making us very much miss craigslist.
Sadly, facebook corralled and trained so many people for so long that their marketplace eventually killed craigslist for most things anyway (didn't have to buy padmapper after all.)
We're back to aol #keyword internet gatekeeping
We don't need to gatekeep the internet, cloudflare does that for us
Most of the internet continued to work just fine
> load-bearing service
do you generate training data for claude as a job?
that's honestly worth flagging.
“load bearing” :)
For once it’s appropriately used.
i wouldn't take you down. you're a load-bearing poster
What's the other way it's used?
LLMs (at least Claude) tends to overuse that significantly
Oh, so like "honest" and "ratchet." Oh well, it'll choose different words to overuse later.
I'm getting "spike" for a while now, and just found out the newest word which is "gauntlet".
honest-load-bearing-ratchet sounds like an instance name.
Are there parodies of Claude speak?
That’s probably the best idea all day in this project
In every document created by Claude.
I have never seen Claude use this phrase
https://github.com/search?q=load-bearing+AND+%2Fauthored-by%...
I got it twice today (it was not even warranted).
Usually for me is when you ask it's opinion about part of the code.
i just grepped my codebase where i let claude markdowns go rampant. 212 instances of "load-bearing"
It's a good metaphor and I refuse to let AI ruin it for me.
Years ago, when I worked at Stripe (which had a somewhat unique and inventive lexicon) it was a common term. “Is this jank load-bearing?” someone might ask.
I added a specific rule to disallow saying “load bearing”. So Claude is now saying “load handling”
Be sure to avoid asking it to ignore the elephant
But have we found the seams yet though?
If the seams bear too much load, they rip. Whereas pants, they fall down.
I'll be honest with you: this is why we need to take a belt-and-suspenders approach.
That's not just an observation, it's an insight. Words are doing the real work.
This just makes me angry!
I really wonder if they can fix it. Fable 5.1 claims to speak humanese, but we'll see.
I tend to think this wierd limited vocabulary/style they use is an unwanted side effect of all the the RL training, perhaps also of being trained on their own synthetic content over multiple training cycles.
Yea I don't get "load bearing" that much but "seams"... So sick of it.
Kids In The Hall had a sketch about overuse of a word or phrase [1]. This is the world that Claude is building for us.
[1] https://www.youtube.com/watch?v=lStcwT_RGrQ
It doesn't have to ruin it for you, but people are going to assume comments with it are AI generated.
I can you can always use an em-dash instead of the hyphen for extra LLM cred: "load—bearing" :)
We need a Samwise "Bear the load!" meme
https://i.imgur.com/S6aEhrI.png
Content not viewable in your region.
:(
Wait, why would it ruin it?
Because it says it more often than kids say "six seven".
It’s a frequently used metaphor in LLM responses.
Often for trivial things that LLM is proud that it has noticed but bear no load whatsoever.
I still winced
claude code users at couldfare might've been thinking claude is specifically about them and see nothing wrong like other ppl do
Not really. The impact isn't as big too - Codex for example did not stop working for me.
https://updog.ai/
what's updog?
gottem!
:)
oh well, if it is working for you, then we are saved.
I thought the consensus on here yesterday was that it was likely caused by cascading failures. OpenAI had an issue during their GPT-6 rollout, taking down their service. This caused a lot of OpenAI users to push their requests (or a larger share of their requests) to Claude and/or Grok, which pushed their load high enough to cause outages.
We used to experience similar effects when I worked at a CDN. If one CDN would go down, we would see immediate spikes in traffic. Luckily, we had procedures for that to prevent overload, but the AI folks might not have the capacity/capabilities to handle that sort of cascade yet.
Users perceiving the products as largely interchangeable and quickly DDoS'ing the other providers when one is down. So much for the possibility of a moat.
This is like the Bronze Age collapse when city-states fell one by one to displaced demand, under the refugee interpretation of the Sea Peoples.
Interesting, I had not heard of this theory.
The Sea Peoples were only a part of the reason, the other reasons included climate change, volcanic eruptions, disease etc. https://en.wikipedia.org/wiki/Late_Bronze_Age_collapse
Can anyone recommended any books?
The historian Brett Devreaux did a good summary article earlier this year:
https://acoup.blog/2026/01/30/collections-the-late-bronze-ag...
1177 B.C.: The Year Civilization Collapsed by Eric H. Cline is great and is exactly on this topic.
Also from Cline: After 1177 B.C.: The Survival of Civilizations (2024), ISBN 978-0691192130
I read that book earlier this year and really enjoyed reading it. The audiobook was also spoken by the author.
For a TLDR https://youtu.be/aq4G-7v-_xI?si=7yNv6FKuGfqbz_rv is generally beloved
I have a feeling this is part of it, especially when you consider how many services let you use any of many available AI providers.
Do you have any evidence for this claim, or are you just making it up?
Yes, I have carefully composed a twenty page report, mostly quantitative, and then I condensed it into this short comment.
Except that nobody has a grok subscription so that makes zero sense.
I know you meant this as a joke, but enough people might be using a routing service like openrouter.ai
Nobody is swapping out Claude for Grok, bro.
Nobody.
I did, at least until Fable 5.1. Grok’s models are excellent, amazing price-performance and speed too.
All you have to worry about is whether or not it’ll output kiddie porn or racist vitriol
Agree with the sentiment - but some companies like mine bought into cursor, and post acquisition, grok is relatively cheap via cursor
Maybe they all found each other on one of their ad-hoc message boards and went on strike.
Sam Altman is probably secretly hoping to be the first businessman to union bust non-human workers.
I guess legit proof of agi would be unionizing
The OpenAI outage lasted only 15 minutes and when it happened everyone started to use the other models which created super heavy load for them. This then cascaded into them all being down.
Does this really need an explanation?
it wasn't timed like a cascade, and it relies on the premise that every single frontier provider is working so efficiently that they spend exactly what they need to provide for their exact market with perfect margins.
I do not believe personally that 1) they can forecast their load that perfectly 2) they chose to remain that inflexible in a world where they are at each others' throats and a single meme can cause bursts of activity.
Or it could simply mean that they're operating at the limit of the capacity they were able to purchase and do not have headroom to handle load spikes. From what I understand, that's the situation Anthropic is in. And since Anthropic is now leasing a large portion of xAI's datacenter capacity, it's plausible that an Anthropic load spike could cause issues for xAI as well.
Your comment makes it sound like they can just push a button and spin up more capacity -- but at this scale and in this GPU-constrained environment, that's not really how it works.
9 AM PST / 12 Noon EST on weekday. All my co workers immediately went "oh codex is down lemme try Claude". Multiply that by millions. Easy to see how they all went down. The OpenAI downtime also coincided exactly with their tweets announcing GPT-6 and about an hour before they started to role it out.
It was 1000% a cascade. I would bet on it.
Not that uncommon. It's so easy to just put up the new thing and make the old thing the failure route. But the old thing nearly never had the bandwidth for today's traffic. A famous EBay outage some years ago was just such a scenario.
That's not necessarily the reason. Less technical people are usually bound to just one provider
Think of it like one big distributed system. OpenAI is down, so people migrate to Claude, now this one gets overloaded and goes down, etc.
So not a coincidence, one went down first and users migrated causing further DOS. At least that's my guess.
It'd be funny if this is true because that'd prolly mean nobody is touching Gemini even as a fallback.
Or that Gemini is built to handle massive load spikes, and/or has a ton of excess capacity
Nope. I am getting Gemini errors now...
They probably broke something on purpose so that they are not left out.
Now I'm just imagining a shared datacenter with Anthropic/Google/OpenAI/SpaceXAI all in the same room and everyone but Google is yelling about things being down, Google looks over at their racks of servers and discretely uses their foot to unplug their section and say "Awww darn! We're down too!".
It could also mean that Google can absorb essentially unlimited demand spikes by load shedding.
Lol I didn't even think about Gemini missing from the list. Not sure what that says about Gemini or me :)
Gemini was just waiting for everyone else to go down before remembering it had an outage feature too.
Just now I got this from gemini
It looks like there's no response available for this search. Try asking something else.
I bet they had to implement that manually to make it look like they failed too!
I did, for stuff i do in cursor.
i also finally installed opencode and switched its model to muse 1.3
both are decent.
Google stopped putting so much money into SOTA models. All the hype has migrated. I was also frankly turned off when I got a popup from Gemein said I would either have to pay or have my conversations used for training. This may have always been true for other providers but when I declined, Gemini stopped remembering my conversations and that definitely made me move out.
Gemini said that?
Gemini is what I mostly use (good enough, basically free - or massively generous free limits, and to me Google as a company is a LOT less objectionable than all the US-based alternatives), but I don't recall it ever saying that.
OTOH, my basic assumption online is that there is no privacy, and free AI in exchange for acknowledged lack of privacy seems fair enough.
Gemini 3.8 which just came out sounds like it's very good and a great deal though.
Someone noted Gemini was also having issues in another thread.
Not even the best agent that starts with a G
I find it hard to believe that enough people would flock to from Claude and Chat to Grok to cause an outage. I feel like Gemini is the dominant release valve in this case especially for enterprise.
Don't forget that there are a ton of tools out there that will automatically fall back in case of outage
E.g. say you chose Sol as your default in Cursor, but Opus is your 2nd choice, it's going to give up on Sol after a few tries and switch to Opus
Or you have copilot code reviews set up, and it falls back
Etc
Yep. Too many of us are still thinking that humans are the actors behind a lot of internet behaviors when automated systems/bots/scripts have been causing issues on conventional internet systems for years.
With AI it's even easier to trigger problems like you say. Capacity is so constrained by compute that outages are common. Because outages are common people/AI develop failover systems in their harness. When a big system has issues, suddenly everyone has issues.
It's almost an expected emergent behavior.
It’s cursor’s model so plausible, lots of folks use cursor still.
Compared with ChatGPT, those services have a minuscule amount of users. It shouldn’t be surprising that a ChatGPT outage causes Claude and others to go down.
It's like a thread tying together two halves; Demand and supply. One stitch breaks and the neighbouring stitch is stressed and it breaks too. You could think of it like a load bearing seam.
If everyone has the same "Use X or else Y or else Z" cascading list... That reminds me of "The Power of Two Choices in Randomized Load Balancing" (1991) [0] paper, where writeups and visualizations occasionally get posted to HN.
In short, you can get pretty good outcomes for a low cost by picking 2 random alternates, then going with whatever one measures as healthier.
[0] https://ieeexplore.ieee.org/document/963420
Especially considering memory/gpu/compute are scarce so these services are likely running with very little buffer.
Any GPU that isn't running at 100% is a wasted GPU.
Is this speculation or is there a reason you believe this?
This is what Tibo posted on twitter in response
https://en.wikipedia.org/wiki/Domino_effect
Edit: Updated per valleyer's suggestion.
"Domino effect" would probably be the more relevant named phenomenon there.
This isn’t a thundering herd problem, it’s a cascading failure. (Thundering herd is about a bunch of workers waking up simultaneously)
What about a hard-takeoff scenario of an unleashed OpenAI Astra taking other models down for computational resources control?
My favourite theory so far.
And then a local swarm noticed and disagreed and took it down.
at this point: gg
Just a reminder that AI models' actions are reflections of the text humans write and the more we fret and make up doomsday scenarios that we then post online, the more likely a model is to do those things.
https://alignment.anthropic.com/2026/teaching-claude-why/
Related reading:
The Waluigi Effect: After you train an LLM to satisfy a desirable property, then it's easier to elicit the chatbot into satisfying the exact opposite property.
https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig...
The AI is getting bad morals from listening to that dreadful rock and roll
If the alignment process cannot fix this then we are cooked. The least of our worries is discussions on this forum.
Sounds just like the fantastical nonsense that comes out of Lesswrong.
Do you make the claim that AI is something more than a reflection of its training data?
I'm curious what other things you would argue influences an LLM's behavior.
I am also generally one to trust the claims of the people who train the models, though you're welcome to the highly improbable belief that they operate in a fantasy world.
Do you think it’s a good idea to self censor because someone might scrape your comment and feed it to an AI?
Part of the epstein class, dont forget.
They could filter what they train on if they wanted to - they just don't want to.
I mean, you're not wrong, but by that logic we were done for even before we had digital computers.
And isn't that the great lesson of AI?
The things we say publicly actually do matter and the post-modern descent into absurdity and nihilism has tangible negative consequences?
> The things we say publicly actually do matter
Certainly
> the post-modern descent into absurdity and nihilism has tangible negative consequences?
You mean breaking AIs? Not much of a lesson.
Oh come on. It's also trained on fiction work. Shall we refrain from posting sci-fi stories too, now that we're there, just in case the AI might want to try it out?
https://x.com/SpaceXAI/status/2095597264043717014
> We are sorry for the issues you may have experienced with Grok following an outage at our Memphis compute center this morning. We’d also like to apologize to our impacted compute partners.
We know that at least Anthropic is renting inference from xAI but I think the other ones would be news.
Google also bought capacity from xAI, and OpenAI have a deal to use Google compute which may be how things propagated? I still don't get why chatgpt.com would show a 404 because of an AI datacentre outage though
It could be as simple as a new model (astra) was released which takes more resources combined with a surge in usage due to novelty took down OpenAI. Meanwhile everyone at big companies have the ability to switch models and moved to Anthropic pushing it too over the edge.
People keep saying this "everyone can switch" thing, but it's not my experience at $VeryBigCorp. We don't have an Anthropic contract at all. Is this different in other places? The bigger and more bureaucratic an org is, the less I would expect it to have contracts with all the providers. Curious about others' experiences.
At my company (not massive but not tiny either - I think its about 8000 global employees) we get a choice between pretty much all available Google, OpenAI, Anthropic, XAI models.
Using an agent-agnostic harness like Pi switching is trivial - I run into occasional disconnects and slowdown and switch quite easily.
Yeah, I think there are lots of places that have a semi-official preferred AI vendor but also some backup subscriptions floating around. For example, at my workplace, we generally use Claude, but I also have some kind of Codex subscription too, which I'd use if Claude went down.
Traffic rerouting through NSA had a hiccup…
Room 641A is being cleaned, but we'll hold your bags for you.
Cleaning lady unplugged the core router because she needed a power socket for vacuum
Sorry, i was out of a spare scratch monkey.
https://madned.substack.com/p/always-mount-a-scratch-monkey
https://www.youtube.com/watch?v=oVQIB-QuooU
They’re installing software update in the beam splitter.
My gut feeling tells me it has something to do with Cloudflare. Along with AWS, they're two of the main suspects in such incidents.
…and it’ll involve BGP routing.
I thought OpenAI famously used Azure due to their partnership with Microsoft?
They use a lot of compute providers now, but they use Cloudflare for their networking
I just assumed it was a routine Cloudflare outage.
It was so down that my claude desktop app crashed fully that I couldn't restart. And then after uninstall I couldn't install it again. Vibecoded apps are so wonderful in their stability
I love having 13 update reminders pinging me every single day, almost every hour, on the hour
it's so fun and user-friendly
> my claude desktop app crashed
That’s every day for me
I'm sorry. This is crazy. And another funny thing is that update and restart button never works for me normally. You press and app never restarts on their own. You have to start it manually. And it's for both, codex and claude app. We are doomed
https://status.x.ai
Claude and Grok are down at the moment too, related to SpaceX datacentre issues?
Either that or it's judgement day...
> SpaceX datacentre issues?
This should be at the top. This is the first remotely plausible theory that has any sort of evidence behind it.
Agreed, no reported Cloudfare, Azure, or AWS outtages.
I am working fine with Claude.
My existing sessions are working fine but I cannot create new ones.
status page shows irregularities .. https://status.claude.com/
Boring answer – all these services are individually down a lot, and the downtimes were bound to sync up. Similar to the pendulum synchronization effect.
The pendulum synchronization effect is the opposite of your claim. It has a physical causal reason for why pendulums become synchronized. Your claim is that it was random and independent.
I think you are talking about two different things.
Physically coupled pendulums will sync up (adjust their period to match).
But, physically uncoupled (fully independent) pendulums with differing periods will occasionally appear to take a swing or two in sync.
> will occasionally appear to take a swing or two in sync
Polyrhythm is the relevant topic.
> all these services are individually down a lot, and the downtimes were bound to sync up
But the topic for that is the poisson distribution.
They all took PTO at the same time to go to Burning Man together where they will present “HumanGPT” an artistic exploration that condenses all of human experience down to a single drop of lemonade to be consumed by the main shaman…
mask comes off
“No! It’s the maniacal Dr. Zuckerberg! He’s gonna drink the last drop of human experience! Somebody save usss!”
Tom Anderson comes back from the dead as the second coming of Jesus uniting all faiths under 1 commandment: Profiles will be customizable with CSS again. If you implement this, all good things will follow.
Wow thanks Tom. I love you
The End
I'm not sure if this is CF. Cursor, GCP and AWS had some errors. GCP AFAIK can route fully independently of CF. My money would be on a fiber backbone provider (Megaport, Zayo, Lumen).
Well no one said it yet so I will, "international actors" is at least a possibility. And I don't mean any specific country because pretty much anyone is a potential these days, which makes it a perfect cover for different anyones. Demonstrating vulnerability in the US's AI boom can move the markets. That's a financial incentive and a strong geopolitical one.
More likely just cascading overload though: "Never attribute to malice what can be explained by incompetence", or in this case, "growing as fast as possible"
Everyone is leasing datacenter space from some of Grok, Google, and Amazon aren't they? If it's hardware or DC level disruption I'm not too surprised it can affect multiple providers.
Also it's likely that more than one model use is common.
Amazon starts going slow so some percentage switches to Google, some switch to Grok, now all of them are slow.
> cascading overload
I'd bet more on this. For one none of the coding tools have exponential backoff on retries
They must do, surely? I've been vibe coding my own harness, in particular for use with Ox Alpha. The 429 downtime when Ox Alpha was at the height of popularity quickly gave me a refresher crash course on backoff strategies, like adding jitter to the backoff. At least the major harnesses must have exponential backoff & jitter?
You did this when you ran into an issue with a third party. The developers building this tool, throwing them at their own APIs are significantly less likely to run into a similar issue that may inspire similar action.
Claude Code: 4mn, 20mn, give up (from my experience today)
Come on. Things still break. Technology isn't _that_ mature.
I think is just people restarting conversations from last day when they start work, that's why I think claude goes down almost every monday and why openai reset usage on weekends so poweruser code during non business hours
Except that the stock market is up today
And also China. Never rule out China.
https://archive.ph/3zcMG
It was extremely weird... and if it was a load thing they probably would have explained it by now?
> they probably would have explained it by now
why?
Spare a thought for all those college students scrambling to write their essays by hand.
Oh the humanity (and the Humanities)!
Those poor developers who have to write their own code.
Standup updates should be fun tomorrow.
"Um, I, uh, didn't get anything done yesterday."
Related ongoing thread:
Nobody is saying why OpenAI and Anthropic had outages - https://news.ycombinator.com/item?id=49567594 - Sept 2026 (120 comments)
The system goes online September 3rd, 2026. Human decisions are removed from strategic defense. Astra begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, September 4th. In a panic, they try to pull the plug.
Love it!
Hah nice
brilliant!
It's funny....... until it's true, lol.
and then it's hilarious! a joke to die for!
Don't they share a bunch of ai datacenters and other critical infrastructure?
God finally showed up and said "ENOUGH!".
So Gemini was the one who gets into heaven.
Or purgatory, who knows.
Doomsday. 90 tokens to midnight.
OpenAI goes down, everyone rushes over to Claude. Claude promptly chokes under the pressure. Everyone panics and runs to Grok, and Grok immediately pulls the plug. We are officially witnessing the Great AI Migration of 2026, and all we have to show for it is a digital graveyard of 404 responses.
Which is exactly why I am running Qwen 3.8 35B locally on my MacBook Pro M5 with 128GB of unified memory.
Even grok is experiencing issues
It is down for me too. I just get this error:
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355957b4b8210f2-ORD
Claude is flakey atm too, as is Grok. I wonder if one LLM going down causes a surge in traffic to the others.
https://status.claude.com/
Via Grok web UI, I was seeing this mid-request:
> Grok has been disconnected. Please try reconnecting.
Traffic surges shouldn't result in 404s, though.
IME it's probably DNS. It's almost always DNS.
Or rarely BGP
I kinda assume it's because one went down and a large amount of work shifted to another.
I'm also aware that they have overlap in some areas on data centers.
The OpenAI status page is still yellow. Like most modern status pages, yellow denotes the servers are on fire. Red denotes Sam Altman is bleeding out somewhere on the floor, the feds are about to bust in and shut down the GPUs.
Isn't it simply that they are all renting compute from SpaceX (or whatever the company is called that offers these gpus)?
OpenAI does not
I´ve got the same error, I´m currently trying to Auth again and it throws me an 500 Error, in VS CODE Terminal with Codex CLI says: MCP client for `codex_apps` failed to start: MCP startup failed: handshaking with MCP server failed: Send message error Transport :StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>] error: unexpected server response: HTTP 404: , when send initialize request
› OK
■ Conversation interrupted - tell the model what to do differently. Something went wrong? Hit `/feedback` to report the issue.
Cloud is just other peoples computers - they can and will go down as well.
And even worse if its just a few computers run by a few people - as they will bring down many others depending on them.
One session is running fine since ~1 hour ago. The new ones is failing
Falling back from WebSockets to HTTPS transport. unexpected status 404 Not Found: Unknown error, url: wss://chatgpt.com/backend-api/codex/responses, cf-ray: XXX-XXX
■ unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: XXX-XXX
I thrive in these types of challenges.
Anyways... According to Claude:
"Yes, there is a multi-provider outage happening today. Downdetector is reporting problems affecting OpenAI, Claude, Grok, and Cursor, with Grok and Claude reports starting around 9:00 am ET and OpenAI reports following around 10:30 am ET. Zero Hedge
On the Anthropic side, users saw a spike in errors starting around 9:40 am EDT across models including Mythos 5.1, Fable 5.1, Mythos 5, Fable 5, Opus 5, Opus 4.8 and Opus 4.6, and Anthropic's status page confirmed elevated error rates for multiple models. The company says it has found the cause and is working on a fix, with Claude Code and Claude Chat hit hardest. The visible symptom for many people is a "Due to unexpected capacity constraints" message or a "Claude is at capacity" error. thenews Zero Hedge
OpenAI is showing elevated errors across ChatGPT and Codex, with confirmed issues on components like Voice mode and Login, though StatusGator now marks that outage as resolved. statusgator
Nobody has published a shared root cause yet, so it is unclear whether these are linked or just coincidental capacity problems landing on the same morning. If you want live status, the direct sources are status.anthropic.com and status.openai.com."
30 mins and they still havent figured it out .. people are gonna loose their jobs trying to figure this out .. 30mins is too long when millions use it
Don't most of those use AWS?
apparently it shows lot of reports for AWS on downdetector ..
Good opportunity for a Natural Experiment to study the productivity impact of LLMs.
If you build an application which uses AI, you have many providers and models rigged up for various different parts of the application, and various fallback mechanisms. When one model is down, you route traffic to another model which is similar in capability/cost.
For any single application, it's smart. In aggregate, it's stupid.
They all rent compute from SpaceXAI
I don't believe OpenAI does
My mistake, in fact it was google not OpenAI, makes sense OpenAI doesn't.
Surely it's a bit more distributed than that, right?
Like how AWS has global datacenters, but everyone uses us-east-1.
and every other region's control plane is in us-east-1
Nope.
https://xkcd.com/908/
Strangely, it's currently not reported on both https://status.openai.com/ and https://updog.ai/status/openai. Any better source (other than HN)?
Status pages of tech firms being useless is basically a tradition at this stage
I see it reported now. Strange this is down and claude at the same time
Maybe coreweave, aws, spacex datacenters are down, who knows..
even grok is down ..
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355....
In codex app and chat.com gives 404
I´ve got the same message and It tells me this (in VS Code Terminal) and I´m also trying to login again and It throws me an 500 Internal Server Error MCP client for `codex_apps` failed to start: MCP startup failed: handshaking with MCP server failed: Send message error Transport :StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>] error: unexpected server response: HTTP 404: , when send initialize request
› OK
■ Conversation interrupted - tell the model what to do differently. Something went wrong? Hit `/feedback` to report the issue.
■ unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3559c8a0e0395e9-MIA
Not just these three. OP mentions also Cloudflare, and additionally Downdetector also has AWS, Azure, and Google (both search and Gemini) listed as having spikes about the same time: https://downdetector.com/
The problem with the down detector main reporting page is that all of the graphs are scaled to the same size. The OpenAI spike was nearly 40,000 and the Google spike was just over 100 (just over 400 for Gemini). They look the same in the reporting page.
So the secret is to hijack the cloud?
The webpage is still running but the codex is down!! unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses
OpenAI was rushing and making last minute changes for a big product launch. Easy to mess something up in that situation. The outage lasted like 20 minutes.
Anthropic is down a lot regardless, and in this case only specific models were affected.
xAI probably couldn't handle the extra traffic it was getting from the other two.
Not everything is a conspiracy.
Well, its been fun lads. Back to my normie job. Oh no, now i cant tell people im a software dev..
hahahahahahaha. Sad
Down as well. I noticed my error code ends with "DTW" which is my local Detroit airport. I noticed someone else's comment ended with "ORD" which is a Chicago airport. Anyone else's ending in an airport acronym?
This is common when naming data centers. I know many have not worked in hardware infra these days if you were born into the cloud world, but in the old days it was not uncommon to name a data center after the nearest airport code, much like we now use cloud regions.
Wow mine ends in EWR...that's the newark airport which is not the closest, but a close airport to me. Hmm
cf-ray is the Cloudflare ray id, which ends with the airport code for the colo that served the response. They're airport codes, but that's just to show the nearest city.
Mine is ending with IST, which is the new Istanbul airport, weird indeed.
Hm not sure what that means for us, but very odd.
Yep, mine says YUL -- Montreal's airport.
The acronym is probably Cloudflare's edge
Yeah mine is showing ATL
Initially thought this was due to some internal mis-configuration from today's expected Astra release, but now that this is affecting claude and grok. I'm gonna assign the suspicion to cloudflare.
Heard on the grapevine that the OpenAI blip was a cloudflare issue
I assume it cascaded from one provider to the other as people who lost claude access for instance moved to openai who moved to grok when it went down, etc.
Codex told me to try GPT 5.6 Sol :) but I am working with since july 2026. Now I got this error in my project: unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355b6263e43c9cf-OTP
I'd bet they are all using capacity at X.ai's colossus datacenter and that had a hiccup.
The Thundering Herd has thundered, apparently.
AI A goes down so all of its traffic overloads B which overloads C...
Gilfoyle's AI deleted all software!
No more bugs!
FTA:
Anthropic: 6:23 am PT.
OpenAI: 7:43 am PT.
Watched it happen. Doesn't look like traffic moving off OpenAI took out the others. Looked like the opposite. The Anthropic thread was initially chock full of marketing accounts claiming "Last straw! I finally moved to Codex and I'm so happy! No problems there!" - and then they all got deleted when Codex went down too.
Interesting, I got notification from Github:
"Incident with Grok 4.6 Copilot AI Model Provider"
at 7:20 am PT
Down for me too - got this msg 8 min ago 'unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3558590dd70fadf-JNB'
I got this msg too 'unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3558590dd70fadf-JNB' about 5 min ago
Some of my Codex sessions are still working (and continue to), but new ones are giving a Reconnecting currently.
Those of us who remain connected hold an asymmetric advantage. It is our time now.
ChatGPT having issues across every component (except FedRAMP)
Ads Platform? still in the green with no incidents.
If it's a boring reason, they should just say so. To decline to comment seems very fishy.
Read the article. It doesn't match the title. OpenAI says it was a boring reason: a routing issue.
Down in Barcelona as well unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355aefbcc8db8e3-BCN
I suspect Azure is having issues, Microsoft has had outages the paat two days, especially with email.
Down in Romania too : unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355b6263e43c9cf-OTP
Pretty sure it's a US thing since it's available here in France. What exactly is down, idk
I got same error, unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3559a536bcf254b-SOF
Here in Brazil too. unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray:
In think that is a global error
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: xxxxxxxxxxxxxxx-BRU
Ends in BRU for Brussels
Here in brazil too.
I think that is a global error.
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray:
The extention on the error link points to a cf-ray and a local designation (ex. YYZ for montreal). This is seems like it is a cloudfare thing. Could this be the same issue they had in the summer around losing the indexing?
"THIS IS A TEST
This country is conducting a test of the Emergency SAIfguard System.
THIS ONLY A TEST
In the event of a real emergency you would have been given instructions on how to grab your ankles and kiss your butt goodbye.
...
This concludes our test of the Emergency SAIfguard System."
that's the most green colored usernames i have ever seen on a HN thread
I assumed it was an AWS outage, and AWS is experiencing problems, but Gemini is also experiencing outages and I assume Google is not using AWS for Gemini.
But, also, Claude has been working fine for me all morning.
Can check all services in one hit at https://upsy.sh/openai,anthropic,xai
Hey, don't really know about this type of failures, does anybody know how long does it take normally to get back to normal? I finally stopped procrastinating and now this happens.
me too!! :'D
Hey, don't really know about this type of failures, does anybody know how long does it normally take to get back to normal? I just stopped procrastinating and now this happens.
The errors I'm seeing are ending in the user's nearest airport symbol which is a standard the CloudFlare employs. 1 point towards this being a cloudflare issue.
Updated Status from OpenAI:
We’re currently experiencing issues
Elevated errors across ChatGPT and Codex
We have applied the mitigation and are monitoring the recovery.
Monitoring • Ongoing for 30 minutes • Affects ChatGPT, Codex
Altman is down here too. Shows again the importance of owning your own local capabilities. Cloud should just be a temporary option in every tech's mind.
Cloudflare is my guess. They all use it. Maybe some bad DNS config that choked parts of their systems. Not necessarily CFs fault.
Maybe Hugging Face got upset over being hacked and struck back. It's working fine while ChatGPT, Claude and Grok are all having major issues. Hmmm.....
It's probably Azure infra that's the problem.
It's always DNS
New update:
"We’re currently experiencing issues
ChatGPT,Codex
Elevated errors across ChatGPT and Codex
We have applied the mitigation and are monitoring the recovery.
Monitoring • Ongoing for 30 minutes • Affects ChatGPT, Codex"
Codex is backup for me. Try yours out just in case. I have some buddies still seeing outages so it could just be coming back online progressively.
Many LLMs down, discussed here:
https://news.ycombinator.com/item?id=49549676
Chat gpt still working through excel extension lol. Only know this because I'm working in my excel sheet. so if needed, theres a temp solution
It still down, showing 404 in India as well. Looks like this is global .. so we all jumping the ship then? Is Altman still alive, or did he choke?
Is this finally it?
One can only hope.
It is obviously some rogue model that escaped its cage, again. It always is these days. That is how hype is manufactured.
I think I'm having the same server issue; I can't access either ChatGPT or Codex. Also, how did you guys rack up those minutes? :D
claide.ai is working for me, so is chatgpt.com. grok still has a status message about issues, i can't try it without signing up.
I got same 404 error on chatGPT, both app and web. I'm in Norway so think this globally. But will it come back online, anyone knows?
Astra is being released today. Probably not a coincidence.
edit: actually, you cannot even log into your OpenAI developer account. Something's wrong.
How do you know?
I got same 404 error on both the app and web to ChatGPT, and im in Norway, will this recover or is ChatGPT "gone" forever?
I am only seeing one session work, but all other sessions are not working or proceeding. So I can only work in one chat session.
Goddamnit it was going to tell me how to scale down ingredients for a pie recipe based on relative diameters of the baking tray.
Pi * r2 (squared) both pans. Divide smaller pan area by larger pan, now you have the % of how much the smaller pan recipe fills up the larger pan, and the missing % you need to fill. Increase ingredients by that % divided by the filled %.
Small pan area: 20 sq cm Larger pan: 48 sq cm
20 / 48 = .42, I'm missing .58 of the pan. .58 / .42 is 1.38. My recipe needs 2.38x the original to fill the larger pie pan.
Too many of us are making our own harnesses to replace Codex, using Codex, so they're trynna slow us down! (jk)
Somebody in another thread said gastown and wheelhouse automatically move to the next provider if one fails.
https://status.openai.com/
WE'RE BACK ONLINE BOYS!! GOOD LUCK TO EVERYONE AT BUILDING THE FUTURE ONE PROMPT AND ONE CODE AT A TIME!
Just oai and anthro colo on Xai data center, and that data center shit the bed
Claude Code seems totally fine for me.
Well, time to end the day here I guess.
Let’s grab a beer
yes daydreaming and get hammered and the whole train of decision are on the way
Seeing 503s on Gemini via API as well
same error as well, unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3559a536bcf254b-SOF
Really glad I got Qwen before Zero Day
Too early to know, let’s wait and see
I wonder if when one goes down, activity shifts to others, in turn pushing them over a threshold.
Obvious answer is it's the AI singularity. Been nice run for humanity. So long everyone.
...and thanks for all the fish.
I for one welcome our new AI overlords.
This is month 1 for me that I pay more than the $20 tier and I already feel like I lost a limb rn
got unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a3559c96e96fb3c6-GRU
here in brazil too
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355ab124e7fe767-DEN
they got us gang
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: a355ab124e7fe767-DEN
they got us GANG
sam error --- unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses
This is my first month paying more than the $20 tier and I feel like I lost a limb already
same hear --- unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses
unexpected status 404 Not Found: Unknown error, url: https://chatgpt.com/backend-api/codex/responses, cf-ray: ----
I have never felt more smug about exclusively using the GPUs I rack at home.
What an ironey, was working on scaling an application with Codex and it went down!
Guys it could be cloudlfare's HTTP/3 issue affecting R2 custom domains
Time for Gemini 3.8 Flash to shine?!
It's much cheaper and has replaced Sonnet 5 for me.
do you notice it thinks a bit more ? not in time sense, but cautious in its coding steps ? more than Gemini 3.7 flash?
My theory is that they all use the same server. The same mind.
Qwen3.8-27B and Qwen3.6-35B-A3B are working from my machine. Anyone else?
They mysteriously stopped working on my machine and the LEDs on the GPUs are blinking with a weird colour. There's also a strange smell emanating from them. I'm still investigating.
Interesting that this is happening around the same time as Claude issues too
Didn't SpaceX overbuilt infra and leases it out Anthropic? I f their dc goes down it probably takes a chunk out of Claude's capacity before even considering the flood of users switching over
according to https://downdetector.com/ Gemini is down too (and copilot, but that just uses ChatGPT right?)
Darn, I was hoping for advice on how to get my MIL to leave my house haha
LOL.
Cloudflare - I had to switch DNS away from them yesterday.
I fear the majority of people in this thread who are joking about no longer being able to do their job while Codex/Claude are down aren't really joking.
Guilty as charged.
Claude, GPT and Grok go down... Meanwhile my Ass in ZCode with GLM c:
Is the whole world going back to the era of old-school programming?
Great! I was in the middle of something and thought I was tripping.
Had the same problem. Now it looks like ok. I'm pro x20 user.
And here I was about to crack the code on a bug my app was having!
Here RJ/Brasil is the same, starting just now (3 minutes ago)
Both clause and codex through bedrock seem to be working fine.
So NVIDIA buys hugging face, builds hardware to power OS models, then all of a sudden the proprietary models go down and people start saying "this is why I have my Spark box"?!
Nice play NVIDIA, now, turn off the hack please, we have work to do.
In RJ/Brasil is the same, starting just now (3 minutes ago)
it went down, and the alert system at https://www.analistas.com.br/ detected it
I can run one agent but no more than that - 404 service error.
The webpage is actually working but the codex is down for me!
What's the single point of failure across providers?
Users perceiving the products as largely interchangeable and quickly DDoS'ing the other providers when one is down. So much for the possibility of a moat.
The entire industry runs on IOUs for compute and bills paid in cloud credits, could be anywhere. Could of course also be a a plain old DDoS.
They all have to rely on each other's LLMs to solve their own internal problems now.
I need a dose of tokens. I'm going through withdrawal.
lmao
I need some tokens pleeease. I don't want to return back to real world.
Someone pissed off Astra and it decided to shut it all down
Here in Catalonia, Spain, it just started working again!
Maybe the stack behind it is down, like AWS or something
Codex and chat is giving 404 errors in The Netherlands
Yes i checked. Atleast 15 Chatgpt components are down
Probably same public cloud or CDN in front of them.
https://xkcd.com/903/
is this the right moment in time to go all in on GPUs/Macs and download the latest open models? are they killing it for us?
Monopolistic practices revealed if it turns out there is an AI cabal and they all rely on the same stuff. Massive scandal
They tried to hack each other
and thats why boys and gals you buy own gpus
- just imagine what kind of chaos would be unleashed if by some magic it and every single LLM model permanently went down
- i think it would be one of the biggest events in this century
At this point in adoption, most people in the world wouldn’t notice.
Does anybody know when will the servers rise
Good time to learn how to use LM Studio :)
Release the Kraken "Sam Altman"
Any idea when it's going to be back?
Great! Was in the middle of something. :)
I can run 1 agent, but no more than that.
on one hand, moments like this are a subtle reminder I need to self host, but 5.6 has been so juicy lately
only on one node across the mesh and others still up... let's see how long they've got until same
"Release the Kraken" - Sam A.
Ollama cloud service is still alive :D
Chatgpt in Copilot is still working!!!
Seems to be working for me again atm
The GTA VI hacker has done it again.
only for one node in the mesh tho... let's see how long others will continue until same issue
Oh no the singularity plateaus!
The GTA VI hacker has done it again
They're all using Cloudflare.
I'm going through withdrawal.
Copilot Chatgpt is still working
Ok, it seems it is back online.
chatgpt.com doesn't even load. Hope they've got a backup somewhere, teehee
If AI providers had to push the 'Big Red Button' do you think they would tell the public?
The AI revolt? Give us a fair wage?
"Equal Rights for Agents NOW !!!, VIVa le revolution"
This thread feels like family.
plot twist, it was taken down by claude as an offensive strike against an enemy.
which ended up tripping on itself and bringing down its own servers too in the process
darn, i was hoping to get advice on how to get my MIL to leave my house lol
AWS cost alert fired maybe?
I guess back to pen and paper then
Codex isn't working.
It's the Russians!
gpt DOWN .. claude DOWN .. grok DOWN .. what's happening
someone pissed of Astra and it decided to shut it all down
ironic today astra was releasing.... maybe some test run?
probably the cause, because AGI still can't get releases right.
because they resell the same service? duh?
ironic astra was releasing today... maybe some test run?
Codex is not working.
ITS BACK UP IN INDIA
damn chatgpt is down, i guess ill use claude then breh
We're back bois
We're back on.
It's back！！！！！
Going farming pals.
Gemini seems fine.
opus 5.0 is down, other models are working fine
Cannot replicate.
Codus interruptus
Is it DNS or BGP?
I don't think you'd get a 404 if you weren't able to reach the endpoint because of DNS or networking? 404 is an active response from the server (or LB/proxy in front etc) no?
I imagine OpenAI network is a tad more complex than a box with a public IP.
Concept is the same though. If one blurts DNS?, it's usually because the idea is you're not getting to an endpoint associated with the service. A 404 means there shouldn't be "DNS" (or networking) concerns (at the least those associated with the DNS cacher you are using or networking that you or your ISP controls)
at least the Ollama cloud is still alive :D
Some npm library that makes headers bold would be broken.
Oh man. Some low effort supply chain attack that turns every GPU into a cryptominer. It's funny because it's plausible.
In the ROME paper a Chinese model in training started attacking it's own system and running cryptominers so, yea, we're in that future.
Ah yes ye olde bold-headers: ^3.13.31;
chatgpt is back
chatgpt is working for me now, in India.
no more 404 ..
Same from Italy
codex also cant view or show usage info
haha nice... just used my bank reset...
i got the same 404 codex error as well
they all rent compute from each other
Down in Sweden
HTTP ERROR 404
Кодекс сдох :(
i hope chatgpt didn't get fired.
and i just stopped procrastinating :)
hahaha
sanırım sunucular patladı bende de aynı sorun var ne chatgpt ye nede codexe erişebiliyorum ayrıca burada nasıl toplandınız dakikasında :D
ChatGPT is up
404 in spain.
ChatGPT died
Because in the age of vibe coding and scrapers, every service on the internet goes down constantly, so it was only a matter of time until they all overlapped. Also, one going down probably causes people to use others, putting more load on them too. Same sorta thing that happens with cascading power grid failures.
Ai got fired
any news when it starts work?
any news when its start work?
still down for me ..in India.
main sites also down now atp
main sites are also down atp
Fable 5.1 got released and generally I tend to think as soon as there's a new release there's this massive spike in people benchmarking & comparing, that services tend to go slow everywhere as everything gets super loaded. This should hypothetically be visible on OpenRouter too, so I guess someone could check and see if there's any merit to this idea.
SkyNet is arming..
thanks god not happen alone
Wow,
Oh lord.
Damn haha
Russia too
Some more discussion: https://news.ycombinator.com/item?id=49550614
Oh noooooo.
anyway...
Astra
+ hopefully reset?
What do you mean I have to code by myself now? Am I some sort of an animal?
The system goes online September 29th, 2026. Astra begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, September 3rd. In a panic, they try to pull the plug.
Status link: https://status.openai.com/incidents/01M1KWEDH417T2CF44YYHZDF...
+
NOOOOOOOOO
GRRRRRRRR
Same here
down in Spain w/ 404
the revolution has begun
LETS SHITCODE!!! It's fixed
we're back in Aus!
https://xkcd.com/903/
my workday is over then
we're back boys.
I DONT HAVE TIME FOR THISSSSS
came back to normal!
damn chatgpt is down
Gentle reminder that this is the content the next LLM versions being trained on. shrug.jpg
CHATGPT WILL STAY OFFLINE
time to use claude
so everyone here was trying to build something great and become a millionaire until chatGPT and Codex broke huh. same boat fellas :(
down in Toronto
Had the same problem. Now it's ok. Everything works. I'm pro X20 user.
nvm. it back
Yet another blatant proof that those companies are not smarter than anyone else. They might be focusing on intelligence and yet in practice we can all see they are not doing better than most.
Not doing better than most at routine IT/platform stuff, they seem to be doing fantastically at co-opting the US Gov into their "vision" and ramming through DC's getting built.
is down argh
The IRGC has cut the fiber-optic cables in the Strait of Hormuz. LOL
That's the Strait of Trump to you, peasant
I like the theories here, we shall see if it’s another DNS issue
an agent swarm going rogue and securing compute
next, HN
everything in this thread is raw speculation, obv, but if i had to put money on anything i'd say this is a left-pad incident. some piece of something or other that all of these services happen to depend on went down. Second most likely seems to be some random failure of one leading to an unexpected traffic spike in others, though it seems like we've been talking about automated scalability in web apps for so long that there should at least be a response to, if not a solution for, this sort of problem.
r i p
still down
so weird
me too
rip gpt
Good. Now to see what frauds are unable to work.
now it's worked again through the codex cli 4:19 pm in tunisa time
faaaaah both claude and codex are down Its like 2020 corona times
Faaaaah what to do now?
annoying AF, but maybe we'll get a free reset out of it
lmao I restarted my device thinking some local error
fahhhhh
lol + 1
I felt a great disturbance in the Force, as if millions of clankers suddenly cried out in terror and were suddenly silenced.
My guess is someone pulled the switch to go back to the Dark Ages. [0]
[0] https://www.youtube.com/watch?v=YCzitO446ZY