In my interpretation, 躺平 (tang ping / "lay flat") and 摆烂 (bai lan / ..."let it rot" I guess) overlap in meanings in the "do nothing" aspect, but in the tang ping / lay flat case, it is out of learned helplessness / nihilism, while bai lan / let it rot case, it is out of purposeful negligence / noncompliance, which suits geohot's vibe
I was hoping for more analysis here :\
like from a technical standpoint, why is the doom and gloom incorrect? I've heard some really interesting arguments for that, and non of them are here.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
When the U.S. first legalized pharmaceutical ads on TV, one of the requirements was that the drug companies had to disclose side effects during the ads. The companies chafed at that, thinking it would make the drugs less appealing, and only included the bare minimum the rules required them to. But their own data eventually showed that consumers actually responded positively to hearing the side effects. They found that people think if a drug is powerful, it must also have strong side effects. So instead of just scaring them, it reassured them the product was good. After learning that, the drug makers started including more side effects in the ads than they were required to.
> Looking at the ongoing Mythos export control issue, it's obviously not good for profits.
This is not obvious at all. Anthropic's biggest investor was the catalyst for this action in referring the "jailbreak" (if you can even call it that) to the government. Now, Anthropic is sitting around a table with government officials who are designing benchmarks that will determine what Anthropic's competitors will be allowed to build. At this point, I have no evidence to rule out the possibility that Anthropic's leadership purposefully sought this outcome.
> Why would a company warn that their technology is dangerous if it wasn't?
Because it makes their technology seem more important and powerful?
Do you usually believe 100% what a company says about itself?
> Mythos export control issue, it's obviously not good for profits
Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle
When people discuss the negative effects of climate change, do you think that boosts fossil fuel company valuations by making their products seem more important and powerful?
Fossil fuel companies do everything in their power to stop that discussion.
However, arms manufacturers are doing everything in their power to make you feel unsafe and weak and scared. That way, you are more likely to buy a gun.
Hell yes. More than a third of Americans would run their two and a half ton pickup trucks on burning copies of An Inconvenient Truth if it were at all possible
> Ban Chinese models because they’re “bad”. (can’t have head to head competition).
The current scale of current + planned data centres only makes sense if the US exports inference worldwide. 30% of your current total electrical generation capacity, and equivalent to 76% of your total electrical demand in 2022. At least, assuming the "33 GW installed, 300 GW more planned" claim I've heard was not itself an AI hallucination.
And at least OpenAI and Anthropic were persuing the same argumemts continuously to their initial founding, before such capture was plausible.
I don't only mean that GPT-2 was, famously, when OpenAI surprised the world with (paraphrased) "we think this is a good point to start practicing not releasing the weights, before models get too dangerous" or as a surprising number of people interpret it even when I link to OpenAI's own post "we think this is … too dangerous".
I mean even before that.
I wouldn't disagree if you say Musk is full of BS, but he did donate to OpenAI when it was created as a nonprofit, and his wealth was 2 orders of magnitude smaller than today. Back then he was talking about risks, "summoning the demon", now he's the one demanding saying "robot army" to justify a bigger compensation package from Tesla.
You could also turn this around: Why would a company put up warnings about their product, or at least more than they're legally obligated to. The tobacco and social media industries certainly didn't go "this product has the risk to do great harm, but we also believe in its potential, so we'll go with a responsible approach".
The only examples of companies voluntarily putting up warnings I can think of are those obviously tongue-in-cheek warnings "this book/game/series may cause addiction because it's just so good" etc.
1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.
2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.
The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
> in fact it does the opposite because of the need for government regulation.
For frontier leaders, regulation is a competitive moat. At trillion dollar scale, compliance teams, legal and lobbyists are a tiny fraction of opex. For emerging startups, early regulation of a new market is a substantial barrier to entry.
That said, I think Anthropic blundered into their current Fable mess unintentionally by underestimating just how much they pissed off the current administration by refusing to let the DoD use Ant's AI's for all the domestic spying and lethal combat ops they want.
I don't think the first position is steel manned by assuming those holding it actually believe there are no potential problems or ways to use the technology badly.
But that belief does not imply the doom rhetoric is necessary or the best way to approach those issues.
that's an inaccurate characterization on #1. he's not saying ai risk is _all_ hype to boost valuations, but that it's misused to serve purposes other than warning.
What's "exponential" about AI development? Model parameter counts? Anthropic doesn't publish those for their own models, last I checked. Datacenter buildouts? Water consumption per request? There just isn't enough evidence that AI smarts is growing all that much, once you account for the scaling of inputs. That's what OP probably means by "nonsensical hype".
Lousy benchmark, they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
Make your own then. It can go on the pile with all the others that keep getting saturated too fast to be useful.
> they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
What?
They made the benchmark last year, and included a bunch of models going back as far as 2019.
When they first announced it, the top end of their tests were things AI could not actually automate, and even now only does erratically. Examples of the tasks SOTA models are now saturating (at the 50% success level, not at 80%) include:
"Prune attention heads of a BERT language model while minimizing accuracy loss on text classification tasks."
"Implement a Python library for the ACE-OAuth standard that can generate and parse messages in CBOR format and encrypt/decrypt access tokens with COSE according to RFC specifications."
"Debug a PyTorch machine learning library with gradient calculation and memory optimization bugs until all tests pass."
"Finetune a large language model to reduce the accuracy of a truth detection probe while maintaining performance on standard benchmarks."
- https://arxiv.org/html/2503.17354v1
They're benchmarking against the time it takes humans to do the same things, which means everything they ask every AI to do must have also been done by a human.
>> Looking at the ongoing Mythos export control issue, it's obviously not good for profits
We must live in different planets. The whole episode about Mythos has been the absolute best free PR move ever designed. At this point everyone in tech assumes Anthropic and Mythos are the absolute best models. All of this got achieved without spending a single $$ in advertisement.
This model is also probably really dangerous, not denying that. But they definitely used it as a huge PR tool.
> The whole episode about Mythos has been the absolute best free PR move ever designed.
Great PR.
For a product nobody can pay for.
And a high chance that when anyone can, the market shrinks from "worldwide internet users" to "the USA only".
And it's known to be a highly dynamic market: based on the historical average, it only takes 11 weeks to be completely trounced by someone else's model, so if you're out of the market that long you may as well not have bothered.
A common line of logic would be that a company would never put a dangerous sticker on their product unless they absolutely have to by regulation. So if they do it, it absolutely means they see a benefit of doing it.
Also why do you think the Mythos thing is bad for profits? Anthropic hyped the model up too hard and didn't even have the compute to serve it properly to all users.
The export control issue has not affected Anthropic's revenue and only made its top of the line products seem far more impressive than they actually are. People don't think Nvidia is a bad company if the government puts export controls on their products.
For Anthropic, valuation and public perception is 100x more important before the IPO than revenue or profitablity.
>Why would a company warn that their technology is dangerous if it wasn't?
Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up.
If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
> If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
What else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.
Even here on HN, Tesla fans try to justify the market cap by pointing at Optimus, even though nobody can buy it and there's already competition for it which you can buy.
The positives aren’t there though, beyond what people are already using it for (pair programming aid, fancy auto complete, refactoring tool). Hence the article’s thesis: the doom justifies the valuation.
I remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.
I remember large banks that caused the failure to be bailed out. And talk of trickle down economics, except that average joe was not bailed outed, but punished.
I remember robo signing of home foreclosures with no checks and people loosing houses despite paying debts.
Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.
Anthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.
Regulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.
They did not wanted export controls, but they did wanted everyone so scared and thus pay for license. The decision backfiring does not imply it was not intended to be self serving.
If the company really think their product is dangerous, they should stop making it increasingly dangerous. Antromorphic claims is dangerous and that they need more money to make it even more dangerous.
And the fearmongering is beneficial for them so far.
> Why would a company warn that their technology is dangerous if it wasn't?
It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big.
The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial to walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's their sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent.
While I'm sure many of the top AI leadership are primarily (or purely) mercenary, I also think some, if not many, other execs and employees sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s>
So... yeah, that's why we see some weirdly whiplash messaging.
Even if Dario and Altman originally believed what they were saying, their scaretactics worked wonders for investment. Their companies are now incredibly incentivized to keep the AI Apocalypse narrative going further and further. It is hard to imagine them stopping, as that may lose them investment.
The last 10 years have been a decade of Big Tech Vaporware. NFTs and the Metaverse were assured to be the future. Once this narrative fails too (which I am almost certain is inevitable) I think society's love affair with Technology being the solver-of-all-things will finally fracture.
I think technology can actually solve a lot more things than it has, the problem is that foundational problem solving does not make a unicorn startup with earthshaking economies of scale in most cases, and especially so if there's a physical world piece involved, and even more so if it's going to go asymptotic.
Just to throw out a random example - could technology, in terms of advances and improvements to allocation, distribution, and consumption, play a part in solving the western US's water issues? Probably. But would it be something that make a trillion dollars and a household name? Maybe not. And could saner policy, like making farmers have to bid for acre feet instead of getting it for basically free + distribution cost also help? Sure.
Likewise, even with AI most software is still crap. Like when have you heard about a doctor who loves their EHR? Like never.
So I think technology could be a solver of at least a lot more things. But we've created a market where people want to exhaust every flavor of flim flam and trend of the moment first. Because we've glorified the business of tech more than the actual improvements and aspirations that should be possible.
It's significantly worse! All NFTs did was separate fools from their money, and crypto in general increased GPU costs. All the Metaverse did was cause Facebook to throw away a bunch of money (lol).
Meanwhile, AI has ruined the whole Internet and inflated the price of everything electronic.
AI is actually useful, in the strict sense. It can do things. If you consider its eventual effects good or bad for humanity is fair either way, but it's still different from NFTs or the "metaverse" nonsense. It's a categorically different thing.
Let's not forget that the inevitable AI crash will nerf even supposedely "diversified" broad market ETFs due to BigTech's insanely inflated valuations and the index gaming of the X, OAI, and Ant IPOs.
The whole internet AND local computing. This may be cool for people on SF salaries but for the vast majority of the world memory has become a luxury good
While it is driving new hardware costs up remember you can run Linux on a potato and just wait it out. My pixel 8a is a phenomenal computer if ignore the hype monster.
Specifically Anthropic's whole PR has all been about danger, safety, doomerism all to make themselves indirectly more important and central to the debate.
Calling meetings in Washington DC in order to let everyone know they made a cyberweapon is part of those PR moves. Then they seem surprised lawmakers actually called them out and asked to stop serving that model.
I know this is the cynical take but I cannot unsee the elephant in the room: This doomerism allowed Anthropic to be the center of every AI conversation right now. Their market cap and upcoming IPO is indirectly benefiting from this.
I also cannot take that Anthropic while letting everyone know that Doom is coming (or is already here), are also the ones that want to decide who can profit from this Doomerism. This is how every benevolent dictators start.
What's the evidence that the doom narrative is connected to valuations? It seems more like a marketing/recruiting strategy. As the post points out, institutional investors generally think the idea of mass unemployment is BS, and they are investing accordingly.
AI obviously has military applications, no doom required to think the military will want a lot of it. I'm asking about the extreme scenarios like "AI is gonna take everyone’s jobs" and the other "totalizing narratives" that geohot is referencing.
If this is true: "AI is gonna take everyone’s jobs" then AI is the terminal technology and everyone who doesn't want to be poor forever needs to throw all their money in it.
Companies say all sorts of things which investors don't take at face value, often for the sake of marketing and recruiting. Meta circa 2020 wanted us to believe we'd be swapping NFTs in a virtual mall and having all our meetings strapped into VR headsets by now. That doesn't mean their market cap at the time was based on an assumption they were right, in fact it is much higher now that the metaverse idea flopped. Unlike that situation where money was being spent on projects that turned out to be mostly pointless, AI labs don't have much of a financial stake in a scenario such as "lots of demand for AI, leading to mass unemployment and/or doomsday" as opposed to one like "lots of demand for AI, leading to the world being mostly the same as before but with more AI and expensive RAM".
I think when chinese models figure out how to partner with SOC 2 providers in the USA in a way that offers a $20-$100-$200 subscription is an easy convenient way, anthropic and open ai will be screwed.
If you look at Anthropic's blogs about their model timelines, there is roughly a ~3m period between a model being in internal preview until release. That means that inside of Anthropic, the next version beyond Mythos/Fable is already in preview, already being tested internally. Despite what Geohot is describing here about GLM, my understanding is that Anthropic employees have spent a significant amount of time grappling with a technology that is considerably ahead of what is available to the public today.
In addition, if you look at the graph of LOC written by Claude vs Ants (I.e. AI vs human), there is an incredibly sharp uptick post-Mythos internal preview. Something like from 30% to 75% of code inside the company being written by AI.
While I sympathize with the viewpoint here, I still have to admit that that there's a very different feeling to working inside of a company where they've had months of time with a model that's at the frontier, quickly changing the way everyone around them works, and that _they themselves_ control the keys to.
If Geohot had those keys, I can be 100% confident he'd be raising the alarm at the top of his lungs about it.
Claude code is a very small fraction of the code written by Anthropic and ultimately, despite being widely used, hardly dents Anthropic’s core priority of LLM performance.
It’s because Anthropic doesn’t publish any of its core AI research that we falsely believe that it isn’t by far the central focus of the majority of the team.
Just to be clear, I’m not supporting their stance nor defending the company. I consider it to be deeply harmful that a private company seeks to advocate for AI safety but then own all the means of production and profit financially from keeping its techniques secret. It’s as if the Manhattan project resulted in a for-profit company selling all atomic technology and deciding on its use.
I can't decide if this is a "techbro doesn't understand human factors" post or not, especially considering geohots reputation.
I do believe it is possible Anthropic are legitimately trying to start political discourse, but if they are they are either sacrificing themselves on purpose or shooting themselves in the foot. Others in the comments here are pointing out that there are many incentives for them to get into politics. Maybe they see possible outcomes worth the short term problems.
If it is just marketing and FUD, it's worth considering that a good lie is usually based in truth.
Let's say Anthropic were a "hippie organisation willing to sacrifice revenue for morals" (or pick your own, I'm just giving an example). Could they play the politics game better?
Journalism and public media in general needs to be a licensed profession with tangible standards, and malpractice suits should be pursued aggressively.
Would solve a lot of problems in the US, actually. Being financially incentivized to gleefully lie and spread misinformation at the expense of others should not be protected speech.
Simple minds want to believe one simple thing and then rationalize everything else to force consistency with that one simple idea.
If you want to believe the simple idea that AI is mostly hype, then you'll get stuck in a multi-year loop talking about stochastic parrots, ridiculous valuations, and doomer scaredycats.
But the real world isn't so simple. Multiple seemingly contradictory things can be true at the same time.
Some AI is useless. Some is incredibly powerful and useful even though it makes mistakes. Some companies are wildly overvalued. Some extremely large and expensive companies will quadruple from here. Some frightening scenarios will look silly in hindsight. Other frightening things will happen that none of the doomers foresee.
It would be great to explore those new ideas and possibilities.
It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
The question is not whether AI is useful. It is useful, full stop.
The question is whether AI is useful enough to justify valuations that dwarf the GDP of all but the top 20 countries in the world. As it stands right now, it's not even close! The leaked OpenAI financials put these AI companies in the same profitability territory as utilities, with zero justification for these crazy valuations.
> The question is not whether AI is useful. It is useful, full stop.
And nukes are useful by some metric too.
AI (llms) is not useful in any way that we as humanity should be pushing for. It’s more harmful than good in every way possible. It’s honestly astounding to me that everyone’s ethics are so weak as to believe that somehow these incredible destructive companies are somehow good for humanity.
Technically, it's the time discounted rate of future profits that determine valuations. If you take it as a given that they will provide exactly as much value as they do today, then, yes, your question has the obvious answer of no. If you think AI will provide greater value in the future, the answer depends on the value you think it will have.
Technically, it's the time discounted rate of future profits that determine valuations
Close, it's the time-discounted expectation of future returns. This seems related to future profits but it need not be. Historically, stocks tend to perform poorly after IPOs. There's no guarantee that (say) Anthropic's stock price would ever recover after a post-IPO drop.
The recent attempts by Anthropic et al. to circumvent the usual rules for inclusion in indices have raised red flags all over the place, with many calling it a naked attempt to raid everyone's pension funds for hundreds of billions in ill-gotten capital.
What you have written does not seem to be in close contact with the OP. He talks positively about the GLM news and whatnot. He is highly skeptical only of the "doom" scenarios, including upending most or a massive amount of jobs, and how that is deployed to keep the investment machine working at such breakneck pace.
> Some AI is useless. Some is incredibly powerful... It would be great to explore those new ideas and possibilities.
I agree, but it's not so mysterious what will win out. Even if the criticism is repeated so often, that's because much of it is still valid.
LLMs are not AGI. A statistical model of language helps fill in gaps. This is super useful for new and much improved UI/UX ideas that converge with better accessibility. Similar is true for generating images, video, audio, etc. There are situations where it's the right tool for expressing an idea.
What we need is a sense of maturity. The limitations are very clear to everyone now, and we're already past the disillusionment. If we can rein in the abuse, there should be a good path forward. The technology is already boring and that's a very good sign.
I love this website because people will write stuff like “I believe the hype because my brain is big, ‘believing the hype’ is actually very complex and cannot be embraced by lesser intellects”
There are reasonable anti- and pro-AI views. There are also unreasonable ones on both sides, and it's best if they are ignored.
However, on a site like HN or Reddit, you're far more likely to hear squawking about "stochastic parrots" or a rant about AI water usage than the mirror on the pro-AI side, making them harder to ignore.
It’s funny that a lot of what gets framed as pro- vs anti- AI arguments would be better described as past/present vs future arguments. If one guy is saying “that’s a stochastic parrot” and the second guy says “it’s going to achieve apotheosis and bring about global communism or feudalism or whatever while maximizing shareholder value” they’re not arguing about technology, they’re arguing about the second guy’s status as a clairvoyant.
You could post “fortune tellers used to be considered fraudsters and charlatans” and reasonably expect a “Get a load of the Luddite over here. Go raise a barn, Josiah!” response on the internet these days despite not mentioning AI or technology at all
> It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Yeah poor you this must be much more tiring than being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it. Sorry dude, hearing that people don't like that messaging and are replying with copypasta talking points must really be rough for you. Praying for you in these trying times.
Regardless of whether you believe in doom or not, it's this kind of wild, willful misunderstanding that makes people roll their eyes and dismiss you. Yes, people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
With the reality-bending financial incentives that these people have, I simply do not trust your or anyone else's assessment of the genuineness of their beliefs. And anyway, it's quite ironic that from godless San Francisco of all places, we are supposed to simply accept that the harmful acts of religious extremists are justified because, well, they do believe in their cause.
The point here is not the correctness of their beliefs. It's about what the actual content of their beliefs is. If someone says that Donald Trump is a secret Shia supremacist and using that to explain his actions, pointing out that that belief about Trump's belief is wrong is not a statement that Trump's beliefs are correct, but that your model of Trump's belief is incorrect.
Right, but it seems to me that you (and many people) are taking their stated beliefs at face value, when there are alternate interpretations of their conduct that are (a) highly plausible, (b) aligned with their incentives, and (c) less weird.
Personally, I do think some of the people in this field have really drunk the Kool-Aid and still believe in the paperclip monster. But for many others, I do not think the past couple of years of plateauing progress has escaped their notice. ESPECIALLY at the leadership level. I think they're larping and they know it. I just think they want the money and the power.
If I were building a technology that I thought posed a risk of exterminating the human race I would simply stop building it. But what do I know, I'm not a genius AI researcher.
> genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
"They are not in a thought bubble YOU are in the thought bubble"
No; I'm only stating that your model of reality is off, and I'm correcting it. No, the folks at the frontier labs do not believe they are trying to scam people with an NFT-tier fraud, and their warnings are not an attempt at marketing hype.
So to be clear, you're saying that ALL the people that you currently know of who work at these companies believe they are actively working on, in your words, something that "...poses a potential existential threat to humanity"
Oh nevermind I get it. They're only intentionally working on building something that they believe will end humanity. Well as long as it's not intentional crypto-tier scamming, it's all good then.
I'm convinced. Consider my model of reality corrected. Thanks homie.
Their usual stance is something along the lines of their company is creating AI correctly--not that AI will inherently destroy humanity--and that them working there helps that end. It's extremely reasonable to question their approach, but here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
>here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
Introducing parameters for the other person, then use those to call them crazy when they operate inside them. Actual DA-tier tactic, my guy. Yikes.
My initial reply was "being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it."
As in, we are constantly told that everything is going to be shit and there's nothing we can do, because there is a ton of moneyed interest in it.
You're the one that made the leap to "crypto" scamming in the reply and introduced the existential threat aspect. Both in the same comment, actually.
Then apparently tried to crazy-make me for (admittedly dickishly) pointing out that trying to build something that is an existential threat to humanity (your words, their belief, none of my words or belief) is actually worse than crypto scamming (again, both things which you introduced to the thread, btw).
But yeah shame on me for not taking you seriously and providing evidence I guess.
Odd how the bar for me is "providing evidence" but you're happy to outright lie that I was going from crypto scamming and existential threats in order to score a rhetorical point by claiming "AI psychosis" in a useless internet thread. Yeah I must be the unhinged one here, surely.
> This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
To quote you, ten minutes before:
> that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it.
It's a fair description of your stance. And, full of sound and fury, signifying nothing.
If they genuinely believe that, then they are literally a conspiracy to destroy the world. If their current actions are the outcome of genuine belief to what they say, then they are indeed massive "midwits" and a literal cult conspiracy. Due to the amounts of money
they siphoned, that would make them dangerous cult regardless of whether their ai predictions are correct.
Calling them scammers is literally giving them benefit of the doubt and trying to avoid conspiratorial thinking. You frame the belief that "these are normal people with normal motivation" to be paranoid conspiratorial thinking. Meanwhile, you claim they are literal conspiracy and cult and that is somehow less paranoid.
Now, dangerous cults and scammers are severely overlapping categories. Most if not all cults scam people. They lie to
get more powerful whenever it suits them. They have true believers (like your friends) fully willing to cause any amount of harm for the visionary in the center. These people may have started as a well meaning do-gooders, but they usually just end up being co-conspirators in crimes.
people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe
The same could have been said (and has been said) about other tech company employees for all sorts of other reasons in alignment with those companies' goals. Don't you remember how much people used to laugh at Tesla employees for worshipping Elon Musk as some kind of god of engineering and entrepreneurial genius? Or Apple employees in the Steve Jobs reality distortion field?
I would have thought at this point that it'd be well known that the employees of all cult-like tech companies can't be trusted to make a sober evaluation of their companies' justified valuation. We can talk about conflicts of interest and we'd barely be getting started! How about biased selection by hiring managers for the most fervent believers in the company's mission from the get-go?
Sure; they might be caught in their own hype cycle. I am pointing out that the repeated claims in this thread (and by Geohot, ironically enough) that they are NFT crypto bros running a scam is incorrect.
Doesn’t have to be a naked scam though. Dario could be caught in the dictator’s dilemma: he was a believer at first but now he’s on the tiger’s back so he dare not get off.
> Can someone write an AI 2027 but instead of some totalizing doom propaganda it talks about the bubble unwinding and what we can do to prevent this kind of crap in the future? How do we build an economy and society that’s sustainable?
I imagine tons of people have written that article. But no one reads it. They're all busy with the doomiest bullshit Facebook or Tiktok will serve them. It's what gets engagement.
No one is clicking on the "None of the things your scared of are scary and here's why" video when it's up against the "$x is the end of the world and will eat your children" agitprop.
There are a few possible explanations of Dario and Anthropic's behavior, if you are not sure if all they want is pump up the valuation like me:
- Anthropic has a cult-like culture. AI Safety is their religion. The AI Constitution is their bible. Dario is their cult leader. Employees are the apostles. They just really really believe their church, and only Dario is qualified to manage the AI safety.
- Asymmetrical risk. If Dario speaks optimistically about AI and he turns out to be wrong, he'd face the rage of many people. If he fans the doomerism of AI and he turns out to be wrong, at most he will be mocked.
- Regulatory capture. After all, pretty much all the AI big shots in Biden government went to Anthropic. They produced the Biden's regulation, and they made it clear that they wanted to pick a few winners to back.
HN is not immune to this. I do not take HN seriously, because empirically so many takes across so many subjects have been wrong in a melodramatic fashion and the “adults”, often people with training and first hand experience have to show up to set the subject matter right before someone gets hurt. But as a barometer of unhinged hype? Only X is comparable.
>>SF wants to come for your inner life and pimp you out and mediate every interaction and there’s not even a so.
I have been saying this for a while. I visited SF a couple months ago and god do people feel empty from the inside. Everything is revolving around AI this and AI that. Half of these people were not paying attention when we were training gradient boost models and now all of these people are 'AI Agents experts'.
I think they are trying to express that everyone and their brother is now an expert in AI. But everyone was already a decorated expert with decades of hands on "AI" experience by the summer of 2023. So "experts in agentic AI" is just the continuation of that trend.
> Remember when everyone died during COVID? And when we finally ended racism during DEI? Can someone write an AI 2027 (...)
Oh no, the obvious strawmans that - despite the author's assertions - people did not actually widely believe in, were strawmans? Crazy, I tell you. It'd seem that people aren't actually as dumb as the author likes to characterize them being.
Sure love this genre of writing. It's a beautifully revealing tour de force in projection and narcissism. "Will somebody please think for all the fools who believe in the obvious nonsense I secretly fear?"
> And like most cults when the end times don’t come they are just going to fade out and everyone forgets.
Actually, that's not what usually happens after a failed prediction. The weak-believers leave but the rest end up having their beliefs reinforced by the event (or lack thereof). It's like an exercise in doublethink, and those who pass are now deeper than ever in the cult.
Even in this article, he claims COVID wasn’t a big deal because, essentially, only a million people died in the US instead of everyone. No one ever claimed 100% mortality.
Most people realize this is hyperbolic rhetorical. Taken in context (instead of nitpicking a minor "detail") he's calling attention to the fact some people think that AI will affect EVERYONE negatively.
This article was not about Covid, which means his sarcastic question serves a purpose to illuminate the issue with judgment on the rise of AI, not argue that "only a million people died" as you claim.
Yet the measures taken far exceeded the damage than if we had let the virus take its course.
The problem wasn't the mortality rate, it's the fact that the media can cleverly turn something in a huge deal by talking endlessly about it. During COVID, there was no other news except the virus on news for 24/7.
The SFBA culture has given me the ick for a while now. Anyone who has done web development in the last few years is inevitably exposed to it. Idk how to describe it without breaking the rules of the site so I’ll just say nothing.
Somehow the Chinese phrases 内卷 https://en.wikipedia.org/wiki/Neijuan and 摆烂 ("let it rot" https://zh-yue.wikipedia.org/wiki/%E6%93%BA%E7%88%9B) are quite appropriately used, even in stylistic tone. Don't know how much Chinese geohot actually speaks but it's a nice touch.
The English article for "let it rot" (or a related phrase) seems to be here: https://en.wikipedia.org/wiki/Tang_ping
In my interpretation, 躺平 (tang ping / "lay flat") and 摆烂 (bai lan / ..."let it rot" I guess) overlap in meanings in the "do nothing" aspect, but in the tang ping / lay flat case, it is out of learned helplessness / nihilism, while bai lan / let it rot case, it is out of purposeful negligence / noncompliance, which suits geohot's vibe
"Involution" is spot on.
Thanks for the references.
I was hoping for more analysis here :\ like from a technical standpoint, why is the doom and gloom incorrect? I've heard some really interesting arguments for that, and non of them are here.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
When the U.S. first legalized pharmaceutical ads on TV, one of the requirements was that the drug companies had to disclose side effects during the ads. The companies chafed at that, thinking it would make the drugs less appealing, and only included the bare minimum the rules required them to. But their own data eventually showed that consumers actually responded positively to hearing the side effects. They found that people think if a drug is powerful, it must also have strong side effects. So instead of just scaring them, it reassured them the product was good. After learning that, the drug makers started including more side effects in the ads than they were required to.
> Looking at the ongoing Mythos export control issue, it's obviously not good for profits.
This is not obvious at all. Anthropic's biggest investor was the catalyst for this action in referring the "jailbreak" (if you can even call it that) to the government. Now, Anthropic is sitting around a table with government officials who are designing benchmarks that will determine what Anthropic's competitors will be allowed to build. At this point, I have no evidence to rule out the possibility that Anthropic's leadership purposefully sought this outcome.
> Why would a company warn that their technology is dangerous if it wasn't?
Because it makes their technology seem more important and powerful?
Do you usually believe 100% what a company says about itself?
> Mythos export control issue, it's obviously not good for profits
Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle
When people discuss the negative effects of climate change, do you think that boosts fossil fuel company valuations by making their products seem more important and powerful?
Fossil fuel companies do everything in their power to stop that discussion.
However, arms manufacturers are doing everything in their power to make you feel unsafe and weak and scared. That way, you are more likely to buy a gun.
Hell yes. More than a third of Americans would run their two and a half ton pickup trucks on burning copies of An Inconvenient Truth if it were at all possible
*why would they warn if it they didn’t believe it.
2 words: regulatory capture.
Ban Chinese models because they’re “bad”. (can’t have head to head competition).
Bad open weight model (can’t let people escape the subscription model).
People have been warning about the dangers of advanced AI since long before open weight competition was a consideration.
Sure, but they’ve either been the kool-aid-drinking p-doomers, or academic discussions about giving them access to weapons.
Now we have real companies whose valuation depends on people using their services.
> 2 words: regulatory capture.
> Ban Chinese models because they’re “bad”. (can’t have head to head competition).
The current scale of current + planned data centres only makes sense if the US exports inference worldwide. 30% of your current total electrical generation capacity, and equivalent to 76% of your total electrical demand in 2022. At least, assuming the "33 GW installed, 300 GW more planned" claim I've heard was not itself an AI hallucination.
And at least OpenAI and Anthropic were persuing the same argumemts continuously to their initial founding, before such capture was plausible.
I don't only mean that GPT-2 was, famously, when OpenAI surprised the world with (paraphrased) "we think this is a good point to start practicing not releasing the weights, before models get too dangerous" or as a surprising number of people interpret it even when I link to OpenAI's own post "we think this is … too dangerous".
I mean even before that.
I wouldn't disagree if you say Musk is full of BS, but he did donate to OpenAI when it was created as a nonprofit, and his wealth was 2 orders of magnitude smaller than today. Back then he was talking about risks, "summoning the demon", now he's the one demanding saying "robot army" to justify a bigger compensation package from Tesla.
You could also turn this around: Why would a company put up warnings about their product, or at least more than they're legally obligated to. The tobacco and social media industries certainly didn't go "this product has the risk to do great harm, but we also believe in its potential, so we'll go with a responsible approach".
The only examples of companies voluntarily putting up warnings I can think of are those obviously tongue-in-cheek warnings "this book/game/series may cause addiction because it's just so good" etc.
I agree. There are 2 groups of people:
1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.
2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.
The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
> in fact it does the opposite because of the need for government regulation.
For frontier leaders, regulation is a competitive moat. At trillion dollar scale, compliance teams, legal and lobbyists are a tiny fraction of opex. For emerging startups, early regulation of a new market is a substantial barrier to entry.
That said, I think Anthropic blundered into their current Fable mess unintentionally by underestimating just how much they pissed off the current administration by refusing to let the DoD use Ant's AI's for all the domestic spying and lethal combat ops they want.
I don't think the first position is steel manned by assuming those holding it actually believe there are no potential problems or ways to use the technology badly.
But that belief does not imply the doom rhetoric is necessary or the best way to approach those issues.
that's an inaccurate characterization on #1. he's not saying ai risk is _all_ hype to boost valuations, but that it's misused to serve purposes other than warning.
Those words come directly from TFA:
> It’s all just nonsensical hype
And he’s referring to completely real and reasonable warnings in anthropic blog posts about the exponential rate of AI development.
What's "exponential" about AI development? Model parameter counts? Anthropic doesn't publish those for their own models, last I checked. Datacenter buildouts? Water consumption per request? There just isn't enough evidence that AI smarts is growing all that much, once you account for the scaling of inputs. That's what OP probably means by "nonsensical hype".
What's "exponential" about AI development?
The METR task-completion time horizons, for one.
https://metr.org/time-horizons/
Lousy benchmark, they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
> Lousy benchmark
Make your own then. It can go on the pile with all the others that keep getting saturated too fast to be useful.
> they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
What?
They made the benchmark last year, and included a bunch of models going back as far as 2019.
When they first announced it, the top end of their tests were things AI could not actually automate, and even now only does erratically. Examples of the tasks SOTA models are now saturating (at the 50% success level, not at 80%) include:
They're benchmarking against the time it takes humans to do the same things, which means everything they ask every AI to do must have also been done by a human.>> Looking at the ongoing Mythos export control issue, it's obviously not good for profits
We must live in different planets. The whole episode about Mythos has been the absolute best free PR move ever designed. At this point everyone in tech assumes Anthropic and Mythos are the absolute best models. All of this got achieved without spending a single $$ in advertisement.
This model is also probably really dangerous, not denying that. But they definitely used it as a huge PR tool.
> The whole episode about Mythos has been the absolute best free PR move ever designed.
Great PR.
For a product nobody can pay for.
And a high chance that when anyone can, the market shrinks from "worldwide internet users" to "the USA only".
And it's known to be a highly dynamic market: based on the historical average, it only takes 11 weeks to be completely trounced by someone else's model, so if you're out of the market that long you may as well not have bothered.
How many non-tech people knew about Anthropic before? How many know about Anthropic since this episode?
Still don't see it? Still don't see how the huge IPO coming up soon could profit from this?
Yes, this was a brilliant PR move (independent from the actual benefit of the model).
> Still don't see it? Still don't see how the huge IPO coming up soon could profit from this?
"Would you like to buy Anthropic shares?"
"Anthropic? Didn't the government ban them?"
"Yes"
"I'll pass"
To me, they're selling the "power" of their product by mentioning the danger. "It's TOO powerful to even release yet!"
Whether or not that level of power exists, that is definitely how they're pitching it.
A common line of logic would be that a company would never put a dangerous sticker on their product unless they absolutely have to by regulation. So if they do it, it absolutely means they see a benefit of doing it.
Also why do you think the Mythos thing is bad for profits? Anthropic hyped the model up too hard and didn't even have the compute to serve it properly to all users.
The export control issue has not affected Anthropic's revenue and only made its top of the line products seem far more impressive than they actually are. People don't think Nvidia is a bad company if the government puts export controls on their products.
For Anthropic, valuation and public perception is 100x more important before the IPO than revenue or profitablity.
> Why would a company warn that their technology is dangerous if it wasn't?
That's a bit like asking, "why would someone claim to have a lot of money if they didn't?" It's step 1 in countless scams.
You can't think of any ways to exploit people by convincing them you have technology which might rule the world soon?
Because it makes them feel important because they are developing something so dangerous everyone else ought to listen to what they say.
It's about ego, not profits.
>Why would a company warn that their technology is dangerous if it wasn't?
Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up.
If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
> If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
Well, that explains Musk.
Why would a company warn that their technology is dangerous if it wasn't?
The whole point of the article is to answer this question, and here's the answer:
Because all the AI doom fear-mongering is driving sky-high valuations. The more the public panics, the more investors open their wallets.
Is there any evidence of this? It seems like a thing people say all the time, but completely unsubstantiated.
What else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.
You cold also drive valuations like Musk does.
Hype up the *positives*.
Even here on HN, Tesla fans try to justify the market cap by pointing at Optimus, even though nobody can buy it and there's already competition for it which you can buy.
The positives aren’t there though, beyond what people are already using it for (pair programming aid, fancy auto complete, refactoring tool). Hence the article’s thesis: the doom justifies the valuation.
In fairness, it is true that there is a correlation such that:
> The more the public panics, the more investors open their wallets.
I don't believe this to be causation, but I can see the correlation.
They banned exporting bad cryptography algorithms/code 30 years ago too.
People just love to talk about anything that tickling their fancies.
It’s all a fugazi.
Setting up for the eventual bailout. Remember - "we can't let banks fail or it'll be a depression"?
I remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.
I remember large banks that caused the failure to be bailed out. And talk of trickle down economics, except that average joe was not bailed outed, but punished.
I remember robo signing of home foreclosures with no checks and people loosing houses despite paying debts.
It’s the same narrative as “the communists are developing nukes so we need to develop first”.
> it's obviously not good for profits.
Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.
Anthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.
Regulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.
They did not wanted export controls, but they did wanted everyone so scared and thus pay for license. The decision backfiring does not imply it was not intended to be self serving.
If the company really think their product is dangerous, they should stop making it increasingly dangerous. Antromorphic claims is dangerous and that they need more money to make it even more dangerous.
And the fearmongering is beneficial for them so far.
> Why would a company warn that their technology is dangerous if it wasn't?
It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big.
The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial to walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's their sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent.
While I'm sure many of the top AI leadership are primarily (or purely) mercenary, I also think some, if not many, other execs and employees sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s>
So... yeah, that's why we see some weirdly whiplash messaging.
Yea, the "doom trolling" seems to be starting to reach its limit.
https://www.nytimes.com/2026/06/17/opinion/ai-dangerous-open...
Can you share a non-paywalled version? Wayback machine doesn't seem to have this.
Even if Dario and Altman originally believed what they were saying, their scaretactics worked wonders for investment. Their companies are now incredibly incentivized to keep the AI Apocalypse narrative going further and further. It is hard to imagine them stopping, as that may lose them investment.
The last 10 years have been a decade of Big Tech Vaporware. NFTs and the Metaverse were assured to be the future. Once this narrative fails too (which I am almost certain is inevitable) I think society's love affair with Technology being the solver-of-all-things will finally fracture.
I think technology can actually solve a lot more things than it has, the problem is that foundational problem solving does not make a unicorn startup with earthshaking economies of scale in most cases, and especially so if there's a physical world piece involved, and even more so if it's going to go asymptotic.
Just to throw out a random example - could technology, in terms of advances and improvements to allocation, distribution, and consumption, play a part in solving the western US's water issues? Probably. But would it be something that make a trillion dollars and a household name? Maybe not. And could saner policy, like making farmers have to bid for acre feet instead of getting it for basically free + distribution cost also help? Sure.
Likewise, even with AI most software is still crap. Like when have you heard about a doctor who loves their EHR? Like never.
So I think technology could be a solver of at least a lot more things. But we've created a market where people want to exhaust every flavor of flim flam and trend of the moment first. Because we've glorified the business of tech more than the actual improvements and aspirations that should be possible.
> NFTs and the Metaverse
AI is not that bad
It's significantly worse! All NFTs did was separate fools from their money, and crypto in general increased GPU costs. All the Metaverse did was cause Facebook to throw away a bunch of money (lol).
Meanwhile, AI has ruined the whole Internet and inflated the price of everything electronic.
AI is actually useful, in the strict sense. It can do things. If you consider its eventual effects good or bad for humanity is fair either way, but it's still different from NFTs or the "metaverse" nonsense. It's a categorically different thing.
Let's not forget that the inevitable AI crash will nerf even supposedely "diversified" broad market ETFs due to BigTech's insanely inflated valuations and the index gaming of the X, OAI, and Ant IPOs.
The whole internet AND local computing. This may be cool for people on SF salaries but for the vast majority of the world memory has become a luxury good
While it is driving new hardware costs up remember you can run Linux on a potato and just wait it out. My pixel 8a is a phenomenal computer if ignore the hype monster.
so, twitter went from 98% shit to 99% shit, and landfills get less smartphones; I'd say it's worth it
> will finally fracture.
If true, it will only be replaced with something else. With what is anyone’s guess.
I came to the same conclusion.
Specifically Anthropic's whole PR has all been about danger, safety, doomerism all to make themselves indirectly more important and central to the debate.
Calling meetings in Washington DC in order to let everyone know they made a cyberweapon is part of those PR moves. Then they seem surprised lawmakers actually called them out and asked to stop serving that model.
I know this is the cynical take but I cannot unsee the elephant in the room: This doomerism allowed Anthropic to be the center of every AI conversation right now. Their market cap and upcoming IPO is indirectly benefiting from this.
I also cannot take that Anthropic while letting everyone know that Doom is coming (or is already here), are also the ones that want to decide who can profit from this Doomerism. This is how every benevolent dictators start.
> This is how every benevolent dictator starts
I don’t think “benevolent” is necessary in that sentence. It’s how many non-benevolent dictatorships started.
> This is how every benevolent dictators start.
Every dictator is benevolent for the people that agree with them.
What's the evidence that the doom narrative is connected to valuations? It seems more like a marketing/recruiting strategy. As the post points out, institutional investors generally think the idea of mass unemployment is BS, and they are investing accordingly.
Dumb thing to ask tbh.
Dangerous new tech with military applications sounds harder to ignore than AI that spits out text.
AI obviously has military applications, no doom required to think the military will want a lot of it. I'm asking about the extreme scenarios like "AI is gonna take everyone’s jobs" and the other "totalizing narratives" that geohot is referencing.
If this is true: "AI is gonna take everyone’s jobs" then AI is the terminal technology and everyone who doesn't want to be poor forever needs to throw all their money in it.
Of course it impacts company valuation.
Companies say all sorts of things which investors don't take at face value, often for the sake of marketing and recruiting. Meta circa 2020 wanted us to believe we'd be swapping NFTs in a virtual mall and having all our meetings strapped into VR headsets by now. That doesn't mean their market cap at the time was based on an assumption they were right, in fact it is much higher now that the metaverse idea flopped. Unlike that situation where money was being spent on projects that turned out to be mostly pointless, AI labs don't have much of a financial stake in a scenario such as "lots of demand for AI, leading to mass unemployment and/or doomsday" as opposed to one like "lots of demand for AI, leading to the world being mostly the same as before but with more AI and expensive RAM".
I think when chinese models figure out how to partner with SOC 2 providers in the USA in a way that offers a $20-$100-$200 subscription is an easy convenient way, anthropic and open ai will be screwed.
You mean 2, 10 and 20? Otherwise at the same price I'm going with big 2
> Remember when everyone died during COVID?
yes
> And when we finally ended racism during DEI?
I fucking hate geohot, I can't believe you monsters are into this shit
I just want to say, as someone who lost literally every single grandparent I had to covid... this dude can go fuck himself.
Like - right up his own asshole.
---
I understand this comment is inappropriate. I think the comment in the article is WAY more fucking out of line than this.
If you look at Anthropic's blogs about their model timelines, there is roughly a ~3m period between a model being in internal preview until release. That means that inside of Anthropic, the next version beyond Mythos/Fable is already in preview, already being tested internally. Despite what Geohot is describing here about GLM, my understanding is that Anthropic employees have spent a significant amount of time grappling with a technology that is considerably ahead of what is available to the public today.
In addition, if you look at the graph of LOC written by Claude vs Ants (I.e. AI vs human), there is an incredibly sharp uptick post-Mythos internal preview. Something like from 30% to 75% of code inside the company being written by AI.
While I sympathize with the viewpoint here, I still have to admit that that there's a very different feeling to working inside of a company where they've had months of time with a model that's at the frontier, quickly changing the way everyone around them works, and that _they themselves_ control the keys to.
If Geohot had those keys, I can be 100% confident he'd be raising the alarm at the top of his lungs about it.
If the March leak of Claude Code was Mythos / Fable in preview ... I'm not sure I'm that worried about AI capabilities.
Seriously -- if you dig through that source code, and then listen to the messaging, it seems hard to keep a straight face.
Also, hasn't this company been claiming that almost all their code is written with AI for significantly longer than "post-Mythos internal preview"?
Claude code is a very small fraction of the code written by Anthropic and ultimately, despite being widely used, hardly dents Anthropic’s core priority of LLM performance.
It’s because Anthropic doesn’t publish any of its core AI research that we falsely believe that it isn’t by far the central focus of the majority of the team.
Just to be clear, I’m not supporting their stance nor defending the company. I consider it to be deeply harmful that a private company seeks to advocate for AI safety but then own all the means of production and profit financially from keeping its techniques secret. It’s as if the Manhattan project resulted in a for-profit company selling all atomic technology and deciding on its use.
I see, thanks for clarification -- where does one find the LoC comparing Claude to human written? Interesting if that's available publicly.
I mean, my company is also having internal test release before going out. That is just a normal process.
I can't decide if this is a "techbro doesn't understand human factors" post or not, especially considering geohots reputation.
I do believe it is possible Anthropic are legitimately trying to start political discourse, but if they are they are either sacrificing themselves on purpose or shooting themselves in the foot. Others in the comments here are pointing out that there are many incentives for them to get into politics. Maybe they see possible outcomes worth the short term problems.
If it is just marketing and FUD, it's worth considering that a good lie is usually based in truth.
Let's say Anthropic were a "hippie organisation willing to sacrifice revenue for morals" (or pick your own, I'm just giving an example). Could they play the politics game better?
Journalism and public media in general needs to be a licensed profession with tangible standards, and malpractice suits should be pursued aggressively.
Would solve a lot of problems in the US, actually. Being financially incentivized to gleefully lie and spread misinformation at the expense of others should not be protected speech.
Simple minds want to believe one simple thing and then rationalize everything else to force consistency with that one simple idea.
If you want to believe the simple idea that AI is mostly hype, then you'll get stuck in a multi-year loop talking about stochastic parrots, ridiculous valuations, and doomer scaredycats.
But the real world isn't so simple. Multiple seemingly contradictory things can be true at the same time.
Some AI is useless. Some is incredibly powerful and useful even though it makes mistakes. Some companies are wildly overvalued. Some extremely large and expensive companies will quadruple from here. Some frightening scenarios will look silly in hindsight. Other frightening things will happen that none of the doomers foresee.
It would be great to explore those new ideas and possibilities.
It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Some AI is useless
The question is not whether AI is useful. It is useful, full stop.
The question is whether AI is useful enough to justify valuations that dwarf the GDP of all but the top 20 countries in the world. As it stands right now, it's not even close! The leaked OpenAI financials put these AI companies in the same profitability territory as utilities, with zero justification for these crazy valuations.
> The question is not whether AI is useful. It is useful, full stop.
And nukes are useful by some metric too.
AI (llms) is not useful in any way that we as humanity should be pushing for. It’s more harmful than good in every way possible. It’s honestly astounding to me that everyone’s ethics are so weak as to believe that somehow these incredible destructive companies are somehow good for humanity.
Technically, it's the time discounted rate of future profits that determine valuations. If you take it as a given that they will provide exactly as much value as they do today, then, yes, your question has the obvious answer of no. If you think AI will provide greater value in the future, the answer depends on the value you think it will have.
Technically, it's the time discounted rate of future profits that determine valuations
Close, it's the time-discounted expectation of future returns. This seems related to future profits but it need not be. Historically, stocks tend to perform poorly after IPOs. There's no guarantee that (say) Anthropic's stock price would ever recover after a post-IPO drop.
The recent attempts by Anthropic et al. to circumvent the usual rules for inclusion in indices have raised red flags all over the place, with many calling it a naked attempt to raid everyone's pension funds for hundreds of billions in ill-gotten capital.
What you have written does not seem to be in close contact with the OP. He talks positively about the GLM news and whatnot. He is highly skeptical only of the "doom" scenarios, including upending most or a massive amount of jobs, and how that is deployed to keep the investment machine working at such breakneck pace.
> Some AI is useless. Some is incredibly powerful... It would be great to explore those new ideas and possibilities.
I agree, but it's not so mysterious what will win out. Even if the criticism is repeated so often, that's because much of it is still valid.
LLMs are not AGI. A statistical model of language helps fill in gaps. This is super useful for new and much improved UI/UX ideas that converge with better accessibility. Similar is true for generating images, video, audio, etc. There are situations where it's the right tool for expressing an idea.
What we need is a sense of maturity. The limitations are very clear to everyone now, and we're already past the disillusionment. If we can rein in the abuse, there should be a good path forward. The technology is already boring and that's a very good sign.
I love this website because people will write stuff like “I believe the hype because my brain is big, ‘believing the hype’ is actually very complex and cannot be embraced by lesser intellects”
There are reasonable anti- and pro-AI views. There are also unreasonable ones on both sides, and it's best if they are ignored.
However, on a site like HN or Reddit, you're far more likely to hear squawking about "stochastic parrots" or a rant about AI water usage than the mirror on the pro-AI side, making them harder to ignore.
It’s funny that a lot of what gets framed as pro- vs anti- AI arguments would be better described as past/present vs future arguments. If one guy is saying “that’s a stochastic parrot” and the second guy says “it’s going to achieve apotheosis and bring about global communism or feudalism or whatever while maximizing shareholder value” they’re not arguing about technology, they’re arguing about the second guy’s status as a clairvoyant.
You could post “fortune tellers used to be considered fraudsters and charlatans” and reasonably expect a “Get a load of the Luddite over here. Go raise a barn, Josiah!” response on the internet these days despite not mentioning AI or technology at all
"I cannot be contradicted because I can hold conflicting viewpoints" was Chomsky's bit, HN can only take credit for imitating it.
“Dario Amodei doesn’t seem like he can tell the future” - Simple. Pedestrian. The roughshod cogitation of a country oaf.
“Maybe he can” - Complex. Divine, possibly? A breathtaking filigree of nuance, like an Alex Grey painting of conceptual allemande.
> It's so boring rehashing the same old tired and worn out ideas like "they're just hyping the danger to pump their shares up."
Yeah poor you this must be much more tiring than being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it. Sorry dude, hearing that people don't like that messaging and are replying with copypasta talking points must really be rough for you. Praying for you in these trying times.
Regardless of whether you believe in doom or not, it's this kind of wild, willful misunderstanding that makes people roll their eyes and dismiss you. Yes, people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
With the reality-bending financial incentives that these people have, I simply do not trust your or anyone else's assessment of the genuineness of their beliefs. And anyway, it's quite ironic that from godless San Francisco of all places, we are supposed to simply accept that the harmful acts of religious extremists are justified because, well, they do believe in their cause.
The point here is not the correctness of their beliefs. It's about what the actual content of their beliefs is. If someone says that Donald Trump is a secret Shia supremacist and using that to explain his actions, pointing out that that belief about Trump's belief is wrong is not a statement that Trump's beliefs are correct, but that your model of Trump's belief is incorrect.
Right, but it seems to me that you (and many people) are taking their stated beliefs at face value, when there are alternate interpretations of their conduct that are (a) highly plausible, (b) aligned with their incentives, and (c) less weird.
Personally, I do think some of the people in this field have really drunk the Kool-Aid and still believe in the paperclip monster. But for many others, I do not think the past couple of years of plateauing progress has escaped their notice. ESPECIALLY at the leadership level. I think they're larping and they know it. I just think they want the money and the power.
If I were building a technology that I thought posed a risk of exterminating the human race I would simply stop building it. But what do I know, I'm not a genius AI researcher.
> genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as marketing hype by crypto bros to increase IPO valuations long ago passed into the realm of conspiracy thinking.
"They are not in a thought bubble YOU are in the thought bubble"
I am absolutely BTFO'd, you got me.
No; I'm only stating that your model of reality is off, and I'm correcting it. No, the folks at the frontier labs do not believe they are trying to scam people with an NFT-tier fraud, and their warnings are not an attempt at marketing hype.
So to be clear, you're saying that ALL the people that you currently know of who work at these companies believe they are actively working on, in your words, something that "...poses a potential existential threat to humanity"
Oh nevermind I get it. They're only intentionally working on building something that they believe will end humanity. Well as long as it's not intentional crypto-tier scamming, it's all good then.
I'm convinced. Consider my model of reality corrected. Thanks homie.
Their usual stance is something along the lines of their company is creating AI correctly--not that AI will inherently destroy humanity--and that them working there helps that end. It's extremely reasonable to question their approach, but here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
>here you've jumped from "NFT crypto bros trying to run a scam" to "monsters excited to annihilate the human race," which is a wild leap, and betrays a point of view that seems more driven by anti-AI psychosis than consideration of any evidence.
This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
Introducing parameters for the other person, then use those to call them crazy when they operate inside them. Actual DA-tier tactic, my guy. Yikes.
My initial reply was "being told constantly that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it."
As in, we are constantly told that everything is going to be shit and there's nothing we can do, because there is a ton of moneyed interest in it.
You're the one that made the leap to "crypto" scamming in the reply and introduced the existential threat aspect. Both in the same comment, actually.
Then apparently tried to crazy-make me for (admittedly dickishly) pointing out that trying to build something that is an existential threat to humanity (your words, their belief, none of my words or belief) is actually worse than crypto scamming (again, both things which you introduced to the thread, btw).
But yeah shame on me for not taking you seriously and providing evidence I guess.
Odd how the bar for me is "providing evidence" but you're happy to outright lie that I was going from crypto scamming and existential threats in order to score a rhetorical point by claiming "AI psychosis" in a useless internet thread. Yeah I must be the unhinged one here, surely.
> This would be a great point if I had introduced either the NFT/crypto comparison OR the "existential threat" parameters, but if you read through the thread, you introduced both.
To quote you, ten minutes before:
> that the apocalypse is coming and your jobs are gone and everything is going to be shitty because a bunch of ultra rich midwits want even more money and there's nothing you can do to stop it.
It's a fair description of your stance. And, full of sound and fury, signifying nothing.
If they genuinely believe that, then they are literally a conspiracy to destroy the world. If their current actions are the outcome of genuine belief to what they say, then they are indeed massive "midwits" and a literal cult conspiracy. Due to the amounts of money they siphoned, that would make them dangerous cult regardless of whether their ai predictions are correct.
Calling them scammers is literally giving them benefit of the doubt and trying to avoid conspiratorial thinking. You frame the belief that "these are normal people with normal motivation" to be paranoid conspiratorial thinking. Meanwhile, you claim they are literal conspiracy and cult and that is somehow less paranoid.
Now, dangerous cults and scammers are severely overlapping categories. Most if not all cults scam people. They lie to get more powerful whenever it suits them. They have true believers (like your friends) fully willing to cause any amount of harm for the visionary in the center. These people may have started as a well meaning do-gooders, but they usually just end up being co-conspirators in crimes.
people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe
The same could have been said (and has been said) about other tech company employees for all sorts of other reasons in alignment with those companies' goals. Don't you remember how much people used to laugh at Tesla employees for worshipping Elon Musk as some kind of god of engineering and entrepreneurial genius? Or Apple employees in the Steve Jobs reality distortion field?
I would have thought at this point that it'd be well known that the employees of all cult-like tech companies can't be trusted to make a sober evaluation of their companies' justified valuation. We can talk about conflicts of interest and we'd barely be getting started! How about biased selection by hiring managers for the most fervent believers in the company's mission from the get-go?
Sure; they might be caught in their own hype cycle. I am pointing out that the repeated claims in this thread (and by Geohot, ironically enough) that they are NFT crypto bros running a scam is incorrect.
Doesn’t have to be a naked scam though. Dario could be caught in the dictator’s dilemma: he was a believer at first but now he’s on the tiger’s back so he dare not get off.
> The only possible conclusion is that it’s designed to cause panic.
Really? That's the only possible conclusion?
Anthropic has _always_ positioned itself as a company that cares about AI safety.
> Can someone write an AI 2027 but instead of some totalizing doom propaganda it talks about the bubble unwinding and what we can do to prevent this kind of crap in the future? How do we build an economy and society that’s sustainable?
I imagine tons of people have written that article. But no one reads it. They're all busy with the doomiest bullshit Facebook or Tiktok will serve them. It's what gets engagement.
No one is clicking on the "None of the things your scared of are scary and here's why" video when it's up against the "$x is the end of the world and will eat your children" agitprop.
There are a few possible explanations of Dario and Anthropic's behavior, if you are not sure if all they want is pump up the valuation like me:
- Anthropic has a cult-like culture. AI Safety is their religion. The AI Constitution is their bible. Dario is their cult leader. Employees are the apostles. They just really really believe their church, and only Dario is qualified to manage the AI safety.
- Asymmetrical risk. If Dario speaks optimistically about AI and he turns out to be wrong, he'd face the rage of many people. If he fans the doomerism of AI and he turns out to be wrong, at most he will be mocked.
- Regulatory capture. After all, pretty much all the AI big shots in Biden government went to Anthropic. They produced the Biden's regulation, and they made it clear that they wanted to pick a few winners to back.
Is "being mocked" not also "facing the rage of many"? What do you imagine with the latter that would be so much worse?
I assume that "you didn't warn us" and "why are you so pessimistic" would lead to different scale of anger.
HN is not immune to this. I do not take HN seriously, because empirically so many takes across so many subjects have been wrong in a melodramatic fashion and the “adults”, often people with training and first hand experience have to show up to set the subject matter right before someone gets hurt. But as a barometer of unhinged hype? Only X is comparable.
>>SF wants to come for your inner life and pimp you out and mediate every interaction and there’s not even a so.
I have been saying this for a while. I visited SF a couple months ago and god do people feel empty from the inside. Everything is revolving around AI this and AI that. Half of these people were not paying attention when we were training gradient boost models and now all of these people are 'AI Agents experts'.
> AI Agents experts
you mean loop engineers?
I think they are trying to express that everyone and their brother is now an expert in AI. But everyone was already a decorated expert with decades of hands on "AI" experience by the summer of 2023. So "experts in agentic AI" is just the continuation of that trend.
I was trying to be funny
> Remember when everyone died during COVID? And when we finally ended racism during DEI? Can someone write an AI 2027 (...)
Oh no, the obvious strawmans that - despite the author's assertions - people did not actually widely believe in, were strawmans? Crazy, I tell you. It'd seem that people aren't actually as dumb as the author likes to characterize them being.
Sure love this genre of writing. It's a beautifully revealing tour de force in projection and narcissism. "Will somebody please think for all the fools who believe in the obvious nonsense I secretly fear?"
> And like most cults when the end times don’t come they are just going to fade out and everyone forgets.
Actually, that's not what usually happens after a failed prediction. The weak-believers leave but the rest end up having their beliefs reinforced by the event (or lack thereof). It's like an exercise in doublethink, and those who pass are now deeper than ever in the cult.
I stopped taking this guy seriously after he proposed "just" digging canals to bring desalinated ocean water into the U.S. desert.
Even in this article, he claims COVID wasn’t a big deal because, essentially, only a million people died in the US instead of everyone. No one ever claimed 100% mortality.
> Remember when everyone died during COVID?
Most people realize this is hyperbolic rhetorical. Taken in context (instead of nitpicking a minor "detail") he's calling attention to the fact some people think that AI will affect EVERYONE negatively.
This article was not about Covid, which means his sarcastic question serves a purpose to illuminate the issue with judgment on the rise of AI, not argue that "only a million people died" as you claim.
COVID was a risk management disaster that should not have escalated to the level that it did if there was not so much political overreach.
Yet the measures taken far exceeded the damage than if we had let the virus take its course.
The problem wasn't the mortality rate, it's the fact that the media can cleverly turn something in a huge deal by talking endlessly about it. During COVID, there was no other news except the virus on news for 24/7.
Great article. Holtz is growing on me ngl…
The SFBA culture has given me the ick for a while now. Anyone who has done web development in the last few years is inevitably exposed to it. Idk how to describe it without breaking the rules of the site so I’ll just say nothing.
> If San Francisco was nuked tomorrow, the world would feel a weight off their shoulders.
Who Would Jesus Nuke?
Did Hotz hit his head or something? Atheistic hedonists oh my!