I also feel a need to amass retirement levels of wealth right ASAP before this all comes crashing down, and then move to somewhere else away from this mess.
The only good way for me to do that is to keep working in AI, because nothing else that I have skills for pays that high.
qurren: Thanks for the reply, that certainly makes sense to me, though I think given the most popular idiocies in the air and their severe distortions on the discourse, it's not a great idea to leave that ambiguous in precisely that way.
It's bizarre that your reply was nearly insta-[dead] and I can't reply to it directly.
1. By "crashing down" I was referring to civilization as we know it, not an economic crash. My views about building wealth aren't about avoiding an economic crash, it's about just being able to have optionality earlier to relocate myself and my family to the least-affected-by-AI part of the world.
My personal view of the modality of failure: I don't think AI will destroy humans, but rather functionally dismantle civilization as we know it; I think we will just increasingly become slaves to the systems they create and the world won't be run in human interests anymore (it already is not run in the bests interest of the non-ultra-wealthy, so it's really just an extension of that: I think the future will just start moving toward s/ultra-wealthy/AI/) and we will increasingly become more like worker ants.
2. I disagree, I think some countries of the world are more likely to value stability of the population than others. I think there are places that are likely to be more safe than others. Also, a notable side effect of current trade wars is that some countries may end up getting left out of a good chunk of the destruction.
Yeah. Let's say they regulate AI to oblivion to avoid dangers, perhaps crashing the economy in the short-term but assuring that no major catastrophe occurs.
All those who worked in AI still have the money they cashed out on, there might not be another opportunity to obtain generational wealth in such a short time again.
Have some shame? Or if genuinely not able to feel that, at least minimally perform it?
As another poster said, hating the player is acceptable, as is self-hate for the things we must do. We're all complex and can't all make the grand gestures, but I would hope people have dissonance about that, and maybe sit with it and speak to it like it's real * shrug*
I already chose a job that I believe is minimally harmful within the AI space. I work in an autonomous vehicle company which I believe puts safety first and is on track to reducing injuries and deaths to humans. This is not the thread where I wish to debate that premise, but that's just my belief so I'm happy to work towards it.
I realize that we are complicit in enabling Anthropic, OpenAI, NVIDIA, TSMC, et al. in the rest of the AI space.
I only have space to think about this so much. I still need to pay rent next month, and I don't have enough saved up to pay rent for the remaining several decades of my life.
I have mad respect for people who can actually throw away a shit ton of financial security for themselves and their families for something they believe in. It’s rare for a reason. Everyone has a price.
If you can get financial security without selling your soul, fewer people would do that, across every industry where selling your soul for security is possible.
I feel people who sell their souls to be in finance or work as corporate attorneys for bad companies deserve criticism, but people who sell their souls to work for an enterprise they themselves believe is at the very least going to destroy civilization absolutely are pathetic and evil.
My personal take is almost all capitalist enterprises are on a path to cause destruction to civilization. AI is not unique here, and every industry is "AI or die" right now.
As for short term survival, I don't have the option to just quit in protest without income, nor do I think that will be effective in accomplishing anything. If anything I will just pigeon-hole myself into becoming a worse future-victim of the system than I already would be.
How valuable will your money be once civilization is destroyed and you, self-admittedly, have no skills that would help you after the collapse? Seems like a better use of your time may be in identifying and developing those skills and cultivating a community around yourself (or joining in one) that can survive your predicted end of civilization.
Money is useful to weather the storm and relocate myself and my family to somewhere less affected by AI, or possibly have some minimal influence over redirection of resources in an AI-controlled future world. It's just insurance and optionality.
> have no skills that would help you after the collapse
That's not what I said. I only have 1 high-paying skill that can help me buy that optionality, and that's software engineering. I don't know of anything else high-paying that I could study quick enough, and other high-paying industries including lawyers and doctors are also predatory in different ways. I don't think the alternatives are better.
As for post-collapse skills, I think the ultimate is just being able to live off the land and move to somewhere that has the right climate and natural resources for that. Moving to another country isn't logistically straightforward, but money can help remove some of those hurdles, both in terms of residence permits and in terms of buying land and housing. So the priority is still financial. The learnings happen in parallel.
> Seems like a better use of your time
Not looking for advice on time management advice. I'm only trying to protect my family's future by building optionality. My individual decisions aren't going to change what the system is doing.
I guess my real confusion is this: You keep predicting the end of civilization in your comments, but you only seem to talk about collecting money to deal with it. Does your end of civilization not also involve the severe loss in value of our current currencies and other financial assets? How do you intend to grease palms to get into other countries if the USD (I'm making an assumption that you live in the US, adjust as appropriate) has become valueless because the US as a civilization has collapsed?
If AI is going to do that, AI needs to at least get good enough to autonomously produce all the chips and energy it needs to execute. That seems like a pretty far way off, considering we don't even have robots that can fold laundry competently.
If AI exists but there are still humans producing chips for it, I don't think you would be able to say that civilization has been destroyed
I am not the relevant one with a position on this. I am referring to people who believe this but work at these companies (especially in capability research).
I’m reacting to this
> I also feel a need to amass retirement levels of wealth right ASAP before this all comes crashing down, and then move to somewhere else away from this mess.
and further context is the WSJ story from yesterday discussing the resignation of anthropic engineers and another anthropic engineer who says it will “kill us all by the end of the decade”.
Essentially I continually hear the perspectives of people who work at AI labs. They frequently brag about escaping the “permanent underclass” and often rationalize ideas like “well better they be the ones in poverty and not me”.
> well better they be the ones in poverty and not me
I'm not this extreme, and don't wish for others to be in poverty.
But I do wish for myself and my family to not be in poverty or suffering.
If the system is designed for these to be in conflict, that's not something I have the power to change, beyond voting for the less-shitty of 2 shitty choices on a ballot. At the end of the day I will prioritize myself and my family, and I think that's natural for 99% of people.
Suspicious in that it's unlikely or suspicious in that it betrays their real motivation?
On the former: It kind of makes sense if they think they can convince regulators that they (and maybe a few select competitors) are the only ones who can keep AI in check. Regulation can work in their favor given how well they're established, financially, compared to the small upstarts they need to keep down.
On the latter: I agree, see above. Promoting the idea that AI can or will kill us all, but also that it can't be stopped except by them, puts them in a position (if they're successful in their persuasion) to get more financial benefits.
Rise in energy prices under increasing AI DCs power draw and wasteful usage of clean water in AI DCs cooling are LEAST of my worries regarding AI.
The scale of damage when occurring is what worries me.
Eg. in old times when someone wanted to eavesdrop someones phone they needed ratio of 1:1 in person count, one keeping up the conversation and another one transcripting it up. Then some more people to intrepret that transcript an make conclusions, maybe even decisions what comes next. That is not the reality we leave in today. Today's scale of data processing and storage capabilities makes the ratio of 1:8298979488 (AI:everyone). I don't want us to ever have AI decision making over those transcripts with the same ratio. Possibility seems clearly to be there soon.
Another eg. in old times when someone wanted to dig into vulnerable system they needed ratio of 1:1 in count, one person having time to dig in one system at a time. I don't want to know how many critical and vulnerable systems we have running today in this planet connected to the same network these AI agents are run. Yes, over time you can use AI to find and patch those vulnerabilities but the scale and speed to do harm (exploiting every system at once) is the problem here.
I’ve heard many of the AI doomer arguments including Bostrom’s Superintelligence.
But it’s still hard for me to take them seriously. These arguments start with reasonable assumptions and then project them wildly. Current LLMs are powerful and can be used dangerously. It’s reasonable to accept a few tumultuous years of change. But I can’t see them reasonably leading to civilization-ending doom.
Are the doomers within the AI industry overly anxious? Or am I missing something?
I am FAR more afraid of violent AI doomers and malevolent people powered by unaligned AI, than I am of violent (or even just extremely disruptive) AI. Remember Joseph from the movie "Contact"?
I still think that aligning AI is a fool's errand though.
Superintelligence by Bostrom came out in 2014, I recommend reading it, at the time I thought it was some of the best sci fi I'd ever consumed, and did not think it would be a public debate within my lifetime. What I find interesting is that respected/knowledgable people have been on TV/Mainstream media saying this for a few years now, Jack Clark has been in front of Congress multiple times. What changed about this particular message that made it so gripping? My little sister msg'd me last night and asked me "Can AI really kill us all? Why can't we just unplug it?". Interesting times....!
This will either be normal crisis, in which case no point in panicking. Or it is existential, in which case, no point in panicking.
If this turns out to be existential, what would anyone be able to do to prepare? Short of politically controlling territory with a population and militia, any plan is merely some blind bet in it not really being existential and you figuring out what will be safe.
It's mostly coming from two out of many American companies that happen to have the highest concentration of people from a very specific movement born in California long before the current AI wave. In fact Anthropic's entire internal culture was built around their views. The fact they operate massive amounts of money doesn't help.
He’s probably referencing effective accelerationism or something associated with the TESCREAL cloud of beliefs
Edit: saw the other responses. I don’t think effective altruism was the right answer. Effective accelerationism, which came out of effective altruism, has all the ideas about human extinction and needing to develop AGI at all costs to save the species insanity.
They have the most creditors, we’re at the top of the S-Curve, and inference margins have a lot of room to contract (and it’s not obvious that A\ or OAI are competitive against DeepSeek in that domain).
Which is itself a side effect of the US media treating most news as a spectacle, combined with the US (historically, but especially currently) being a phenomenally good place to be incredibly rich because much of the government will in a way that many developed nations would not allow.
Every company that is not OpenAI or Anthropic is sufficiently behind the frontier to feel ethically shielded from the weight of advancing the frontier.
wow. that's really quite backwards. the more money you raise and the more money you want, the bigger the story you have to tell. apparently after 1T you've got to raise the stakes to extinction level event.
Capital concentration, they have access and lots of control over the media, their CEOs made themselves public personalities, they for some reasons decided to cosplay as Bond villains, they are linked to Trump (and lot of people hate Trump). Just a few reasons, I’m sure we can think of more
Jacob Coxon is listed as a 2021 fellow on Newspeak House's official site. Media has called Newspeak a hub of "civic hackers" aiming to improve/hack democratic tools and processes.
I think that more people are finally realizing that the best plan of major AI companies sounds more or less like:
- We'll build an AGI able to perform most cognitive tasks at least at human level (including, of course, the most fulfilling)
- This will wreak havoc in the job market, but people can still compete for jobs requiring physical labor, until robots take also most of these jobs
- Then, we'll build an artificial super-intelligence way smarter than any human, and we'll be forever the pet species of a silicon god operating according to "human values"
- In the process, there's a non-negligible probability that humanity will go extinct
The goal may be to build a super-intelligence, but the plan is incredibly dumb !
And it's all largely baseless FUD. As someone else said, it's either a tractable challenge, in which case there's no point for alarm, or it's a nontraversable existential challenge, in which case there's no point for alarm. The people who claim it is the latter have no real empirical basis on which to base such an opinion other than simply fearing AND mistrusting the unknown.
The best way to stifle AI growth would be to deprive the labs of capital, and yet these AI safetyists never seem to seriously attack the money side.
It gives away the game entirely, they want regulatory capture and the politicians boosting it want millions in donations from the IPOs, there's big money in this type of AI grifting.
The SEC should heed the AI labs own warnings and forbid any AI lab doing doom marketing from doing an IPO, for safety of course.
Deprive who of capital? The folks behind Deepseek, Kimi, Qwen, etc. or only the likes of Anthropic and OpenAI? You can wipe out the mosquitos around your house but if you don't get the entire neighborhood you may as well not bother.
> The best way to stifle AI growth would be to deprive the labs of capital, and yet these AI safetyists never seem to seriously attack the money side.
That's because you're up against billions if not trillions of dollars desperately searching for any kind of return on investment, and there aren't that many industries left now that China has everyone by the balls.
As much as I'd love to see some enforcement action before the AI bros turn the world into another horror show like 2007ff... it won't happen. And just like we didn't have any meaningful consequences back then, this time the ones responsible (hello Sam Altman) will get away scot-free as well.
One resignation distracted from the constant failures of AI in order to boost its value yet again, and to advocate for a particular AI company as saintly.
I have yet to see a positive improvement in the world that could be blamed on AI. Why don't they fix the DMVs of the world? The answer is that they would if they could, it would be great PR, but they can't. They can't even operate a phone for the corner drugstore.
I'm bullish on LLMs and they're defeating any optimism I have daily. I should see every program on my computer improving, but they're getting worse. I should see every website getting cleaner while heavy, crappy frameworks put in for developer convenience are being removed and being replaced with feather-light, minimal code and new designs that anticipate my needs. Nope, even getting worse. This is coding, where they're best.
Can somebody tell me anything that AI has improved in my life? The costs are obvious and massive. Just one improvement to hold onto my optimism.
"They" don't care about "fixing" the DMV. It's nice to think that "they would if they could" but in fact, the anti-government party actively cripples and dismantles government services so that people lose faith in government's ability to improve lives. And what would "fixing" it look like, anyway? Lower cost to operate or lower time-in-queue? Those are in conflict and as much as I would appreciate a better experience there when I have to go (once every 5 years), I think few people would be willing to pay $X/year for that.
On the other hand, my personal computer setup is better than it's been in a long time. Other peoples' software that I run has not improved much, but I've managed to fix several long-standing issues by just giving Fable access to my configuration and telling it my intent.
I've also vibe-coded several applications which are lighter-weight versions of things like Pinboard, Descript, and Weechat. I'm not releasing them because I've been doing OSS for 10 years and I've felt first-hand how the economics of open-source is skewed. And in this day and age where anyone can vibe up their own custom app, it doesn't feel worth it anymore.
What this tells me is that there's not proper incentive for the software you use to get "better" according to your values. You have implicit metrics of "cleanliness", "minimalism", "anticipatory design", and I share those values, but having worked as a SWE for 30 years, I have never worked for a corporation which encouraged those values. If I managed to inflict them on the corporate software I created, it was a "yes and" situation and I only did it because I (or other engineers) cared, never because management wanted it.
So the problem is not "AI is failing us" but rather "the system is failing to use these awesome tools to improve our lives". We can use AI to try to improve the system itself, but our system is not a github repo you can deploy to prod. It's a bit trickier to vibe up the improvements to our incentive structure (though I am trying to do it a bit myself).
Internal tooling is basically free. You may not care, but for me that’s a big deal. I don’t have to spend 6 months negotiating with an internal software team to build a trivial plugin that saves me a few hours a week. I ask the genie, and I get it for a couple dollars.
The only improvement I’ve seen is speed of development, but it’s come at the cost of quality. Hence pretty much every single piece of software a consumer uses becoming significantly worse over the past two years.
It’s especially worsened by every leader doing their damndest to force something related to AI into their workflows and products, regardlesss of if it’s a good fit, lest their own leaders or investors decide they are dinosaurs who need to be replaced.
Fake psyop slop.
Guy worked there for 6 weeks, almost no X followers when he posted his “I quit” thread, WSJ then posted their story around this post 18 mins later.
I'm pretty skeptical as well. As soon as I saw his interview on NBC News last night I texted a friend that this guy is about to get a $500 million seed round for an AI alignment startup.
The entire interview he's just grinning like he has the "answer" to all these companies he alleges don't take safety seriously.
Which makes you question if this was the plan all along or he was totally naive not to think this though. Imagine if he did this with Aylo (formerly known as MindGeek) :)
I'm an AI engineer, I feel similarly.
I also feel a need to amass retirement levels of wealth right ASAP before this all comes crashing down, and then move to somewhere else away from this mess.
The only good way for me to do that is to keep working in AI, because nothing else that I have skills for pays that high.
> before this all comes crashing down
You do not feel similarly to the resigning guy if what you are worried about is an economic crash from this not turning out
> move to somewhere else away from this mess
You do not feel similarly to the resigning guy if you think there is anywhere you could move to run away from what you are worried about
qurren: Thanks for the reply, that certainly makes sense to me, though I think given the most popular idiocies in the air and their severe distortions on the discourse, it's not a great idea to leave that ambiguous in precisely that way.
It's bizarre that your reply was nearly insta-[dead] and I can't reply to it directly.
1. By "crashing down" I was referring to civilization as we know it, not an economic crash. My views about building wealth aren't about avoiding an economic crash, it's about just being able to have optionality earlier to relocate myself and my family to the least-affected-by-AI part of the world.
My personal view of the modality of failure: I don't think AI will destroy humans, but rather functionally dismantle civilization as we know it; I think we will just increasingly become slaves to the systems they create and the world won't be run in human interests anymore (it already is not run in the bests interest of the non-ultra-wealthy, so it's really just an extension of that: I think the future will just start moving toward s/ultra-wealthy/AI/) and we will increasingly become more like worker ants.
2. I disagree, I think some countries of the world are more likely to value stability of the population than others. I think there are places that are likely to be more safe than others. Also, a notable side effect of current trade wars is that some countries may end up getting left out of a good chunk of the destruction.
Yeah. Let's say they regulate AI to oblivion to avoid dangers, perhaps crashing the economy in the short-term but assuring that no major catastrophe occurs.
All those who worked in AI still have the money they cashed out on, there might not be another opportunity to obtain generational wealth in such a short time again.
[flagged]
What would you do in their situation?
Have some shame? Or if genuinely not able to feel that, at least minimally perform it?
As another poster said, hating the player is acceptable, as is self-hate for the things we must do. We're all complex and can't all make the grand gestures, but I would hope people have dissonance about that, and maybe sit with it and speak to it like it's real * shrug*
> at least minimally perform it
I already chose a job that I believe is minimally harmful within the AI space. I work in an autonomous vehicle company which I believe puts safety first and is on track to reducing injuries and deaths to humans. This is not the thread where I wish to debate that premise, but that's just my belief so I'm happy to work towards it.
I realize that we are complicit in enabling Anthropic, OpenAI, NVIDIA, TSMC, et al. in the rest of the AI space.
I only have space to think about this so much. I still need to pay rent next month, and I don't have enough saved up to pay rent for the remaining several decades of my life.
What exactly would this performative shame entail? How would it change any outcome?
Ensuring your financial wellbeing into an uncertain future seems to be self-respecting.
[dead]
Don’t hate the player.
I have mad respect for people who can actually throw away a shit ton of financial security for themselves and their families for something they believe in. It’s rare for a reason. Everyone has a price.
If you can get financial security without selling your soul, fewer people would do that, across every industry where selling your soul for security is possible.
I feel people who sell their souls to be in finance or work as corporate attorneys for bad companies deserve criticism, but people who sell their souls to work for an enterprise they themselves believe is at the very least going to destroy civilization absolutely are pathetic and evil.
> destroy civilization
My personal take is almost all capitalist enterprises are on a path to cause destruction to civilization. AI is not unique here, and every industry is "AI or die" right now.
As for short term survival, I don't have the option to just quit in protest without income, nor do I think that will be effective in accomplishing anything. If anything I will just pigeon-hole myself into becoming a worse future-victim of the system than I already would be.
How valuable will your money be once civilization is destroyed and you, self-admittedly, have no skills that would help you after the collapse? Seems like a better use of your time may be in identifying and developing those skills and cultivating a community around yourself (or joining in one) that can survive your predicted end of civilization.
Money is useful to weather the storm and relocate myself and my family to somewhere less affected by AI, or possibly have some minimal influence over redirection of resources in an AI-controlled future world. It's just insurance and optionality.
> have no skills that would help you after the collapse
That's not what I said. I only have 1 high-paying skill that can help me buy that optionality, and that's software engineering. I don't know of anything else high-paying that I could study quick enough, and other high-paying industries including lawyers and doctors are also predatory in different ways. I don't think the alternatives are better.
As for post-collapse skills, I think the ultimate is just being able to live off the land and move to somewhere that has the right climate and natural resources for that. Moving to another country isn't logistically straightforward, but money can help remove some of those hurdles, both in terms of residence permits and in terms of buying land and housing. So the priority is still financial. The learnings happen in parallel.
> Seems like a better use of your time
Not looking for advice on time management advice. I'm only trying to protect my family's future by building optionality. My individual decisions aren't going to change what the system is doing.
I guess my real confusion is this: You keep predicting the end of civilization in your comments, but you only seem to talk about collecting money to deal with it. Does your end of civilization not also involve the severe loss in value of our current currencies and other financial assets? How do you intend to grease palms to get into other countries if the USD (I'm making an assumption that you live in the US, adjust as appropriate) has become valueless because the US as a civilization has collapsed?
These all read like excuses for being a moral coward.
How is AI going to destroy civilization?
If AI is going to do that, AI needs to at least get good enough to autonomously produce all the chips and energy it needs to execute. That seems like a pretty far way off, considering we don't even have robots that can fold laundry competently.
If AI exists but there are still humans producing chips for it, I don't think you would be able to say that civilization has been destroyed
I am not the relevant one with a position on this. I am referring to people who believe this but work at these companies (especially in capability research).
I’m reacting to this
> I also feel a need to amass retirement levels of wealth right ASAP before this all comes crashing down, and then move to somewhere else away from this mess.
and further context is the WSJ story from yesterday discussing the resignation of anthropic engineers and another anthropic engineer who says it will “kill us all by the end of the decade”.
Essentially I continually hear the perspectives of people who work at AI labs. They frequently brag about escaping the “permanent underclass” and often rationalize ideas like “well better they be the ones in poverty and not me”.
> well better they be the ones in poverty and not me
I'm not this extreme, and don't wish for others to be in poverty.
But I do wish for myself and my family to not be in poverty or suffering.
If the system is designed for these to be in conflict, that's not something I have the power to change, beyond voting for the less-shitty of 2 shitty choices on a ballot. At the end of the day I will prioritize myself and my family, and I think that's natural for 99% of people.
You can absolutely hate the player. You can even hate yourself if you know you’d make the same choice.
We’re all humans, we have agency. We’re not some pachinko ball bouncing through life.
It seems deeply suspicious that a company benefitting from regulatory capture would artificially stoke fears of AI and propaganda about it.
Suspicious in that it's unlikely or suspicious in that it betrays their real motivation?
On the former: It kind of makes sense if they think they can convince regulators that they (and maybe a few select competitors) are the only ones who can keep AI in check. Regulation can work in their favor given how well they're established, financially, compared to the small upstarts they need to keep down.
On the latter: I agree, see above. Promoting the idea that AI can or will kill us all, but also that it can't be stopped except by them, puts them in a position (if they're successful in their persuasion) to get more financial benefits.
Rise in energy prices under increasing AI DCs power draw and wasteful usage of clean water in AI DCs cooling are LEAST of my worries regarding AI.
The scale of damage when occurring is what worries me.
Eg. in old times when someone wanted to eavesdrop someones phone they needed ratio of 1:1 in person count, one keeping up the conversation and another one transcripting it up. Then some more people to intrepret that transcript an make conclusions, maybe even decisions what comes next. That is not the reality we leave in today. Today's scale of data processing and storage capabilities makes the ratio of 1:8298979488 (AI:everyone). I don't want us to ever have AI decision making over those transcripts with the same ratio. Possibility seems clearly to be there soon.
Another eg. in old times when someone wanted to dig into vulnerable system they needed ratio of 1:1 in count, one person having time to dig in one system at a time. I don't want to know how many critical and vulnerable systems we have running today in this planet connected to the same network these AI agents are run. Yes, over time you can use AI to find and patch those vulnerabilities but the scale and speed to do harm (exploiting every system at once) is the problem here.
I’ve heard many of the AI doomer arguments including Bostrom’s Superintelligence.
But it’s still hard for me to take them seriously. These arguments start with reasonable assumptions and then project them wildly. Current LLMs are powerful and can be used dangerously. It’s reasonable to accept a few tumultuous years of change. But I can’t see them reasonably leading to civilization-ending doom.
Are the doomers within the AI industry overly anxious? Or am I missing something?
I am FAR more afraid of violent AI doomers and malevolent people powered by unaligned AI, than I am of violent (or even just extremely disruptive) AI. Remember Joseph from the movie "Contact"?
I still think that aligning AI is a fool's errand though.
Superintelligence by Bostrom came out in 2014, I recommend reading it, at the time I thought it was some of the best sci fi I'd ever consumed, and did not think it would be a public debate within my lifetime. What I find interesting is that respected/knowledgable people have been on TV/Mainstream media saying this for a few years now, Jack Clark has been in front of Congress multiple times. What changed about this particular message that made it so gripping? My little sister msg'd me last night and asked me "Can AI really kill us all? Why can't we just unplug it?". Interesting times....!
This will either be normal crisis, in which case no point in panicking. Or it is existential, in which case, no point in panicking.
If this turns out to be existential, what would anyone be able to do to prepare? Short of politically controlling territory with a population and militia, any plan is merely some blind bet in it not really being existential and you figuring out what will be safe.
Why is all this drama coming only from the American companies?
It's mostly coming from two out of many American companies that happen to have the highest concentration of people from a very specific movement born in California long before the current AI wave. In fact Anthropic's entire internal culture was built around their views. The fact they operate massive amounts of money doesn't help.
> from a very specific movement born in California long before the current AI wave
Can you be more specific? This is new to me, but I'm not familiar with Californian history.
Effective Altruism: https://en.wikipedia.org/wiki/Effective_altruism
It has some interesting ideas, and also gave us people like Sam Bankman-Fried.
He’s probably referencing effective accelerationism or something associated with the TESCREAL cloud of beliefs
Edit: saw the other responses. I don’t think effective altruism was the right answer. Effective accelerationism, which came out of effective altruism, has all the ideas about human extinction and needing to develop AGI at all costs to save the species insanity.
EA.
After searching the closest I got was "Effective altruism". Is that what you tried to say?
https://en.wikipedia.org/wiki/Effective_altruism
Edit: Another comment suggests Effective accelerationism.
https://en.wikipedia.org/wiki/Effective_accelerationism
Because they have the most advanced models.
They have the most creditors, we’re at the top of the S-Curve, and inference margins have a lot of room to contract (and it’s not obvious that A\ or OAI are competitive against DeepSeek in that domain).
Because Anthropic and OpenAI are the primary drivers of the AI space?
America is a stage for the world. Nobody knows or cares about drama happening in European tech companies.
I say this neither negatively or positively, it's just that US news is, for now, the only international news, in both tech and politics.
Which is itself a side effect of the US media treating most news as a spectacle, combined with the US (historically, but especially currently) being a phenomenally good place to be incredibly rich because much of the government will in a way that many developed nations would not allow.
Every company that is not OpenAI or Anthropic is sufficiently behind the frontier to feel ethically shielded from the weight of advancing the frontier.
wow. that's really quite backwards. the more money you raise and the more money you want, the bigger the story you have to tell. apparently after 1T you've got to raise the stakes to extinction level event.
"it's just marketing" cool story bro
Capital concentration, they have access and lots of control over the media, their CEOs made themselves public personalities, they for some reasons decided to cosplay as Bond villains, they are linked to Trump (and lot of people hate Trump). Just a few reasons, I’m sure we can think of more
Jacob Coxon is listed as a 2021 fellow on Newspeak House's official site. Media has called Newspeak a hub of "civic hackers" aiming to improve/hack democratic tools and processes.
What is "RSI"? This is the first I'm hearing of this alleged wildfire.
Recursive Self Improvement
Crazy that it's not defined anywhere in this post while also being an existing thing (Repetitive stress injury).
Clearly: Repetitive Strain Injury :D
Something we all suffer from in this industry !
Recursive Self Improvement
ad infinitum, nothing to do with "anybody's" HN username ;)
I think that more people are finally realizing that the best plan of major AI companies sounds more or less like:
- We'll build an AGI able to perform most cognitive tasks at least at human level (including, of course, the most fulfilling)
- This will wreak havoc in the job market, but people can still compete for jobs requiring physical labor, until robots take also most of these jobs
- Then, we'll build an artificial super-intelligence way smarter than any human, and we'll be forever the pet species of a silicon god operating according to "human values"
- In the process, there's a non-negligible probability that humanity will go extinct
The goal may be to build a super-intelligence, but the plan is incredibly dumb !
And it's all largely baseless FUD. As someone else said, it's either a tractable challenge, in which case there's no point for alarm, or it's a nontraversable existential challenge, in which case there's no point for alarm. The people who claim it is the latter have no real empirical basis on which to base such an opinion other than simply fearing AND mistrusting the unknown.
The best way to stifle AI growth would be to deprive the labs of capital, and yet these AI safetyists never seem to seriously attack the money side.
It gives away the game entirely, they want regulatory capture and the politicians boosting it want millions in donations from the IPOs, there's big money in this type of AI grifting.
The SEC should heed the AI labs own warnings and forbid any AI lab doing doom marketing from doing an IPO, for safety of course.
Deprive who of capital? The folks behind Deepseek, Kimi, Qwen, etc. or only the likes of Anthropic and OpenAI? You can wipe out the mosquitos around your house but if you don't get the entire neighborhood you may as well not bother.
> The best way to stifle AI growth would be to deprive the labs of capital, and yet these AI safetyists never seem to seriously attack the money side.
That's because you're up against billions if not trillions of dollars desperately searching for any kind of return on investment, and there aren't that many industries left now that China has everyone by the balls.
As much as I'd love to see some enforcement action before the AI bros turn the world into another horror show like 2007ff... it won't happen. And just like we didn't have any meaningful consequences back then, this time the ones responsible (hello Sam Altman) will get away scot-free as well.
One resignation distracted from the constant failures of AI in order to boost its value yet again, and to advocate for a particular AI company as saintly.
I have yet to see a positive improvement in the world that could be blamed on AI. Why don't they fix the DMVs of the world? The answer is that they would if they could, it would be great PR, but they can't. They can't even operate a phone for the corner drugstore.
I'm bullish on LLMs and they're defeating any optimism I have daily. I should see every program on my computer improving, but they're getting worse. I should see every website getting cleaner while heavy, crappy frameworks put in for developer convenience are being removed and being replaced with feather-light, minimal code and new designs that anticipate my needs. Nope, even getting worse. This is coding, where they're best.
Can somebody tell me anything that AI has improved in my life? The costs are obvious and massive. Just one improvement to hold onto my optimism.
"They" don't care about "fixing" the DMV. It's nice to think that "they would if they could" but in fact, the anti-government party actively cripples and dismantles government services so that people lose faith in government's ability to improve lives. And what would "fixing" it look like, anyway? Lower cost to operate or lower time-in-queue? Those are in conflict and as much as I would appreciate a better experience there when I have to go (once every 5 years), I think few people would be willing to pay $X/year for that.
On the other hand, my personal computer setup is better than it's been in a long time. Other peoples' software that I run has not improved much, but I've managed to fix several long-standing issues by just giving Fable access to my configuration and telling it my intent.
I've also vibe-coded several applications which are lighter-weight versions of things like Pinboard, Descript, and Weechat. I'm not releasing them because I've been doing OSS for 10 years and I've felt first-hand how the economics of open-source is skewed. And in this day and age where anyone can vibe up their own custom app, it doesn't feel worth it anymore.
What this tells me is that there's not proper incentive for the software you use to get "better" according to your values. You have implicit metrics of "cleanliness", "minimalism", "anticipatory design", and I share those values, but having worked as a SWE for 30 years, I have never worked for a corporation which encouraged those values. If I managed to inflict them on the corporate software I created, it was a "yes and" situation and I only did it because I (or other engineers) cared, never because management wanted it.
So the problem is not "AI is failing us" but rather "the system is failing to use these awesome tools to improve our lives". We can use AI to try to improve the system itself, but our system is not a github repo you can deploy to prod. It's a bit trickier to vibe up the improvements to our incentive structure (though I am trying to do it a bit myself).
Internal tooling is basically free. You may not care, but for me that’s a big deal. I don’t have to spend 6 months negotiating with an internal software team to build a trivial plugin that saves me a few hours a week. I ask the genie, and I get it for a couple dollars.
The only improvement I’ve seen is speed of development, but it’s come at the cost of quality. Hence pretty much every single piece of software a consumer uses becoming significantly worse over the past two years.
It’s especially worsened by every leader doing their damndest to force something related to AI into their workflows and products, regardlesss of if it’s a good fit, lest their own leaders or investors decide they are dinosaurs who need to be replaced.
It did not.
Fake psyop slop. Guy worked there for 6 weeks, almost no X followers when he posted his “I quit” thread, WSJ then posted their story around this post 18 mins later.
I'm pretty skeptical as well. As soon as I saw his interview on NBC News last night I texted a friend that this guy is about to get a $500 million seed round for an AI alignment startup.
The entire interview he's just grinning like he has the "answer" to all these companies he alleges don't take safety seriously.
> The entire interview he's just grinning like he has the "answer" to all these companies he alleges don't take safety seriously.
Can also be attributed to stress, nervousness no?
What is your source for 6 weeks? I see since 2024
Which makes you question if this was the plan all along or he was totally naive not to think this though. Imagine if he did this with Aylo (formerly known as MindGeek) :)
Yeah 2 vague articles from 1 guy isn't a "firestorm"
Jacob Coxon was at Anthropic since 2024, so more than 6 weeks.