I'm not sure what happens (I think no one does), but with every other technology, I can opt-out if I don't want to use it.
Even if I get unemployed, I don't see how I can opt-out when it comes to everyone everywhere using it.
Governments historically have considered humans as assets - it's even called "human capital", and they require us for tax revenue and to run the state. What happens when we aren't needed? Even if the AI is magically aligned to the state, I don't see how I end up the winning side of that. It's not like we're getting a voice in how this is shoved down our throats today. Why would it get any better when AGI can do everything humans can do? What still gives us a voice in the world?
The 'voice' anybody has is some function of their capital, ability to leverage it and ability to prevent its erosion by theft, fraud, seduction/politics scams and honeytrapping. Humans can still acquire capital in the future in several ways, although their ability to not be parted from it is in doubt.
1: verbal/literary labour with your vocal cords, or the use of an output tool, to playback existing or newly created works in environments where mechanical methods are disallowed or disfavored,
2: aesthetic labour through precise recreations of other's mental labour in the performing arts,
3: legal labour in acting as an agent to help or hinder existing legal processes in systems that give humans standing,
4: biological labour as a substrate for growing certain transplantable products,
5: smuggling labour (either non-invasive or surgically invasive),
6: political labour to incrementally sway electoral results in any polity where you are still enfranchised,
7: security labour as a canary for infohazards (such as 'diplomat' programs produced by another ASI) inside of a quarantined environment,
8: frontman work as a deniable patsy to enable economic forgery, deception, revenge or warfare by some hypergraph of patron AGIs and the superorganisms they compose with, or without, your knowledge of what your work entails.
Some of these have a strong factor of supply and demand. You can increase your market value by reducing the supply of other humans competing for these revenue sources.
National governments which can print their own currency don't require people for tax revenue. They use taxes to compel people to acquire their currency to give it value. If they can instead compel AIs to acquire their currency, they won't need humans for that either.
I'm not sure if I'm misunderstanding you, but the purpose of taxes isn't to "compel people to acquire their currency", it's to remove that currency from circulation and prevent the currency they've printed from inflating itself out of existence.
Reducing inflation is indeed one of the reasons, but what I was getting at is that the government doesn't "need" people for tax revenue to pay for its services, contrary to popular belief. It needs taxes to make its money valuable.
L. Randall Wray, "What Are Taxes For? The MMT Approach" (2014):
> Sovereign government does not really need revenue in its own currency in order to spend. It is the tax that 'drives' the currency. If we eliminated the tax, people probably would not immediately abandon use of the currency, but the main driver for its use would be gone.
I agree and I think uncertainty causes people to miss this. A dystopian world is good for no-one. The shitty things people do to each other are scarcity motivated and call me delusionally optimistic but I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
> I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
They will have one of the strongest incentive: to remain at the top. Moreover, it will not be that hard. In Soviet Union one honest socialist with four revolvers (major general Vasily Blokhin) could easily kill up to a thousand people per day personally (if he were laboriously working overtime). And back then there wasn't even a global governmental surveillance or AI-drones with explosives.
> They will have one of the strongest incentive: to remain at the top.
At the top of what? After all the massacres the remaining billionaires and their families would be all that's left of humanity. Who would want to rule over a human-empty, machine-led Earth for the rest of their lives? Not to mention the attitude and morality of their families and those close to them.
I can think of several billionaires who would be quite fine ruling over a devastated and machine-led earth. Their families are chosen or bred to agree with their amorality, and they and their dynastic families would rule with pleasure.
Yes, they can print, but they can also tax capital. It's not actually impossible. We already do it a tiny bit, we've done it more in the past, and contrary to the dogma blasted from the billionaire-owned media outlets and drilled in billionaire-think-tank-approved Econ 101 curricula the universe will not actually explode if we do it again.
We're assuming the oligarchs control the government so they can be rich and powerful and use that to lead lavish lives. Well with these hypothetical AGI imbued robots, they could just construct lavish lives directly.
Not much different from the feudal era. You still need access to good lands (energy production, minerals) but an army of computers and robots instead of serfs and knights. Like Asimov's world of Solaria.
Depends on the political system. Under socialism authority will simply improve the environment by reducing your carbon footprint with eight grams of lead. Under capitalism you're already not needed to anybody.
> Why would it get any better when AGI can do everything humans can do?
Because that "better" we are building ourselves with our own hands. How it can get worse? Only if authority forbid us from building ourselves like we always did and decide to improve the environment.
The issue with devaluation of human work is that the automation doesn't have to be real. All you have to do is make nearly all employers agree that a particular job can pay less.
And that's the case today - new AI being a sufficient programmer replacement is enough of a statement to do layoffs with record profits without impact on stock prices.
> What’s the last major life decision you made without at least consulting an AI?
This seems like asking what's the last major decision you made without at least doing a web search. There's nothing wrong with doing a search. The trouble is when you rely too much on what you read, without checking it.
The cynical view would be that all of our politicians are already bought and paid for, and existing political establishments would block any "serious" candidates who support those sorts of policies.
Personally I think that's likely the case _today_, but who knows if that would hold if/when there comes a time when UBI becomes an economic necessity.
> But wouldn't people just vote for UBI in this case?
Why in North Korea people just does not vote for UBI? For UBI government need control over majority of resources generated by the public. But if government has that control, what people vote for no longer matters.
I'm not sure why the people in power would actually be making their own decisions; if the other country's leaders depend on superintelligence to make their realpolitik moves, you'd expect yours would need to do the same to match them.
So, the governance of a country comes down to AI alignment. You can't let the other guys get a leadership advantage through AI, or you'll start losing the competition.
When we get ASI, we're at the end of human decision making. Now's the time that we have to make sure ASI decision making is to our liking.
Ok, then back up a step: Why distribute the power to vote on the basis of people’s fundamental equal dignity instead of descending into warlord-driven feudalism?
Voting has to have the implication of violent force from the polity to dissuade abrogation of the vote. If people need not apply for a functioning economy, and Palantir/Flock like systems reduce the violence potential of the people, then what incentives remain for the state to be subjected to the people's votes?
The state is made up of human beings too. I don't see why they would want most of humanity to starve. Even a power-hungry psycho wants to have people to rule over, and I don't think the average politician's neurotic traits are so extreme as to want to eliminate 99% of humanity and inherit the Earth.
Positive feedback loop (or how it called in English, idk). You just want to stay at the top, and the better political instruments you have for that, the worse governor you can allow yourself to be. And the worse governor you are, the more you develop and use such instruments. And then you can could be even worse governor. And so on, until you kill 99% of humanity because the were strongly opposing your position on the top.
Social safety nets have come about because labor movements were able to use their leverage over capital to extract concessions. When the labor movements have no more leverage, capital won't give it to them out of the goodness of their hearts.
I’ve never had this worry. When I was funemployed last year, I spent my time reading books and writing code and hanging out with friends. If the future is an infinite UBI-funded vacation, I know what I’ll do. “Before the Singularity, read books and throw house parties; after the singularity, etc.”
yeah it's obvious the author has a really high IQ who can keep himself busy with intellectual pursuits and a nice social life of other smart people . His experience will not map to the general population, who find themselves with idle time. Also, a barebones UBI will be hardly sufficient to cover other expenses unless the AI also solves housing, healthcare, insurance, childcare, transportation etc.
I would say most people do in fact need a life of purpose and fulfillment through some aim larger than themselves that has intrinsic value, can reinforce self worth, etc. I think very few people will find purpose and fulfillment through and endless stream of games, movies and parties. The question is what do those people lacking purpose do with their time as they try to find it, or how do they cope with a life of purposelessness?
Kohei Saito writes about this at length in Capital in the Anthropocene. Most obvious solution is to build an economy around social labor that has traditionally been externalized to households by capital. Start collectively paying people to take care of each other, children, the commons, public infrastructure and so on.
As ecological concerns and automation eliminate a large chunk of production it's pointless to maintain a system that only knows how to finance exactly that stuff, so socialize the gains of those and start redirecting them to socially necessary labor. There's much more to do than just playing boardgames.
Any steady-state economy without an overlord of some kind keeping a balance (a Potemkin steady-state economy) will have some subset of actors who randomly stumble into scenarios of creative destruction and gain disproportionate rewards from it. Unless the gains are zeroed out repeatedly, this will snowball right back into the kind of mixed economy we have. A good book on the landscape the economy arises organically out of is Beinhocker.
You can have a steady-state economy even under the most classical of economics like Solow, simply when capital stock stays constant because investment equals depreciation.
And it's no coincidence that Saito is Japanese because that's what Japan has been living in. They've not grown in decades. Without overlords, it's an adaption to a world where the average citizen is 50, ecology cannot sustain consumption and significantly more time will be dedicated to mending the environment, taking care of old people, and putting technology to work to make that possible.
The nature of novel innovation makes it basically unpredictable, I'm less worried about providing a safety net for video game playing pleasure maxxers and far more concerned about that orphan crushing machine we have currently (as an orphan.)
I think the only main critique that comes to mind is the primitivists, who argue that automation has massively destroyed our freedom and psychologically destabilized us (among other things); you really can't meaningfully opt out of whatever technology exists and it forces someone to use it somewhere. It also ultimately takes away certain freedoms that previously existed and creates lots of new problems that didn't exist before (like increased mental health problems in industrialized nations). And so this problem has already happened with the Industrial Revolution - AI is basically just a continuation of these problems, not perhaps different in kind but degree.
> Alignment
is an issue to me, but any currently existing technology can also be misused to some extent
> Control
again we're not in control of current technology, new things are developed that other people have no control over but just "adapt" to
> by letting the AI write, you are giving up most of what makes up thinking
I guess I haven't been viewing things this way but have been filtering ways I anticipate AI should be used and not used as. In other words, I think it's a great use of AI to share a prompt and AI output as a way of dialogue "writing". For example, I might want to give you 10 ideas on something to bake, and AI outputs 10 ideas that are actually valuable, so sharing the list does add value. But in how I would "write" with AI, it would be clear where I am prompting and where the AI takes over. I also think that a person should read the AI output and essentially indicate points of agreement or disagreement especially if there is some serious defect in the AI output.
> The discourse around software engineering has gotten dumber
Good and bad; it's enabled people with less SE experience (or intelligence even) to talk more about SE and become a meaningful participant in the process of software creation. But it takes less specialized skill for some of these things than some of the previously developed skills, and that can have a different feel to it.
> Deferring to AI
Again I guess I am filtering a bad use of this. AI frequently gives "good" answers that I would have manually hunted for previously on a search engine, and I still can sift through what it outputs. So I think it's great as a brainstorm tool that I somewhat freely choose from. I can understand how other people may not be using it in the same way as I use it though, more like an aide rather than a smarter entity I am taking orders from.
> Stagnation
I guess it's ironic because although I also focus on "stagnation", I guess there is indeed a lot of actual "progress" going on. It's more of a political issue perhaps than a technological issue.
And even then, reaching "I, Robot" levels is unrealistic. Machines are fragile. It would take a very long time to bootstrap the entire automated robot construction and maintenance supply chain, which would also require energy production and resource collection too.
Until then it is vulnerable to disruption. So of course you will need the robot army to forcibly take and defend resources around the world. How will that war turn out?
Robot labor will be more valuable than humans for a very long time. Heck, we still have many towns in the US that have humans running around throwing trash bags into the back of the truck, because they haven't switched to the standardized containers and hydrologic arms. Let's not even get started on the rest of the "developing" world.
Any non-maritime part of the supply chain can be extremely well-defended from disruption as long as you marry agentic defense planning based on existing crime prediction machine learning SaaS platforms to loyal human puppets with boots and guns (police, ICE, military).
>Before LLMs, what was there to be excited about in the near future? Solar power? Self-driving cars? The energy transition? It’s laughable.
I, for one, would have welcomed our green energy revolution overlords.
Also, VR/AR, before the Great Powers deep-sixed any real efforts. Growing up on .hack, GitS, and SAO, it really is a bummer. How come I must know, where the techbros lead me to go?
TL;DR: the same tired points you've read a million times already
"It might be slop but at least it's human slop!" - would the author argue in turn, probably:
> People used to argue online, flame each other, get angry. And that wasn’t good, but at least it was human. [0]
Another perspective though could be that one's thoughts were never meant to be guaranteed high-entropy curio for others to encounter, so they're really under no obligation to be found "original", if that even means anything. [1] And sure, that is true.
Just also not any more true than the former.
[0] emphasis theirs, I don't even need to mock it like this; the secondhand embarrassment comes built-in
[1] this is to say, these may be genuinely candid and original thoughts from the author, that simply happen to coincide with the many thoughts already expressed in this genre of writing
One thing the doom debate tends to skip: the failures that hurt people right now are boring, not exotic. I spend my time probing AI systems that are already deployed and talking to real users, and the actual failure modes are mundane - a system that reassures when it should escalate, follows its script while the situation changes, sounds confident the whole way through. No superintelligence required, just an ordinary model given authority and no tripwires. If the conversation centered on the systems already in production instead of hypothetical future ones, a lot of the disagreement about timelines would matter less. The dangerous stuff is already here; it's just unglamorous.
All it would actually take to avert a jobs crisis is for the per-token price of LLM generation to increase to the point where humans are competitive in dollar terms for intellectual work.
This could happen all by itself if the major LLM players coordinated (i.e., formed a cartel), or if they were forced to to recoup investments (we all know this is coming to some degree, at least for the western companies). Alternatively it could be forced into place if ~all major nations' governments start taxing AI use (which seems eminently a good idea to me in any case).
There's still the issue of local LLMs, but this could be addressed with taxes on GPU hardware, etc.
> You might object: if we’re all jobless, who’s paying for everything? This is trivially answered: the state acts like the heart, taxes are venous blood and welfare is oxygenated arterial blood. The government pays Raytheon for missiles, the money cascades down the economy through factories, aluminium smelters, mines, transport companies, all staffed by AIs buying and selling from each other. The government takes a cut of all economic activity, pays out welfare, the unemployed masses buy food and pay rent, the supermarkets, farms, logistics network, etc. are all staffed by AI.
I don't think this person has a firm enough grasp of economics to be writing on the subject.
I'm not sure what happens (I think no one does), but with every other technology, I can opt-out if I don't want to use it.
Even if I get unemployed, I don't see how I can opt-out when it comes to everyone everywhere using it.
Governments historically have considered humans as assets - it's even called "human capital", and they require us for tax revenue and to run the state. What happens when we aren't needed? Even if the AI is magically aligned to the state, I don't see how I end up the winning side of that. It's not like we're getting a voice in how this is shoved down our throats today. Why would it get any better when AGI can do everything humans can do? What still gives us a voice in the world?
The 'voice' anybody has is some function of their capital, ability to leverage it and ability to prevent its erosion by theft, fraud, seduction/politics scams and honeytrapping. Humans can still acquire capital in the future in several ways, although their ability to not be parted from it is in doubt.
1: verbal/literary labour with your vocal cords, or the use of an output tool, to playback existing or newly created works in environments where mechanical methods are disallowed or disfavored,
2: aesthetic labour through precise recreations of other's mental labour in the performing arts,
3: legal labour in acting as an agent to help or hinder existing legal processes in systems that give humans standing,
4: biological labour as a substrate for growing certain transplantable products,
5: smuggling labour (either non-invasive or surgically invasive),
6: political labour to incrementally sway electoral results in any polity where you are still enfranchised,
7: security labour as a canary for infohazards (such as 'diplomat' programs produced by another ASI) inside of a quarantined environment,
8: frontman work as a deniable patsy to enable economic forgery, deception, revenge or warfare by some hypergraph of patron AGIs and the superorganisms they compose with, or without, your knowledge of what your work entails.
Some of these have a strong factor of supply and demand. You can increase your market value by reducing the supply of other humans competing for these revenue sources.
[dead]
National governments which can print their own currency don't require people for tax revenue. They use taxes to compel people to acquire their currency to give it value. If they can instead compel AIs to acquire their currency, they won't need humans for that either.
I'm not sure if I'm misunderstanding you, but the purpose of taxes isn't to "compel people to acquire their currency", it's to remove that currency from circulation and prevent the currency they've printed from inflating itself out of existence.
Reducing inflation is indeed one of the reasons, but what I was getting at is that the government doesn't "need" people for tax revenue to pay for its services, contrary to popular belief. It needs taxes to make its money valuable.
L. Randall Wray, "What Are Taxes For? The MMT Approach" (2014):
> Sovereign government does not really need revenue in its own currency in order to spend. It is the tax that 'drives' the currency. If we eliminated the tax, people probably would not immediately abandon use of the currency, but the main driver for its use would be gone.
The government is made up of human beings too. I doubt most of them are psycho to such a degree as to let 99% of humanity die for no real reason.
At the risk of being incredibly banal, the current US government shows you absolutely can consolidate enough of such psychos.
Game theory is a helluva thing.
All you need is 1% of humanity to be psycho enough to let the other 99% die out.
And I do not have much faith in the charity of the most powerful 1%. Charitable people often do not make it into that group.
I agree and I think uncertainty causes people to miss this. A dystopian world is good for no-one. The shitty things people do to each other are scarcity motivated and call me delusionally optimistic but I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
> I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
They will have one of the strongest incentive: to remain at the top. Moreover, it will not be that hard. In Soviet Union one honest socialist with four revolvers (major general Vasily Blokhin) could easily kill up to a thousand people per day personally (if he were laboriously working overtime). And back then there wasn't even a global governmental surveillance or AI-drones with explosives.
> They will have one of the strongest incentive: to remain at the top.
At the top of what? After all the massacres the remaining billionaires and their families would be all that's left of humanity. Who would want to rule over a human-empty, machine-led Earth for the rest of their lives? Not to mention the attitude and morality of their families and those close to them.
I can think of several billionaires who would be quite fine ruling over a devastated and machine-led earth. Their families are chosen or bred to agree with their amorality, and they and their dynastic families would rule with pleasure.
Yes, they can print, but they can also tax capital. It's not actually impossible. We already do it a tiny bit, we've done it more in the past, and contrary to the dogma blasted from the billionaire-owned media outlets and drilled in billionaire-think-tank-approved Econ 101 curricula the universe will not actually explode if we do it again.
We're assuming the oligarchs control the government so they can be rich and powerful and use that to lead lavish lives. Well with these hypothetical AGI imbued robots, they could just construct lavish lives directly.
Not much different from the feudal era. You still need access to good lands (energy production, minerals) but an army of computers and robots instead of serfs and knights. Like Asimov's world of Solaria.
If humans aren’t needed, neither is government/state.
> What happens when we aren't needed?
Depends on the political system. Under socialism authority will simply improve the environment by reducing your carbon footprint with eight grams of lead. Under capitalism you're already not needed to anybody.
> Why would it get any better when AGI can do everything humans can do?
Because that "better" we are building ourselves with our own hands. How it can get worse? Only if authority forbid us from building ourselves like we always did and decide to improve the environment.
The issue with devaluation of human work is that the automation doesn't have to be real. All you have to do is make nearly all employers agree that a particular job can pay less.
And that's the case today - new AI being a sufficient programmer replacement is enough of a statement to do layoffs with record profits without impact on stock prices.
> What’s the last major life decision you made without at least consulting an AI?
This seems like asking what's the last major decision you made without at least doing a web search. There's nothing wrong with doing a search. The trouble is when you rely too much on what you read, without checking it.
> if humans are economically useless, the state does not need them. Why pay out UBI to people who have neither economic nor political power?
Forgive me if I'm naïve. But wouldn't people just vote for UBI in this case?
The cynical view would be that all of our politicians are already bought and paid for, and existing political establishments would block any "serious" candidates who support those sorts of policies.
Personally I think that's likely the case _today_, but who knows if that would hold if/when there comes a time when UBI becomes an economic necessity.
> But wouldn't people just vote for UBI in this case?
Why in North Korea people just does not vote for UBI? For UBI government need control over majority of resources generated by the public. But if government has that control, what people vote for no longer matters.
The US can't even vote for universal healthcare. I have real trouble seeing a path for UBI.
I'm not sure why the people in power would actually be making their own decisions; if the other country's leaders depend on superintelligence to make their realpolitik moves, you'd expect yours would need to do the same to match them.
So, the governance of a country comes down to AI alignment. You can't let the other guys get a leadership advantage through AI, or you'll start losing the competition.
When we get ASI, we're at the end of human decision making. Now's the time that we have to make sure ASI decision making is to our liking.
Ok, then back up a step: Why distribute the power to vote on the basis of people’s fundamental equal dignity instead of descending into warlord-driven feudalism?
Voting has to have the implication of violent force from the polity to dissuade abrogation of the vote. If people need not apply for a functioning economy, and Palantir/Flock like systems reduce the violence potential of the people, then what incentives remain for the state to be subjected to the people's votes?
The state is made up of human beings too. I don't see why they would want most of humanity to starve. Even a power-hungry psycho wants to have people to rule over, and I don't think the average politician's neurotic traits are so extreme as to want to eliminate 99% of humanity and inherit the Earth.
Positive feedback loop (or how it called in English, idk). You just want to stay at the top, and the better political instruments you have for that, the worse governor you can allow yourself to be. And the worse governor you are, the more you develop and use such instruments. And then you can could be even worse governor. And so on, until you kill 99% of humanity because the were strongly opposing your position on the top.
[dead]
Social safety nets have come about because labor movements were able to use their leverage over capital to extract concessions. When the labor movements have no more leverage, capital won't give it to them out of the goodness of their hearts.
Do the downvoters of this comment have an alternative set of examples from history or do they just dislike this uncomfortable fact?
What leverage would you have to enforce the will of the vote on those that control things?
Who says you get a vote against AI-powered drones though.
Good luck having votes
https://en.wikipedia.org/wiki/2026_Nicaraguan_general_electi...
> what if GPT-3 hadn’t worked? What would we have to look forward to, in this alternate 2026?
One would hope maybe boring things like healthcare, better living conditions, better functioning societies, and so on.
Not as exciting as flying cars and thinking machines for sure.
I’ve never had this worry. When I was funemployed last year, I spent my time reading books and writing code and hanging out with friends. If the future is an infinite UBI-funded vacation, I know what I’ll do. “Before the Singularity, read books and throw house parties; after the singularity, etc.”
yeah it's obvious the author has a really high IQ who can keep himself busy with intellectual pursuits and a nice social life of other smart people . His experience will not map to the general population, who find themselves with idle time. Also, a barebones UBI will be hardly sufficient to cover other expenses unless the AI also solves housing, healthcare, insurance, childcare, transportation etc.
You don't need high IQ to enjoy reading or hikink, most people could and should decide to like it more.
For those with low IQ like me, there's video games, movies, piracy, parties, and hiking.
I would say most people do in fact need a life of purpose and fulfillment through some aim larger than themselves that has intrinsic value, can reinforce self worth, etc. I think very few people will find purpose and fulfillment through and endless stream of games, movies and parties. The question is what do those people lacking purpose do with their time as they try to find it, or how do they cope with a life of purposelessness?
And probably AI will make those even more addictive
> His experience will not map to the general population
Well, I can't solve everyone's lives.
Kohei Saito writes about this at length in Capital in the Anthropocene. Most obvious solution is to build an economy around social labor that has traditionally been externalized to households by capital. Start collectively paying people to take care of each other, children, the commons, public infrastructure and so on.
As ecological concerns and automation eliminate a large chunk of production it's pointless to maintain a system that only knows how to finance exactly that stuff, so socialize the gains of those and start redirecting them to socially necessary labor. There's much more to do than just playing boardgames.
Any steady-state economy without an overlord of some kind keeping a balance (a Potemkin steady-state economy) will have some subset of actors who randomly stumble into scenarios of creative destruction and gain disproportionate rewards from it. Unless the gains are zeroed out repeatedly, this will snowball right back into the kind of mixed economy we have. A good book on the landscape the economy arises organically out of is Beinhocker.
You can have a steady-state economy even under the most classical of economics like Solow, simply when capital stock stays constant because investment equals depreciation.
And it's no coincidence that Saito is Japanese because that's what Japan has been living in. They've not grown in decades. Without overlords, it's an adaption to a world where the average citizen is 50, ecology cannot sustain consumption and significantly more time will be dedicated to mending the environment, taking care of old people, and putting technology to work to make that possible.
The nature of novel innovation makes it basically unpredictable, I'm less worried about providing a safety net for video game playing pleasure maxxers and far more concerned about that orphan crushing machine we have currently (as an orphan.)
> Automation has been good
I think the only main critique that comes to mind is the primitivists, who argue that automation has massively destroyed our freedom and psychologically destabilized us (among other things); you really can't meaningfully opt out of whatever technology exists and it forces someone to use it somewhere. It also ultimately takes away certain freedoms that previously existed and creates lots of new problems that didn't exist before (like increased mental health problems in industrialized nations). And so this problem has already happened with the Industrial Revolution - AI is basically just a continuation of these problems, not perhaps different in kind but degree.
> Alignment
is an issue to me, but any currently existing technology can also be misused to some extent
> Control
again we're not in control of current technology, new things are developed that other people have no control over but just "adapt" to
> by letting the AI write, you are giving up most of what makes up thinking
I guess I haven't been viewing things this way but have been filtering ways I anticipate AI should be used and not used as. In other words, I think it's a great use of AI to share a prompt and AI output as a way of dialogue "writing". For example, I might want to give you 10 ideas on something to bake, and AI outputs 10 ideas that are actually valuable, so sharing the list does add value. But in how I would "write" with AI, it would be clear where I am prompting and where the AI takes over. I also think that a person should read the AI output and essentially indicate points of agreement or disagreement especially if there is some serious defect in the AI output.
> The discourse around software engineering has gotten dumber
Good and bad; it's enabled people with less SE experience (or intelligence even) to talk more about SE and become a meaningful participant in the process of software creation. But it takes less specialized skill for some of these things than some of the previously developed skills, and that can have a different feel to it.
> Deferring to AI
Again I guess I am filtering a bad use of this. AI frequently gives "good" answers that I would have manually hunted for previously on a search engine, and I still can sift through what it outputs. So I think it's great as a brainstorm tool that I somewhat freely choose from. I can understand how other people may not be using it in the same way as I use it though, more like an aide rather than a smarter entity I am taking orders from.
> Stagnation
I guess it's ironic because although I also focus on "stagnation", I guess there is indeed a lot of actual "progress" going on. It's more of a political issue perhaps than a technological issue.
Actual AGI might be really bad but thankfully we are not even close to building it and I doubt I’ll see it in my lifetime (imo ofc)
And even then, reaching "I, Robot" levels is unrealistic. Machines are fragile. It would take a very long time to bootstrap the entire automated robot construction and maintenance supply chain, which would also require energy production and resource collection too.
Until then it is vulnerable to disruption. So of course you will need the robot army to forcibly take and defend resources around the world. How will that war turn out?
Robot labor will be more valuable than humans for a very long time. Heck, we still have many towns in the US that have humans running around throwing trash bags into the back of the truck, because they haven't switched to the standardized containers and hydrologic arms. Let's not even get started on the rest of the "developing" world.
Any non-maritime part of the supply chain can be extremely well-defended from disruption as long as you marry agentic defense planning based on existing crime prediction machine learning SaaS platforms to loyal human puppets with boots and guns (police, ICE, military).
If you're betting the farm on there being a wall on the current trajectory of LLMs, I suggest you hedge that bet.
If I wasn’t betting the farm I’d go into debt arming myself somewhere in northern Siberia with enough rations to live a lifetime.
Since neither of us are doing this, we’re apparently both betting our lives on the current trajectory of LLMs hitting a wall soon and fast.
LLMs are very limited and have finite usefulness. The idea of AGI is a grift.
>Before LLMs, what was there to be excited about in the near future? Solar power? Self-driving cars? The energy transition? It’s laughable.
I, for one, would have welcomed our green energy revolution overlords.
Also, VR/AR, before the Great Powers deep-sixed any real efforts. Growing up on .hack, GitS, and SAO, it really is a bummer. How come I must know, where the techbros lead me to go?
TL;DR: the same tired points you've read a million times already
"It might be slop but at least it's human slop!" - would the author argue in turn, probably:
> People used to argue online, flame each other, get angry. And that wasn’t good, but at least it was human. [0]
Another perspective though could be that one's thoughts were never meant to be guaranteed high-entropy curio for others to encounter, so they're really under no obligation to be found "original", if that even means anything. [1] And sure, that is true.
Just also not any more true than the former.
[0] emphasis theirs, I don't even need to mock it like this; the secondhand embarrassment comes built-in
[1] this is to say, these may be genuinely candid and original thoughts from the author, that simply happen to coincide with the many thoughts already expressed in this genre of writing
One thing the doom debate tends to skip: the failures that hurt people right now are boring, not exotic. I spend my time probing AI systems that are already deployed and talking to real users, and the actual failure modes are mundane - a system that reassures when it should escalate, follows its script while the situation changes, sounds confident the whole way through. No superintelligence required, just an ordinary model given authority and no tripwires. If the conversation centered on the systems already in production instead of hypothetical future ones, a lot of the disagreement about timelines would matter less. The dangerous stuff is already here; it's just unglamorous.
[dead]
All it would actually take to avert a jobs crisis is for the per-token price of LLM generation to increase to the point where humans are competitive in dollar terms for intellectual work.
This could happen all by itself if the major LLM players coordinated (i.e., formed a cartel), or if they were forced to to recoup investments (we all know this is coming to some degree, at least for the western companies). Alternatively it could be forced into place if ~all major nations' governments start taxing AI use (which seems eminently a good idea to me in any case).
There's still the issue of local LLMs, but this could be addressed with taxes on GPU hardware, etc.
> You might object: if we’re all jobless, who’s paying for everything? This is trivially answered: the state acts like the heart, taxes are venous blood and welfare is oxygenated arterial blood. The government pays Raytheon for missiles, the money cascades down the economy through factories, aluminium smelters, mines, transport companies, all staffed by AIs buying and selling from each other. The government takes a cut of all economic activity, pays out welfare, the unemployed masses buy food and pay rent, the supermarkets, farms, logistics network, etc. are all staffed by AI.
I don't think this person has a firm enough grasp of economics to be writing on the subject.