> The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language.
> The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it...
> The third warning was about environmental cost.
> The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit.
> The fifth warning was the one Google cared about most. Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them.
Personally I'm not convinced on the first two. The third is obviously a concern. The fourth seems logical, but I'm sure what the impact is, if any. The fifth is a problem, I suppose, but one that already exists in so many other capacities.
There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes.
It's incredibly depressing that the concept of "bias" has been shrunken down to solely mean "bad attitudes about an ethnic or gender ground" (and perhaps on the right, "bad attitudes about conservatives")
Bias could mean so, so many other things. Was the amyloid hypothesis incorrect? How should we use semicolons? How do you know when meetings waste more time than not? etc. People understand the world via mental shortcuts, via theory-rather-than-fact. We're stuck doing this because we're limited in so many ways. We are so biased about so many things, and this could interact in so many interesting ways. But damned if anyone cares about that. The only thing they seem to care about is how you feel about the "right" or "wrong" groups of people. It's a catastrophic waste of time and energy.
Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confidence in statements (like, "the antecedent of the feminine pronoun in this sentence, is it the 'nurse' or the 'doctor'").
There's plenty of research into biases in LLMs, and there should be; it's a fundamentally new branch of computer science that could have profound impacts on how we automate and regiment social decisions in the future (like extending credit). The bias concern is well taken in those settings. But it has very little to do with the overwhelming majority of day-to-day LLM use; Claude and ChatGPT are not indoctrinating into the manosphere users asking about discounted cash flow formulae.
I had a good laugh when Haiku's thinking summarization referred to mayor Mamdani as a, quote, "known anti-Zionist." :-)
(Context: I asked it to write fake Reddit comments, because I was curious about how realistic they could be. The colorful phrase occurred during its reasoning about the requested subjects.)
Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.
The bias concerns in Gebru's paper cover pre-LLM systems. For all we know, modern frontier models might mitigate many of the concerns the paper brings up. It's hard to know. The logic used in summaries like the one we're commenting on is conclusory: centuries of prejudice are encoded in the total corpus of human language, language models are trained on that corpus, ergo language models must be biased.
When a researcher discovers that smoking is damaging to the lungs, do they need to provide a solution that allows people to smoke without damaging their lungs? Would their inability to provide a solution take anything away from the research?
Acute would imply that we should flat out stop. Chronic would imply looking for plans to work on it. Acute and chronic would imply that we should both stop and take action to address damages.
If you’re referring to a solution to large datasets without not being auditable, she actually did provide a solution. Something to do with data sheets for these training data sets similar to those provided for hardware components. At least, if my memory serves me.
I was more irked by the diversity of teams developing these concern. Which, feels like a benign enough concern, but not one where you can just stop progress.
Worse, I think it is a ridiculously safe bet that the US was home to the most diverse teams you could get for this sort of work. Asking the good faith participants to stop participating would have decreased the stated goal.
If the criticism can't distill up from "bad things could happen", it just isn't useful to keep paying people to come up with that kind of critique.
And it isn't like we stopped paying attention to these concerns, is it? Nor were they completely blind siding us at the time. The question was largely of what to do about them.
The question also whether large-scale utilization of LLMs (and also the prerequisite increased training processes) should proceed before these issues were addressed. Clearly, we collectively answered "yes" without any actual reasoning (and arguably, without any collective decision making either).
This feels incoherent. I'm game to agree that there were and are poor decisions being made. But are you proposing that we could have stopped all progress until these vague concerns were addressed?
For some of the concerns, like language understanding, I can't bring myself to think that many of the experts out there were doing any better than these models can do today. Quite the contrary.
And do you think that that would not have been counter to the concern over diversity of teams working on it?
Or concerns over bias going away by having the US attempt to abstain? Good luck with that. It sucks, but China and Russia should stand as stark examples that it turns out you can take strong control over the internet.
During the time that this paper was written agents were not really a thing. I would be more concerned about centralisation of work itself as a bigger concern
Yeah, I think it's pretty clear that LLMs are more than mere "stochastic parrots" - they can prove theorems, follow instructions, and complete complex tasks.
This was the most notable claim of the paper, and it's aged very poorly.
Are they, though? I think what LLMs proved is that proving theorems, following instructions and solving complex problems - intelligent behaviour - does not need any kind of understanding, but only ability to recombine things in a stochastic matter. Which basically just means that these things weren't as special as people had thought.
We've clearly crossed a threshold at which "stochastic" is no longer doing the work Gebru (and, more importantly, the acolytes of this paper; I shouldn't tar Gebru with what they've done with the work) expected it to do. Lots of important processes are stochastic, including at some levels human thought itself. Advocates who deploy the term "stochastic" seem to believe it impeaches the technology, which is kind of embarrassing to see.
I think you have already decided that LLMs cannot possibly understand. Therefore anything they do must not have required understanding in the first place. It's circular logic.
Careful, you're responding to a summary of the Stochastic Parrot paper, but not the paper itself, which isn't structured this way.
For instance, the paper doesn't raises model collapse (not using that term) as a risk, a possibility. It doesn't predict it with certainty, unlike this summary, which appears to believe something like it has actually occurred.
It seems that the main issue with AI is often not what sci-fi or EA-adjacent prophets are trying to warn us about, but the insidious dangers of the failure modes.
We are collectively not well calibrated to deal with systems that seems capable but fails in surprising ways.
Commercial planes are still under the responsibility and control of highly trained human pilots, even if I am pretty sure that full automation would be technically feasible, even without relying on modern AI, I don't think any companies would be comfortable with the liability.
The first issue I have with the article is the title. I followed this whole saga very closely when it happened, and while I definitely understand the nuance of her separation, I agree with Google that Gebru wasn't fired - she quit.
I do not understand what universe you must live in to think you can come to your employer and make a large list of demands (including demands that can easily be taken as subtle or not so subtle threats to your colleagues), say "if you don't meet these demands then I'm going to quit, and quit loudly", and then when the company accepts your proposal by saying "OK, fine, we don't accept your demands so we're accepting your resignation", and then you try to backtrack with a surprised Pikachu face and then cry loudly about how Google fired you. Seriously, where I come from the response would be "get bent."
I also would highlight that the biggest complaint in the paper was how LLMs amplified bias. Google was laughed at for one of its Gemini releases from just a few years back (can't remember if it was called Gemini then) where one commenter noted "it is extremely difficult to get Google's AI to believe white people exist", as they so obviously overcorrected on the racial bias issue where image generation was creating black Nazis and Asian medieval kings of England.
This paper has not held up, like, at all. The first half of it recites Woke 1.0 principles, like a concern that LMs will thwart efforts to "decolonialize education by shifting to oral histories" in order to avoid the biases of "text". The second half of it makes predictions from axioms about LMs not truly understanding text that nobody would take seriously today.
There's philosophical grappling to be done, as with the Ted Chiang post on the front page right now, about what it is LLMs are actually doing (I'm mostly with Chiang on those core philosophical issues). But Gebru went way past that, attacking their underlying utility. The coherency of GPT 5.5 responses are not simply tricks of the mind, and frontier models (leaving aside Grok, if you want to call it a frontier model) have not in fact been engines for bias.
I don't want to say this has not happened, but where's the evidence of anything in this article?
According to the article she resigned, which is very different from getting fired, so what is the information the author has to substantiate this claim?
The warnings:
Personally I'm not convinced on the first two. The third is obviously a concern. The fourth seems logical, but I'm sure what the impact is, if any. The fifth is a problem, I suppose, but one that already exists in so many other capacities.There has been plenty of research that shows LLMs encode social biases. It seems pretty obvious even before looking at the research that training on the whole internet will end up encoding widely-held social biases and stereotypes.
https://arxiv.org/pdf/2508.07111
https://github.com/angl1n/social-bias-llm-vlm
It's incredibly depressing that the concept of "bias" has been shrunken down to solely mean "bad attitudes about an ethnic or gender ground" (and perhaps on the right, "bad attitudes about conservatives")
Bias could mean so, so many other things. Was the amyloid hypothesis incorrect? How should we use semicolons? How do you know when meetings waste more time than not? etc. People understand the world via mental shortcuts, via theory-rather-than-fact. We're stuck doing this because we're limited in so many ways. We are so biased about so many things, and this could interact in so many interesting ways. But damned if anyone cares about that. The only thing they seem to care about is how you feel about the "right" or "wrong" groups of people. It's a catastrophic waste of time and energy.
Have you read through the sources on that Github link? It's a set of sociology cites establishing that bias exists (something no serious person ever disputed), followed by a couple papers showing mechanistic descriptions of how bias could propagate through an LLM. The paper you call out specifically takes last-generation open-weights models and attempts to trick them into revealing biases through their level of confidence in statements (like, "the antecedent of the feminine pronoun in this sentence, is it the 'nurse' or the 'doctor'").
There's plenty of research into biases in LLMs, and there should be; it's a fundamentally new branch of computer science that could have profound impacts on how we automate and regiment social decisions in the future (like extending credit). The bias concern is well taken in those settings. But it has very little to do with the overwhelming majority of day-to-day LLM use; Claude and ChatGPT are not indoctrinating into the manosphere users asking about discounted cash flow formulae.
(Maybe Grok is though.)
I confess I laughed harder at the Grok comment than I wish I had. Sad to remember that some strawmen are given life and promoted by people. Actively.
I had a good laugh when Haiku's thinking summarization referred to mayor Mamdani as a, quote, "known anti-Zionist." :-)
(Context: I asked it to write fake Reddit comments, because I was curious about how realistic they could be. The colorful phrase occurred during its reasoning about the requested subjects.)
> There has been plenty of research that shows LLMs encode social biases.
At the risk of stepping into a hornets nest: is that different than "knowledge"?
Or maybe, what would it mean if an LLM had no social biases? (Would we ever agree that was the case?)
Yes, it would be extremely bad if the statistical weight of the total corpus of training data caused a system using an LLM to make decisions about extending credit to offer worse terms (say) to women.
And papers on bias amplification in ML predate LLMs. I remember this specific one which was a spotlight paper at EMNLP:
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, Zhao et al.
https://arxiv.org/abs/1707.09457
The bias concerns in Gebru's paper cover pre-LLM systems. For all we know, modern frontier models might mitigate many of the concerns the paper brings up. It's hard to know. The logic used in summaries like the one we're commenting on is conclusory: centuries of prejudice are encoded in the total corpus of human language, language models are trained on that corpus, ergo language models must be biased.
More than not being entirely sure what the impact is, I don't see any suggestion at what to do about it?
When a researcher discovers that smoking is damaging to the lungs, do they need to provide a solution that allows people to smoke without damaging their lungs? Would their inability to provide a solution take anything away from the research?
To conflate AI with smoking is just not helpful. At all.
Or are you saying that there are acute harms from AI that are being ignored?
Acute, chronic - why would it matter?
Why is it unhelpful to conflate AI with smoking?
And yes, lots of people are saying "there are harms from AI that are being ignored".
Acute would imply that we should flat out stop. Chronic would imply looking for plans to work on it. Acute and chronic would imply that we should both stop and take action to address damages.
What harms from AI are people ignoring?
If you’re referring to a solution to large datasets without not being auditable, she actually did provide a solution. Something to do with data sheets for these training data sets similar to those provided for hardware components. At least, if my memory serves me.
I was more irked by the diversity of teams developing these concern. Which, feels like a benign enough concern, but not one where you can just stop progress.
Worse, I think it is a ridiculously safe bet that the US was home to the most diverse teams you could get for this sort of work. Asking the good faith participants to stop participating would have decreased the stated goal.
Why should the person identifying the problem provide a solution? This doesn't make sense.
If the criticism can't distill up from "bad things could happen", it just isn't useful to keep paying people to come up with that kind of critique.
And it isn't like we stopped paying attention to these concerns, is it? Nor were they completely blind siding us at the time. The question was largely of what to do about them.
The question also whether large-scale utilization of LLMs (and also the prerequisite increased training processes) should proceed before these issues were addressed. Clearly, we collectively answered "yes" without any actual reasoning (and arguably, without any collective decision making either).
This feels incoherent. I'm game to agree that there were and are poor decisions being made. But are you proposing that we could have stopped all progress until these vague concerns were addressed?
For some of the concerns, like language understanding, I can't bring myself to think that many of the experts out there were doing any better than these models can do today. Quite the contrary.
And do you think that that would not have been counter to the concern over diversity of teams working on it?
Or concerns over bias going away by having the US attempt to abstain? Good luck with that. It sucks, but China and Russia should stand as stark examples that it turns out you can take strong control over the internet.
During the time that this paper was written agents were not really a thing. I would be more concerned about centralisation of work itself as a bigger concern
Yeah, I think it's pretty clear that LLMs are more than mere "stochastic parrots" - they can prove theorems, follow instructions, and complete complex tasks.
This was the most notable claim of the paper, and it's aged very poorly.
Are they, though? I think what LLMs proved is that proving theorems, following instructions and solving complex problems - intelligent behaviour - does not need any kind of understanding, but only ability to recombine things in a stochastic matter. Which basically just means that these things weren't as special as people had thought.
We've clearly crossed a threshold at which "stochastic" is no longer doing the work Gebru (and, more importantly, the acolytes of this paper; I shouldn't tar Gebru with what they've done with the work) expected it to do. Lots of important processes are stochastic, including at some levels human thought itself. Advocates who deploy the term "stochastic" seem to believe it impeaches the technology, which is kind of embarrassing to see.
I think you have already decided that LLMs cannot possibly understand. Therefore anything they do must not have required understanding in the first place. It's circular logic.
The second point is only true if you don't do any RL, right?
Careful, you're responding to a summary of the Stochastic Parrot paper, but not the paper itself, which isn't structured this way.
For instance, the paper doesn't raises model collapse (not using that term) as a risk, a possibility. It doesn't predict it with certainty, unlike this summary, which appears to believe something like it has actually occurred.
It seems that the main issue with AI is often not what sci-fi or EA-adjacent prophets are trying to warn us about, but the insidious dangers of the failure modes.
We are collectively not well calibrated to deal with systems that seems capable but fails in surprising ways.
Commercial planes are still under the responsibility and control of highly trained human pilots, even if I am pretty sure that full automation would be technically feasible, even without relying on modern AI, I don't think any companies would be comfortable with the liability.
The first issue I have with the article is the title. I followed this whole saga very closely when it happened, and while I definitely understand the nuance of her separation, I agree with Google that Gebru wasn't fired - she quit.
I do not understand what universe you must live in to think you can come to your employer and make a large list of demands (including demands that can easily be taken as subtle or not so subtle threats to your colleagues), say "if you don't meet these demands then I'm going to quit, and quit loudly", and then when the company accepts your proposal by saying "OK, fine, we don't accept your demands so we're accepting your resignation", and then you try to backtrack with a surprised Pikachu face and then cry loudly about how Google fired you. Seriously, where I come from the response would be "get bent."
I also would highlight that the biggest complaint in the paper was how LLMs amplified bias. Google was laughed at for one of its Gemini releases from just a few years back (can't remember if it was called Gemini then) where one commenter noted "it is extremely difficult to get Google's AI to believe white people exist", as they so obviously overcorrected on the racial bias issue where image generation was creating black Nazis and Asian medieval kings of England.
What is/was the source of this rather than random tumblr?
This May 26th Twitter post ...maybe? Account now suspended https://x.com/heygurisingh/status/2059251382960734593
(http://web.archive.org/web/20260526123243/https://twitter.co...)
Looks like the dude got suspended for being a bot: https://piunikaweb.com/2026/05/28/x-suspend-accounts-ai-repl...
(direct link: https://x.com/nikitabier/status/2059789636885790911 )
This paper has not held up, like, at all. The first half of it recites Woke 1.0 principles, like a concern that LMs will thwart efforts to "decolonialize education by shifting to oral histories" in order to avoid the biases of "text". The second half of it makes predictions from axioms about LMs not truly understanding text that nobody would take seriously today.
There's philosophical grappling to be done, as with the Ted Chiang post on the front page right now, about what it is LLMs are actually doing (I'm mostly with Chiang on those core philosophical issues). But Gebru went way past that, attacking their underlying utility. The coherency of GPT 5.5 responses are not simply tricks of the mind, and frontier models (leaving aside Grok, if you want to call it a frontier model) have not in fact been engines for bias.
I don't want to say this has not happened, but where's the evidence of anything in this article?
According to the article she resigned, which is very different from getting fired, so what is the information the author has to substantiate this claim?
I agree. Why is someone's lazy Tumblr hot take getting upvoted here? Are people considering it a good conversation starter or something?
“…training a single large language model produced emissions equivalent to the lifetime output of 5 cars” 5 cars?? sacrement!
The deafening silence in the comment section says it all.
This doesn't confirm their bias.
I don't see any substantiation of anything stated in that blog post.