I would argue that it's not just potential; it's actively happening, and a lot of us here noticed and discussed it years ago.
The phenomenon of correcting people because Chat Gippity said x or y was the beginning. Now people repeat what the machine said as though it originated from them, and this has been totally normalized. It permeates everything. People feel empowered by it, but they have no intent or ability to verify. This is normal. It's another source of information, but it's vetted by probability at best, yet also misinterpreted and internalized at worst.
People plagiarize and behave as though it's their own work with total confidence and no shame whatsoever. Speaking to teachers about this is mind-blowing. This is very real and present. These people believe they're doing 'the work' in many cases. Some are aware it's a farce, many are not.
It has jammed a lever into D-K and cranked it up into something even worse, in my opinion.
I have a couple of people in my team at work whose first port of call for any hard question is to ask AI, and they proudly proclaim that AI said it can be done - all you need to do is x, y and z.
Having had to actually work on something that their AI said was a nice easy 7-step process... yeah, two weeks' work was just laying the foundations.
The giveaway was my Medical Professional father thinking that AI was really good at things outside of his area of expertise, and really bad at things inside of his area of expertise.
You might be doing the same thing, if you think that your medical professional father would have any understanding of how an LLM works, an area in which he has no expertise.
To him, and most other people who don't know anything about the tech and how it works, it's probably just a magic intelligent box that can answer many things that he can't.
The marketing says it's this close to AGI and taking all our jobs, so that must be true!
Which is weird, because AI being bad at things in my expertise (programming) makes me distrust it 10x harder for things outside my expertise. At this point, unless an LLM can give me a reference that I can follow back to a trustworthy primary source (and unless that source says what the LLM synthesized from it), I automatically discard anything it says. It's simply wrong too frequently for me to do otherwise.
From a software POV, I feel like it makes it easier to implement stuff. Whether that stuff is good or bad. If you know what you're doing, vet the output, and use it properly, you can get a nice productivity boost while still producing good code. If you don't know what you're doing, you are prone to go down rabbit hole after rabbit hole of unwise decisions.
I would just like to lightly push back on that point. That bad code? That dead end? That month’s worth of tokens spent on a runaway loop? Those weren’t dead ends. Those were learning points. Experiences carried forward, etched in your mind. So take heart. We need both successes and failures to grow as people. And you are growing. I can see it.
Let me play devil's advocate; D-K people often deceive others because of their articulation ability. If one can elaborate on their thoughts in a well organized and cohesive manner, society tends to give them the benefit of the doubt that they are not speaking from pure grandiosity. But the more AI puts everyone's ability to simply articulate on the same pedestal, the more it becomes a feature we will tend to overlook, forcing us to judge ideas on their merit alone.
I’ve been referring to it as “Dunning-Kruger As A Service” for quite some time. Interesting that other folks are coming to the same conclusion.
There’s a subtlety here, tho. It doesn’t mean (at all) that’s it useless. Just that those who don’t know what they’re doing are going to be amplifying their incompetence.
The entire internet has been a Dunning-Kruger amplifier since Altavista and Webcrawler. We're going to keep getting better at it.
I recommend the book "The Information" (2011, James Gleick) for a grounded history of how we've slowly gone from using smoke signals and drums to transmit information to how we crossed the inflection point between being limited in how much information we could consume to instead being limited in how well we could curate the information we consume. Curation will continue to improve, and misplaced confidence is merely one downstream effect.
I also recommend, for those struggling with these effects, the book "Amusing Ourselves to Death" (1985, Neil Postman, updated in 2005 by his son), to understand the concept of Information Action Ratio - how to best target your curation.
Anecdotally, I see this every day. I have seen otherwise intelligent people suggest that AI can defeat cause and effect, or that it's self aware, or secretly deterministic. Their enthusiasm borders on religious, or magical thinking.
It has graduated into being an active and fairly precisely targeted thought shaping tool.
I worked on Ads and Feed Ranking at FB/IG and we never dreamed of the scope for shaping behavior and opinion that is now routinely deployed by frontier and near frontier vendors. RLHF is basically feed ranking in the first place, preference gradient with no ground truth referee, late SFT on amplifying data sets, and affine injections into the residual stream with a fluent, earnest base model that the public has been conditioned to regard as omniscient and wise?
Yeah that's fucking mind control when applied at scale. We did some sketchy shit a decade ago, this is next level.
I would argue that it's not just potential; it's actively happening, and a lot of us here noticed and discussed it years ago.
The phenomenon of correcting people because Chat Gippity said x or y was the beginning. Now people repeat what the machine said as though it originated from them, and this has been totally normalized. It permeates everything. People feel empowered by it, but they have no intent or ability to verify. This is normal. It's another source of information, but it's vetted by probability at best, yet also misinterpreted and internalized at worst.
People plagiarize and behave as though it's their own work with total confidence and no shame whatsoever. Speaking to teachers about this is mind-blowing. This is very real and present. These people believe they're doing 'the work' in many cases. Some are aware it's a farce, many are not.
It has jammed a lever into D-K and cranked it up into something even worse, in my opinion.
I have a couple of people in my team at work whose first port of call for any hard question is to ask AI, and they proudly proclaim that AI said it can be done - all you need to do is x, y and z.
Having had to actually work on something that their AI said was a nice easy 7-step process... yeah, two weeks' work was just laying the foundations.
We need an, "your absolutly wrong" advocat diaboli agent in every conversation..
I've been having this thought for the last month.
The giveaway was my Medical Professional father thinking that AI was really good at things outside of his area of expertise, and really bad at things inside of his area of expertise.
You might be doing the same thing, if you think that your medical professional father would have any understanding of how an LLM works, an area in which he has no expertise.
To him, and most other people who don't know anything about the tech and how it works, it's probably just a magic intelligent box that can answer many things that he can't.
The marketing says it's this close to AGI and taking all our jobs, so that must be true!
Gell-Mann Amnesia effect
https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect
google graduate
Which is weird, because AI being bad at things in my expertise (programming) makes me distrust it 10x harder for things outside my expertise. At this point, unless an LLM can give me a reference that I can follow back to a trustworthy primary source (and unless that source says what the LLM synthesized from it), I automatically discard anything it says. It's simply wrong too frequently for me to do otherwise.
Sadly a lot of times the sources are just LLM generated themselves...
I'm exactly the same, but unfortunately I think we're in the minority.
People are absolutely dying to outsource their thinking.
Which area of programming is it bad at? AFAIK it’s bad at embedded programming, but boring web/crud stuff it is very good at.
From a software POV, I feel like it makes it easier to implement stuff. Whether that stuff is good or bad. If you know what you're doing, vet the output, and use it properly, you can get a nice productivity boost while still producing good code. If you don't know what you're doing, you are prone to go down rabbit hole after rabbit hole of unwise decisions.
I would just like to lightly push back on that point. That bad code? That dead end? That month’s worth of tokens spent on a runaway loop? Those weren’t dead ends. Those were learning points. Experiences carried forward, etched in your mind. So take heart. We need both successes and failures to grow as people. And you are growing. I can see it.
Is this satire?
[dead]
Let me play devil's advocate; D-K people often deceive others because of their articulation ability. If one can elaborate on their thoughts in a well organized and cohesive manner, society tends to give them the benefit of the doubt that they are not speaking from pure grandiosity. But the more AI puts everyone's ability to simply articulate on the same pedestal, the more it becomes a feature we will tend to overlook, forcing us to judge ideas on their merit alone.
I’ve been referring to it as “Dunning-Kruger As A Service” for quite some time. Interesting that other folks are coming to the same conclusion.
There’s a subtlety here, tho. It doesn’t mean (at all) that’s it useless. Just that those who don’t know what they’re doing are going to be amplifying their incompetence.
Wouldn’t that imply that it makes smarter people feel dumber? I haven’t heard of or seen such an effect from LLMs. Have you?
If it only amplifies half the effect, I don’t think TFA is an accurate claim.
Somebody saying LLM makes them feel dumb is the top comment on a previous discussion: https://news.ycombinator.com/item?id=45876744
Interesting, thanks for the link!
Related:
https://news.ycombinator.com/item?id=45876744 - "LLMs are steroids for your Dunning-Kruger" (bytesauna.com)
392 points | 7 months ago | 301 comments
https://news.ycombinator.com/item?id=45851483 - "AI is Dunning-Kruger as a service" (christianheilmann.com)
268 points | 7 months ago | 199 comments
The entire internet has been a Dunning-Kruger amplifier since Altavista and Webcrawler. We're going to keep getting better at it.
I recommend the book "The Information" (2011, James Gleick) for a grounded history of how we've slowly gone from using smoke signals and drums to transmit information to how we crossed the inflection point between being limited in how much information we could consume to instead being limited in how well we could curate the information we consume. Curation will continue to improve, and misplaced confidence is merely one downstream effect.
I also recommend, for those struggling with these effects, the book "Amusing Ourselves to Death" (1985, Neil Postman, updated in 2005 by his son), to understand the concept of Information Action Ratio - how to best target your curation.
I'm absolutely right.
Anecdotally, I see this every day. I have seen otherwise intelligent people suggest that AI can defeat cause and effect, or that it's self aware, or secretly deterministic. Their enthusiasm borders on religious, or magical thinking.
They believe the marketing.
Billionaire hype men couldn't be wrong, could they? Otherwise how could they get so rich??
I find AI (gemini specifically) generally debunks all my profound stoner ideas pretty well.
Potentially ?
Definitely is.
Potentially? From what I can tell, Google's AI overview is one of the most widely sited sources now.
It was a Dunning-Kruger amplifier.
It has graduated into being an active and fairly precisely targeted thought shaping tool.
I worked on Ads and Feed Ranking at FB/IG and we never dreamed of the scope for shaping behavior and opinion that is now routinely deployed by frontier and near frontier vendors. RLHF is basically feed ranking in the first place, preference gradient with no ground truth referee, late SFT on amplifying data sets, and affine injections into the residual stream with a fluent, earnest base model that the public has been conditioned to regard as omniscient and wise?
Yeah that's fucking mind control when applied at scale. We did some sketchy shit a decade ago, this is next level.
This is two-tweet hot take about DK and Idiocracy (we live in a society).
Yeah we know that LLMs tend towards sycophancy.
Discussing DK has a real Matthew 7:3-5 vibe about it.
[flagged]