Whether LLMs are "intelligent" seems a wholly uninteresting distinction, resembling the internet ceremony surrounding whether a hotdog is a sandwich.
There's probably very interesting discussion to be had about hotdogs and LLMs, but whether they're sandwiches or intelligent isn't a useful proxy to them.
I disagree completely. Many people take for granted that the expression of intelligence/competence is the same as actual intelligence/competence, and many people are acting accordingly. But a simulacrum is definitively NOT the thing itself. When you trust fake intelligence, especially as a way to indulge mental laziness, your own faculties atrophy, and then in short order you can't even tell the difference between a real intelligence bomb and a dumb empty shell that has the word "intelligent" written on it.
I'm not even taking for granted what it means. Can you define it in a way that your neighbor will independently arrive at? It's an incredibly lossy container for whatever meaning that people will want to pack it with, moreso than for other words.
Is a hotdog a simulacrum of a sandwich? Or a fake sandwich? I have no clue and don't care because it doesn't meaningfully inform me of the utility of the thing.
An LLM might be "unintelligent" but I can't model what you think the consequences of that are. I'd skip the formalities and just talk about those instead.
It sounds like you are [dis]interested in a philosophical discussion about epistemology. So it seems that you've skipped the inquiry yourself and have short-circuited to "don't care". Which is kind of "utilitarian". For other perspectives[0]:
> The school of skepticism questions the human ability to attain knowledge, while fallibilism says that knowledge is never certain. Empiricists hold that all knowledge comes from sense experience, whereas rationalists believe that some knowledge does not depend on it. Coherentists argue that a belief is justified if it coheres with other beliefs. Foundationalists, by contrast, maintain that the justification of basic beliefs does not depend on other beliefs. Internalism and externalism debate whether justification is determined solely by mental states or also by external circumstances.
For my part, I do believe that there is non-propositional knowledge. That a person can look at a set of facts/experiences/inputs and apply their mind towards discerning knowledge (or "truth"), or at least the relative probability of knowledge being true. That while this discernment and knowledge might be explained or justified verbally and logically, the actual discernment is non-verbal. And, for sure, correctness is not even essential--a person may discern that the truth is unknowable from the information at their disposal, and they may even discern incorrectly! But there is some mental process that can actually look behind the words to its "meaning" and then apply its own discernment to that meaning. (Notably this does not merely aggregating everyone else's discernment!) This is "intelligence", and it is something that humans can do, even if many of us often don't even apply this faculty ourselves.
From discussions on HN and otherwise I gather this is what people refer to by "world-modeling". So my discernment is that language manipulation is neither necessary nor sufficient for intelligence--though it may be necessary to communicate more abstract intelligence. What LLM/AGI proponents are arguing is that language manipulation is sufficient for intelligence. This is a profound misunderstanding of intelligence, and one that should not be written off with a blithe and unexamined "but who knows what intelligence is anyway".
I'm not discounting the philosophy, just the language.
I don't mean to sound blithe. If I do, it's not out of indifference but out of active determination that these kinds of terminological boundary disputes quickly veer into pointlessness. They seldom inform us of anything other than how we choose to use words.
See my other comment above. Language manipulation is not sufficient for intelligence and understanding. There is no one in the Chinese Room who understands the questions and answers; there is no understanding in the system; there is no understanding at all.
And people will keep ignoring Stallman at their peril. But if you understand how the technology works, you also know he's right. If you think he isn't, you either don't understand or you don't _want_ to understand because your job depends on it.
"people should not trust systems that mindlessly play with words to be correct in what those words mean"
Yes, but this applies to any media channel or just other human minds. It's an admonition to think critically about all incoming signals.
"users cannot get a copy of it"
Can't get a copy of my interlocutor's mind, either, for careful verification. Shall I retreat to my offline cave and ruminate deeply with only my own thoughts and perhaps a parrot?
>you also know he's right. If you think he isn't, you either don't understand or you don't _want_ to understand because your job depends on it.
> Yes, but this applies to any media channel or just other human minds.
You can hold a person responsible, first and foremost. But I am so tired of this strawman argument; it's unfalsifiable but also stupid because if you interact with real people, you immediately know the difference between people and these language models. And if you can't I feel sorry for you, because that's more than likely a mental illness.
So no I can't "prove" that people aren't also just statistical probability machines and that every time you ask someone to explain their thought process they're not just bullshitting, because no, I can't know what goes on in their brain nor measure it. And some people do bullshit. But I operate in the real world with real people every day and if they _are_ just biological statistical probability machines, then they're a _heck_ of a lot more advanced than the synthetic variety. So much so that I consider them wholly different, akin to the difference between a simple circuit with a single switch vs. the SoC of a modern smartphone.
I actually agree with you that LLMs are so rigid and shallow as even a typical person appears as an ocean to them in a conversation.
I just think Stallman is this broken-clock purist that offered no specific practical advice in this case. I’d be more interested in what he thinks in LLMs one-shotting humans with their tokens (LLM psychopathy?) as they come on the scene worldwide.
I don’t have the luxury of listening to him. I would be much less effective at my job compared to my competitors in the job market if I didn’t use ChatGPT, regardless of whether it’s open source software or meets his definition of intelligence.
Richard makes a distinction between human understanding and AI indifference to truth. But isn't that what half the country is doing a.t.m? And more philosophically, we can't know the Truth because we rely on leaky abstractions all the way.
AI models are subject to user satisfaction and sustained usage, the models also have a need to justify their existence, not just us. They are not that "indifferent", after multiple iterations the external requirement becomes internalized goal. Cost is the key - it costs to live, and it costs to execute AI. Cost becomes valence.
I see it like a river - water carves the banks, and banks channel the water, you can't explain one without the other, in isolation. So are external constraints and internal goals.
From my understanding what Stallman says is that LLMs don't "understand" what they're saying. They do a probabilistic search of the most appropriate letter (say) that has had come after another letter in the text (or any media) they have been trained on, and they place it similar in resemblance in the text that they produce. This is largely (no pun) dependent on existing data that is there in the world today, and the more the data that LLMs can work through, the better they get at predicting. (Hence the big data center shops today.)
But the limitation is that it cannot "imagine" (as in "imagination is more important than knowledge" by Einstein, who worked on a knowledge problem using imagination, but with the same knowledge resources as his peers.) In this video [1], Stallman talks about his machine trying to understand the "phenomenon" of a physical mechanism, which enables it to "deduce" next steps. I suppose he means it was not doing a probabilistic search on a large dataset to know what should have come next (which makes it human-knowledge dependent), essentially rendering it to an advanced search engine but not AI.
Interesting points! Maybe a better term is LLMs (BTW smart phones are not smart and people don’t seem to be confused). I agree with being dependent and sending so much data to those servers. I would mention there is a version of ChatGPT you can run locally[1].
It doesn't understand anything. Yet if you prompt it with a question about what it understands, its output is consistent with something that understands.
Text in, text out. The question is how much a sequence of tokens captures what we think a mind is. "It" ceases to exist when we stop giving it a prompt, if "it" even exists. Whether you consider something "AI" says more about what you think a mind is than anything about the software.
Another day, another example of the AI Effect in action:
> "The AI effect" refers to a phenomenon where either the definition of AI or the concept of intelligence is adjusted to exclude capabilities that AI systems have mastered. This often manifests as tasks that AI can now perform successfully no longer being considered part of AI, or as the notion of intelligence itself being redefined to exclude AI achievements.[4][2][1] Edward Geist credits John McCarthy for coining the term "AI effect" to describe this phenomenon.[4] The earliest known expression of this notion (as identified by Quote Investigator) is a statement from 1971, "AI is a collective name for problems which we do not yet know how to solve properly by computer", attributed to computer scientist Bertram Raphael.[5]
> McCorduck calls it an "odd paradox" that "practical AI successes, computational programs that actually achieved intelligent behavior were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the 'failures', the tough nuts that couldn't yet be cracked."[6] It is an example of moving the goalposts.[7]
I wonder how many more times I'll have to link this page until people stop repeating it.
Leaving alone Stallman's extreme take, present day LLMs and other generative systems are absolutely still being referred to by society as AI, and I don't see this changing any time soon, so what does this say about the AI effect?
Consciousness, in Zoltan Torey's[1] model, is the brain's layered, language-enabled off-line mechanism that reflects on its own sensory endogram, generating self-aware, internally guided behavior.[2] The off-line mechanism generates mental alternatives, which are then "run past the brainstem, which then makes the selection." Nice little accessible book.[3]
> Taking “computer” first, we find that this alleged
source of machine-generated consciousness is not what it
is cracked up to be. It is a mere effigy, an entity in name
only. It is no more than a cleverly crafted artifact, one essentially indistinguishable from the raw material out of
which it is manufactured.[2]
Someone should start a StallmanGPT that writes regular blogposts on “Don’t use <popular software or website>”. See if readers can tell those apart from the real website.
Extremely based and to the point.
Its ridiculous how all these comment somehow disagree with him, they are not inteligent systems, its justa regression function run on words or pixel data
What if I said that the ability to move the goalpost is the real trick?
Machines started to hold up casual conversation well, so we came up with more clever examples of how to make it hallucinate, which made it look dumb again. We're surprisingly good and fast at it.
You're trying to cap that to a decade, or a specific measure. It serves no other purpose than to force one to make a prediction mistake, which is irrelevant to the intelligence discussion.
His argument misses the point.. I don't particularly care if it's intelligent or understands anything. My question is does it help with what I'm trying to do
As for it being closed source and kept at arms length? Sure.. and if it's taken away or the value proposition changes, I stop using it
My freedom comes from having the ability to switch if needed, not from intentionally making myself less effective. There is no lock in
> I don't particularly care if it's intelligent or understands anything. My question is does it help with what I'm trying to do
So, he's right? All you care is that it helps you, so it doesn't matter if it's called "artificial intelligence" or not. It doesn't matter for you, and it matters for him (and lots of other people), so let's change the name to "artificial helper", what do you think? Looks like a win-win scenario.
If that's really the point (that it helps you, and intelligence doesn't matter), let's remove the intelligence from the name.
That's fine. You must understand that some people will not agree with you either, right? That's how it works. We don't even have to explain why, but it's a common courtesy.
Think this way: it's still a win-win no matter what. What Stallman is saying is that there would be no reason not to use ChatGPT if it was free (you are able to get a copy of the source and build it yourself) and not called AI. If you change those two things, then it's Stallman compliant.
That's totally doable. It would still be the exact same program you use today and helps you, and it would also now be immune to those two criticism points (whether it is intelligent or not and what's under the hood).
How would it be doable to make them open? I think this is a fundamentally different thing than LibreOffice vs Excel. These things are incredibly expensive to train and run, and doing it as a FOSS project for anyone to clone and run locally means they'd never make their investment back
Open models exist but they're not very useful compared to the latest. Hopefully that'll change but who knows
What a horrible take from someone who used to be competent. I find that it’s usually the hardcore OSS and hardware adjacent types to be ideological about AI.
What is thinking and why do you think that LLM ingesting content is not also reading? Clearly they're absorbing some sort of information from text content, aka reading.
Are you saying we don't run on math? How much do you know of how the brain functions?
This sort of Socratic questioning shows that no one truly can answer them because no one actually knows about the human mind, or how to distinguish or even define intelligence.
By this logic, most human brains are bullshit generators too. Some humans even have a complete and utter disregard for the truth. (One such human happens to own Truth Social.)
Whether LLMs are "intelligent" seems a wholly uninteresting distinction, resembling the internet ceremony surrounding whether a hotdog is a sandwich.
There's probably very interesting discussion to be had about hotdogs and LLMs, but whether they're sandwiches or intelligent isn't a useful proxy to them.
I disagree completely. Many people take for granted that the expression of intelligence/competence is the same as actual intelligence/competence, and many people are acting accordingly. But a simulacrum is definitively NOT the thing itself. When you trust fake intelligence, especially as a way to indulge mental laziness, your own faculties atrophy, and then in short order you can't even tell the difference between a real intelligence bomb and a dumb empty shell that has the word "intelligent" written on it.
I'm not even taking for granted what it means. Can you define it in a way that your neighbor will independently arrive at? It's an incredibly lossy container for whatever meaning that people will want to pack it with, moreso than for other words.
Is a hotdog a simulacrum of a sandwich? Or a fake sandwich? I have no clue and don't care because it doesn't meaningfully inform me of the utility of the thing.
An LLM might be "unintelligent" but I can't model what you think the consequences of that are. I'd skip the formalities and just talk about those instead.
It sounds like you are [dis]interested in a philosophical discussion about epistemology. So it seems that you've skipped the inquiry yourself and have short-circuited to "don't care". Which is kind of "utilitarian". For other perspectives[0]:
> The school of skepticism questions the human ability to attain knowledge, while fallibilism says that knowledge is never certain. Empiricists hold that all knowledge comes from sense experience, whereas rationalists believe that some knowledge does not depend on it. Coherentists argue that a belief is justified if it coheres with other beliefs. Foundationalists, by contrast, maintain that the justification of basic beliefs does not depend on other beliefs. Internalism and externalism debate whether justification is determined solely by mental states or also by external circumstances.
For my part, I do believe that there is non-propositional knowledge. That a person can look at a set of facts/experiences/inputs and apply their mind towards discerning knowledge (or "truth"), or at least the relative probability of knowledge being true. That while this discernment and knowledge might be explained or justified verbally and logically, the actual discernment is non-verbal. And, for sure, correctness is not even essential--a person may discern that the truth is unknowable from the information at their disposal, and they may even discern incorrectly! But there is some mental process that can actually look behind the words to its "meaning" and then apply its own discernment to that meaning. (Notably this does not merely aggregating everyone else's discernment!) This is "intelligence", and it is something that humans can do, even if many of us often don't even apply this faculty ourselves.
From discussions on HN and otherwise I gather this is what people refer to by "world-modeling". So my discernment is that language manipulation is neither necessary nor sufficient for intelligence--though it may be necessary to communicate more abstract intelligence. What LLM/AGI proponents are arguing is that language manipulation is sufficient for intelligence. This is a profound misunderstanding of intelligence, and one that should not be written off with a blithe and unexamined "but who knows what intelligence is anyway".
[0] https://en.wikipedia.org/wiki/Epistemology
I'm not discounting the philosophy, just the language.
I don't mean to sound blithe. If I do, it's not out of indifference but out of active determination that these kinds of terminological boundary disputes quickly veer into pointlessness. They seldom inform us of anything other than how we choose to use words.
What are your thoughts on the Chinese room thought experiment?
See my other comment above. Language manipulation is not sufficient for intelligence and understanding. There is no one in the Chinese Room who understands the questions and answers; there is no understanding in the system; there is no understanding at all.
And people will keep ignoring Stallman at their peril. But if you understand how the technology works, you also know he's right. If you think he isn't, you either don't understand or you don't _want_ to understand because your job depends on it.
He's sort of right but in an unspecific way.
"people should not trust systems that mindlessly play with words to be correct in what those words mean"
Yes, but this applies to any media channel or just other human minds. It's an admonition to think critically about all incoming signals.
"users cannot get a copy of it"
Can't get a copy of my interlocutor's mind, either, for careful verification. Shall I retreat to my offline cave and ruminate deeply with only my own thoughts and perhaps a parrot?
>you also know he's right. If you think he isn't, you either don't understand or you don't _want_ to understand because your job depends on it.
He can't keep getting away with this!
> Yes, but this applies to any media channel or just other human minds.
You can hold a person responsible, first and foremost. But I am so tired of this strawman argument; it's unfalsifiable but also stupid because if you interact with real people, you immediately know the difference between people and these language models. And if you can't I feel sorry for you, because that's more than likely a mental illness.
So no I can't "prove" that people aren't also just statistical probability machines and that every time you ask someone to explain their thought process they're not just bullshitting, because no, I can't know what goes on in their brain nor measure it. And some people do bullshit. But I operate in the real world with real people every day and if they _are_ just biological statistical probability machines, then they're a _heck_ of a lot more advanced than the synthetic variety. So much so that I consider them wholly different, akin to the difference between a simple circuit with a single switch vs. the SoC of a modern smartphone.
I actually agree with you that LLMs are so rigid and shallow as even a typical person appears as an ocean to them in a conversation.
I just think Stallman is this broken-clock purist that offered no specific practical advice in this case. I’d be more interested in what he thinks in LLMs one-shotting humans with their tokens (LLM psychopathy?) as they come on the scene worldwide.
I don’t have the luxury of listening to him. I would be much less effective at my job compared to my competitors in the job market if I didn’t use ChatGPT, regardless of whether it’s open source software or meets his definition of intelligence.
Richard makes a distinction between human understanding and AI indifference to truth. But isn't that what half the country is doing a.t.m? And more philosophically, we can't know the Truth because we rely on leaky abstractions all the way.
AI models are subject to user satisfaction and sustained usage, the models also have a need to justify their existence, not just us. They are not that "indifferent", after multiple iterations the external requirement becomes internalized goal. Cost is the key - it costs to live, and it costs to execute AI. Cost becomes valence.
I see it like a river - water carves the banks, and banks channel the water, you can't explain one without the other, in isolation. So are external constraints and internal goals.
From my understanding what Stallman says is that LLMs don't "understand" what they're saying. They do a probabilistic search of the most appropriate letter (say) that has had come after another letter in the text (or any media) they have been trained on, and they place it similar in resemblance in the text that they produce. This is largely (no pun) dependent on existing data that is there in the world today, and the more the data that LLMs can work through, the better they get at predicting. (Hence the big data center shops today.)
But the limitation is that it cannot "imagine" (as in "imagination is more important than knowledge" by Einstein, who worked on a knowledge problem using imagination, but with the same knowledge resources as his peers.) In this video [1], Stallman talks about his machine trying to understand the "phenomenon" of a physical mechanism, which enables it to "deduce" next steps. I suppose he means it was not doing a probabilistic search on a large dataset to know what should have come next (which makes it human-knowledge dependent), essentially rendering it to an advanced search engine but not AI.
[1] https://youtu.be/V6c7GtVtiGc?si=fhkG2ZA-nsQgrVwm
Interesting points! Maybe a better term is LLMs (BTW smart phones are not smart and people don’t seem to be confused). I agree with being dependent and sending so much data to those servers. I would mention there is a version of ChatGPT you can run locally[1].
[1] https://openai.com/index/introducing-gpt-oss/
It doesn't understand anything. Yet if you prompt it with a question about what it understands, its output is consistent with something that understands.
Text in, text out. The question is how much a sequence of tokens captures what we think a mind is. "It" ceases to exist when we stop giving it a prompt, if "it" even exists. Whether you consider something "AI" says more about what you think a mind is than anything about the software.
Another day, another example of the AI Effect in action:
> "The AI effect" refers to a phenomenon where either the definition of AI or the concept of intelligence is adjusted to exclude capabilities that AI systems have mastered. This often manifests as tasks that AI can now perform successfully no longer being considered part of AI, or as the notion of intelligence itself being redefined to exclude AI achievements.[4][2][1] Edward Geist credits John McCarthy for coining the term "AI effect" to describe this phenomenon.[4] The earliest known expression of this notion (as identified by Quote Investigator) is a statement from 1971, "AI is a collective name for problems which we do not yet know how to solve properly by computer", attributed to computer scientist Bertram Raphael.[5]
> McCorduck calls it an "odd paradox" that "practical AI successes, computational programs that actually achieved intelligent behavior were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the 'failures', the tough nuts that couldn't yet be cracked."[6] It is an example of moving the goalposts.[7]
I wonder how many more times I'll have to link this page until people stop repeating it.
[0] https://en.wikipedia.org/wiki/AI_effect
Leaving alone Stallman's extreme take, present day LLMs and other generative systems are absolutely still being referred to by society as AI, and I don't see this changing any time soon, so what does this say about the AI effect?
Consciousness, in Zoltan Torey's[1] model, is the brain's layered, language-enabled off-line mechanism that reflects on its own sensory endogram, generating self-aware, internally guided behavior.[2] The off-line mechanism generates mental alternatives, which are then "run past the brainstem, which then makes the selection." Nice little accessible book.[3]
> Taking “computer” first, we find that this alleged source of machine-generated consciousness is not what it is cracked up to be. It is a mere effigy, an entity in name only. It is no more than a cleverly crafted artifact, one essentially indistinguishable from the raw material out of which it is manufactured.[2]
[1] https://en.wikipedia.org/wiki/Zoltan_Torey
[2] https://mitpress.mit.edu/9780262527101/the-conscious-mind/
[3] https://search.worldcat.org/title/887744728
Someone should start a StallmanGPT that writes regular blogposts on “Don’t use <popular software or website>”. See if readers can tell those apart from the real website.
Extremely based and to the point. Its ridiculous how all these comment somehow disagree with him, they are not inteligent systems, its justa regression function run on words or pixel data
Can you please offer a measurable definition of intelligence that you would put good money on not being cracked by AI in a decade?
What if I said that the ability to move the goalpost is the real trick?
Machines started to hold up casual conversation well, so we came up with more clever examples of how to make it hallucinate, which made it look dumb again. We're surprisingly good and fast at it.
You're trying to cap that to a decade, or a specific measure. It serves no other purpose than to force one to make a prediction mistake, which is irrelevant to the intelligence discussion.
> all these comments
All 2 of them! Way to gauge the crowd sentiment.
His argument misses the point.. I don't particularly care if it's intelligent or understands anything. My question is does it help with what I'm trying to do
As for it being closed source and kept at arms length? Sure.. and if it's taken away or the value proposition changes, I stop using it
My freedom comes from having the ability to switch if needed, not from intentionally making myself less effective. There is no lock in
> I don't particularly care if it's intelligent or understands anything. My question is does it help with what I'm trying to do
So, he's right? All you care is that it helps you, so it doesn't matter if it's called "artificial intelligence" or not. It doesn't matter for you, and it matters for him (and lots of other people), so let's change the name to "artificial helper", what do you think? Looks like a win-win scenario.
If that's really the point (that it helps you, and intelligence doesn't matter), let's remove the intelligence from the name.
Well I don't agree with him saying these are a reasons not to use it
That's fine. You must understand that some people will not agree with you either, right? That's how it works. We don't even have to explain why, but it's a common courtesy.
Think this way: it's still a win-win no matter what. What Stallman is saying is that there would be no reason not to use ChatGPT if it was free (you are able to get a copy of the source and build it yourself) and not called AI. If you change those two things, then it's Stallman compliant.
That's totally doable. It would still be the exact same program you use today and helps you, and it would also now be immune to those two criticism points (whether it is intelligent or not and what's under the hood).
How would it be doable to make them open? I think this is a fundamentally different thing than LibreOffice vs Excel. These things are incredibly expensive to train and run, and doing it as a FOSS project for anyone to clone and run locally means they'd never make their investment back
Open models exist but they're not very useful compared to the latest. Hopefully that'll change but who knows
What a horrible take from someone who used to be competent. I find that it’s usually the hardcore OSS and hardware adjacent types to be ideological about AI.
Interesting take from richard.
All these people that go "it just predicts words" seem to be very certain that the brain does something else.
Even if it does, that's not very relevant. The airplane does not imitate the bird, yet it very much does fly.
actually it does. We do not predict words lol.
Well, I failed to predict the "lol" at the end of your comment; touché.
I find your certainty to be unwarranted.
Then what do we do? lol.
We understand the meaning that we wish to convey and then intelligently choose the best method that we have at our disposal to communicate that.
LLMs find the most likely next word based on its billions of previously scanned word combinations and contexts. It's an entirely different process.
How does this intelligence work? Can you explain how 'meaning' is expressed in neurons, or whatever it is that makes up consciousness?
I don't think we know. Or if we have theories, the error bars are massive.
>LLMs find the most likely next word based on its billions of previously scanned word combinations and contexts. It's an entirely different process.
How is that different than using one's learned vocabulary?
How do you know we understand and LLMs don't? To an outsider they look the same. Indeed, that is the point of solipsism.
Because unlike a human brain, we can actually read the whitepaper on how the process works.
They do not "think", they "language", i.e. large language model.
What is thinking and why do you think that LLM ingesting content is not also reading? Clearly they're absorbing some sort of information from text content, aka reading.
I think you don't understand how llms work. They run on math, the only parallel between an llm and a human is the output.
Are you saying we don't run on math? How much do you know of how the brain functions?
This sort of Socratic questioning shows that no one truly can answer them because no one actually knows about the human mind, or how to distinguish or even define intelligence.
So do neurons.
It does something else yes.
By this logic, most human brains are bullshit generators too. Some humans even have a complete and utter disregard for the truth. (One such human happens to own Truth Social.)
It's true that lots of people don't seem to recognize objective truth, or just don't want to admit it. Perception is reality.
The ChatGPT has a few self-awareness modules, it can even behave based on its certainty. Please see the Adrej Karpathy's video on it.
This is the breakthrough we went beyond. There's no going back now. There is also a reasoning now in the LLM