I listened to this podcast and it was maddening. A few of his frames :
AI is just software, nothing new to see here.
AI safety is primarily a sandboxing problem.
There is no collective action problem, and every company should just slow down if they think they need to slow down.
There is no need for regulation because the existing incentives in the market keep companies from acting badly, which is why no company has ever done anything bad.
If something bad does happen, then we can regulate after the fact.
We will end up creating more jobs than we destroy, so don't worry about it.
Our kids might forget a whole bunch of stuff or never learn it in the first place. But don't worry, they'll come up with new things to learn instead.
The only way to get safety is to move faster because then we will more quickly arrive at safety.
Recursive self-improvement is just what we've always done.
The real danger is alarmism that might scare the public and the young people.
It’s feeling less and less like human progress and more and more like progress for a small group at the top of our economic structure who are hoovering up everything for themselves.
Who gives a fuck if some techno barons can deploy novel software at the drop of a hat if it means I lose my career in an environment without social safety nets.
It’s not progress from my perspective when my life quality regresses.
lol, no. It’s currently being deployed in feverish attempt to remove jobs.
I see company doing layoffs left and right because they can do the same or more with less employees. I’m not seeing any companies feverishly hiring engineers because the productivity per hour of labor is higher.
There is also no guarantee that AI will lead to any social safety net, they are orthogonal concepts.
The only thing that would lead to a social safety net is political will and currently all the political will behind the technology firms building AI appears to be firmly behind dismantling democracy and becoming oligarchical leaders of their little fiefdoms while the rest of us can get fucked.
Two questions: 1. Have the "fud-spreaders" actually held anything back so far? Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies. 2. Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
Have the "fud-spreaders" actually held anything back so far?
I don't know. Astroturf campaigns to subvert data center construction seem to have had an impact, for example, but ultimately I can only fall back on vague vibes picked up from the news and social media. I don't know the specific success rate.
Some of those campaigns seem justified to me (why in the world do these things have to be built in populated areas?) while other arguments made by the anti-data center movement are certainly propagated in bad faith (Water usage? Really?)
Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies.
That's just so much kayfabe. Don't listen to what they say, watch what they do.
Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
I can't speak for Huang as I don't know him personally, but he has to walk a tightrope between an infantilized POTUS with essentially unlimited power to wreck his business, and his own stockholders who have elevated his company's market cap to a level that probably doesn't make sense even to him. As far as I can tell he's just going the obvious, predictable, and safe route of trying to sell as many shovels and tents as he can while the selling is good.
I see a bigger downside in the precautionary principle than in accelerationism. We are going to face immense societal disruption due to AI over the next couple of decades. That is as unavoidable as an earthquake. It is going to happen regardless of who prevails in the political and social arguments going on today. There is no possible way that every actor in every country on Earth is going to be convinced or compelled to "pace the frontier." Consequently, all that matters is how we react, adapt, and take advantage of the new ground rules of reality.
As usual, there will be those who thrive and those who get steamrolled. But I see no reason why we won't all end up better off in the end, as with the rest of the history of technology since the invention of fire.
I think this is overly optimistic; it assumes universities reacted instantly and pivoted to teaching coding with AI as soon as it was feasible. In actuality, university curriculums cannot change quickly, and AI capabilities are changing much faster. In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
For CS students I agree. For non-CS students, who need programming as a way to use computers as a tool, I think they do need a solid understanding of what a computer does and what it's limitations are (and what it's capable of), but for their day jobs (once they graduate), they will most likely be getting AI to do everything for them. As long as they can reason about what's happening, to understand the results and to improve them, this seems likely to be the path in the future.
This is an introductory programming course, designed for non-CS students e.g. engineers and scientists.
> Learning outcomes
> 2. Explain key concepts in AI-assisted programming, including Large Language Models (LLMs), prompting, problem decomposition, and top-down design.
> 3. Apply the workflow of AI-assisted programming and prompt-engineering techniques to guide and improve code generated by AI assistants.
This course used to be non-AI (last year), and they rewrote recently to incorporate AI tools, as they realised the writing's on the wall for non-programmers.
It must be quite challenging to write curriculum when the underlying technology (AI) is changing so quickly.
It also is naive to think, if AI is shrinking head count, that companies will seek anything other than seniors.
2008 triggered a glut of cheap experienced workers that slowly reengaged juniors but this time that the entire ladder is going to be more valuable than any grad.
In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
How many of them can read x64 or ARM assembly emitted by their compilers?
It’s how I deeply understand what a RAM lookup vs having it already in a register means for optimization.
Me, too, but we should both understand what we're talking about: a hobby.
It's like teaching cursive to schoolkids. All well and good, but don't you dare complain about limited classroom time for instruction in other, more important subjects.
Understanding how computer architecture works is a hobby? Is unimportant?
I built and deployed embedded systems robotics that protect water and oil pipeline infrastructure that costs billions of dollars if there is a failure.
That is not a hobby, that’s a career.
The tools I built just helped fix the fresh water access for my entire city of 2 million people. Explain to me what’s more important than people having access to water.
for a guy who is in the middle of it all, the opinion seems sophomoric at best. LLMs are based on heuristics and they will always make mistakes no matter how better they get. You'd need someone to 'fine-tune' the conversation with the LLM.
>"The first chip Huang worked on had 200 transistors, each of which he said he knew by name, while today’s engineers assemble systems from chips containing hundreds of trillions of them without ever working at that level. “Some of the lower-level knowledge is gone,” he acknowledged, and he later described AI as “clearly” a new abstraction level in the same progression."
love love this theoretically but in practice, at least in this “phase 1 of AI craze, unknown how long it’ll last” that someone coming out of Uni with deep knowledge of ABCDs of SWE will be unemployable unless she/he is AI native with harnesses coming out of their ears
With the massive layoffs there have been massive spikes of suicides and od deaths that will only get worse. Articles like this are simply trying to cover it up.
I listened to this podcast and it was maddening. A few of his frames :
AI is just software, nothing new to see here.
AI safety is primarily a sandboxing problem.
There is no collective action problem, and every company should just slow down if they think they need to slow down.
There is no need for regulation because the existing incentives in the market keep companies from acting badly, which is why no company has ever done anything bad.
If something bad does happen, then we can regulate after the fact.
We will end up creating more jobs than we destroy, so don't worry about it.
Our kids might forget a whole bunch of stuff or never learn it in the first place. But don't worry, they'll come up with new things to learn instead.
The only way to get safety is to move faster because then we will more quickly arrive at safety.
Recursive self-improvement is just what we've always done.
The real danger is alarmism that might scare the public and the young people.
Nothing bad can happen, it can only good happen.
>We will end up creating more jobs than we destroy, so don't worry about it.
The destroyed jobs and the persons doing them will be different from the new ones created..
>If something bad does happen, then we can regulate after the fact.
How does regulating after the fact undo the harms caused?
> they'll come up with new things to learn instead.
Like they forgot how to communicate face to face, but have learned to communicate via social media?
I unironically agree with most of these points.
The implication being that you only ironically agree with the rest?
Happy path coding is certainly the most fun.
A whole bunch of fluff without any evidence to back it up.
'Maddening' is letting a few FUD-spreaders hold back human progress.
It’s feeling less and less like human progress and more and more like progress for a small group at the top of our economic structure who are hoovering up everything for themselves.
Who gives a fuck if some techno barons can deploy novel software at the drop of a hat if it means I lose my career in an environment without social safety nets.
It’s not progress from my perspective when my life quality regresses.
Who else is going to save you? You might as well shake your fist at a tsunami.
AI is the ultimate social safety net, if you'll just get out of the way of the people weaving it.
what an insane thing to say
lol, no. It’s currently being deployed in feverish attempt to remove jobs.
I see company doing layoffs left and right because they can do the same or more with less employees. I’m not seeing any companies feverishly hiring engineers because the productivity per hour of labor is higher.
There is also no guarantee that AI will lead to any social safety net, they are orthogonal concepts.
The only thing that would lead to a social safety net is political will and currently all the political will behind the technology firms building AI appears to be firmly behind dismantling democracy and becoming oligarchical leaders of their little fiefdoms while the rest of us can get fucked.
Who?
Two questions: 1. Have the "fud-spreaders" actually held anything back so far? Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies. 2. Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
Have the "fud-spreaders" actually held anything back so far?
I don't know. Astroturf campaigns to subvert data center construction seem to have had an impact, for example, but ultimately I can only fall back on vague vibes picked up from the news and social media. I don't know the specific success rate.
Some of those campaigns seem justified to me (why in the world do these things have to be built in populated areas?) while other arguments made by the anti-data center movement are certainly propagated in bad faith (Water usage? Really?)
Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies.
That's just so much kayfabe. Don't listen to what they say, watch what they do.
Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?
I can't speak for Huang as I don't know him personally, but he has to walk a tightrope between an infantilized POTUS with essentially unlimited power to wreck his business, and his own stockholders who have elevated his company's market cap to a level that probably doesn't make sense even to him. As far as I can tell he's just going the obvious, predictable, and safe route of trying to sell as many shovels and tents as he can while the selling is good.
I see a bigger downside in the precautionary principle than in accelerationism. We are going to face immense societal disruption due to AI over the next couple of decades. That is as unavoidable as an earthquake. It is going to happen regardless of who prevails in the political and social arguments going on today. There is no possible way that every actor in every country on Earth is going to be convinced or compelled to "pace the frontier." Consequently, all that matters is how we react, adapt, and take advantage of the new ground rules of reality.
As usual, there will be those who thrive and those who get steamrolled. But I see no reason why we won't all end up better off in the end, as with the rest of the history of technology since the invention of fire.
Several technologies have made us far worse off.
But your shoes fit a lot better if you use the fluoroscope..
> sell as many shovels and tents as he can while the selling is good.
sounds like, based on your next-to-last paragraph, he’ll be good on this for next couple of decades
I think this is overly optimistic; it assumes universities reacted instantly and pivoted to teaching coding with AI as soon as it was feasible. In actuality, university curriculums cannot change quickly, and AI capabilities are changing much faster. In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
I'm not sure if AI use even aligns with the purpose of the university program, which is usually understanding.
For CS students I agree. For non-CS students, who need programming as a way to use computers as a tool, I think they do need a solid understanding of what a computer does and what it's limitations are (and what it's capable of), but for their day jobs (once they graduate), they will most likely be getting AI to do everything for them. As long as they can reason about what's happening, to understand the results and to improve them, this seems likely to be the path in the future.
Some unis have: https://programsandcourses.anu.edu.au/2027/course/COMP1730
This is an introductory programming course, designed for non-CS students e.g. engineers and scientists.
> Learning outcomes
> 2. Explain key concepts in AI-assisted programming, including Large Language Models (LLMs), prompting, problem decomposition, and top-down design.
> 3. Apply the workflow of AI-assisted programming and prompt-engineering techniques to guide and improve code generated by AI assistants.
This course used to be non-AI (last year), and they rewrote recently to incorporate AI tools, as they realised the writing's on the wall for non-programmers.
It must be quite challenging to write curriculum when the underlying technology (AI) is changing so quickly.
It also is naive to think, if AI is shrinking head count, that companies will seek anything other than seniors.
2008 triggered a glut of cheap experienced workers that slowly reengaged juniors but this time that the entire ladder is going to be more valuable than any grad.
In my experience, some students are using AI to write code, but then have no idea what's going on. The "skill" of typing "Write a function to do [basic thing]" will not make them AI-native or help them in the workplace.
How many of them can read x64 or ARM assembly emitted by their compilers?
How many of them will ever need to?
There's your answer.
I learned how to code assembly. Knowing it makes me a better developer.
It’s how I deeply understand what a RAM lookup vs having it already in a register means for optimization.
These are things you need to know if you want to work on high performance applications or in limited embedded systems.
So, yes, many of us need to and it’s important we keep teaching it to future students.
It’s how I deeply understand what a RAM lookup vs having it already in a register means for optimization.
Me, too, but we should both understand what we're talking about: a hobby.
It's like teaching cursive to schoolkids. All well and good, but don't you dare complain about limited classroom time for instruction in other, more important subjects.
Understanding how computer architecture works is a hobby? Is unimportant?
I built and deployed embedded systems robotics that protect water and oil pipeline infrastructure that costs billions of dollars if there is a failure.
That is not a hobby, that’s a career.
The tools I built just helped fix the fresh water access for my entire city of 2 million people. Explain to me what’s more important than people having access to water.
>How many of them will ever need to?
This is a category error. LLMs are probabilistic. The ones run by an AI company over API, even more so.
Compilers are not.
I don't think reading disassembly is actually that weird. I spend a lot of time doing it at work.
for a guy who is in the middle of it all, the opinion seems sophomoric at best. LLMs are based on heuristics and they will always make mistakes no matter how better they get. You'd need someone to 'fine-tune' the conversation with the LLM.
They’re all backpedaling hard on all that “AI is going to replace you” hype.
>"The first chip Huang worked on had 200 transistors, each of which he said he knew by name, while today’s engineers assemble systems from chips containing hundreds of trillions of them without ever working at that level. “Some of the lower-level knowledge is gone,” he acknowledged, and he later described AI as “clearly” a new abstraction level in the same progression."
Jensen gets it!
Related:
https://en.wikipedia.org/wiki/Coupling_(computer_programming...
https://en.wikipedia.org/wiki/Abstraction_layer
https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
https://en.wikipedia.org/wiki/Tower_of_Babel
https://en.wikipedia.org/wiki/Prat%C4%ABtyasamutp%C4%81da
Universities need to be "hardcore" non-AI.
It is so, so easy to pick up LLM aided development.
What is way harder is self-managed slop mitigation which is only achieved with employees who know the A,B,C's of software development.
love love this theoretically but in practice, at least in this “phase 1 of AI craze, unknown how long it’ll last” that someone coming out of Uni with deep knowledge of ABCDs of SWE will be unemployable unless she/he is AI native with harnesses coming out of their ears
With the massive layoffs there have been massive spikes of suicides and od deaths that will only get worse. Articles like this are simply trying to cover it up.