Author here. The argument is not that AI is a bubble or that current systems are mere autocomplete. I expect them to transform software, science, and knowledge work.
The narrower claim is that increasingly accurate reconstruction of cognitive structure embedded in human artifacts does not, by itself, explain how transformer-based systems become open-ended superintelligence.
This is the first of three essays. I’m especially interested in arguments that identify the missing causal mechanism, or show where my account of the ceiling fails.
If the argument is correct, it also drives a stake in the heart of any claims that AI represents an existential threat to humanity because of ASI. I don't make that connection explicitly in the essay, but I'm making it here.
Author here. The argument is not that AI is a bubble or that current systems are mere autocomplete. I expect them to transform software, science, and knowledge work.
The narrower claim is that increasingly accurate reconstruction of cognitive structure embedded in human artifacts does not, by itself, explain how transformer-based systems become open-ended superintelligence.
This is the first of three essays. I’m especially interested in arguments that identify the missing causal mechanism, or show where my account of the ceiling fails.
If the argument is correct, it also drives a stake in the heart of any claims that AI represents an existential threat to humanity because of ASI. I don't make that connection explicitly in the essay, but I'm making it here.