If the investment currently driving new models ceases for whatever reason, there's plenty of ways to burn weights into read-only hardware that's much more energy efficient and compact, we don't even need new lithography nodes for smaller transistors.
Only a few are attempting this today, because the model update cycle is so fast compared to mass production of hardware, that any wights you burn into hardware are likely obsolete before you can ship them.
If you want to get sci-fi about it: what happens when we can take a biopsy from someone to get some stem cells, use it to tissue culture a brain organoid, use a nano-electrode "neural lace" to send the right electrical impulses to exploit biological processes so as to wire up the synapses to encode the same patterns as some AI model (probably not a Transformer architecture, those are situationally good for computers and not likely the best for biology), then implant it in the head of whoever you got the tissue sample from?
With currently reported model sizes, at 1 param ~= 1 synapse, this would be around 1% of your normal brain volume.
Everyone could skip further education while also getting to at least "pass the exams" level knowledge in… how many subjects can LLMs already pass exams in? Does anyone even check this since GPT-4?
> but strictly thinking about capabilities.
Yes, trivially so.
If the investment currently driving new models ceases for whatever reason, there's plenty of ways to burn weights into read-only hardware that's much more energy efficient and compact, we don't even need new lithography nodes for smaller transistors.
Only a few are attempting this today, because the model update cycle is so fast compared to mass production of hardware, that any wights you burn into hardware are likely obsolete before you can ship them.
If you want to get sci-fi about it: what happens when we can take a biopsy from someone to get some stem cells, use it to tissue culture a brain organoid, use a nano-electrode "neural lace" to send the right electrical impulses to exploit biological processes so as to wire up the synapses to encode the same patterns as some AI model (probably not a Transformer architecture, those are situationally good for computers and not likely the best for biology), then implant it in the head of whoever you got the tissue sample from?
With currently reported model sizes, at 1 param ~= 1 synapse, this would be around 1% of your normal brain volume.
Everyone could skip further education while also getting to at least "pass the exams" level knowledge in… how many subjects can LLMs already pass exams in? Does anyone even check this since GPT-4?
• https://en.wikipedia.org/wiki/Cerebral_organoid
• https://en.wikipedia.org/wiki/Long-term_potentiation
Nothing in the laws of physics prevents it. Heavy regulation might.