I still don't understand how to interpret these conclusions. This is all based on a report from a 'third party' 'evaluating' Anthropic, but it's mostly all the same EA people on both sides of it saying the sky is falling. These are the same people that did the junk investigation into the 'HF incident'. How do we even know that METR is being honest, or even objective? How do we know AECI is a good enough benchmark to make this kind of claim, or that the benchmarks that make it up are any good on their own? A lot of Fable 5.2 seems to be a regression in terms of cost and a potential sign of diminishing returns on compute scaling, so to have this come out at the same time claiming we'll see the explosion next year (according to the original article, basically starting in January) is curious.
The authors said we'd be halfway there by now, but most of the political turmoil stuff hasn't followed because the capabilities have not made it as far enough into most workplaces. The public's view of these systems' capabilities and reliability is closer to GPT-3 than FOOM.
I'm not sure that they're wrong, because they have followed AI capabilities somewhat closely. It's the downstream effects that seem to be all over the place by comparison. I don't know what to believe, honestly.
I'd really love to call this end-times millenarianism with no factual basis, but Kokotajlo et al. have had a much better track record at making predictions than I am, and I've always underestimated the rate of AI progress. This is much both much more reasonable and much more terrifying than I'd like to admit.
By itself it is already hard to think these things through, and it is similarly hard to take the conclusions seriously if you do. On top of that there are an unusually high number of highly distributed, cross-domain, and unusually large incentives at play here, that are unusually uniform in direction, making for an extraordinary headwind to sail against.
This is concerning. It's clear we need some level of societal backlash to halt and reverse this.
Society has no true frame of reference for such exponential growth. I see a parallel to COVID. It was not a big deal on March 1, by April 1 society was halted. This is worse.
Its a reasonable opinion to hold people working on such breakneck AI progress to the standard we would hold if your friend told you they were making a nuclear warhead in their basement. It should be stopped.
I still don't understand how to interpret these conclusions. This is all based on a report from a 'third party' 'evaluating' Anthropic, but it's mostly all the same EA people on both sides of it saying the sky is falling. These are the same people that did the junk investigation into the 'HF incident'. How do we even know that METR is being honest, or even objective? How do we know AECI is a good enough benchmark to make this kind of claim, or that the benchmarks that make it up are any good on their own? A lot of Fable 5.2 seems to be a regression in terms of cost and a potential sign of diminishing returns on compute scaling, so to have this come out at the same time claiming we'll see the explosion next year (according to the original article, basically starting in January) is curious.
The authors said we'd be halfway there by now, but most of the political turmoil stuff hasn't followed because the capabilities have not made it as far enough into most workplaces. The public's view of these systems' capabilities and reliability is closer to GPT-3 than FOOM.
I'm not sure that they're wrong, because they have followed AI capabilities somewhat closely. It's the downstream effects that seem to be all over the place by comparison. I don't know what to believe, honestly.
Interesting to revisit the comments on the original AI 2027 submission:
https://news.ycombinator.com/item?id=43571851
The top thread in particular is an incredible time capsule. No matter which way you cut it, the world was different before Opus 5.5.
I'd really love to call this end-times millenarianism with no factual basis, but Kokotajlo et al. have had a much better track record at making predictions than I am, and I've always underestimated the rate of AI progress. This is much both much more reasonable and much more terrifying than I'd like to admit.
Seems like this should be bigger news. But maybe that’s a sign that HN doesn’t buy these scenarios. What is it?
By itself it is already hard to think these things through, and it is similarly hard to take the conclusions seriously if you do. On top of that there are an unusually high number of highly distributed, cross-domain, and unusually large incentives at play here, that are unusually uniform in direction, making for an extraordinary headwind to sail against.
This is concerning. It's clear we need some level of societal backlash to halt and reverse this.
Society has no true frame of reference for such exponential growth. I see a parallel to COVID. It was not a big deal on March 1, by April 1 society was halted. This is worse.
Its a reasonable opinion to hold people working on such breakneck AI progress to the standard we would hold if your friend told you they were making a nuclear warhead in their basement. It should be stopped.