Speaking as someone responding to security@ for a notable FOSS project (C code base dating back to 1996), a meta-comment:
Our current quota is, ca. 20% of AI-found "critical" issues are in fact critical. 3-5% of reported "high" are also critical (i.e. unfortunately it's not possible to rely on AI being excessively alarmist, it does under-classify some issues.) This is with erring towards higher severity though; we didn't get anything that'd e.g. make HN front page.
Simultaneously, ca. 35% of reported "critical" are not even actual security issues. (May be biased due to the nature of our project, AI seems to have issues understanding security boundaries.)
Possible some of this is due to "they're prompting it wrong". Hard to tell since we don't have access ourselves.
I've not seen major differences between models & vendors, but we mostly get reports from Anthropic. (We currently just treat them as functionally interchangeable, which is a data point in itself.)
> The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
On a sidenote: I'm guesstimating the total cost on maintenance side will be 1x…10x of this. (Mostly humans to review fixes. And this isn't made cheaper by AI in my experience; to get a reasonable level of assurance regarding larger design aspects & co, you just end up spending a similar amount in tokens as the involved humans' time would cost.)
> In a run in mid-October 2026, our pipeline using an upcoming cyber model dynamically reproduced 1,724 vulnerabilities and generated PoCs for 291 information leak vulnerabilities and 948 working local privilege escalation (LPE) exploits for a commonly used Linux distribution. The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
Impressive if these pan out to be actual issues. Last time this sort of claim was made with mythos most of the issues were nothing burgers.
Not saying LLMs aren't pushing hard in security research right now, just that the tangibles seems to come largely from a knowledgeable person pushing an agent towards the find and not some autonomous process.
I maintain a Linux build for an IoT product, and it's been a brutal few months trying to keep up with kernel patches. We're trying to go through certification right now, and one of the questions is effectively "are there any known vulnerabilities, and if so justify why they don't matter". Every time I manage to answer the question, a new set of CVEs comes out and we have to update everything before we can answer it again.
I imagine the people who are committed to keeping LTS versions of the kernel patched are having a terrible time right now. I guess at least AI tools make the patching part easier.
> I guess at least AI tools make the patching part easier.
You'd hope so because if it turns out there is an asymmetry between "finding the hole" and "filling the hole" in terms of model capability that's a certified Bad Time(TM) for everyone.
Hell even if there isn't, We still have decades of cruft to fix and we already knew before LLM's that in general defending is harder than attacking, they only have to be lucky once, we have to be lucky every time.
One thing I don't understand about security issues. If you look at the physics the rowhammer "bug" is not a cpu flaw, and not really a memory flaw either. This was known when capacitor based memory was invented, and was put forward as a disadvantage in the very first designs. It's physics. If you do any kind of DDR ram (including, I think, HBM), there is no way to avoid rowhammer-style bit flips. And fixed systems ... aren't fixed. Not even fixed systems currently being built.
Capacitor based memory will be, and will remain, vulnerable to this. Capacitors being charged and discharged will generate radio waves and radio waves will change the state of other capacitors. There is no avoiding this, there is only minimizing it, and at the scales we're talking about, there's only minimizing it somewhat.
There are probably a lot of patterns we haven't seen yet. Like memory write patterns that change other memory by having the waves interfere.
You can only make it more difficult by thinking better about memory layout, and change it a bit by thinking about the physical layout of memory cells. But you're only changing the patterns. It's not possible to prevent it. You cannot fix this. Not now, not yesterday, not in 100 years.
It's currently too difficult for human to think about (even though people are slowly publishing about yet another system being vulnerable to rowhammer style attacks that work a little different from the existing ones)
So why isn't it "game over", at least for the web, for phones and apps, for ...? That bug isn't solved an nothing short of switching everything to SRAM, at incredible cost, will fix it, not just in legacy systems, but new systems too, now and in the future.
The physics makes the rowhammer impossible to fix in practice, but you can add various mitigations: scrambling, ECC, detection and response, etc... On the application level, you can also make it so that it is not possible to do a rowhammer in the first place. Fix everything that can be fixed and mitigate what can't be fixed.
And there is another security issue that can't be fixed, and is much worse than rowhammer: people. But we can't remove people, just like we can't remove DRAM for the performance level we need, so we mitigate the risk. It is not game over because we have other layers of security.
Rowhammer is most certainly a hardware flaw. DRAM manufacturers have pushed cells too close together to meet the specifications claimed on their datasheets. Increasing cell sizes would be the correct hardware fix for the issue. Engineering is accepting the reality of physics to build things that are useful.
Unfortunately, we live in a world where the few percentage points on the production cost of DRAM needed to completely mitigate rowhammer is sufficient to ensure that defective DRAM continues to be sold well after the industry became aware of the problem. This is one of the instances where a class action lawsuit might actually do some good.
> Increasing cell sizes would be the correct hardware fix for the issue. Engineering is accepting the reality of physics to build things that are useful.
Which they're not doing, in fact they're doing the opposite since what you're suggesting is both expensive and makes things (a lot slower). Oh and while I'm complaining: thousands of people are investigating how to place DDR much closer to other logic that also emits radio waves.
Oh ... and even if you did move the capacitors apart, that would make it more difficult. But by using more cells to hammer you can still make it fail. That would even be counterbalanced because the frequency needed for triggering rowhammer would go down as well, so it'd be a more forgiving environment for exploits.
Speaking as someone responding to security@ for a notable FOSS project (C code base dating back to 1996), a meta-comment:
Our current quota is, ca. 20% of AI-found "critical" issues are in fact critical. 3-5% of reported "high" are also critical (i.e. unfortunately it's not possible to rely on AI being excessively alarmist, it does under-classify some issues.) This is with erring towards higher severity though; we didn't get anything that'd e.g. make HN front page.
Simultaneously, ca. 35% of reported "critical" are not even actual security issues. (May be biased due to the nature of our project, AI seems to have issues understanding security boundaries.)
Possible some of this is due to "they're prompting it wrong". Hard to tell since we don't have access ourselves.
I've not seen major differences between models & vendors, but we mostly get reports from Anthropic. (We currently just treat them as functionally interchangeable, which is a data point in itself.)
> The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
On a sidenote: I'm guesstimating the total cost on maintenance side will be 1x…10x of this. (Mostly humans to review fixes. And this isn't made cheaper by AI in my experience; to get a reasonable level of assurance regarding larger design aspects & co, you just end up spending a similar amount in tokens as the involved humans' time would cost.)
>Our current quota is, ca. 20% of AI-found "critical" issues are in fact critical.
What percentage of ALL returned AI security bugs are critical?
Source blog post: https://openai.com/index/a-thousand-fewer-paths-to-root/
> In a run in mid-October 2026, our pipeline using an upcoming cyber model dynamically reproduced 1,724 vulnerabilities and generated PoCs for 291 information leak vulnerabilities and 948 working local privilege escalation (LPE) exploits for a commonly used Linux distribution. The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
[dupe] Discussion on source: https://news.ycombinator.com/item?id=50372929
Impressive if these pan out to be actual issues. Last time this sort of claim was made with mythos most of the issues were nothing burgers.
Not saying LLMs aren't pushing hard in security research right now, just that the tangibles seems to come largely from a knowledgeable person pushing an agent towards the find and not some autonomous process.
Imagine being a Linux maintainer waking up to this. I'd go straight back to bed.
I maintain a Linux build for an IoT product, and it's been a brutal few months trying to keep up with kernel patches. We're trying to go through certification right now, and one of the questions is effectively "are there any known vulnerabilities, and if so justify why they don't matter". Every time I manage to answer the question, a new set of CVEs comes out and we have to update everything before we can answer it again.
I imagine the people who are committed to keeping LTS versions of the kernel patched are having a terrible time right now. I guess at least AI tools make the patching part easier.
> I guess at least AI tools make the patching part easier.
You'd hope so because if it turns out there is an asymmetry between "finding the hole" and "filling the hole" in terms of model capability that's a certified Bad Time(TM) for everyone.
Hell even if there isn't, We still have decades of cruft to fix and we already knew before LLM's that in general defending is harder than attacking, they only have to be lucky once, we have to be lucky every time.
It's going to be a rough few years.
> and if so justify why they don't matter"
"Oh, they matter, but we gotta push out fixes for the 4000 actively exploited holes we have already"
One thing I don't understand about security issues. If you look at the physics the rowhammer "bug" is not a cpu flaw, and not really a memory flaw either. This was known when capacitor based memory was invented, and was put forward as a disadvantage in the very first designs. It's physics. If you do any kind of DDR ram (including, I think, HBM), there is no way to avoid rowhammer-style bit flips. And fixed systems ... aren't fixed. Not even fixed systems currently being built.
Capacitor based memory will be, and will remain, vulnerable to this. Capacitors being charged and discharged will generate radio waves and radio waves will change the state of other capacitors. There is no avoiding this, there is only minimizing it, and at the scales we're talking about, there's only minimizing it somewhat.
There are probably a lot of patterns we haven't seen yet. Like memory write patterns that change other memory by having the waves interfere.
You can only make it more difficult by thinking better about memory layout, and change it a bit by thinking about the physical layout of memory cells. But you're only changing the patterns. It's not possible to prevent it. You cannot fix this. Not now, not yesterday, not in 100 years.
It's currently too difficult for human to think about (even though people are slowly publishing about yet another system being vulnerable to rowhammer style attacks that work a little different from the existing ones)
So why isn't it "game over", at least for the web, for phones and apps, for ...? That bug isn't solved an nothing short of switching everything to SRAM, at incredible cost, will fix it, not just in legacy systems, but new systems too, now and in the future.
Security works by layers.
The physics makes the rowhammer impossible to fix in practice, but you can add various mitigations: scrambling, ECC, detection and response, etc... On the application level, you can also make it so that it is not possible to do a rowhammer in the first place. Fix everything that can be fixed and mitigate what can't be fixed.
And there is another security issue that can't be fixed, and is much worse than rowhammer: people. But we can't remove people, just like we can't remove DRAM for the performance level we need, so we mitigate the risk. It is not game over because we have other layers of security.
Rowhammer is most certainly a hardware flaw. DRAM manufacturers have pushed cells too close together to meet the specifications claimed on their datasheets. Increasing cell sizes would be the correct hardware fix for the issue. Engineering is accepting the reality of physics to build things that are useful.
Unfortunately, we live in a world where the few percentage points on the production cost of DRAM needed to completely mitigate rowhammer is sufficient to ensure that defective DRAM continues to be sold well after the industry became aware of the problem. This is one of the instances where a class action lawsuit might actually do some good.
> Increasing cell sizes would be the correct hardware fix for the issue. Engineering is accepting the reality of physics to build things that are useful.
Which they're not doing, in fact they're doing the opposite since what you're suggesting is both expensive and makes things (a lot slower). Oh and while I'm complaining: thousands of people are investigating how to place DDR much closer to other logic that also emits radio waves.
Oh ... and even if you did move the capacitors apart, that would make it more difficult. But by using more cells to hammer you can still make it fail. That would even be counterbalanced because the frequency needed for triggering rowhammer would go down as well, so it'd be a more forgiving environment for exploits.
If the volume of bugs being found continues, maybe the rate of bugs will slow, but increase on the release of a sufficiently capable new model.
The vibe coded nature of letting LLMs write patches without understanding the impacts, could invite bandaids on top of bandaids.
[flagged]