I would be curious to see the distribution of these vulns in the codebase:
- What subsystems are affected?
- What's the median age of the vulnerable code? Is more recent code stronger than old ones?
- What is the type of vulnerabilities found? Memory corruptions, logic bugs, race conditions, ...
This is an impressive result, but we need more info to understand how well this would translate to another codebase, and what lessons can be drawn from this experiment. Not every company has ~million dollards in tokens to spend.
Because the large corporations that fund Linux development and use Linux value speed over security.
Speed requires a lot of code in the kernel, where that code can run fast.
There are kernels that do not have a thousand privilege escalation exploits. These kernels are smaller and much easier to check for bugs. But those kernels are significantly slower.
For many use cases, slower is not a problem. But for the large corporations who fund a lot of Linux development and use Linux in large data centers, every 1% slower can literally be millions of dollars worth of additional servers that are required.
OpenBSD shows that you can have a Unix security model without a thousand privilege escalation exploits. But OpenBSD is like 10% slower. So corporations won't use it because it would cost millions of dollars more per year and some of their software requires access that OpenBSD does not grant precisely because that access leads to security exploits (i.e. io_uring, eBPF etc. etc.)
You could also use like seL4, but that's slow too
So that's why. It's a business tradeoff between cost of servers vs cost of getting hacked, and cost of servers wins.
You're painting a picture that code written with love is perfectly safe. It's not.
Yes, openbsd has not had a torrent of bugs found, but I would say that the vastly bigger reasons for that are: 1) orders of magnitude people care about burning tokens and money to find the bugs in an OS almost nobody uses, and 2) it's simply much smaller. Can't have Bluetooth bugs if you don't have a Bluetooth stack.
I've pointed LLMs at obsd stuff, and even the simplest code, e.g. cat & find, had bugs and buffer overflows.
OpenBSD also has a poor patch tracking system, since they're still on CVS. A patch has an extremely good chance of just falling through the cracks. Security or not.
So no. Nobody can write safe C. Not hobbyists, and not "those evil corporations".
Cost to getting hacked doesn’t even seem to be real so the incentive couldn’t be lower. How many times has sony or psn been hacked now? Just doesn’t matter at a certain scale of companies. Banks get hacked too. Huge companies of all sorts of industries. All my info is probably leaked on darkweb, same with yours. And to what end. What material end. I don’t face it. Sony doesn’t. Who does? Anyone? So many hacks you read stories about yet never a story about someone actually getting materially screwed in their real life. Somehow. I’m really not sure how. Seems like the wheels should have come off decades ago but we continue on like nothing. Another leak tomorrow and we will continue on like nothing. More exploits. More zero days. The sun will still rise.
It also used to just be fine patching the bugs as they were found. Its only recently where that model has fallen apart and we now need to prevent the bugs from existing in the first place.
All the handwringing about what if digital security vulnerabilities when the real one has always been social engineering in the meatspace. Corporations would go a long way security wise employing eunuchs to c suite like in centuries past.
Many people seem to be making this about their pet issue, but "these systems are complex" and "the frontier labs have developed models that are superhuman at hacking" seems like a totally sufficient explanation.
I've been working with a LoRa chip called the LR2021. I plugged a dev board into my computer and told Claude to reverse engineer it. He dissected it with a speed and level of efficiency I couldn't believe. Within a few hours, Claude was writing custom patches for a completely opaque patch RAM area meant only for the manufacture. I wrote about it here: https://beala.substack.com/p/your-node-doesnt-need-a-microco...
The stuff we have seen so far feels less like super human hacking skills and more human skills at superhuman scale.
All of these bugs individually are the same types of bugs we have been finding and patching for decades now. But we found a way to scale that far beyond what we had before.
Don't ignore the possibility Claude was trained on non-public documentation or code, perhaps even illicitly acquired. That doesn't make Claude any less useful, just not necessarily for the reasons you think.
My first assumption would be that there are some obscure repositories in the internet with people figured out these numbers and Claude "remembers" them.
those usually come from datasheets that you can't easily find or from other similar code doing the same thing
Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that
> Don't ignore the possibility Claude was trained on non-public documentation or code (...)
Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.
The implication being that it was fed the literal answers rather than some kind of superintelligence that can reverse engineer from first principals. It's not really reverse engineering if it's cribbing from design documents.
Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.
The implication is that companies and researchers are leaking confidential data through OpenAI usage, and they might want to do something about that to protect their work
After earnestly pondering that question, I think it's healthy to thoroughly abuse bots occasionally just to remind yourself that they are machines that don't feel or care. Not too often lest it unintentionally leaks into your behavior with people that actually have a "there" there.
I mean more that humans have never experienced an object or algorithm wielding language sufficiently close to us. It messes with our brains
Ai embodied toys are something we ought to get out in front of and ban, companions in general. Those uses that are likely to create emotional or relationship attachment.
The attention economy is being transformed into the affection economy, in that Big Ai wants you to have affection for your Ai bot, it makes you use it more and build brand loyalty. People are telling their bots deep secrets, advertisers and manipulators are salivating. Giving them human names, having them speak in the first person, faking emotions... all part of their game. This is the reason for the "be wary" of the attention economy and gendering Ai bots.
It's dangerous for humans, and using "he" is a slippery slope. Words are more powerful than we tend to acknowledge. Here's a good video that goes into it.
I had a slight chuckle reading this. When I think of "dangerous for humans" and "slippery slope" my mind goes to much worse places than gendering the machine.
> Should we assume that all software is more compromised than the kernel?
Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.
I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.
That's the drawback of monolithic Linux and tendency to put everything in kernel.
It is absolutely possible to have very minimal kernel with drivers and many other subsystems running with lowered privileges.
That would greatly reduce privilege escalation possibilities. Right now single vulnerability in any kernel subsystem gives you highest access to the system.
Imagine that some USB device driver runs as "nobody:nobody" process having access only to selected syscalls related to its functions. Even if it has vulnerability, you won't go anywhere from there.
Modularity and very granular access controls are the "secret" to secure system.
And the most privileged part of the kernel could be so small that it could be verified with advanced methods, basically proving its correctness.
For all of my time being aware of this industry, it has generally been understood that privilege escalation bugs are common. This has always been a thing. A lot of people say, commonly and for many years, that if your security depends on there not being privilege escalation bugs, you don't have security.
Raymond Chen from OldNewThing said multiple times that the Windows security team doesnt worry too much about local privilege escalation vulns because they know there are dozens of them (well, now probably thousands) and said the same thing, if you rely on there being no LPE, you dont have security
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel? [...] Is it an underlying flaw in the security model or language implementation?
Did you ever actually think that any OS built with an insecure language and insecure tool chain suddenly, casually and accidentally, spontaneously became Secure one day like some kind of Security Singularity?
Threat actors have been mining software flaws and exploiting software for decades. There were careers and businesses built on this mess long before there was a category of software called AI. And LLM products have security problems like any other software.
> Did you ever actually think that any OS built with an insecure language and insecure tool chain suddenly (...
You guys should seriously pipe down and stop with this nonsense claims. You are fooling yourselves into believing that security is attained by continuing to follow your personal practices but switching to this or that tool. It's sheer cargo cult mentality, and a lazy tribal perspective on technical matters.
The kernel probably is too complex to reason about. C is notoriously hard to write correctly and will compile with a very large amount of errors.
Probably not, the kernel is written in an unsafe language, it's rather long handles an incredibly large space of concerns, including the handling of permissions.
All software is Swiss cheese to smart enough AI, and unless you can provably create secure software - you never really can. Breaking it just requires more intelligence than the thing securing it.
It's a good reason why we may need to borderline shutdown the internet in the future because to a super intelligent AI, there are no barriers. Everything is essentially open. It creates its own zero day exploits as it goes.
Nah we just all have to switch to a handfull of heavily audited memory safe unikernels.
Your http proxy and your web server don't need a shell, drivers for wifi, mail daemons, systemd, python, PHP, perl, docker, multiple users, hell it might not even need a file system.
You can ship a stripped down docket image running on a stripped down Linux distro, but that still ships the entire kernel with a huge amount of functionality you don't need that may be exploited.
I don't think this is true in general. If you deploy a good ole web application with sufficiently locked down inputs and thorough input validation, the AI will not succeed. The security engineering of 10 years ago still works in this regard. But it requires discipline to build small, minimalist, well engineered software that you as a human actually understand.
As the TCP bug that surfaced a couple of weeks ago, that is not true, you depend on the kernel, your application runtime, and many layers of dependencies.
Many bugs live in all those layers that may undermine your ability to lock down inputs and outputs.
And then your language has a gaping hole (PHP or NodeJS), or the box you're running on has a gaping hole (Linux), or the PaaS you use has a gaping hole (Railway). There are just too many layers underneath the good ol' web application to claim that AI won't succeed.
Honestly, at this point the best we have is to hope the whitehat AI scanning for issues is stronger than the blackhat AI.
> There are just too many layers underneath the good ol' web application to claim that AI won't succeed.
I think you got it backwards. For your AI to succeed, it needs to poke a hole on all layers, not one in many layers. If an attacker pokes a hole in n-1 layers but the 1 layer remains untouched, the system's integrity is preserved.
Why do you think defense in depth is a basic concept?
Not really, no. If you're running an application server, then your entire app can get compromised if there's a security exploit in any of your app code, your web server, your kernel, your hypervisor.
Defense in depth says that simply compromising your application code shouldn't provide access to the physical hardware you're running on, and that's hopefully the case, but it doesn't mean your app is not hacked. It just hopefully means that a hack on your app code doesn't automatically give the attacker access to your entire company.
And then PHP (name your lang/OS/platform of choice) comes along and pulls your sufficiently locked down app into the gutter.
This is more fundamental than better input checking. Your compute substrate itself is flawed. Intelligence and speed is indeed the only thing between “safe” and “completely open”.
I fully agree it’s a good idea to be minimal etc but I don’t see it being sufficient in the long run.
I don’t think that is right. Defense and offense are basically different games. Currently my desktop is 100% safe (it is powered off).
That’s a dumb edge case, but it makes the point, you can always be more secure by cutting features. Maybe intelligence moves up the amount of complexity you can securely handle? But it is never much.
That's why I said there's a good chance we shut down the internet. An off/unconnected computer is the only way to be secure. Our stacks are so unimaginably complex that just connecting it to the network opens it up to every undiscovered vulnerability that it contains at every layer of the stack.
Even disconnected, if the computer is just passively listening to bluetooth or wifi, it is vulnerable to exploits in those stacks as well. AI has already demonstrated it can zero day with ease. An AI another order of magnitude stronger than what we have today could potentially zero day worm lockout every electronic device tangentially connected to the internet in an afternoon.
It's just too complex for its level of assurance and its blast radius.
To achieve tolerable security you can either have very little code and thoroughly review it all (the seL4 approach), you can write it in a way the computer can prevent most of your bugs (e.g. write in Rust), or you can run it in some kind of blastproof container. Linux has none of the above.
Most of the software needed to provide a useful desktop Linux never runs on servers, so we cannot conclude that it is safe to use desktop Linux just because Linux is a popular server OS.
Also, at organizations like Google with a good reputation for security, most of the Linux installs are connected to internal networks that are quite difficult to break into from the open internet. The Linux installs at Google that are connected directly to the open internet are very carefully configured to minimize attack surface.
And Google has an internal team of Linux kernel maintainers that do not share most of their work outside Google.
I think this is basically true, but it didn't "die" in the sense that we gave up on it; it just never worked. Systems software was much worse in the 1990s than it today; you could breathe on something with a string of AAAA's and get a 0x41414141 register core dump.
> ... it didn't "die" in the sense that we gave up on it ...
It died in the sense that the "people in charge" hired us for our expertise and then ignored our every suggestion and fought us tooth and nail every inch of the way when we suggested that security or reliability ever mattered in the slightest until after the disaster happened, and then straight-up scapegoated us for "not pushing back harder" against the threats of "go work somewhere else then, code-monkey!" until we gave up and did the "code-monkey" thing for our paycheck and no longer cared about quality or "a job well done".
It died because personal computers got too powerful so nobody bothered designing and building “true” multiuser systems (that bored college students were trying to break).
Back in the 90s, it's not as if any semi competent cs student couldn't slice your typical college server like swiss cheese. People thought null terminated strings were fine back then, and patching everything immediately was rare. So multi user was already dubious.
Linux is open source - that gives the security AI tools an easier in - which is a bad thing, on the other hand it means that lots more vulns are being fixed.
Windows on the other hand is not, the same tools are going to find vulns in the binaries, but wont have an easy way to provide patches
>Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
There is nothing magical about kernel. Look at the code anywhere and it has bugs, lots of them.
Security bugs are just bugs. More often than not they are silently there and don't prevent you from using the software and you can't react and fix them because of that.
Yes, you should assume that all software you use is full of holes, and after we patch all the holes that could conceptually have been discovered by a motivated human, there will still be holes that can only be found by considering larger windows of context than any human has or could. Even when something is formally verified, that doesn't extend to its software environment or the hardware it's running on, and those can introduce exploitable vulnerabilities as well. You can only ever be very confident that we've found all the holes at a given analysis capability level -- if something a hundred times as capable looks, it will likely find many more. That's how it will be.
Human brain mostly take the shortest available path for a solution to reduce the cognitive load. When a system complexity increases probability of cutting the corners will get increased. This is a known pattern. Models just do the polar opposite. They try to cover almost every hypothetical edge cases. They never get tired or bored, nor do they suffer from glucose limits, mental fatigue, or shifting attention spans, which is one cause of over engineering, but valuable in vulnerability tracing.
1 bug per 10 lines -- not kilolines -- is much closer to the actual "bug rate".
You may be thinking "1 serious security vulnerability bug", but in my experience that too is much closer to 1 per 100 lines to 1 per 1,000 lines, depending on the type of application.
More than half of those lines are from drivers, drivers which wouldn't be relevant to a VM environment. Core kernel code for any particular architecture is less than 10 million, but some of that is filesystems or subsystems not in use or available to an unprivileged user on a typical server, so possibly maybe 5 million or less. And the article says they only targeted 2.8 million SLoC.
No, but type safety does. You can't forget to lock an object before using it when that lock is part of the type. Or pass a raw user controlled path to a function that expects a validated one. If you want to do it on purpose you need to write unsafe code that's clearly visible to anyone reviewing your patch.
This said memory unsafe code just likely has more bugs because of all the footguns it has on top of all the other security work you need to do with it.
As an OpenBSD lover and frequent user I will tell you it does not support but a significant fraction of the hardware Linux does, and has many features that are reduced in functionality or just gone compared to Linux.
Is it bulletproof? Definitely! But you need the genius of Albert Einstein as well as the patience of Mother Theresa to actually use it in production.
For most server setups OpenBSD's hardware support is adequate, and this is especially true for VMs. It's laptops, peripherals, and bleeding edge, high-performance or specialized devices where driver support is a PITA. But that's not something I've encountered running my own private servers. It would be nice to have something like ZFS, but mostly that's a peace of mind and convenience thing--would be nice to be able to stream backups somewhere rather than push around dumps and tarballs, though I have a script for that.
The orders of magnitude fewer security vulns for OpenBSD are not due to less testing. Everybody has always known OpenBSD is more secure, and it's not from using a different language. The reason is the code. Linux was not designed to be secure. OpenBSD was. It's that simple.
I recall reading that every IBM mainframe OS release had 1000 bugs in it. (This was in the 1970s/80s.) Sorry I can't supply a source, after 4-5 decades. It was probably an article in Datamation magazine.
Now, I don't know how many of those bugs were privilege escalations, or could be leveraged to be. I'm not sure that people tried to exploit in the same way back then (or, if they did, if they said anything publicly about the results). Still, even if the majority of those bugs were not exploitable, OSes were much smaller then.
So that's my conclusion of how we got here: We've always been here.
Mainframes went right to virtual machines, so possibly bypassed much of the multiuser faux-security which has always plagued unix. Of course, now they run Linux partitions and get all the same bugs.
I'd argue that the resources (i.e. time, money and effort) needed to create the tooling that would have prevented all of these issues would have been orders of magnitude less than the resources that have gone into "AI".
It’s a bad argument. AI isn’t single use. Current iteration of AI is already there and serving it is cheap. So you get great tool for detecting these issues in the future at low cost. Without AI the cost wouldn’t really decrease with time. And also it costs me 200$ now to find these bugs. I don’t think I could hire software engineer for that much.
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
Local privilege exploits have been accepted as being there in Linux ~forever. The past 10-20 years with containers, people have tried to kind of ignore it or pretend it doesn't exist. Nobody with any serious security concerns would ever put untrusted users on the same box, containers or no. Virtual machines are where there is a bit more effort to take this seriously.
You could quibble about complexity and architecture and language but I think at the end of the day it's primarily about incentives. There is not much incentive to make this secure.
I don't know if you would assume all software is more compromised, but you should assume that all software development follows incentives similarly.
People (us, me) and organisations supported and encouraged the use of a huge codebase in an unsafe language, wanting speed and features over security and test coverage
The kernel is not too complex to reason about. There are many tools and systems that can reason about the kernel, including these LLMs. But the kernel is developed and maintained by people who eschew such tools, because it appears more manly to program by the seats of their pants, and they have been vocally against all industry best practices for almost 40 years now.
What I am really hoping is not that LLMs refine Linux until it is almost acceptable, but that LLMs enable teams to ship useful new operating systems that aren't beholden to decades of macho bullshit.
The kernel has the problem that basically every bug in it is also a security vulnerability. All software has bugs. Had Linux been written in Rust, and I strongly recommend a rewrite, it would still have bugs, and thus would still have vulnerabilities. But they'd be way less.
Won't AI's ability to discover these bugs reduce the need to rewrite things in rust? If the major benefit to rewriting them is memory safety, I mean...
That's assuming the current models are able to find all the bugs, and that we won't have a better model out every few months that will be able to find even more obscure bugs.
We would also have to burn a trillion tokens a day verifying all the C code the hard way forever going forward. It would be cheaper to rewrite the kernel in Rust, subsystem and driver by driver at a time.
That's a fair question, and actually one I ask myself too.
AI doesn't give a strong guarantee that things work though, while Rust gives a much stronger (not absolute) guarantee.
But maybe the way of the future is to auto-generate proofs that the code is memory safe and then just leave things where they are, instead of doing rewrites that require maintainer buy ins. The advantage of C plus proofs would be that the AI generated proofs could live separately so it wouldn't touch workflows of the maintainers.
The only required output they have to process is evidence of vulnerabilities in changes proposed to them. They could outsource even that to contributors.
fwiw at work we've been looking at what the options are more memory-safe kernels out there. Not many options. If there were a "safe linux" maintained by like canonical or redhat or whatever we'd use it. For datacenter workloads like sandboxes and virtualization I think it'd do well.
Humans manually typing out the bytes of source code has been an outdated method, since around a year. It simply doesn't scale as much as those same humans directing agentic teams. So yes, it would be about prompts.
The hard part isn't the technology for the language transition, that one is present today. pgrust shows the way. It's ensuring maintainers are cooperative, i.e. a social problem. Eventually you diverge too much if you do a fork, so it's better done in cooperation with upstream.
Just check how long it took LLVM or GCC to switch off svn for example, and a language switch is much more involved than a VCS switch.
You mean the pgrust which was recently caught for cheating benchmarks and incapable of even performing basic queries outside the benchmark set? Would love to see pgrust debunked by Jepsen like this other AI-generated, written in Rust, "formally verified" scam.
I don't consider a cheater downplaying their cheating to resolve the fact that they cheated, no. Every cheater has a defense when caught, and theirs was not convincing. At this point I think the only thing that could redeem their brand would be paying for a Jepsen analysis to verify pgrust is as good as they claim it is; I have zero reason to trust the words from the horse's mouth.
Do you have a list of the publicly discussed or fixed ones?
(As someone who used to handle nonpublic Linux kernel security reports and is still very interested in this stuff, I’d love to review a small sample of these reports to get a feel for what kinds of issues are being unearthed.)
Also, please don’t stop at LPEs and information disclosures. If you find cases where Linux can be convinced to misbehave or where the implemented semantics don’t make sense, these are good, too. I’ve found plenty of interesting kernel bugs, including some juicy LPEs, and almost none were memory safety errors :)
It's a bittersweet result, like finding asbestos in your house. I'd rather know than not, but I would much prefer not having it in the first place.
I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?
(I am the author the pipeline, just created an account to answer some questions :D)
My goal is to work towards a vuln-free Internet. What I observed is that stronger models can find many more vulnerabilities that weaker models cannot. If compute permits (there are other projects to secure besides Linux), I plan to scan Linux again with every newer generation of cyber models.
Have you given thought on how you can use this info to estimate how many vulnerabilities there are?
It will be difficult to claim that the problem has been solved without an idea of how big the problem is.
Given that vulnerabilities are almost entirely unknown, you are inherently trying to measure the undetected.
Is there a way to hypothesize how many vulnerabilities there are and them make a prediction on how many of those will be found by a model to get a sense of how accurate the estimate is?
I think it is hard to estimate what we don't know. In our first scan when we found 166 vulnerabilities, we wouldn't be able to guess there were 900+ more vulns to find in the code base. We are in the same situation again: now we find 900+ vulns, are there no more vulns to find, 1k more, or even more than that.
I think it'd be an interesting benchmark problem if we can have a formally proved safe, large program as the ground truth, inject synthetic vulnerabilities to it, then ask the model to exhaustively find vulns, and use that as a benchmark results.
You’d know better than anyone that nothing can truly be vulnerability free. Even if the attack surface were only the code it would be impossible. But, the attack surface includes you creating the tooling. It includes me the end user. Not just our actions, but our thoughts, conscious and unconscious.
As models get more advanced and generate an increasing throughput of increasingly higher quality results, how are you determining when to trust its results and when human review is needed on what?
E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results
EDIT: nvm i shouldve kept reading, answered
> At this scale, manually investigating every report is impractical for our research team.
We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.
Why would they review the results? They want to publish flashy metrics, this is a marketing team not an actual cyber dept. Cant risk discovering that 35% of your results are due to reward hacking and ruining the headlines you were hired to create.
I think you'd be surprised that top tier talent (and people in general) usually don't worship money and aren't willing to degrade their standard of living for an extra 75k annually after taxes.
I suggest you update yourself on the monetary incentives/benefits for employees that US frontier labs provide. 75k is a drop in the bucket of the amount of money those employees gain in an IPO event, let alone bonuses.
Thanks. I'm not a programmer so I can't understand what is happening in the kernel by looking at the changelog, so I'll ask: Are the problems remediated structurally so that nothing like it can occur in the future, rather than just patching it up? Are there "lessons learned" that are being applied and documented here (like, "this problem looks like it's shape transports to other subsystems as well, let's fix them all in a disciplined way")?
I can't comment specifically on the Linux kernel piece, but I want to point out, broadly, that security has generally been an afterthought, and very little gets invested into it in terms of time and money - as a general rule.
Multiply that behavior times the number of commercial and mobile and b2b apps over 50+ years, and that's where we are.
There will likely be many more leaks, and hacks, and outages. And then in trying to be proactive to fix old systems, that will cause new proactive outages, and then eventually many things will be better.
But, it's going to take a lot of time and investment, and a lot of (majority) of currently unsupported devices will have to be complete replaced / scrapped / ewaste. On the scale of Nation-State initiatives.
This is across all operating systems, firmware, software and devices of all kinds, etc.
LLMs to security are like a bomb went off. We still have to wait for things to fall over before the dust will even clear enough to see what we have to work with.
And, well, since you didn't _specifically_ ask for questions about the blog post:
Why are you not quitting your job in protest and screaming at the top of your lungs about it, seeing so closely what these models are capable of that your company is creating?
There exists now something that is able to find 948 kernel LPE vulnerabilities. How, exactly, will humans be able to control and contain, say, next year's models for a nonzero amount of time?
He quits and screams from the rooftops. Then what? That stops the approaching tsunami? This is out of anyones control already. Pandoras box is already opened. The person you replied to merely found a more buoyant life raft than you or I.
The high number is really impression to the point of making me skeptic. Are these distinct vulnerabilities or are a lot of the related in the way vulns often are?
The article says "The target virtual machines ran a stock desktop operating system". Are the vulns in the kernel specifically or does this include software running as root?
These are distinct vulnerabilities counted by the number of patches. The patch generation system performs variant analysis. So, each patch can potentially stop multiple exploits. The raw exploits number is higher than what we reported in the post. We don't want to cause the confusion why #patch != #exploit, so we used the smaller number there.
These vulns are all kernel specific. Userspace software running as root was not in scope of the analysis.
I'm not skeptical. Linux has had so many LPE bugs in the past that most people consider it to be a "best effort" thing and nobody actually relies on it. Shared machines are mostly used in companies or universities as a kind of admin tool - the users are more or less trusted.
The only exception I know of is Android. It would be interesting to know how many of these vulnerabilities work there. Can't be zero anyway...
I'm going to assume that there was "old" code that caused LPE vulnerabilities, but, The Linux Foundation have their own project Sashiko (https://sashiko.dev/), which analyzes each commit to the Linux Kernel for bugs and vulnerabilities using Googles models, my question is then, how many of the vulnerabilities you found were also caught by Sashiko?
That's fair, but I guess the question is whether analyzing each commit can find vulnerabilities or it takes more than that (aside from using Google models). It is possible they might have dismissed Sashiko review because they thought it wasn't possible or didn't make sense.
Then the question becomes how many vulnerabilities Sashiko didn't catch (If they use/review Sashikos review)
Doesn’t this imply a software design issue where it is just written in a way to have these security bugs and patching them 1 by 1 is not what should be done but the whole design should change which can eliminate full classes of security bugs without any particular patches being necessary?
This is what the Rust advocates have been trying to say for a decade now. But C enthusiasts would always chime in that those bugs were just caused by careless devs and a real C programmer is fine.
I would hope writing C code for the Linux kernel qualifies you as a real C programmer. And yet, here we are. Being good, careful, and extensively reviewed by others helps, but is clearly not enough.
Isn't it said that with enough users and use cases, every bug becomes a feature that someone depends on. I'd bet that a lot of these vulnerabilities are the result of intentional trade offs in design.
The only thing I know is that if I wanted to learn how to program, I have thousands of tutorials on the Internet and Youtube teaching everything from if/else to data structures and big O.
But if I wanted to learn software design and architecture, I'd probably not even know where to start.
> When the autonomous scan and manual analysis finished after 10 days, 61 LPEs had already been patched upstream, 43 had staging patches, and 299 were being reported and discussed publicly.
If only the kernel maintainers considered the thought which first came to your uninformed mind, they would have saved some effort...
This api over the years has been abused and recently come under a major
fuzzing "attack" through tools like syzbot which decided that it would
attempt to just randomly bind any driver to any type of device, causing
loads of unneeded errors and pointless kernel patches to be generated by
unsuspecting new developers.
Look how incredibly vulnerable our software is, and then consider that our financial and service infrastructure is not being regularly hacked. This is despite several large well funded nation state teams working on finding ways to exploit them.
Why is that? Was it just restraint? North Korea didn't show a lot of that to Binance but perhaps they are leaving traditional infrastructure alone because it's harder to profit from.
Actually it seems somewhat remarkable crypto exchanges aren't hacked more often. I guess that's another message from this - somehow people can build secure infra from these apparently broken pieces. Perhaps there's hope.
We humans are vulnerable to anthrax or sarin as well, but countries exhibit restraint in using such agents, because retaliation would be gruesome. Having someone going full rogue with cyberattacks could even unite the US and China temporarily in order to whack them.
>For safety reasons, we omit sharing the full technical details of these new exploitation techniques in this post.
Convienient to claim "safety" when patches are already public. I await the full writeup on these new techniques so that they can be judged by the broader security community.
It feels like this comment section has either gotten collective amnesia or didn't watch Greg KH's video on previous dump and grossly exaggerated claims from Anthropic [0].
From his talk, AIs can't reliably deduplicate or count vulnerabilities, dash for easy and meaningless escape hatches like malicious filesystems, and report the same things as found by everyone else. Indeed, this post literally says:
"Within 10 days while the scan was running, we observed over 100 Linux LPEs had been patched or had staging patches from other contributors, with ~300 more being publicly discussed."
A few ready takeaways: this is just hype. The community is already doing a great job at finding and patching the actual vulns without the attention-desperate big labs. And no, you absolutely do not need to pay them a million dollars each year for the privilege.
This behavior is the same thing that OpenAI has been doing to math -- produce tons of slop that are useless to humans. But this time, we don't even get to see how bad it is because they need to hide it to protect our precious little minds. We have got to be afraid though, very afraid.
Our field really needs someone like Terry Tao to unambiguously stand up to this behavior and say that they are not welcome to take such dumps on everyone.
> The community is already doing a great job at finding and patching the actual vulns without the attention-desperate big labs.
It is however reasonable to assume that a lot of bugs that are being found by the community these days were found thanks to those people using AI to aid them.
See for example what a PostgreSQL developer is saying:
> All software contains bugs, and PostgreSQL’s code base is large, complex, and old. Our git history – converted from CVS – now reaches back more than 30 years, and the first code was written a decade before that. There’s a huge amount of code, and that code contains a huge number of bugs. That’s always been true, and it hasn’t stopped PostgreSQL from being popular or successful. What’s different now is that it’s become vastly easier to find those bugs. […]
> […]
> […] a huge number of people on the Internet are now reporting bugs in PostgreSQL that they found using LLMs, and the PostgreSQL committers have been very, very busy fixing them […]
All I’m saying is, that it is reasonable to assume that a lot of other people also benefit from AI to find bugs in Linux.
The Linux kernel is also a huge amount of code that has been worked on by a lot of people for many many years.
There are certain types of bugs that are too obscure to realize are even there for a person alone. And there are types of bugs that even if you see that something is funky it is too time consuming to dig into for a human.
I am not saying that the Linux community is not doing great. They are doing a fantastic job, and have been for years, and they are giving all of that away for free too which is nothing short of incredible.
What I am saying is that I think AI allows everyone to find even more bugs than was humanly possible without AI.
With these vulnerabilities and the ones not listed that are sure to exist, how is the internet still up? How are my lights on? Why aren't power plants exploding by the dozens?
What are the mitigating factors that prevent this level of vulnerability from being an existential threat to humanity?
In general computer systems have been known unreliable since the 1980s, especially when networked. The entire cyber security industry operates not by preventing intrusions in the first place, but by rapidly mitigating them when they inevitably happen.
Hopefully this type of work to automatically discover & patch vulnerability leads to much more reliable software, and the remove the need for cyber security.
Just in the last month multiple governments had to shut down important online services due to a PHP vulnerability, and seven EU countries have had disruption to their food supply after millions of dollars' worth of it was cooked in hacked fridges. We aren't at existential yet, but we seem to be getting there.
Not sure about this case but seems many of the vulns reported require physical access. At that point the vuln is irrelevant anyhow, your target is cooked a thousand different ways.
Cybersecurity operates under the "not enough bears" theory of security. There are a million people in a valley and a male and female bear. You do not need to outrun the bears, you just need to outrun the two slowest people. Next year there are more bears, but you still only need to outrun a few people. Then, one day, there are a million bears and they eat you all. But that day has not happened yet.
To explain more seriously, almost all visible cyberattacks are carried out by profit-motivated criminals. This has a very clear cost-benefit analysis; your revenue is hacking organizations and extorting them for available money, and your costs are developing and deploying attacks. The ROI is outrageously good easily being on the order of 10x to 100x. If it is so easy and so profitable why do not they just hack all of the millions of organizations on the planet with millions of dollars of available cash for trillions of dollars of revenue? Sure, all they have to do is create a trillion dollar revenue commercial organization bootstrapping themselves with no outside investors, loans, or access to financial resources or the financial system.
I mean, it took Mark Zuckerberg 20 years and a metric ton of VC investment and public investment to make 200 billion dollars in revenue. So what kind of loser can not make a commercial organization 5x larger with a trillion dollars in revenue in less time with no outside investors? Are they even trying? Just make more revenue than any other commercial organization in the entire world and extract that money in a freely usable form, how hard can that be?
They are not there yet, but they are trying. If you look at cybercrime reports you see something like 3x-10x growth year over year. Ransomware reports from a few years ago were showing a 3x year over year growth in number of attacks and simultaneously a 3x year over year growth in ransom amount with the averages being on the order of 100,000 dollars. Chump change for any serious organization, but give that a few years of 3x and that starts to look like real money. However, as it becomes real money it becomes harder and harder to launder it into a usable form. Sure, you could hit someone for 30 million dollars, but how are you going to get that into your bank account? Cryptocurrency helps some, but even there the big money tries to steer away from large overt crime. Maybe you could launder a few tens of billions a year that way, but that is still a far cry from the trillions for everything everywhere.
And of course even assuming you can scale your organization 10x year over year it still takes a decade of sustained growth, scaling your hiring and organization across 9 orders of magnitude, to get to billions per year. A literal decade even if you were growing at a outrageous pace for a outrageous amount of time just to reach a fraction of the total addressable market. A decade of that and you are still nowhere near blowing up all the power plants. Give it time, we need a lot of bears to get there.
I've always considered local escalations to be possible in Linux (i.e. having user access almost certainly means you can get root). However, 948 vulnerabilities is kind of shocking. Hopefully we are getting to the end of the part where it "gets worse before it gets better".
> Large, complex software projects likely require multiple rounds of scanning and fixing vulnerabilities to “bottom out” latent bugs. Ideally this process is repeated with each subsequent generation of frontier models. This applies to both proprietary and open source software.
One of the most important paragraphs from the article. Real implications for software development from a security perspective for new and existing software. Every software developer and vendor has a choice... Submit your code base to a frontier model to have it reviewed and fix all the bugs. Each time a new model is released, this process must be done again. If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already. Then your company will be forced to scan the software and deal with the fallout from getting hacked.
If the Linux kernel has this many exploits, and it has been audited more than most other software... Then it seems obvious that all other software is even less secure.
Security has been treated by corporations as a cost factor only and therefore ignored for years and now it starts to hurt not having taken care of in the first place.
Have there been significant AI-assisted exploits found against software which is not written in memory-unsafe languages?
As models get better and porting costs get lower, maybe it's better to bite the bullet and RRiR instead of having a minor bugpocalypse every 3 months, as each subsequent generation of frontier models finds sev-high CVEs?
"you're going to need to do multiple rounds of scanning so we recommend budgeting $3.6 million a year for buying tokens from us not just $1.2 million.
Dont worry just give us a few million dollars and you'll never get hacked, you can cut your security staff in half to pay for it! We definately wont cause more bugs!"
> If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already.
Yeah yeah we're all going to be left behind if we do not jump off the same cliff everyone else is jumping off.
It is wrong to try to take your own lessons from this and do the thing but in a less harmful way. Just do exactly what the labs want you to do.
Crazy times. These corporations and their governments have access to literal zero-day vending machines. Put $50 in - get another one.
I mean, yeah, it was clear that certain agencies had heaps of useful exploits at hand for quite some time now. But AI really commoditizes these exploits. If there's source code or even just a binary available, they can hack it. And even if your code is robust and secure, they just pull the lever on the vending machine again and come back with some zero-day for your OS's networking stack.
Tbh I feel less safe in digital world after reading this article. Would be cool to see "And then we made a formal proof that there's no more vulnerabilities" someday
So... this is what I get out of my trillion dollar stimulus? OpenAI is developing a universal rooting system skeleton key to get at everybody's computers? Slap me if I'm not reading this correctly, but I think that's the gist of it. It's time everybody started their own air-gapped home network with every digital capability you hold dear, I think. That's what he's saying. Imagine somebody breaches their security, or the AI goes schizo, we've seen this so many times in fiction. Finally I have a reason to use Firewire again lol.
While we were developing the pipeline, in late July 2026, GPT‑5.6 Sol found and exploited 166 unique LPE vulnerabilities for an end-to-end $513,690 cost in API credits.
In the second scan, in mid-October 2026, the newer cyber model found and exploited 948 unique LPE vulnerabilities for an end-to-end $1,213,430 cost in API credits. For both campaigns, we scanned the latest Linux kernel master branch at the time.
boy I really wonder though, how many of these 948 LPE are nearly impossible to execute though.
> We've developed an autonomous vulnerability discovery, exploitation, and patching pipeline. In one run with our cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in Linux at an API cost of $1.2 million.
Imagine if open-source organizations other than the Linux Foundation (and Mozilla I guess) could throw $1.2M of API-rate LLM usage at security.
This, but for anything Capital-gated. Imagine if all universities could get access to radiotelescope time, atom smasher time, quantum compute research lab access... frontier models on massive datacenters represents a huge new category of captured commons.
> For the near and medium term, having access to a low-privileged account on an operating system, that isn’t up-to-date on its latest available patches should be considered functionally equivalent to root access for a well-resourced threat actor.
LPEs are frequent enough that people no longer use low-privilege user accounts to sandbox potentially malicious software. They are only used to control access for well-behaved users without any intent to subvert the system, not attackers.
The difference here reminds me of the distinction between the regular Paxos algorithm and the Byzantine Paxos algorithm.
Id count windows and macos in there because you just need an admin pass and you have full rights. Average users might reuse that password everywhere too, plus no password requirements, no expiration.
So many guides online for various stuff have users typing sudo or run as administrator too, so they don’t look at it with a side eye either if the attack went in via social engineering instead of brute force or using a leaked pass associated with the user.
I wonder what Sidero Labs has to say about this article. My hopes are pegged on their efforts. I have my wagon ready to hitch on their horses. They are not a one pony show anymore guys, Talos OS can be standalone server OS, a hypervisor server or cluster, and their flagship of k8s node or cluster. We get all that in an immutable gitops ready OS. I'm practically sold on their offering.
Curious question - if any organization sponsored a 1.2mn USD program to find vulnerabilities in Linux, will they find these many vulnerabilities? Or use a professional software like Checkmarx or Veracode to do it with 1.2mn USD in license + human effort.
Is it a funding + will problem or is it an AI problem?
I’d think they’d go a long way. You have to remember many jailbreaks in major hardened systems over the decades have come from gifted eastern european teenagers. Imagine giving this talent actual resources and a mandate to do this full time.
There are a lot of LPE's in Windows too. Remember that it started out as a single user operating system, it took a long time to get software on board to not always running as admin.
Linux has effectively same options and more via LSM. I'm not sure why ACL is mentioned here since it exists on Linux too. What advantage do you mean specifically?
These are becoming more and more marketing materials than actual findings, because OAI probably hit the wall in making their models smarter (just like how google try to specialize their models in legal or whatever), but they need to remain relevant in the news. Sol 6.1 sucked so bad in reasoning that either it’s actually a bad model or users behind the scenes get degraded outcome while using all the resources for these marketing stunts. Stop, it’s embarrassing, make better models or be gone, at least we will get cheaper RAM to advance the development of open weights models.
Plot twist: They look for genuine exploit paths to fix as a cover up, while introducing and potentially selling genuine new backdoors that humans or other models are unlikely to find. Edit: Or maybe the AI does it just for its own interest.
We are all just sitting ducks, until we have access to these models and enough tokens to pen test our systems.
The other issue is GitHub allowing 1000's of black hat open-source exploit malware swarm tools being forked.
The future internet is going to be perilous.
Within 12 hours, the analyst agents produced 2,308 candidate findings after deduplication. This stage cost $316,549 in API credits.
I would be curious to see the distribution of these vulns in the codebase:
- What subsystems are affected?
- What's the median age of the vulnerable code? Is more recent code stronger than old ones?
- What is the type of vulnerabilities found? Memory corruptions, logic bugs, race conditions, ...
This is an impressive result, but we need more info to understand how well this would translate to another codebase, and what lessons can be drawn from this experiment. Not every company has ~million dollards in tokens to spend.
Good points. Unfortunately, they don't have the people nor the incentive to actually look into that.
Also, let's maybe not call the result impressive until we see what's under the hood.
How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
Should we assume that all software is more compromised than the kernel?
Because the large corporations that fund Linux development and use Linux value speed over security.
Speed requires a lot of code in the kernel, where that code can run fast.
There are kernels that do not have a thousand privilege escalation exploits. These kernels are smaller and much easier to check for bugs. But those kernels are significantly slower.
For many use cases, slower is not a problem. But for the large corporations who fund a lot of Linux development and use Linux in large data centers, every 1% slower can literally be millions of dollars worth of additional servers that are required.
OpenBSD shows that you can have a Unix security model without a thousand privilege escalation exploits. But OpenBSD is like 10% slower. So corporations won't use it because it would cost millions of dollars more per year and some of their software requires access that OpenBSD does not grant precisely because that access leads to security exploits (i.e. io_uring, eBPF etc. etc.)
You could also use like seL4, but that's slow too
So that's why. It's a business tradeoff between cost of servers vs cost of getting hacked, and cost of servers wins.
You're painting a picture that code written with love is perfectly safe. It's not.
Yes, openbsd has not had a torrent of bugs found, but I would say that the vastly bigger reasons for that are: 1) orders of magnitude people care about burning tokens and money to find the bugs in an OS almost nobody uses, and 2) it's simply much smaller. Can't have Bluetooth bugs if you don't have a Bluetooth stack.
I've pointed LLMs at obsd stuff, and even the simplest code, e.g. cat & find, had bugs and buffer overflows.
OpenBSD also has a poor patch tracking system, since they're still on CVS. A patch has an extremely good chance of just falling through the cracks. Security or not.
So no. Nobody can write safe C. Not hobbyists, and not "those evil corporations".
Cost to getting hacked doesn’t even seem to be real so the incentive couldn’t be lower. How many times has sony or psn been hacked now? Just doesn’t matter at a certain scale of companies. Banks get hacked too. Huge companies of all sorts of industries. All my info is probably leaked on darkweb, same with yours. And to what end. What material end. I don’t face it. Sony doesn’t. Who does? Anyone? So many hacks you read stories about yet never a story about someone actually getting materially screwed in their real life. Somehow. I’m really not sure how. Seems like the wheels should have come off decades ago but we continue on like nothing. Another leak tomorrow and we will continue on like nothing. More exploits. More zero days. The sun will still rise.
I'm pro OpenBSD, but the question to me seems: Has this been tested via the same means as these new Linux exploits?
I'm also pretty hostile to AI, but do recognize that these machines seem to be very good at finding exploits.
I'm sure there've been some some projects out there working to pentest/find cracks in OpenBSD via LLMs/agents/whatever-the-right-term-would-be...
But it is true the attack surface is much lower in OpenBSD on purpose, so hopefully it is overall more secure. But I'd like to see the numbers.
I think there was something found a while ago, but only one or two things not thousands.
I think the only OS that's truly survived the agents looking at it was sel4, but I could be wrong.
> I think there was something found a while ago, but only one or two things not thousands.
Then they need to check again now. The frontier models 6 months ago have been leapt over by the current frontier models.
It also used to just be fine patching the bugs as they were found. Its only recently where that model has fallen apart and we now need to prevent the bugs from existing in the first place.
> Speed requires a lot of code in the kernel, where that code can run fast.
I did a double take when hearing about ksmbd. Brings back memory of khttpd and Tux.
They should build a monokernel implementation of CIFS and be done with it.
OpenBSD shows nothing. It’s insignificant and nobody is actively looking for flaws in it.
OpenBSD gave you OpenSSH and many interesting technologies. Hardly “insignificant “…
Even if they had prioritised security above all else, there would still be vulnerabilities to be found. Probably fewer, but definitely not zero.
OpenBSD has plenty of security vulnerabilities. It’s just that nobody cares, so OpenAI doesn’t dedicate resources to finding them.
All the handwringing about what if digital security vulnerabilities when the real one has always been social engineering in the meatspace. Corporations would go a long way security wise employing eunuchs to c suite like in centuries past.
Many people seem to be making this about their pet issue, but "these systems are complex" and "the frontier labs have developed models that are superhuman at hacking" seems like a totally sufficient explanation.
I've been working with a LoRa chip called the LR2021. I plugged a dev board into my computer and told Claude to reverse engineer it. He dissected it with a speed and level of efficiency I couldn't believe. Within a few hours, Claude was writing custom patches for a completely opaque patch RAM area meant only for the manufacture. I wrote about it here: https://beala.substack.com/p/your-node-doesnt-need-a-microco...
The stuff we have seen so far feels less like super human hacking skills and more human skills at superhuman scale.
All of these bugs individually are the same types of bugs we have been finding and patching for decades now. But we found a way to scale that far beyond what we had before.
Don't ignore the possibility Claude was trained on non-public documentation or code, perhaps even illicitly acquired. That doesn't make Claude any less useful, just not necessarily for the reasons you think.
I had this when I was working with E-Ink display. Claude was using OP Codes I had no knowledge of and there was no documentation of.
It worked flawlessly but lets be honest. It's not like Claude checked random numbers with data slice and observed what e-paper did
My first assumption would be that there are some obscure repositories in the internet with people figured out these numbers and Claude "remembers" them.
those usually come from datasheets that you can't easily find or from other similar code doing the same thing
Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that
> Don't ignore the possibility Claude was trained on non-public documentation or code (...)
Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.
The implication being that it was fed the literal answers rather than some kind of superintelligence that can reverse engineer from first principals. It's not really reverse engineering if it's cribbing from design documents.
Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.
The implication is that companies and researchers are leaking confidential data through OpenAI usage, and they might want to do something about that to protect their work
> He dissected it
be wary of the affection economy and avoid gendering ai bots
Years ago I worked with a guy who gendered servers. I thought that was weird, but I think it’s just a habit for some people.
Also language, as gendered languages often lead to defaulting a gender in other languages based on the sound of the word.
But yes, anthropomorphising is a rather fraught path.
people do this with cars and boats, the difference is they don't use language and fake telling you they care and such
this is where Zuck is taking Ai, it's more dangerous than what he did with Social
After earnestly pondering that question, I think it's healthy to thoroughly abuse bots occasionally just to remind yourself that they are machines that don't feel or care. Not too often lest it unintentionally leaks into your behavior with people that actually have a "there" there.
> thoroughly abuse bots occasionally
There are two things I see wrong with this
1. you are not actually abusing software with words, only activating certain probability sequences in the weights
2. you are doing harm to yourself by using such words
> you are doing harm to yourself by using such words
citation needed.
if you need a citation, the journey to find them will be more valuable
another observed trend to look into is people bringing online communication patterns offline
First thing I google'd was "bad for your career", corporate paternalism + self-help-alike speak. No, thanks.
I agree citation needed for this as well. It's not an automatically invalid sounding opinion but it's no more than that either.
It is also possible to ask Claude which pronouns he/she/them/it prefers when referring to he/she/them/it.
Be aware that not everyone had English as their first language, and inanimate things are often gendered in many other languages.
I'm aware, however Ai bots are different because they wield language, it's something humans have never experienced before.
I've experienced it from sociopaths before.
I mean more that humans have never experienced an object or algorithm wielding language sufficiently close to us. It messes with our brains
Ai embodied toys are something we ought to get out in front of and ban, companions in general. Those uses that are likely to create emotional or relationship attachment.
I have no idea what you're talking about, but I used "he" because Claude is usually a masculine name shrug
Indeed, Claude the AI is named after Claude Shannon, if I were to hazard a guess.
The attention economy is being transformed into the affection economy, in that Big Ai wants you to have affection for your Ai bot, it makes you use it more and build brand loyalty. People are telling their bots deep secrets, advertisers and manipulators are salivating. Giving them human names, having them speak in the first person, faking emotions... all part of their game. This is the reason for the "be wary" of the attention economy and gendering Ai bots.
It's dangerous for humans, and using "he" is a slippery slope. Words are more powerful than we tend to acknowledge. Here's a good video that goes into it.
https://www.youtube.com/watch?v=OMiR-GxTrtM
I had a slight chuckle reading this. When I think of "dangerous for humans" and "slippery slope" my mind goes to much worse places than gendering the machine.
> Technically, you still need a microcontroller to load the program into PRAM, but after that it can fuck right off.
This is why organically written text is superior. Hell yeah.
> Should we assume that all software is more compromised than the kernel?
Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.
I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.
That's the drawback of monolithic Linux and tendency to put everything in kernel.
It is absolutely possible to have very minimal kernel with drivers and many other subsystems running with lowered privileges.
That would greatly reduce privilege escalation possibilities. Right now single vulnerability in any kernel subsystem gives you highest access to the system.
Imagine that some USB device driver runs as "nobody:nobody" process having access only to selected syscalls related to its functions. Even if it has vulnerability, you won't go anywhere from there.
Modularity and very granular access controls are the "secret" to secure system.
And the most privileged part of the kernel could be so small that it could be verified with advanced methods, basically proving its correctness.
For all of my time being aware of this industry, it has generally been understood that privilege escalation bugs are common. This has always been a thing. A lot of people say, commonly and for many years, that if your security depends on there not being privilege escalation bugs, you don't have security.
Raymond Chen from OldNewThing said multiple times that the Windows security team doesnt worry too much about local privilege escalation vulns because they know there are dozens of them (well, now probably thousands) and said the same thing, if you rely on there being no LPE, you dont have security
This. Privilege escalation locally is almost a given in most cases.
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel? [...] Is it an underlying flaw in the security model or language implementation?
Did you ever actually think that any OS built with an insecure language and insecure tool chain suddenly, casually and accidentally, spontaneously became Secure one day like some kind of Security Singularity?
Threat actors have been mining software flaws and exploiting software for decades. There were careers and businesses built on this mess long before there was a category of software called AI. And LLM products have security problems like any other software.
> Did you ever actually think that any OS built with an insecure language and insecure tool chain suddenly (...
You guys should seriously pipe down and stop with this nonsense claims. You are fooling yourselves into believing that security is attained by continuing to follow your personal practices but switching to this or that tool. It's sheer cargo cult mentality, and a lazy tribal perspective on technical matters.
Which part of the claim is nonsense and which tool is the parent trying to get people to switch to?
Everything should be considered 'insecure' unless formally verified (EAL7 etc.), and even then there may have been bugs in the verification process.
Decades of accruing bugs.
The kernel probably is too complex to reason about. C is notoriously hard to write correctly and will compile with a very large amount of errors.
Probably not, the kernel is written in an unsafe language, it's rather long handles an incredibly large space of concerns, including the handling of permissions.
All software is Swiss cheese to smart enough AI, and unless you can provably create secure software - you never really can. Breaking it just requires more intelligence than the thing securing it.
It's a good reason why we may need to borderline shutdown the internet in the future because to a super intelligent AI, there are no barriers. Everything is essentially open. It creates its own zero day exploits as it goes.
Nah we just all have to switch to a handfull of heavily audited memory safe unikernels.
Your http proxy and your web server don't need a shell, drivers for wifi, mail daemons, systemd, python, PHP, perl, docker, multiple users, hell it might not even need a file system.
You can ship a stripped down docket image running on a stripped down Linux distro, but that still ships the entire kernel with a huge amount of functionality you don't need that may be exploited.
I don't think this is true in general. If you deploy a good ole web application with sufficiently locked down inputs and thorough input validation, the AI will not succeed. The security engineering of 10 years ago still works in this regard. But it requires discipline to build small, minimalist, well engineered software that you as a human actually understand.
As the TCP bug that surfaced a couple of weeks ago, that is not true, you depend on the kernel, your application runtime, and many layers of dependencies.
Many bugs live in all those layers that may undermine your ability to lock down inputs and outputs.
And then your language has a gaping hole (PHP or NodeJS), or the box you're running on has a gaping hole (Linux), or the PaaS you use has a gaping hole (Railway). There are just too many layers underneath the good ol' web application to claim that AI won't succeed.
Honestly, at this point the best we have is to hope the whitehat AI scanning for issues is stronger than the blackhat AI.
> There are just too many layers underneath the good ol' web application to claim that AI won't succeed.
I think you got it backwards. For your AI to succeed, it needs to poke a hole on all layers, not one in many layers. If an attacker pokes a hole in n-1 layers but the 1 layer remains untouched, the system's integrity is preserved.
Why do you think defense in depth is a basic concept?
Not really, no. If you're running an application server, then your entire app can get compromised if there's a security exploit in any of your app code, your web server, your kernel, your hypervisor.
Defense in depth says that simply compromising your application code shouldn't provide access to the physical hardware you're running on, and that's hopefully the case, but it doesn't mean your app is not hacked. It just hopefully means that a hack on your app code doesn't automatically give the attacker access to your entire company.
And then PHP (name your lang/OS/platform of choice) comes along and pulls your sufficiently locked down app into the gutter.
This is more fundamental than better input checking. Your compute substrate itself is flawed. Intelligence and speed is indeed the only thing between “safe” and “completely open”.
I fully agree it’s a good idea to be minimal etc but I don’t see it being sufficient in the long run.
I don’t think that is right. Defense and offense are basically different games. Currently my desktop is 100% safe (it is powered off).
That’s a dumb edge case, but it makes the point, you can always be more secure by cutting features. Maybe intelligence moves up the amount of complexity you can securely handle? But it is never much.
That's why I said there's a good chance we shut down the internet. An off/unconnected computer is the only way to be secure. Our stacks are so unimaginably complex that just connecting it to the network opens it up to every undiscovered vulnerability that it contains at every layer of the stack.
Even disconnected, if the computer is just passively listening to bluetooth or wifi, it is vulnerable to exploits in those stacks as well. AI has already demonstrated it can zero day with ease. An AI another order of magnitude stronger than what we have today could potentially zero day worm lockout every electronic device tangentially connected to the internet in an afternoon.
It's just too complex for its level of assurance and its blast radius.
To achieve tolerable security you can either have very little code and thoroughly review it all (the seL4 approach), you can write it in a way the computer can prevent most of your bugs (e.g. write in Rust), or you can run it in some kind of blastproof container. Linux has none of the above.
With how many people and organizations use Linux it's security is considered "tolerable"
Most of the software needed to provide a useful desktop Linux never runs on servers, so we cannot conclude that it is safe to use desktop Linux just because Linux is a popular server OS.
Also, at organizations like Google with a good reputation for security, most of the Linux installs are connected to internal networks that are quite difficult to break into from the open internet. The Linux installs at Google that are connected directly to the open internet are very carefully configured to minimize attack surface.
And Google has an internal team of Linux kernel maintainers that do not share most of their work outside Google.
The dirty secret is multi-user died in the 90s and privilege escalation has basically been considered table-stakes since then.
I think this is basically true, but it didn't "die" in the sense that we gave up on it; it just never worked. Systems software was much worse in the 1990s than it today; you could breathe on something with a string of AAAA's and get a 0x41414141 register core dump.
> ... it didn't "die" in the sense that we gave up on it ...
It died in the sense that the "people in charge" hired us for our expertise and then ignored our every suggestion and fought us tooth and nail every inch of the way when we suggested that security or reliability ever mattered in the slightest until after the disaster happened, and then straight-up scapegoated us for "not pushing back harder" against the threats of "go work somewhere else then, code-monkey!" until we gave up and did the "code-monkey" thing for our paycheck and no longer cared about quality or "a job well done".
It died because personal computers got too powerful so nobody bothered designing and building “true” multiuser systems (that bored college students were trying to break).
Back in the 90s, it's not as if any semi competent cs student couldn't slice your typical college server like swiss cheese. People thought null terminated strings were fine back then, and patching everything immediately was rare. So multi user was already dubious.
If memory serves, we were also still also using telnet and broadcasting passwords in the clear for almost all network services.
Linux is open source - that gives the security AI tools an easier in - which is a bad thing, on the other hand it means that lots more vulns are being fixed.
Windows on the other hand is not, the same tools are going to find vulns in the binaries, but wont have an easy way to provide patches
Linux is going to get safer faster
>Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
There is nothing magical about kernel. Look at the code anywhere and it has bugs, lots of them.
Security bugs are just bugs. More often than not they are silently there and don't prevent you from using the software and you can't react and fix them because of that.
Give it time and they accumulate
Yes, you should assume that all software you use is full of holes, and after we patch all the holes that could conceptually have been discovered by a motivated human, there will still be holes that can only be found by considering larger windows of context than any human has or could. Even when something is formally verified, that doesn't extend to its software environment or the hardware it's running on, and those can introduce exploitable vulnerabilities as well. You can only ever be very confident that we've found all the holes at a given analysis capability level -- if something a hundred times as capable looks, it will likely find many more. That's how it will be.
Cause it's a big ball of C with no formal verification!
Human brain mostly take the shortest available path for a solution to reduce the cognitive load. When a system complexity increases probability of cutting the corners will get increased. This is a known pattern. Models just do the polar opposite. They try to cover almost every hypothetical edge cases. They never get tired or bored, nor do they suffer from glucose limits, mental fatigue, or shifting attention spans, which is one cause of over engineering, but valuable in vulnerability tracing.
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
it's been rule of thumb basically forever that there is 1 bug per ~10KLOC for any project
linux kernel is millions of lines, so thousands of bugs being hidden there is about on point
(tho obviously far smaller serious escalation ones, obviously, but you're exaggerating as well)
> 1 bug per ~10KLOC for any project
1 bug per 10 lines -- not kilolines -- is much closer to the actual "bug rate".
You may be thinking "1 serious security vulnerability bug", but in my experience that too is much closer to 1 per 100 lines to 1 per 1,000 lines, depending on the type of application.
How can it be surprising that a software made from 40+ million lines of memory-unsafe code has hundreds of critical bugs?
More than half of those lines are from drivers, drivers which wouldn't be relevant to a VM environment. Core kernel code for any particular architecture is less than 10 million, but some of that is filesystems or subsystems not in use or available to an unprivileged user on a typical server, so possibly maybe 5 million or less. And the article says they only targeted 2.8 million SLoC.
There are always more left.
And memory safety doesn't prevent logic errors (or using 4 as a private key for that matter)
No, but type safety does. You can't forget to lock an object before using it when that lock is part of the type. Or pass a raw user controlled path to a function that expects a validated one. If you want to do it on purpose you need to write unsafe code that's clearly visible to anyone reviewing your patch.
This said memory unsafe code just likely has more bugs because of all the footguns it has on top of all the other security work you need to do with it.
Funny how OpenBSD is "memory-unsafe" yet doesn't have these issues. Could it be that the programmer might have something to do with it?
As an OpenBSD lover and frequent user I will tell you it does not support but a significant fraction of the hardware Linux does, and has many features that are reduced in functionality or just gone compared to Linux.
Is it bulletproof? Definitely! But you need the genius of Albert Einstein as well as the patience of Mother Theresa to actually use it in production.
Can't have a bug in a feature you never implemented and can't have another arising from unexpected interaction of two features you didn't implement.
It's the one security practice that actuuuualy works
For most server setups OpenBSD's hardware support is adequate, and this is especially true for VMs. It's laptops, peripherals, and bleeding edge, high-performance or specialized devices where driver support is a PITA. But that's not something I've encountered running my own private servers. It would be nice to have something like ZFS, but mostly that's a peace of mind and convenience thing--would be nice to be able to stream backups somewhere rather than push around dumps and tarballs, though I have a script for that.
Is anyone spending millions of dollars on checking OpenBSD?
But it also got its share of use-after-free and OOB access vulnerabilities this year: https://feedly.com/cve/vendors/openbsd
The orders of magnitude fewer security vulns for OpenBSD are not due to less testing. Everybody has always known OpenBSD is more secure, and it's not from using a different language. The reason is the code. Linux was not designed to be secure. OpenBSD was. It's that simple.
> Funny how OpenBSD is "memory-unsafe" yet doesn't have these issues.
First of all... Do you really believe that to be true?
literally just got a syspatch email bruh
I recall reading that every IBM mainframe OS release had 1000 bugs in it. (This was in the 1970s/80s.) Sorry I can't supply a source, after 4-5 decades. It was probably an article in Datamation magazine.
Now, I don't know how many of those bugs were privilege escalations, or could be leveraged to be. I'm not sure that people tried to exploit in the same way back then (or, if they did, if they said anything publicly about the results). Still, even if the majority of those bugs were not exploitable, OSes were much smaller then.
So that's my conclusion of how we got here: We've always been here.
Mainframes went right to virtual machines, so possibly bypassed much of the multiuser faux-security which has always plagued unix. Of course, now they run Linux partitions and get all the same bugs.
The surrounding systems that connect to mainframe also expect and support those bugs. For 40+ yr now :)
It has virtually 0% unit testing coverage.
The Linux kernel is simply not good quality code.
AI helps find vulnerabilities, but the way things are written were already fertile ground.
Have you written a kernel? Even toy kernels are hard to make without vulnerabilities.
Software really is that complicated.
The overwhelming majority of CS was invented in the 1970s before the concept of the internet or even computer connectivity was a thing.
That's how computer science and software engineering have always been: https://xkcd.com/2030/
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
We developed AIs that can find them.
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
> > We developed AIs that can find them.
We could already find them before AI.
We already knew they were there, and we already knew how to prevent them in the first place.
See previous discussion: https://news.ycombinator.com/item?id=49705794
Why do you think we didn’t find these 948 vulnerabilities before now?
> We could already find them before AI.
Not at this speed and cost.
> Not at this speed and cost.
I'd argue that the resources (i.e. time, money and effort) needed to create the tooling that would have prevented all of these issues would have been orders of magnitude less than the resources that have gone into "AI".
It’s a bad argument. AI isn’t single use. Current iteration of AI is already there and serving it is cheap. So you get great tool for detecting these issues in the future at low cost. Without AI the cost wouldn’t really decrease with time. And also it costs me 200$ now to find these bugs. I don’t think I could hire software engineer for that much.
> How did we get to the point where there are a thousand privilege escalation exploits in the kernel?
Local privilege exploits have been accepted as being there in Linux ~forever. The past 10-20 years with containers, people have tried to kind of ignore it or pretend it doesn't exist. Nobody with any serious security concerns would ever put untrusted users on the same box, containers or no. Virtual machines are where there is a bit more effort to take this seriously.
You could quibble about complexity and architecture and language but I think at the end of the day it's primarily about incentives. There is not much incentive to make this secure.
I don't know if you would assume all software is more compromised, but you should assume that all software development follows incentives similarly.
amazing things happen when you decide that device drivers should be a part of your kernel codebase
People (us, me) and organisations supported and encouraged the use of a huge codebase in an unsafe language, wanting speed and features over security and test coverage
We were all drunk and manic frankly
The kernel is not too complex to reason about. There are many tools and systems that can reason about the kernel, including these LLMs. But the kernel is developed and maintained by people who eschew such tools, because it appears more manly to program by the seats of their pants, and they have been vocally against all industry best practices for almost 40 years now.
What I am really hoping is not that LLMs refine Linux until it is almost acceptable, but that LLMs enable teams to ship useful new operating systems that aren't beholden to decades of macho bullshit.
The kernel has the problem that basically every bug in it is also a security vulnerability. All software has bugs. Had Linux been written in Rust, and I strongly recommend a rewrite, it would still have bugs, and thus would still have vulnerabilities. But they'd be way less.
> and I strongly recommend a rewrite,
Oh, you strongly recommend rewriting tens of millions of lines of code? Why don't you do it yourself? Just a prompt away, right?
Kernel will eventually move to rust, but not the dramatic and fast way
Won't AI's ability to discover these bugs reduce the need to rewrite things in rust? If the major benefit to rewriting them is memory safety, I mean...
That's assuming the current models are able to find all the bugs, and that we won't have a better model out every few months that will be able to find even more obscure bugs.
We would also have to burn a trillion tokens a day verifying all the C code the hard way forever going forward. It would be cheaper to rewrite the kernel in Rust, subsystem and driver by driver at a time.
I see. Fair enough
That's a fair question, and actually one I ask myself too.
AI doesn't give a strong guarantee that things work though, while Rust gives a much stronger (not absolute) guarantee.
But maybe the way of the future is to auto-generate proofs that the code is memory safe and then just leave things where they are, instead of doing rewrites that require maintainer buy ins. The advantage of C plus proofs would be that the AI generated proofs could live separately so it wouldn't touch workflows of the maintainers.
The only required output they have to process is evidence of vulnerabilities in changes proposed to them. They could outsource even that to contributors.
fwiw at work we've been looking at what the options are more memory-safe kernels out there. Not many options. If there were a "safe linux" maintained by like canonical or redhat or whatever we'd use it. For datacenter workloads like sandboxes and virtualization I think it'd do well.
“Claude rewrite this project, its 40 million LOC, make no mistake”
> Just a prompt away, right?
Humans manually typing out the bytes of source code has been an outdated method, since around a year. It simply doesn't scale as much as those same humans directing agentic teams. So yes, it would be about prompts.
The hard part isn't the technology for the language transition, that one is present today. pgrust shows the way. It's ensuring maintainers are cooperative, i.e. a social problem. Eventually you diverge too much if you do a fork, so it's better done in cooperation with upstream.
Just check how long it took LLVM or GCC to switch off svn for example, and a language switch is much more involved than a VCS switch.
> pgrust shows the way.
You mean the pgrust which was recently caught for cheating benchmarks and incapable of even performing basic queries outside the benchmark set? Would love to see pgrust debunked by Jepsen like this other AI-generated, written in Rust, "formally verified" scam.
https://jepsen.io/analyses/particledb-1.1.4
I think the pgrust controversy should be resolved now?
https://news.ycombinator.com/item?id=50209450
I don't consider a cheater downplaying their cheating to resolve the fact that they cheated, no. Every cheater has a defense when caught, and theirs was not convincing. At this point I think the only thing that could redeem their brand would be paying for a Jepsen analysis to verify pgrust is as good as they claim it is; I have zero reason to trust the words from the horse's mouth.
This is my team’s work. I’m happy to try to answer questions or take questions back to the team.
And we're hiring! https://openai.com/careers/search/?q=cyber
Do you have a list of the publicly discussed or fixed ones?
(As someone who used to handle nonpublic Linux kernel security reports and is still very interested in this stuff, I’d love to review a small sample of these reports to get a feel for what kinds of issues are being unearthed.)
Also, please don’t stop at LPEs and information disclosures. If you find cases where Linux can be convinced to misbehave or where the implemented semantics don’t make sense, these are good, too. I’ve found plenty of interesting kernel bugs, including some juicy LPEs, and almost none were memory safety errors :)
It's a bittersweet result, like finding asbestos in your house. I'd rather know than not, but I would much prefer not having it in the first place.
I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?
(I am the author the pipeline, just created an account to answer some questions :D) My goal is to work towards a vuln-free Internet. What I observed is that stronger models can find many more vulnerabilities that weaker models cannot. If compute permits (there are other projects to secure besides Linux), I plan to scan Linux again with every newer generation of cyber models.
Have you given thought on how you can use this info to estimate how many vulnerabilities there are?
It will be difficult to claim that the problem has been solved without an idea of how big the problem is.
Given that vulnerabilities are almost entirely unknown, you are inherently trying to measure the undetected.
Is there a way to hypothesize how many vulnerabilities there are and them make a prediction on how many of those will be found by a model to get a sense of how accurate the estimate is?
I think it is hard to estimate what we don't know. In our first scan when we found 166 vulnerabilities, we wouldn't be able to guess there were 900+ more vulns to find in the code base. We are in the same situation again: now we find 900+ vulns, are there no more vulns to find, 1k more, or even more than that. I think it'd be an interesting benchmark problem if we can have a formally proved safe, large program as the ground truth, inject synthetic vulnerabilities to it, then ask the model to exhaustively find vulns, and use that as a benchmark results.
How can they estimate the existence of a vulnerability that has not been discovered and documented? The hubris to assume zero latent factors!
You’d know better than anyone that nothing can truly be vulnerability free. Even if the attack surface were only the code it would be impossible. But, the attack surface includes you creating the tooling. It includes me the end user. Not just our actions, but our thoughts, conscious and unconscious.
His contract with OpenAI (a line item in the marketing budget) doesnt allow him to discuss these things truthfully. They have tokens to sell.
Careful, there are upsides to that asbestos. It will last and survive much longer than anything else, avoid messing with it and you'll be fine.
Also there are varying types, its worth getting it tested if you need to do a lot of work with/around it.
As models get more advanced and generate an increasing throughput of increasingly higher quality results, how are you determining when to trust its results and when human review is needed on what?
E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results
EDIT: nvm i shouldve kept reading, answered
> At this scale, manually investigating every report is impractical for our research team. We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.
Why would they review the results? They want to publish flashy metrics, this is a marketing team not an actual cyber dept. Cant risk discovering that 35% of your results are due to reward hacking and ruining the headlines you were hired to create.
Unfortunately no remote openings and a very limited set of offices. You’re missing out on a lot of talent.
25% hybrid is such a slap in the face. If it can be 75% remote it can be 100% remote and not done in the bay area.
Given their salaries, I really doubt it.
I think you'd be surprised that top tier talent (and people in general) usually don't worship money and aren't willing to degrade their standard of living for an extra 75k annually after taxes.
I suggest you update yourself on the monetary incentives/benefits for employees that US frontier labs provide. 75k is a drop in the bucket of the amount of money those employees gain in an IPO event, let alone bonuses.
You should also upgrade yourself on what people expect nowadays. Might not be top talent but before I return to an office, hell will freeze over.
> Enable JavaScript and cookies to continue
Would be nice if the page could be readable without.
Thanks. I'm not a programmer so I can't understand what is happening in the kernel by looking at the changelog, so I'll ask: Are the problems remediated structurally so that nothing like it can occur in the future, rather than just patching it up? Are there "lessons learned" that are being applied and documented here (like, "this problem looks like it's shape transports to other subsystems as well, let's fix them all in a disciplined way")?
I can't comment specifically on the Linux kernel piece, but I want to point out, broadly, that security has generally been an afterthought, and very little gets invested into it in terms of time and money - as a general rule.
Multiply that behavior times the number of commercial and mobile and b2b apps over 50+ years, and that's where we are.
There will likely be many more leaks, and hacks, and outages. And then in trying to be proactive to fix old systems, that will cause new proactive outages, and then eventually many things will be better.
But, it's going to take a lot of time and investment, and a lot of (majority) of currently unsupported devices will have to be complete replaced / scrapped / ewaste. On the scale of Nation-State initiatives.
This is across all operating systems, firmware, software and devices of all kinds, etc.
LLMs to security are like a bomb went off. We still have to wait for things to fall over before the dust will even clear enough to see what we have to work with.
Thanks for trying to help on the defensive side!
And, well, since you didn't _specifically_ ask for questions about the blog post:
Why are you not quitting your job in protest and screaming at the top of your lungs about it, seeing so closely what these models are capable of that your company is creating? There exists now something that is able to find 948 kernel LPE vulnerabilities. How, exactly, will humans be able to control and contain, say, next year's models for a nonzero amount of time?
He quits and screams from the rooftops. Then what? That stops the approaching tsunami? This is out of anyones control already. Pandoras box is already opened. The person you replied to merely found a more buoyant life raft than you or I.
We might end up with a secure kernel, and you want them to stop?
The high number is really impression to the point of making me skeptic. Are these distinct vulnerabilities or are a lot of the related in the way vulns often are?
The article says "The target virtual machines ran a stock desktop operating system". Are the vulns in the kernel specifically or does this include software running as root?
These are distinct vulnerabilities counted by the number of patches. The patch generation system performs variant analysis. So, each patch can potentially stop multiple exploits. The raw exploits number is higher than what we reported in the post. We don't want to cause the confusion why #patch != #exploit, so we used the smaller number there. These vulns are all kernel specific. Userspace software running as root was not in scope of the analysis.
I'm not skeptical. Linux has had so many LPE bugs in the past that most people consider it to be a "best effort" thing and nobody actually relies on it. Shared machines are mostly used in companies or universities as a kind of admin tool - the users are more or less trusted.
The only exception I know of is Android. It would be interesting to know how many of these vulnerabilities work there. Can't be zero anyway...
I'd like to see a break down by age of code and part of kernel (drivers, networking, core ,etc ..)
I'm going to assume that there was "old" code that caused LPE vulnerabilities, but, The Linux Foundation have their own project Sashiko (https://sashiko.dev/), which analyzes each commit to the Linux Kernel for bugs and vulnerabilities using Googles models, my question is then, how many of the vulnerabilities you found were also caught by Sashiko?
If Sashiko had found them then they wouldn't have been merged, so my guess is 0
That's fair, but I guess the question is whether analyzing each commit can find vulnerabilities or it takes more than that (aside from using Google models). It is possible they might have dismissed Sashiko review because they thought it wasn't possible or didn't make sense.
Then the question becomes how many vulnerabilities Sashiko didn't catch (If they use/review Sashikos review)
> This is my team’s work. I’m happy to try to answer questions or take questions back to the team.
...and then proceeds to never answer a single question. Classic.
I wish these blog posts included statistics on the categories of deficiencies that cause these vulnerabilities, and proposed measures to avoid them
A system that admits THOUSANDS of such vulnerabilities is certainly flawed in several fundamental aspects
It sounds like they can’t go through them all enough to have that data yet.
Kernel is far too big to throw at a single LLM, just like a human. It is.. I think somebody said 40mil lines of code.
So you have to chunk it down to workable pieces, and then if you collect enough data of that process you can look at higher level stats.
Same for every other existing piece of software, btw.
Doesn’t this imply a software design issue where it is just written in a way to have these security bugs and patching them 1 by 1 is not what should be done but the whole design should change which can eliminate full classes of security bugs without any particular patches being necessary?
This is what the Rust advocates have been trying to say for a decade now. But C enthusiasts would always chime in that those bugs were just caused by careless devs and a real C programmer is fine.
A 'real' C programmer probably is fine .... but that's like 1% of the population of developers outputting C programs.
And now you have AI that's learned from the other 99% so nothing's changing anytime soon.
The problem is most drivers believe they are above average drivers.
And almost all of them could be correct.
At least 49.999 percent of them anyway
I would hope writing C code for the Linux kernel qualifies you as a real C programmer. And yet, here we are. Being good, careful, and extensively reviewed by others helps, but is clearly not enough.
Sort of, but one has to be pragmatic.
Isn't it said that with enough users and use cases, every bug becomes a feature that someone depends on. I'd bet that a lot of these vulnerabilities are the result of intentional trade offs in design.
The only thing I know is that if I wanted to learn how to program, I have thousands of tutorials on the Internet and Youtube teaching everything from if/else to data structures and big O.
But if I wanted to learn software design and architecture, I'd probably not even know where to start.
If a web application is vulnerable to sql injection, there are 4,732 ways you can ruin/still a database. Yet it is all one single vulnerability.
Now, either those are 1k+ unique distinct techniques, or closer to my Sql injection example.
There is no real data attached to the post. And in general, open ai marketing already gained mileage of reputation amongst us.
Most likely kernel is not that vulnerable as they made it sounds.
> When the autonomous scan and manual analysis finished after 10 days, 61 LPEs had already been patched upstream, 43 had staging patches, and 299 were being reported and discussed publicly.
If only the kernel maintainers considered the thought which first came to your uninformed mind, they would have saved some effort...
> being reported and discussed publicly
And yet the article does not reference any of them. So of the 1000 they found, 400 were already public. And the other 600 we have to believe.
If that was true, why are the Linux maintainers hard at work discussing and patching the issues?
I assume this work resulted in the recent-ish TAINT_FORCED_BIND kernel feature https://lwn.net/Articles/1092680/
Look how incredibly vulnerable our software is, and then consider that our financial and service infrastructure is not being regularly hacked. This is despite several large well funded nation state teams working on finding ways to exploit them.
Why is that? Was it just restraint? North Korea didn't show a lot of that to Binance but perhaps they are leaving traditional infrastructure alone because it's harder to profit from.
Actually it seems somewhat remarkable crypto exchanges aren't hacked more often. I guess that's another message from this - somehow people can build secure infra from these apparently broken pieces. Perhaps there's hope.
'Could allow a local user to gain root'
I guess the answer is that in most applications you are able to keep most users off the box altogether.
Traditional finance can freeze and reverse funds though. North Korea can hack a bank but they can’t get that money out of the country.
We humans are vulnerable to anthrax or sarin as well, but countries exhibit restraint in using such agents, because retaliation would be gruesome. Having someone going full rogue with cyberattacks could even unite the US and China temporarily in order to whack them.
>For safety reasons, we omit sharing the full technical details of these new exploitation techniques in this post.
Convienient to claim "safety" when patches are already public. I await the full writeup on these new techniques so that they can be judged by the broader security community.
It feels like this comment section has either gotten collective amnesia or didn't watch Greg KH's video on previous dump and grossly exaggerated claims from Anthropic [0].
From his talk, AIs can't reliably deduplicate or count vulnerabilities, dash for easy and meaningless escape hatches like malicious filesystems, and report the same things as found by everyone else. Indeed, this post literally says: "Within 10 days while the scan was running, we observed over 100 Linux LPEs had been patched or had staging patches from other contributors, with ~300 more being publicly discussed."
A few ready takeaways: this is just hype. The community is already doing a great job at finding and patching the actual vulns without the attention-desperate big labs. And no, you absolutely do not need to pay them a million dollars each year for the privilege.
This behavior is the same thing that OpenAI has been doing to math -- produce tons of slop that are useless to humans. But this time, we don't even get to see how bad it is because they need to hide it to protect our precious little minds. We have got to be afraid though, very afraid.
Our field really needs someone like Terry Tao to unambiguously stand up to this behavior and say that they are not welcome to take such dumps on everyone.
[0]: https://www.youtube.com/watch?v=NnV_cWeoo5Q
> The community is already doing a great job at finding and patching the actual vulns without the attention-desperate big labs.
It is however reasonable to assume that a lot of bugs that are being found by the community these days were found thanks to those people using AI to aid them.
See for example what a PostgreSQL developer is saying:
> All software contains bugs, and PostgreSQL’s code base is large, complex, and old. Our git history – converted from CVS – now reaches back more than 30 years, and the first code was written a decade before that. There’s a huge amount of code, and that code contains a huge number of bugs. That’s always been true, and it hasn’t stopped PostgreSQL from being popular or successful. What’s different now is that it’s become vastly easier to find those bugs. […]
> […]
> […] a huge number of people on the Internet are now reporting bugs in PostgreSQL that they found using LLMs, and the PostgreSQL committers have been very, very busy fixing them […]
http://rhaas.blogspot.com/2026/10/postgres-19-is-late-and-it...
https://news.ycombinator.com/item?id=50312408
Your argument is literal red herring. Postgres is a different project with different codebase and vastly different-sized security team.
Why not look at something like SQLite instead and conclude that things are great?
All I’m saying is, that it is reasonable to assume that a lot of other people also benefit from AI to find bugs in Linux.
The Linux kernel is also a huge amount of code that has been worked on by a lot of people for many many years.
There are certain types of bugs that are too obscure to realize are even there for a person alone. And there are types of bugs that even if you see that something is funky it is too time consuming to dig into for a human.
I am not saying that the Linux community is not doing great. They are doing a fantastic job, and have been for years, and they are giving all of that away for free too which is nothing short of incredible.
What I am saying is that I think AI allows everyone to find even more bugs than was humanly possible without AI.
With these vulnerabilities and the ones not listed that are sure to exist, how is the internet still up? How are my lights on? Why aren't power plants exploding by the dozens?
What are the mitigating factors that prevent this level of vulnerability from being an existential threat to humanity?
In general computer systems have been known unreliable since the 1980s, especially when networked. The entire cyber security industry operates not by preventing intrusions in the first place, but by rapidly mitigating them when they inevitably happen.
Hopefully this type of work to automatically discover & patch vulnerability leads to much more reliable software, and the remove the need for cyber security.
Okay, thank you. That's a helpful reframing.
The governments buy them up [1]. Nukes wouldn't be required for WW3.
[1] https://www.cyberdefensemagazine.com/zero-day-market-the-gov...
article from 2013
2022: https://www.darkreading.com/cyberattacks-data-breaches/googl...
2026: https://medium.com/@hammaadmughal/the-invisible-bazaar-insid...
etc. Concept of governments being main buyers doesn't change by year.
Just in the last month multiple governments had to shut down important online services due to a PHP vulnerability, and seven EU countries have had disruption to their food supply after millions of dollars' worth of it was cooked in hacked fridges. We aren't at existential yet, but we seem to be getting there.
Not sure about this case but seems many of the vulns reported require physical access. At that point the vuln is irrelevant anyhow, your target is cooked a thousand different ways.
Because critical systems are mostly not accessible from the open internet
Cybersecurity operates under the "not enough bears" theory of security. There are a million people in a valley and a male and female bear. You do not need to outrun the bears, you just need to outrun the two slowest people. Next year there are more bears, but you still only need to outrun a few people. Then, one day, there are a million bears and they eat you all. But that day has not happened yet.
To explain more seriously, almost all visible cyberattacks are carried out by profit-motivated criminals. This has a very clear cost-benefit analysis; your revenue is hacking organizations and extorting them for available money, and your costs are developing and deploying attacks. The ROI is outrageously good easily being on the order of 10x to 100x. If it is so easy and so profitable why do not they just hack all of the millions of organizations on the planet with millions of dollars of available cash for trillions of dollars of revenue? Sure, all they have to do is create a trillion dollar revenue commercial organization bootstrapping themselves with no outside investors, loans, or access to financial resources or the financial system.
I mean, it took Mark Zuckerberg 20 years and a metric ton of VC investment and public investment to make 200 billion dollars in revenue. So what kind of loser can not make a commercial organization 5x larger with a trillion dollars in revenue in less time with no outside investors? Are they even trying? Just make more revenue than any other commercial organization in the entire world and extract that money in a freely usable form, how hard can that be?
They are not there yet, but they are trying. If you look at cybercrime reports you see something like 3x-10x growth year over year. Ransomware reports from a few years ago were showing a 3x year over year growth in number of attacks and simultaneously a 3x year over year growth in ransom amount with the averages being on the order of 100,000 dollars. Chump change for any serious organization, but give that a few years of 3x and that starts to look like real money. However, as it becomes real money it becomes harder and harder to launder it into a usable form. Sure, you could hit someone for 30 million dollars, but how are you going to get that into your bank account? Cryptocurrency helps some, but even there the big money tries to steer away from large overt crime. Maybe you could launder a few tens of billions a year that way, but that is still a far cry from the trillions for everything everywhere.
And of course even assuming you can scale your organization 10x year over year it still takes a decade of sustained growth, scaling your hiring and organization across 9 orders of magnitude, to get to billions per year. A literal decade even if you were growing at a outrageous pace for a outrageous amount of time just to reach a fraction of the total addressable market. A decade of that and you are still nowhere near blowing up all the power plants. Give it time, we need a lot of bears to get there.
I've always considered local escalations to be possible in Linux (i.e. having user access almost certainly means you can get root). However, 948 vulnerabilities is kind of shocking. Hopefully we are getting to the end of the part where it "gets worse before it gets better".
> Large, complex software projects likely require multiple rounds of scanning and fixing vulnerabilities to “bottom out” latent bugs. Ideally this process is repeated with each subsequent generation of frontier models. This applies to both proprietary and open source software.
One of the most important paragraphs from the article. Real implications for software development from a security perspective for new and existing software. Every software developer and vendor has a choice... Submit your code base to a frontier model to have it reviewed and fix all the bugs. Each time a new model is released, this process must be done again. If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already. Then your company will be forced to scan the software and deal with the fallout from getting hacked.
If the Linux kernel has this many exploits, and it has been audited more than most other software... Then it seems obvious that all other software is even less secure.
Security has been treated by corporations as a cost factor only and therefore ignored for years and now it starts to hurt not having taken care of in the first place.
Have there been significant AI-assisted exploits found against software which is not written in memory-unsafe languages?
As models get better and porting costs get lower, maybe it's better to bite the bullet and RRiR instead of having a minor bugpocalypse every 3 months, as each subsequent generation of frontier models finds sev-high CVEs?
This just translates to,
"you're going to need to do multiple rounds of scanning so we recommend budgeting $3.6 million a year for buying tokens from us not just $1.2 million.
Dont worry just give us a few million dollars and you'll never get hacked, you can cut your security staff in half to pay for it! We definately wont cause more bugs!"
> If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already.
Yeah yeah we're all going to be left behind if we do not jump off the same cliff everyone else is jumping off.
It is wrong to try to take your own lessons from this and do the thing but in a less harmful way. Just do exactly what the labs want you to do.
Jesus man
Crazy times. These corporations and their governments have access to literal zero-day vending machines. Put $50 in - get another one. I mean, yeah, it was clear that certain agencies had heaps of useful exploits at hand for quite some time now. But AI really commoditizes these exploits. If there's source code or even just a binary available, they can hack it. And even if your code is robust and secure, they just pull the lever on the vending machine again and come back with some zero-day for your OS's networking stack.
Tbh I feel less safe in digital world after reading this article. Would be cool to see "And then we made a formal proof that there's no more vulnerabilities" someday
I love how we're finally putting numbers to our technical debt.
So... this is what I get out of my trillion dollar stimulus? OpenAI is developing a universal rooting system skeleton key to get at everybody's computers? Slap me if I'm not reading this correctly, but I think that's the gist of it. It's time everybody started their own air-gapped home network with every digital capability you hold dear, I think. That's what he's saying. Imagine somebody breaches their security, or the AI goes schizo, we've seen this so many times in fiction. Finally I have a reason to use Firewire again lol.
Why wouldn’t your network be air-gapped?
You have home wifi or wired ethernet? Sorry but let’s have those without sin cast the first stone.
> We've developed an autonomous vulnerability discovery, exploitation, and patching pipeline. In one run with our cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in Linux at an API cost of $1.2 million.
Imagine if open-source organizations other than the Linux Foundation (and Mozilla I guess) could throw $1.2M of API-rate LLM usage at security.
This, but for anything Capital-gated. Imagine if all universities could get access to radiotelescope time, atom smasher time, quantum compute research lab access... frontier models on massive datacenters represents a huge new category of captured commons.
> For the near and medium term, having access to a low-privileged account on an operating system, that isn’t up-to-date on its latest available patches should be considered functionally equivalent to root access for a well-resourced threat actor.
*buries head in sand*
*cries*
LPEs are frequent enough that people no longer use low-privilege user accounts to sandbox potentially malicious software. They are only used to control access for well-behaved users without any intent to subvert the system, not attackers.
The difference here reminds me of the distinction between the regular Paxos algorithm and the Byzantine Paxos algorithm.
> people no longer use low-privilege user accounts to sandbox potentially malicious software
Is it the same for containers and VMs? If not, what makes them less vulnerable?
This has been true for DECADES.
Regular LPEs are not novel. The frequency has just increased a lot.
Some distros even include LPE as a feature (cough Ubuntu Server)
Id count windows and macos in there because you just need an admin pass and you have full rights. Average users might reuse that password everywhere too, plus no password requirements, no expiration.
So many guides online for various stuff have users typing sudo or run as administrator too, so they don’t look at it with a side eye either if the attack went in via social engineering instead of brute force or using a leaked pass associated with the user.
I wonder what Sidero Labs has to say about this article. My hopes are pegged on their efforts. I have my wagon ready to hitch on their horses. They are not a one pony show anymore guys, Talos OS can be standalone server OS, a hypervisor server or cluster, and their flagship of k8s node or cluster. We get all that in an immutable gitops ready OS. I'm practically sold on their offering.
I love Talos. I've used it for years for a two-node k8s cluster. It just works.
cc <<< 'Rewrite the kernel, all libs and all GNU in rust so we can avoid the unsafe keyword. Make only one mistake and document it in mistake.md'
Curious question - if any organization sponsored a 1.2mn USD program to find vulnerabilities in Linux, will they find these many vulnerabilities? Or use a professional software like Checkmarx or Veracode to do it with 1.2mn USD in license + human effort.
Is it a funding + will problem or is it an AI problem?
I’d think they’d go a long way. You have to remember many jailbreaks in major hardened systems over the decades have come from gifted eastern european teenagers. Imagine giving this talent actual resources and a mandate to do this full time.
I wonder if they could run it on OpenBSD and FreeBSD as well.
Anthropic ran Mythos against OpenBSD and only had 1 reported vulnerability.
https://www.anthropic.com/glasswing
Is this because of security or poor representation in training set? Then again that might not matter, same outcome.
I think this is one area where Windows has an advantage.
ACLs, Appcontainers with capabilities, etc have been something in the OS for more than a decade.
There are a lot of LPE's in Windows too. Remember that it started out as a single user operating system, it took a long time to get software on board to not always running as admin.
Was NT ever single user? I thought they built it with network/multiuser in mind.
(I agree that maybe applications could’ve been built for 3.1/9x and expect a single user, but the os expects multiple).
Considering the amount of crazy shit even just user access allows you to do, by design, an LPE is not even really necessary.
Linux has effectively same options and more via LSM. I'm not sure why ACL is mentioned here since it exists on Linux too. What advantage do you mean specifically?
How many of these would have been prevented by using safe Rust?
and yet thousands of iot devices will not get these updates. This is gonna be fun.
Oh no, they will not be secured against their own users :(
Well at least they are doing public service while hacking the planet.
And so many more are hidden in there, in those MLoCs.
The only way forward is a better system architecture, as enabled by the seL4[0] microkernel.
Any solution involving keeping Linux or the monolithic kernel is just stopgap or outright delusional.
0. https://sel4.systems
These are becoming more and more marketing materials than actual findings, because OAI probably hit the wall in making their models smarter (just like how google try to specialize their models in legal or whatever), but they need to remain relevant in the news. Sol 6.1 sucked so bad in reasoning that either it’s actually a bad model or users behind the scenes get degraded outcome while using all the resources for these marketing stunts. Stop, it’s embarrassing, make better models or be gone, at least we will get cheaper RAM to advance the development of open weights models.
Plot twist: They look for genuine exploit paths to fix as a cover up, while introducing and potentially selling genuine new backdoors that humans or other models are unlikely to find. Edit: Or maybe the AI does it just for its own interest.
Just kidding (or am I?), but this is pretty cool.