Hoping gpu's find their way to adding support too. It boggles my mind having io-uring which feels so kernel centric not kernel offload juggling and moving and processing buffers that aren't in main memory.
This is great work by the Redox team. This might be over the heads of many HNers who are undergoing skill atrophy in real time and addicted to agents, given that 4 hours and less than 10 comments on a highly technical blog post about Rust and asynchronous I/O in Redox.
This project is vehemently against AI usage of any kind, so I can trust that the contents of this post has been checked and written by someone who has read the code and knows what they are doing.
> The Linux io_uring interface allows the user to submit one or more I/O operations to the kernel, which are then processed in an asynchronous fashion without blocking the userspace process. Redox, being a microkernel, handles I/O in userspace scheme processes rather than in the kernel.
io_uring is incredibly fast for async I/O but is also a massive security disaster in the Linux kernel. As long as Redox is able to show a secure interface for async I/O the better. Likely part of their long term goals for the OS.
> While it has substantially improved over the years, IPC is still quite slow on Redox, and this is important to optimize as ring buffers can only offset this overhead to a certain extent. This has been a focus of our work to improve system performance.
I quite like this targeted and thoughtful approach for a wholistic improvement starting with ring buffers in redox. They need to compare the async executor benchmark results with Linux and continue from there for improvements.
> The optimization made here is that the kernel zeroes the vector registers if the caller and callee entered the kernel through syscalls, rather than saving/restoring (using vzeroall, ldmxcsr etc. as required for correctness).
Interesting approach for complete correctness over performance in this area and this tells me that they are not taking going to risk on overlying on rust's "guarantees" to help them prevent races or data reuse after free bugs in the Redox kernel.
One of the most technically sound and researched posts I've read, but unfortunately it has been buried under low effort slop posts on HN.
(I don't have a horse in this race, I just like being well informed and had reconciled myself with io_uring being secure as a consequence of the Redox article.)
The Redox article is underselling it somewhat. Part of the problem is Linux's historical syscall restriction systems (seccomp) did not (does not?) apply to io_uring, so io_uring is (was?) all or nothing on async versions of syscalls it implements. Also, io_uring is rapidly adding API surface compared to other parts of the kernel. With these factors combined, it has had repeated security problems since 2019 and has lead to, e.g., Android disabling io_uring entirely for applications.
> in the past year, there has been a clear trend: 60% of the submissions exploited the io_uring component of the Linux kernel (we paid out around 1 million USD for io_uring alone). Furthermore, io_uring vulnerabilities were used in all the submissions which bypassed our mitigations.
I would think that Android applications also don't really benefit from uring anyway since human-facing applications tend to not actually do all that much. Like for networking, they're not trying to process millions of connections/flows at 10s of Gbit/s. They'd be processing 1-20 flows at like a few Mbit/s. The types of processes that might benefit tend to also be relatively controlled (e.g. a database or application server).
More like io_urine, it leaks everywhere and if pisses you off in both ways. Even more by hooking it up against DMABUF. We didn't learn anything from the 90's...
In the abstract there's nothing necessarily inherently insecure about io_uring. In practice on Linux it's a security disaster because it hooks into and interacts with subsystems in ways never anticipated when they were originally written, exposing and creating all kinds of bugs. Claims that the wave of bugs is now in the past were... premature. They haven't really stopped, though part of that is because io_uring hasn't stopped growing and becoming more complex itself.
>io_uring is incredibly fast for async I/O but is also a massive security disaster in the Linux kernel
Torvalds bound it up with... DMABUF. Guess what will happen.
On Linux' future memory subsystem imported from a branch, the Asterinas' one in Rust it's miles ahead and far more secure. Like Windows 98 compared to NT.
I get some hesitancy but do folks really think this Fear Uncertainty and Doubt, that being conservative is going to pay off in the long run?
The wins on the book for io-uring are incredible. Massive. It looks like a miracle. It's completely changed what a box can do. Orders of magnitude more. With vastly less cpu.
Has it had problems? Sure yes, it did. Is it still perfect? Probably not.
For cloud providers running arbitrary user code in containers yeah maybe the fear of the unknown makes sense. That the risk of some undisclosed vulnerability is going to get exploited against you and your customers. But most people aren't running arbitrary code on their systems. And most people are not going to be the target when there is a high profile vulnerability. We have, from what I can tell, no evidence any of these vulnerabilities from the past have ever had consequences. They're not good! But people really are batting so hard to declare an L, wind themselves and others up so hard to justify outrage. I find it ridiculous. Preposterous.
The wins are already a landslide. Issues get tackled. The rate and severity of issues has tapered to a trickle. The design fundamentally makes sense as a consumer, a programmer using it. The hardware obviously loves it. Io-uring is going to keep winning, keep cleaning house.
And what is going to happen to the dissenters standing outside year after year waving these protest banners? I can't help but think that their fate is to look ever more ridiculous. That over time the io-uring protest looks ever more an out of touch farce, completely disconnected from the silent majority's very happy better place, that 30x's performance and that makes sense to us all and that has never personally hurt a one of us in any way.
KSMBd has the same fate (albeit radically less upside), the same endless fearmongering attack hounds who will probably never ever stop yelling about it either. For what? It's so unclear. Because sometimes new things do take time to settle some. Because it's fun to yell at things, to steal the attention of the room, to feel better than the crowd. History never dies, grudges and people's bad impressions of things never get updated, negative experiences and feelings have such potency to sear they ways into our souls, to mark us forever. We are thrown off our poles and forever tilted against things, lost entirely to possibility of reevaluation, reconsideration, reflection on what the new contemporary is. https://en.wikipedia.org/wiki/Planck%27s_principle
We can and should be excited for no longer needing to cross the syscall barrier as incessantly. We can and should be excited for the radical efficiency. We can and should be excited for mechanistic sympathy, for these circular buffers that work very well on the hardware. Generally we should, for our soul's sake, work to behold a possibility of success. To allow room even in darkness for maybe perhaps possibly some light and grace and success some day. I am definitely interested in hearing tales of what else could be, other than io-uring. I know and use iouring and it makes sense to me. It works great, from what I can see. But if there's other ideas out there for how else we get to winning, I want to know! To engage in imagining the yes perhaps. Unvetted but here's Asterinas. https://github.com/asterinas/asterinas
That is indeed the chemical reaction that creates rust, and they intentionally named their OS such for that very reason, as they are writing this OS in the Rust programming language. But I am not sure why you’d just comment that and nothing more. For a moment I thought you might be a bot instructed to “write a relevant comment for this title”. But your other comments in other threads seem human to me.
Alternative take: I did look up that exact word while reading the article. It’s been a minute since I saw it in a chemistry context. For me, it was nice to see someone else with the same question and that they took the time to post a concise definition.
Interesting read keeping in mind this is two degrees of separation from NVMe queues and their atomic doorbell head/tail registers.
Maybe not coincidental that io-uring added DMA-PRIME support, & started with NVMe (in upcoming Linux 7.4). https://www.phoronix.com/news/DMA-BUF-IO-uring-Linux-7.4
Hoping gpu's find their way to adding support too. It boggles my mind having io-uring which feels so kernel centric not kernel offload juggling and moving and processing buffers that aren't in main memory.
This is great work by the Redox team. This might be over the heads of many HNers who are undergoing skill atrophy in real time and addicted to agents, given that 4 hours and less than 10 comments on a highly technical blog post about Rust and asynchronous I/O in Redox.
This project is vehemently against AI usage of any kind, so I can trust that the contents of this post has been checked and written by someone who has read the code and knows what they are doing.
> The Linux io_uring interface allows the user to submit one or more I/O operations to the kernel, which are then processed in an asynchronous fashion without blocking the userspace process. Redox, being a microkernel, handles I/O in userspace scheme processes rather than in the kernel.
io_uring is incredibly fast for async I/O but is also a massive security disaster in the Linux kernel. As long as Redox is able to show a secure interface for async I/O the better. Likely part of their long term goals for the OS.
> While it has substantially improved over the years, IPC is still quite slow on Redox, and this is important to optimize as ring buffers can only offset this overhead to a certain extent. This has been a focus of our work to improve system performance.
I quite like this targeted and thoughtful approach for a wholistic improvement starting with ring buffers in redox. They need to compare the async executor benchmark results with Linux and continue from there for improvements.
> The optimization made here is that the kernel zeroes the vector registers if the caller and callee entered the kernel through syscalls, rather than saving/restoring (using vzeroall, ldmxcsr etc. as required for correctness).
Interesting approach for complete correctness over performance in this area and this tells me that they are not taking going to risk on overlying on rust's "guarantees" to help them prevent races or data reuse after free bugs in the Redox kernel.
One of the most technically sound and researched posts I've read, but unfortunately it has been buried under low effort slop posts on HN.
Could you elaborate on io_uring being a security disaster? The Redox article has the following to say:
> In its early days, io_uring did have a few security issues and memory leaks, but these have since been addressed
The article also links to a GitHub discussion where someone explains how io_uring is actually all right (see https://github.com/axboe/liburing/discussions/1047#discussio... ).
(I don't have a horse in this race, I just like being well informed and had reconciled myself with io_uring being secure as a consequence of the Redox article.)
The Redox article is underselling it somewhat. Part of the problem is Linux's historical syscall restriction systems (seccomp) did not (does not?) apply to io_uring, so io_uring is (was?) all or nothing on async versions of syscalls it implements. Also, io_uring is rapidly adding API surface compared to other parts of the kernel. With these factors combined, it has had repeated security problems since 2019 and has lead to, e.g., Android disabling io_uring entirely for applications.
https://security.googleblog.com/2023/06/learnings-from-kctf-...
> in the past year, there has been a clear trend: 60% of the submissions exploited the io_uring component of the Linux kernel (we paid out around 1 million USD for io_uring alone). Furthermore, io_uring vulnerabilities were used in all the submissions which bypassed our mitigations.
I would think that Android applications also don't really benefit from uring anyway since human-facing applications tend to not actually do all that much. Like for networking, they're not trying to process millions of connections/flows at 10s of Gbit/s. They'd be processing 1-20 flows at like a few Mbit/s. The types of processes that might benefit tend to also be relatively controlled (e.g. a database or application server).
7.0 added task level filtering using regular bpf filters.
https://lwn.net/Articles/1054225
Great!
More like io_urine, it leaks everywhere and if pisses you off in both ways. Even more by hooking it up against DMABUF. We didn't learn anything from the 90's...
In the abstract there's nothing necessarily inherently insecure about io_uring. In practice on Linux it's a security disaster because it hooks into and interacts with subsystems in ways never anticipated when they were originally written, exposing and creating all kinds of bugs. Claims that the wave of bugs is now in the past were... premature. They haven't really stopped, though part of that is because io_uring hasn't stopped growing and becoming more complex itself.
>io_uring is incredibly fast for async I/O but is also a massive security disaster in the Linux kernel
Torvalds bound it up with... DMABUF. Guess what will happen.
On Linux' future memory subsystem imported from a branch, the Asterinas' one in Rust it's miles ahead and far more secure. Like Windows 98 compared to NT.
> Guess what will happen.
Throughput for storage networking and GPUs will continue gaining yet more orders of magnitude and radically increased efficiency?
I look at this and cheer. https://www.phoronix.com/news/DMA-BUF-IO-uring-Linux-7.4
I get some hesitancy but do folks really think this Fear Uncertainty and Doubt, that being conservative is going to pay off in the long run?
The wins on the book for io-uring are incredible. Massive. It looks like a miracle. It's completely changed what a box can do. Orders of magnitude more. With vastly less cpu.
Has it had problems? Sure yes, it did. Is it still perfect? Probably not.
For cloud providers running arbitrary user code in containers yeah maybe the fear of the unknown makes sense. That the risk of some undisclosed vulnerability is going to get exploited against you and your customers. But most people aren't running arbitrary code on their systems. And most people are not going to be the target when there is a high profile vulnerability. We have, from what I can tell, no evidence any of these vulnerabilities from the past have ever had consequences. They're not good! But people really are batting so hard to declare an L, wind themselves and others up so hard to justify outrage. I find it ridiculous. Preposterous.
The wins are already a landslide. Issues get tackled. The rate and severity of issues has tapered to a trickle. The design fundamentally makes sense as a consumer, a programmer using it. The hardware obviously loves it. Io-uring is going to keep winning, keep cleaning house.
And what is going to happen to the dissenters standing outside year after year waving these protest banners? I can't help but think that their fate is to look ever more ridiculous. That over time the io-uring protest looks ever more an out of touch farce, completely disconnected from the silent majority's very happy better place, that 30x's performance and that makes sense to us all and that has never personally hurt a one of us in any way.
KSMBd has the same fate (albeit radically less upside), the same endless fearmongering attack hounds who will probably never ever stop yelling about it either. For what? It's so unclear. Because sometimes new things do take time to settle some. Because it's fun to yell at things, to steal the attention of the room, to feel better than the crowd. History never dies, grudges and people's bad impressions of things never get updated, negative experiences and feelings have such potency to sear they ways into our souls, to mark us forever. We are thrown off our poles and forever tilted against things, lost entirely to possibility of reevaluation, reconsideration, reflection on what the new contemporary is. https://en.wikipedia.org/wiki/Planck%27s_principle
We can and should be excited for no longer needing to cross the syscall barrier as incessantly. We can and should be excited for the radical efficiency. We can and should be excited for mechanistic sympathy, for these circular buffers that work very well on the hardware. Generally we should, for our soul's sake, work to behold a possibility of success. To allow room even in darkness for maybe perhaps possibly some light and grace and success some day. I am definitely interested in hearing tales of what else could be, other than io-uring. I know and use iouring and it makes sense to me. It works great, from what I can see. But if there's other ideas out there for how else we get to winning, I want to know! To engage in imagining the yes perhaps. Unvetted but here's Asterinas. https://github.com/asterinas/asterinas
It is a shame to mix ad hominem with technical commentary
[dead]
This is cool stuff !!
Would love to see what other things could benefit from this beyond storage. Maybe like BT or similar wireless.
Redox: Reduction and Oxidation
That is indeed the chemical reaction that creates rust, and they intentionally named their OS such for that very reason, as they are writing this OS in the Rust programming language. But I am not sure why you’d just comment that and nothing more. For a moment I thought you might be a bot instructed to “write a relevant comment for this title”. But your other comments in other threads seem human to me.
Alternative take: I did look up that exact word while reading the article. It’s been a minute since I saw it in a chemistry context. For me, it was nice to see someone else with the same question and that they took the time to post a concise definition.
Yes, a lot of people who don't deal with chem stuff a lot of the time may not know what Redox means or may it have forgotten.