I built something similar recently on top of Incus via Pulumi. I also wanted to avoid libvirt's mountain of XML, and Incus is essentially a lightweight and friendlier interface to QEMU, with some nice QoL features. I'm quite happy with it, though the manifest format is not as fleshed out as what you have here.
What's nice about Pulumi is that I can use the Incus Terraform provider from a number of languages saner than HCL. I went with Python, since I also wanted to expose a unified approach to provisioning, which Pyinfra handles well. This allows me to keep the manifest simple, while having the flexibility to expose any underlying resource. I think it's a solid approach, though I still want to polish it a bit before making a public release.
I took a slightly different approach in that I don't want to use YAML as the authoritative source. Many projects abuse it, and end up creating a DSL on top of it with all sorts of hacks to achieve the flexibility of a programming language. Pulumi and Pyinfra already provide user-friendly primitives and idempotent(ish) APIs that work much better than YAML. I simply want to expose some (opinionated) building blocks to make them easy to use, and allow users to customize them and add their own as needed. E.g. I definitely don't want to write any shell scripts inside YAML. :)
BTW, Pulumi already supports YAML[1], which can be used with any provider. But to me it's too verbose and generic, and of course, it lacks the provisioning primitives.
Your path makes a ton of sense too! You get typed languages, state management, and the Incus team's work on the QEMU layer. The tradeoff I wasn't willing to make is the daemon + state store: Incus wants to own the VM lifecycle the way libvirtd does, and once you have that you're back to "two sources of truth" if you ever shell out. Holos is deliberately stateless on the host; Everything lives under one directory per project, rm -rf is a valid uninstall (though it will abandon the VMs if running). Different answer to the same frustration. Would genuinely like to see your thing when it's public.
This is exactly what I dislike about incus. But what I do like about incus is how I can easily spin up and configure VMs directly using the CLI, without preparing a config first (I hate yaml).
So would be nice if holos could replicate that docker/incus CLI functionality, like say "holos run -d --name db ubuntu:noble bash -c blah".
Docker does have both docker run and docker compose... Holos is compose-only right now. A holos run on top of the same VM machinery isn't a huge lift; mostly it's deciding what the sensible defaults are (image, cloud-init user, port, volume scope); Noted. it's not in 0.1 but I hear the ask.
> Incus wants to own the VM lifecycle the way libvirtd does, and once you have that you're back to "two sources of truth" if you ever shell out.
That's true. But I didn't want to reinvent what Incus or any hypervisor abstraction does. I simply wanted to add some sugar on top that allows me to easily declare infra using small abstractions, and to tie in the provisioning aspect along the way. I still use Incus directly, and can benefit from their work, as you say. State is also managed by Pulumi, so really, there are 3 places for it to exist. There are some challenges with this, of course, but I think the tradeoff is worth it.
Good luck with your project, I'll be keeping an eye on it. I'll probably make a Show HN post when I release mine. Cheers!
it reminds me of https://github.com/dockur/windows with its compose-style YAML over QEMU/KVM. The difference i'm seeing is scope: dockur ships curated OS images (Windows/macOS), while holos looks more like a generic single-host VM runner. Is that a fair read? also curious any plans to support running unattended processes for OS installs?
True and decent read. Holos is a generic runner underneath, and I'm not trying to compete on the curated-OS-image front; the assumption is you bring a cloud image and cloud-init does the rest.
Unattended installs for Linux work today through cloud-init (that's most of what the YAML drives). Windows/macOS would need a different path.. autounattend.xml injection for Windows, custom payload for macOS; I haven't built that. If there's interest, a provisioner: type block that picks the right unattend mechanism based on OS isn't crazy, but it's not on the near-term list.
My biggest issue with QEMU is trying to keep up with seemingly arbitrary changes to the command-line options format over time, and keeping up scripts that can launch an image across different versions for testing.
Originally if you wanted e.g. a SoundBlaster16 device you could use -device sb16. Then it changed to -soundhw sb16. And now it's -audio driver=none,model=sb16; this has been happening with several different classes of options over the years and I haven't found any good documentation of all the differences in one place. If anyone knows, I'd appreciate it.
I like it! is there a way to use both virt-manager and virsh with this or do we have to pick one. I'd be keen to migrate my VMs to this compose based system if it was a simple import job.
Thanks! Honest answer: nope. Holos talks to QEMU directly and skips libvirt entirely — that's deliberate (no XML, no daemon, state lives in a single directory per project), but I like the idea of being able to import vm's. I will try to find some time to add that feature.
Fair — the YAML shape is similar. The differences that matter to me: GPU passthrough works without fighting the provider plugin ecosystem, healthchecks gate depends_on instead of you sleeping in a provisioner, and there's no Ruby, no vagrant reload, no box format.
Honestly, if you just sold it as vagrant but with all the weird stuff fixed and actually made to play nice with the rest of the stack that I'd actually want to use, that would be a pretty compelling package to me
I did use an LLM to workshop my comms; I tend to go on and on so I put my my comments through a LLM to clean up my replies. the opinions and the project are mine, but the polish isn't. I'll knock it off for the rest of this thread.
Very cool, thanks for sharing.
I built something similar recently on top of Incus via Pulumi. I also wanted to avoid libvirt's mountain of XML, and Incus is essentially a lightweight and friendlier interface to QEMU, with some nice QoL features. I'm quite happy with it, though the manifest format is not as fleshed out as what you have here.
What's nice about Pulumi is that I can use the Incus Terraform provider from a number of languages saner than HCL. I went with Python, since I also wanted to expose a unified approach to provisioning, which Pyinfra handles well. This allows me to keep the manifest simple, while having the flexibility to expose any underlying resource. I think it's a solid approach, though I still want to polish it a bit before making a public release.
check out https://github.com/lnussbaum/incant
Ah, that's neat as well.
I took a slightly different approach in that I don't want to use YAML as the authoritative source. Many projects abuse it, and end up creating a DSL on top of it with all sorts of hacks to achieve the flexibility of a programming language. Pulumi and Pyinfra already provide user-friendly primitives and idempotent(ish) APIs that work much better than YAML. I simply want to expose some (opinionated) building blocks to make them easy to use, and allow users to customize them and add their own as needed. E.g. I definitely don't want to write any shell scripts inside YAML. :)
BTW, Pulumi already supports YAML[1], which can be used with any provider. But to me it's too verbose and generic, and of course, it lacks the provisioning primitives.
[1]: https://www.pulumi.com/docs/iac/languages-sdks/yaml/
Your path makes a ton of sense too! You get typed languages, state management, and the Incus team's work on the QEMU layer. The tradeoff I wasn't willing to make is the daemon + state store: Incus wants to own the VM lifecycle the way libvirtd does, and once you have that you're back to "two sources of truth" if you ever shell out. Holos is deliberately stateless on the host; Everything lives under one directory per project, rm -rf is a valid uninstall (though it will abandon the VMs if running). Different answer to the same frustration. Would genuinely like to see your thing when it's public.
This is exactly what I dislike about incus. But what I do like about incus is how I can easily spin up and configure VMs directly using the CLI, without preparing a config first (I hate yaml).
So would be nice if holos could replicate that docker/incus CLI functionality, like say "holos run -d --name db ubuntu:noble bash -c blah".
Docker does have both docker run and docker compose... Holos is compose-only right now. A holos run on top of the same VM machinery isn't a huge lift; mostly it's deciding what the sensible defaults are (image, cloud-init user, port, volume scope); Noted. it's not in 0.1 but I hear the ask.
> Incus wants to own the VM lifecycle the way libvirtd does, and once you have that you're back to "two sources of truth" if you ever shell out.
That's true. But I didn't want to reinvent what Incus or any hypervisor abstraction does. I simply wanted to add some sugar on top that allows me to easily declare infra using small abstractions, and to tie in the provisioning aspect along the way. I still use Incus directly, and can benefit from their work, as you say. State is also managed by Pulumi, so really, there are 3 places for it to exist. There are some challenges with this, of course, but I think the tradeoff is worth it.
Good luck with your project, I'll be keeping an eye on it. I'll probably make a Show HN post when I release mine. Cheers!
it reminds me of https://github.com/dockur/windows with its compose-style YAML over QEMU/KVM. The difference i'm seeing is scope: dockur ships curated OS images (Windows/macOS), while holos looks more like a generic single-host VM runner. Is that a fair read? also curious any plans to support running unattended processes for OS installs?
True and decent read. Holos is a generic runner underneath, and I'm not trying to compete on the curated-OS-image front; the assumption is you bring a cloud image and cloud-init does the rest. Unattended installs for Linux work today through cloud-init (that's most of what the YAML drives). Windows/macOS would need a different path.. autounattend.xml injection for Windows, custom payload for macOS; I haven't built that. If there's interest, a provisioner: type block that picks the right unattend mechanism based on OS isn't crazy, but it's not on the near-term list.
My biggest issue with QEMU is trying to keep up with seemingly arbitrary changes to the command-line options format over time, and keeping up scripts that can launch an image across different versions for testing.
Originally if you wanted e.g. a SoundBlaster16 device you could use -device sb16. Then it changed to -soundhw sb16. And now it's -audio driver=none,model=sb16; this has been happening with several different classes of options over the years and I haven't found any good documentation of all the differences in one place. If anyone knows, I'd appreciate it.
I like it! is there a way to use both virt-manager and virsh with this or do we have to pick one. I'd be keen to migrate my VMs to this compose based system if it was a simple import job.
Thanks! Honest answer: nope. Holos talks to QEMU directly and skips libvirt entirely — that's deliberate (no XML, no daemon, state lives in a single directory per project), but I like the idea of being able to import vm's. I will try to find some time to add that feature.
Interesting. Wonder could it be modified to work with proxmox?
With Proxmox it could be as well Docker compose for LXC.
sure has been awhile since i thought about vagrant
Fair — the YAML shape is similar. The differences that matter to me: GPU passthrough works without fighting the provider plugin ecosystem, healthchecks gate depends_on instead of you sleeping in a provisioner, and there's no Ruby, no vagrant reload, no box format.
Honestly, if you just sold it as vagrant but with all the weird stuff fixed and actually made to play nice with the rest of the stack that I'd actually want to use, that would be a pretty compelling package to me
Stop generating comments, please.
I did use an LLM to workshop my comms; I tend to go on and on so I put my my comments through a LLM to clean up my replies. the opinions and the project are mine, but the polish isn't. I'll knock it off for the rest of this thread.
You should not use an LLM on Hacker News, period.
> Don't post generated comments or AI-edited comments. HN is for conversation between humans.
https://news.ycombinator.com/newsguidelines.html#generated
What is it using to create the vm without libvirt?
See https://github.com/zeroecco/holos/blob/5f5e60f753c5e99c421db...
Qemu command-line calls
another player in this space built on incus: https://github.com/lnussbaum/incant