Hey HN! I've read HN for years but this is my first post.
db-branch uses ZFS copy-on-write to create, destroy, and reset MySQL and PostgreSQL clones in seconds, even for terabyte-scale databases. It's self-hosted and doesn't depend on any cloud vendor.
Background
Per-PR preview environments are common now, but the database is usually still a shared one. PlanetScale, Neon, and Aurora's fast clone solved this with Git-like branching, even for huge datasets: cut a branch in an instant, throw it away when you're done.
The catch is that these are managed services tied to specific cloud vendors. Cost can be a barrier, fully air-gapped (on-prem) environments have constraints, and sometimes you just want the same workflow on your laptop. db-branch is meant for those cases.
There's one more use case I care about: letting an AI agent such as Claude Code freely experiment with SQL against a database that looks like real data. Pointing an agent at production or staging directly is dangerous. With a branch, even if the agent runs something destructive (a DROP, a massive UPDATE), you just throw the branch away or roll back to a snapshot. Being a disposable DB sandbox for AI agents is one of the goals of this project.
Architecture
Storage and processes: Using ZFS snapshot/clone, db-branch instantly creates a dataset from a snapshot of a cleanly shut down MySQL/PostgreSQL instance, then starts a process for it using a systemd template unit, switching ports per branch. Each branch runs as its own independent DB process.
Dynamic routing: ProxySQL sits in front of the branch processes, so clients don't have to change port numbers. Branches are routed by connecting user or by a query comment (/db_branch=.../). This works for PostgreSQL as well as MySQL, using ProxySQL 3.x's PostgreSQL protocol support.
Clustering: db-branch clusters are built with Raft and ZFS send/recv, and routing inside the cluster is also done by ProxySQL.
Limitations and trade-offs
- How the base snapshot is taken: For consistency, snapshots are taken from a cleanly shut down DB. This doesn't mean stopping your production DB. The idea is to stop a read replica and snapshot that, or to use an instance restored from a regular dump/backup as the base.
- Memory is the real cost: CoW keeps disk usage low, but every branch is a separate DB process, so per-process memory grows with the number of branches. So we recommend keeping the DB-side caches (InnoDB buffer pool, PostgreSQL shared_buffers) small and giving most of the memory to the ZFS ARC. Clones share unmodified blocks, so data cached in the ARC is shared across branches, instead of every branch holding its own copy of the same data in its own DB cache. Even so, if you run dozens of branches at the same time, you need to plan the host's RAM accordingly.
- Heavy writes still cost disk: CoW only shares blocks that haven't been modified, so a branch that writes a lot consumes that much space.
Features
- Create and destroy MySQL and PostgreSQL branches
- Take snapshots of a branch as save points, and restore from them
- Upgrade verification for MySQL / PostgreSQL (upgrade version feature)
- SQL routing via ProxySQL
- Cluster setup
- [Commercial module] Recording of executed queries and restore to any point (Lite Plan / History Module)
To keep the project sustainable, the core branching engine and the clustering feature are released as fully open source, free of charge.
More advanced features, namely the per-query recording and restore (History Module), plus enterprise user management, SSO, and auditing, are planned as modules of a commercial plan. Only some of these are available so far, and you can use them for a one-month trial period.
(Whispering: if you're only playing around in the local Multipass environment, you can just tear down the VM and recreate it every month and keep using it basically forever... shhh )
Is it possible to merge changes from a clone/branch back to the "official copy"?
So nice to have a way to get cheap db clones, I just set up this at work recently to do the same thing https://postgres.ai/docs/database-lab
Postgres 18 has file_copy_method = clone for this, is that used underneath or did you implement your own own?
This is interesting. Does it hook into Postgres and MySQL’s MVCC?
Hey HN! I've read HN for years but this is my first post.
db-branch uses ZFS copy-on-write to create, destroy, and reset MySQL and PostgreSQL clones in seconds, even for terabyte-scale databases. It's self-hosted and doesn't depend on any cloud vendor.
Background
Per-PR preview environments are common now, but the database is usually still a shared one. PlanetScale, Neon, and Aurora's fast clone solved this with Git-like branching, even for huge datasets: cut a branch in an instant, throw it away when you're done.
The catch is that these are managed services tied to specific cloud vendors. Cost can be a barrier, fully air-gapped (on-prem) environments have constraints, and sometimes you just want the same workflow on your laptop. db-branch is meant for those cases.
There's one more use case I care about: letting an AI agent such as Claude Code freely experiment with SQL against a database that looks like real data. Pointing an agent at production or staging directly is dangerous. With a branch, even if the agent runs something destructive (a DROP, a massive UPDATE), you just throw the branch away or roll back to a snapshot. Being a disposable DB sandbox for AI agents is one of the goals of this project.
Architecture
Storage and processes: Using ZFS snapshot/clone, db-branch instantly creates a dataset from a snapshot of a cleanly shut down MySQL/PostgreSQL instance, then starts a process for it using a systemd template unit, switching ports per branch. Each branch runs as its own independent DB process.
Dynamic routing: ProxySQL sits in front of the branch processes, so clients don't have to change port numbers. Branches are routed by connecting user or by a query comment (/db_branch=.../). This works for PostgreSQL as well as MySQL, using ProxySQL 3.x's PostgreSQL protocol support.
Clustering: db-branch clusters are built with Raft and ZFS send/recv, and routing inside the cluster is also done by ProxySQL.
Limitations and trade-offs
- How the base snapshot is taken: For consistency, snapshots are taken from a cleanly shut down DB. This doesn't mean stopping your production DB. The idea is to stop a read replica and snapshot that, or to use an instance restored from a regular dump/backup as the base. - Memory is the real cost: CoW keeps disk usage low, but every branch is a separate DB process, so per-process memory grows with the number of branches. So we recommend keeping the DB-side caches (InnoDB buffer pool, PostgreSQL shared_buffers) small and giving most of the memory to the ZFS ARC. Clones share unmodified blocks, so data cached in the ARC is shared across branches, instead of every branch holding its own copy of the same data in its own DB cache. Even so, if you run dozens of branches at the same time, you need to plan the host's RAM accordingly. - Heavy writes still cost disk: CoW only shares blocks that haven't been modified, so a branch that writes a lot consumes that much space.
Features
- Create and destroy MySQL and PostgreSQL branches - Take snapshots of a branch as save points, and restore from them - Upgrade verification for MySQL / PostgreSQL (upgrade version feature) - SQL routing via ProxySQL - Cluster setup - [Commercial module] Recording of executed queries and restore to any point (Lite Plan / History Module)
Try it
You can try it on a local Mac or Windows machine using Multipass: https://github.com/db-branch/db-branch#evaluate-locally-maco...
Monetization and license
To keep the project sustainable, the core branching engine and the clustering feature are released as fully open source, free of charge. More advanced features, namely the per-query recording and restore (History Module), plus enterprise user management, SSO, and auditing, are planned as modules of a commercial plan. Only some of these are available so far, and you can use them for a one-month trial period.
(Whispering: if you're only playing around in the local Multipass environment, you can just tear down the VM and recreate it every month and keep using it basically forever... shhh )