Even with a large database servers (10s of CPU cores, 100s of gigabytes of RAM) bottlenecks arise pretty quickly.
Err, do they? For what percent of real world use cases?
The database can scale to handle more traffic by adding replicas. An extreme example of this is OpenAI's use of 50 replicas on a single Primary.
So an extreme example is OpenAI needing 50 replicas, but we're doing five blades ... err, we're doing 768 servers because the need arose "pretty quickly"?
When we needed to store a petabyte of data (one million gigabytes), we'd need many more shards
For who? The United States government? How many end-users are running 1PB Postgres database on DBaaS?
> A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding
Did you max out the capacity of the best server you can buy?
Such a database can serve millions of customers (the numbers given).
You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer inspect the essential state of your system is the day your company better be included in NASDAQ and ready to pay a few hundred engineers 300k salaries.
I found the article very helpful from a technical perspective, and didn't focus on the number too much, as it could easily be swapped and the decision-making process for when to shard is kinda out of scope.
But I hadn't considered this, so thanks for pushing back. Good to keep in mind their incentives.
I will say, since their product is a proxy whose interface is a single SQL connection, you should in theory be able to do dev queries through that black box, much the same as application queries? What is so scary here that it would require a hundred engineers?
> you should in theory be able to do dev queries through that black box
Because it’s a leaky abstraction which is trying to make guarantees over network connections which are extremely difficult to make within the same kernel.
A few questions I would start with:
- is the system even ACID compliant?
In my reading of this article, no.
- is my sql feature set limited? Will it enforce all constraints? Or are their cross-shard limitations?
For example the article doesn’t discuss transactions or how they would roll back, or how they guarantee a consistent view of data.
Next:
- how does it respond to error?
- how does it respond to load?
This is a complex system and complexity breeds bugs. But now you don’t have the tools or procedures to investigate those bugs because you can’t poke the system at desk with tools. You can’t run experiments; you can’t even see all the data.
The more I read the more I'm struggling to understand the benefit of a router like this that sits on top of a monolithic SQL, vs a truly distributed DB like cockroach.
Like you I'd love to learn more about the internals of their actual SQL engine, which is just barely touched on in the article. The idea of ripping out the layer of a SQL interpreter that does just enough to route it to a real server tickles my brain in the same way as when I learned how node.js ripped the js interpreter out of a browser.
It depends on who you are paying for it, but generally a distributed system is harder to reason about, harder to fix, has weirder edge cases, and much more easily get into situations where it requires even MORE expertise to fix than just having a big honking server.
When you start calculating things that are not just the server, the single server looks cheaper and cheaper. How do you get a consistent backup? How do you do DR? How do you tune queries when it could go to this node or that node? Now writes are going to be significantly slower if you need multi-node commit because no matter what you are racing the speed of light on the network.
Lol that isn't a database server. Come on, if it has hard drives rather than (good) SSD in 2026, it's not a server you should be using for databases unless it's for something with next to no load, and you are also running your application server on the same machine.
What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences?
What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions?
On cross-shard reads: how do you do sorting? And cross-shard joins?
I’d love to be proven wrong, but I suspect the 768 servers look like 1 only on the very surface, and you’ll get wildly different characteristics from cross-shard and single-shard queries.
I personally would prefer if they _didn’t_ look like 1 if they can’t behave like 1.
Of course 768 servers NEVER behave as 1. This is physically impossible.
Global services using relational dbs typically severely restrict queries that run against the cluster. So no joins, no intervals, no grouping, etc.
Transactional queries are usually limited to something like "get a single record, preferably from cache". For many typical web services this can go VERY FAR. Only a handful of global services needs more than a few dozen database servers and a caching cluster. In fact, i have seen major businesses running off a pair of very big postgres instances.
Analytical stuff is extracted into dedicated storages optimized for throughput, like Snowflake or Redshift or BigQuery.
This seems like the important distinction: making the infrastructure look like one database to the application is different from making it behave like one unrestricted relational database.
At what point does hiding the sharding become counterproductive? I imagine teams still need a fairly deep understanding of shard keys, query routing, and failure modes to avoid accidentally expensive cross-shard operations.
Distributed systems introduce severe restrictions on what can be reasonably done at scale. Having a single connection string is one thing, being able to do a massive JOIN is another (and should only be ever done in analytical databases).
The question is not "when sharding becomes counter-productive" but "when it starts making sense".
With sharding something somewhere has to know how to route queries to subsets of data. So it is a complexity price paid for being able to scale. If one can avoid paying this cost then he should.
And USUALLY cross-shard queries are not just expensive but simply impossible in operational clusters. Like, if you do COUNT on a table, you only count within a single db shard table.
Looks like it's not drop-in, you need to heavily modify your data and indexes structurally to make sharding performant. Cross-shard joins are possible but need to be designed for explicitly (to be fair, indexes also need to be designed explicitly in regular SQL). IDK if a drop-in solution is possible or if this is an information-theoretical limitation.
I sometimes feel that when the industry moved from pets to cattle, what really happened is that the cattle turned out to need an exotic farm to live on, negating the savings. You can have a few honking servers or you can hand massage exotic k8s setups on your fleet. Pick your poison, but dont delude yourself that the TCO of the latter is lower than the former.
Looks like the GIF is fully built out in code. It's really nice to look at, well made, and easy to understand too.
I wonder what program or code they used. I'd love to know.
p.sI thought it was a GIF, but it's an iframe. That was a nice little surprise.
Yeah, I'm looking at it in developer mode. It's using a GSAP timeline approach to update SVG properties. I'm curious how they handle security and caching for something like this. It looks like they're using Tailwind, at least. but this approach is really clean and nice.
It really feels like the best way to learn is by studying other people's code.
I didn't explain myself very well. With iframes, there can be security concerns in general. Though in practice, there's usually no real issue. It just surfaces the surface level risk that JavaScript could be tampered with, but it's almost never something to actually worry about.
As for caching strategy, cache invalidation issues do pop up from time to time. Since the filenames usually include a content hash, I'm guessing they're using a Vite style caching strategy. Cache invalidation might almost never be a problem, but you still plan for it just in case.
i do wonder how something like this can be generally implemented. i presume this must only support a subset of SQL/plpgsql, as some things would be.. utterly insane to manage manually. e.g., if i have a table with a btree-gist overlap constraint, or some inclusion-exclusion check-constraint (or literally any constraint that requires multiple rows to be fully determined - there are quite a lot of them), how on earth does this work?
there's a reason why postgres writing is (mostly) serialised (asterisk) to a single writer (asterisk asterisk). something something ACID, but in short by having multiple writers improves availability, but weakens integrity.
Even with a large database servers (10s of CPU cores, 100s of gigabytes of RAM) bottlenecks arise pretty quickly.
Err, do they? For what percent of real world use cases?
The database can scale to handle more traffic by adding replicas. An extreme example of this is OpenAI's use of 50 replicas on a single Primary.
So an extreme example is OpenAI needing 50 replicas, but we're doing five blades ... err, we're doing 768 servers because the need arose "pretty quickly"?
When we needed to store a petabyte of data (one million gigabytes), we'd need many more shards
For who? The United States government? How many end-users are running 1PB Postgres database on DBaaS?
I disagree with the opening premise:
> A single database server cannot handle such demand, so we must spread the queries and data out across many servers with database sharding
Did you max out the capacity of the best server you can buy?
Such a database can serve millions of customers (the numbers given).
You always want to scale up the other parts first, request handlers, caching, etc. The day you can no longer inspect the essential state of your system is the day your company better be included in NASDAQ and ready to pay a few hundred engineers 300k salaries.
Well, they are selling this thing so they don't want you to buy a big server (with a read replica) as that's much cheaper.
I found the article very helpful from a technical perspective, and didn't focus on the number too much, as it could easily be swapped and the decision-making process for when to shard is kinda out of scope.
But I hadn't considered this, so thanks for pushing back. Good to keep in mind their incentives.
I will say, since their product is a proxy whose interface is a single SQL connection, you should in theory be able to do dev queries through that black box, much the same as application queries? What is so scary here that it would require a hundred engineers?
> you should in theory be able to do dev queries through that black box
Because it’s a leaky abstraction which is trying to make guarantees over network connections which are extremely difficult to make within the same kernel.
A few questions I would start with:
- is the system even ACID compliant?
In my reading of this article, no.
- is my sql feature set limited? Will it enforce all constraints? Or are their cross-shard limitations?
For example the article doesn’t discuss transactions or how they would roll back, or how they guarantee a consistent view of data.
Next:
- how does it respond to error?
- how does it respond to load?
This is a complex system and complexity breeds bugs. But now you don’t have the tools or procedures to investigate those bugs because you can’t poke the system at desk with tools. You can’t run experiments; you can’t even see all the data.
Thank you, good points. I'm just learning about this in real time.
It looks like it does support transactions, but they basically destroy the performance benefits: https://vitess.io/docs/faq/sharding/advanced/can-i-use-vites...
The more I read the more I'm struggling to understand the benefit of a router like this that sits on top of a monolithic SQL, vs a truly distributed DB like cockroach.
Like you I'd love to learn more about the internals of their actual SQL engine, which is just barely touched on in the article. The idea of ripping out the layer of a SQL interpreter that does just enough to route it to a real server tickles my brain in the same way as when I learned how node.js ripped the js interpreter out of a browser.
Most non FAANG orgs could probably serve all their customers from postgres on a laptop.
Surely the cost of running sql server on premium hardware with replication would be more than running on commodity hardware with sharding?
It depends on who you are paying for it, but generally a distributed system is harder to reason about, harder to fix, has weirder edge cases, and much more easily get into situations where it requires even MORE expertise to fix than just having a big honking server.
When you start calculating things that are not just the server, the single server looks cheaper and cheaper. How do you get a consistent backup? How do you do DR? How do you tune queries when it could go to this node or that node? Now writes are going to be significantly slower if you need multi-node commit because no matter what you are racing the speed of light on the network.
Only if your engineering resources and free and unbounded.
Even then I would probably use those resources to optimize software instead.
In the physical world of trucks and cranes no company would make that mistake to try to save 30-80k.
Hetzner has this for 100usd a month, 256gb ram can handle quite a lot of traffic already
€84.70 max. per month €0.1357 per hour CPU Intel Xeon E5-1650V3 RAM 256 GB Drives 2 × 6.0 TB Enterprise HDD Location #FSN1-DC1 Information IPv4 ECC iNIC
Lol that isn't a database server. Come on, if it has hard drives rather than (good) SSD in 2026, it's not a server you should be using for databases unless it's for something with next to no load, and you are also running your application server on the same machine.
What about sequences? The example shows an auto-incrementing user ID. How’s that possible without contention between all shards? Is the proxy responsible for sequences?
What about foreign keys? Do they all have to live on the same shard? How do you do distributed transactions?
On cross-shard reads: how do you do sorting? And cross-shard joins?
I’d love to be proven wrong, but I suspect the 768 servers look like 1 only on the very surface, and you’ll get wildly different characteristics from cross-shard and single-shard queries.
I personally would prefer if they _didn’t_ look like 1 if they can’t behave like 1.
Of course 768 servers NEVER behave as 1. This is physically impossible.
Global services using relational dbs typically severely restrict queries that run against the cluster. So no joins, no intervals, no grouping, etc.
Transactional queries are usually limited to something like "get a single record, preferably from cache". For many typical web services this can go VERY FAR. Only a handful of global services needs more than a few dozen database servers and a caching cluster. In fact, i have seen major businesses running off a pair of very big postgres instances.
Analytical stuff is extracted into dedicated storages optimized for throughput, like Snowflake or Redshift or BigQuery.
This seems like the important distinction: making the infrastructure look like one database to the application is different from making it behave like one unrestricted relational database.
At what point does hiding the sharding become counterproductive? I imagine teams still need a fairly deep understanding of shard keys, query routing, and failure modes to avoid accidentally expensive cross-shard operations.
Yes!
Distributed systems introduce severe restrictions on what can be reasonably done at scale. Having a single connection string is one thing, being able to do a massive JOIN is another (and should only be ever done in analytical databases).
The question is not "when sharding becomes counter-productive" but "when it starts making sense".
With sharding something somewhere has to know how to route queries to subsets of data. So it is a complexity price paid for being able to scale. If one can avoid paying this cost then he should.
And USUALLY cross-shard queries are not just expensive but simply impossible in operational clusters. Like, if you do COUNT on a table, you only count within a single db shard table.
I had this question too, even their "Database Scaling" course doesn't get to it.
https://vitess.io/docs/faq/sharding/advanced/
Looks like it's not drop-in, you need to heavily modify your data and indexes structurally to make sharding performant. Cross-shard joins are possible but need to be designed for explicitly (to be fair, indexes also need to be designed explicitly in regular SQL). IDK if a drop-in solution is possible or if this is an information-theoretical limitation.
> The example shows an auto-incrementing user ID. How’s that possible without contention between all shards?
Why not divide ID range (63bit?) by the maximum planned number of shards and then set on each shard it's min/max value so ranges will not overlap?
low-traffic auto increment like user creation can be relatively performant by reserving batches up front, then using those in each shard.
But the use-case for this is mostly if you do not need, or have very limited use for anything cross-shard.
A 767 servers KV store should be enough for everyone
Sibling post has author answering questions in comments: https://news.ycombinator.com/item?id=48925420
Took me an embarrassing amount of time to realize this is an ad.
I sometimes feel that when the industry moved from pets to cattle, what really happened is that the cattle turned out to need an exotic farm to live on, negating the savings. You can have a few honking servers or you can hand massage exotic k8s setups on your fleet. Pick your poison, but dont delude yourself that the TCO of the latter is lower than the former.
what did they use to make those diagrams/animations?
How does Sharding works when You do complex joins ? Seems tricky , runs on each server and gets data back and aggregate it again?
Looks like the GIF is fully built out in code. It's really nice to look at, well made, and easy to understand too. I wonder what program or code they used. I'd love to know.
p.sI thought it was a GIF, but it's an iframe. That was a nice little surprise.
Specifically, it's an JS-controlled/animated SVG embedded in an iframe.
Yeah, I'm looking at it in developer mode. It's using a GSAP timeline approach to update SVG properties. I'm curious how they handle security and caching for something like this. It looks like they're using Tailwind, at least. but this approach is really clean and nice.
It really feels like the best way to learn is by studying other people's code.
What kind of security and caching concerns do they need to handle to animate an SVG?
I didn't explain myself very well. With iframes, there can be security concerns in general. Though in practice, there's usually no real issue. It just surfaces the surface level risk that JavaScript could be tampered with, but it's almost never something to actually worry about.
As for caching strategy, cache invalidation issues do pop up from time to time. Since the filenames usually include a content hash, I'm guessing they're using a Vite style caching strategy. Cache invalidation might almost never be a problem, but you still plan for it just in case.
i do wonder how something like this can be generally implemented. i presume this must only support a subset of SQL/plpgsql, as some things would be.. utterly insane to manage manually. e.g., if i have a table with a btree-gist overlap constraint, or some inclusion-exclusion check-constraint (or literally any constraint that requires multiple rows to be fully determined - there are quite a lot of them), how on earth does this work?
there's a reason why postgres writing is (mostly) serialised (asterisk) to a single writer (asterisk asterisk). something something ACID, but in short by having multiple writers improves availability, but weakens integrity.
Load balancers, microservices and horizontal scaling?
Previously: https://news.ycombinator.com/item?id=48925420