The top concern I've gotten from dev teams when proposing HA distributed postgres (e.g through RDS Aurora global) is that eventual consistency is not suitable for many workloads.
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
Assuming it works the same way as their MySQL product, Vitess:
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
Same here, first thing I need to know when considering a distributed system is how consistency is handled. If it's eventual consistency, what is the replication lag like? If it's strong consistency, can their network handle that? What happens when a node goes down?
I love to see this but as a heavy Vitess/MySQL user I do fear the split focus from PlanetScale. Hoping to see continued improvements on the Vitess side as well.
Selfish doubts aside, congrats to Planetscale on the launch!
don't worry, we still give a lot of love to vitess. at the end of the day they are both databases. the beauty of doing both is we can take learnings from each product and apply it to the other.
We went as far as we could on GCP before PlanetScale. Spanner is amazing but also amazingly expensive. And CloudSQL, also amazing as long as you don’t need write scaling. PlanetScale (Vitess+MySQL plus their branching / deployment and monitoring tools) is just not a combination offered on GCP. And while I didn’t use Aurora, from talking to and exploring AWS it doesn’t really have this combination either.
The largest database to ever run on Aurora MySQL runs on PlanetScale, and the largest database to ever run on Aurora limitless also runs on PlanetScale.
I see no mention of foreign keys, or any other constraints, across shards. If, as I suspect, they're not implemented, it would still be useful but at the level of Spanner 10 years ago.
The top concern I've gotten from dev teams when proposing HA distributed postgres (e.g through RDS Aurora global) is that eventual consistency is not suitable for many workloads.
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
Assuming it works the same way as their MySQL product, Vitess:
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
Therefore, Consistency guarantees are based on implementing application-specific partitioning (correctly)?
Yeah. Vitess does have some options for atomic cross-shard commits using an additional database to track two-phase commits, but that's expensive so you probably want to avoid that. https://vitess.io/docs/25.0/reference/features/distributed-t...
Same here, first thing I need to know when considering a distributed system is how consistency is handled. If it's eventual consistency, what is the replication lag like? If it's strong consistency, can their network handle that? What happens when a node goes down?
I love to see this but as a heavy Vitess/MySQL user I do fear the split focus from PlanetScale. Hoping to see continued improvements on the Vitess side as well.
Selfish doubts aside, congrats to Planetscale on the launch!
don't worry, we still give a lot of love to vitess. at the end of the day they are both databases. the beauty of doing both is we can take learnings from each product and apply it to the other.
Obviously welcome more tools for Postgres and sharding but it’s a tough hill to climb against the hyper clouds. Best wishes to the team.
We went as far as we could on GCP before PlanetScale. Spanner is amazing but also amazingly expensive. And CloudSQL, also amazing as long as you don’t need write scaling. PlanetScale (Vitess+MySQL plus their branching / deployment and monitoring tools) is just not a combination offered on GCP. And while I didn’t use Aurora, from talking to and exploring AWS it doesn’t really have this combination either.
The largest database to ever run on Aurora MySQL runs on PlanetScale, and the largest database to ever run on Aurora limitless also runs on PlanetScale.
Pretty exciting to see a sharded Postgres product in the market.
The pitch is compelling. I wonder how many teams will be able to operate sharded database setups in production as a result of this.
The first question came to my mind was, how would this compare against Citus.
I actually wrote a bit about this on our blog! https://planetscale.com/blog/the-history-of-postgres-shardin...
Independent of the product, love the branding design for this page.
is this open-sourced?
Some more discussion: https://news.ycombinator.com/item?id=49645686
Shouldn't this be folded into that? Another one https://news.ycombinator.com/item?id=49645763
I see no mention of foreign keys, or any other constraints, across shards. If, as I suspect, they're not implemented, it would still be useful but at the level of Spanner 10 years ago.
Cross shard transactions are “coming soon”. So, this is just like having multiple postgres clusters.
Cross shard transactions are supported, but we don't have support for atomic 2pc cross shard transactions, its WIP
only planet scale? Rookie numbers.
I am looking for interdimensional and interuniversal scale. Which of you trust fund babies has a startup which is working on this problem?