This comes across as a 100% vibe-coded system that could really just be a table or two in a standard SQL database with some care taken with indexing and partitioning (when you reach that point).
What am I missing? What would compel someone to abandon a battle tested system like PSQL for a critical storage engine for this?
You can do an append-only table like this in any SQL database. You can use composite keys to achieve the query+natural sorting abilities over a single field. The effort to replicate this feature in any other environment is so minimal that it the 'vendor lock-in' makes this a non-starter in my eyes.
Hi - its not - more about 20% - for tests and non critical things.
Focusing on your abandonment part etc etc:
- Its not really trying to achieve that, this is simply a more efficient model, with a good storage coupling (for chat).
- What is being achieved here is a cohesive solution for chat specific workloads and optimising for it.
One of the benefits is speed (throughput, latency) - and very cheap calls.
More so - under sql, and for developers who have different demands and mostly changing demands - the effort is quite complicated situation (I speak from my own experience)
The focus here is a simple downmost layer - that allow all the varieties of top layers & requirements to be achieved easily.
In my blog post - i share a variety of these.
Also product is not ready - but thought i share this idea - as i think it can be of benefit to most devs who are starting out with a product.
You technically dont even need progressdb to achieve it - You can do it with a backend based pebble or rocksdb database.
>For example, to build shared threads like OpenAI’s, you might end up creating a new collection, a caching layer, or an index just to connect messages that logically belong together.
Which messages belongs together depends on the context and when you discover that the fancy thread store you built your app around can't return a list of MY messages without a full sequential scan of every single thread you'll wish you could solve it with an index.
Hi - dont know how to answer you directly - but yes the store has an index already, at a timed sequence for example. And a key called thread:thread_meta.
This statement is in no way saying you dont need an index, its simply saying, this is the variety of ways people might end up choosing.
Which is just 1 example of the type of work - a thread model & compatible store can fix and fix more efficiently.
In no way fancy too - this is just redis with a disc and some steps imo.
This comes across as a 100% vibe-coded system that could really just be a table or two in a standard SQL database with some care taken with indexing and partitioning (when you reach that point).
What am I missing? What would compel someone to abandon a battle tested system like PSQL for a critical storage engine for this?
You can do an append-only table like this in any SQL database. You can use composite keys to achieve the query+natural sorting abilities over a single field. The effort to replicate this feature in any other environment is so minimal that it the 'vendor lock-in' makes this a non-starter in my eyes.
Hi - its not - more about 20% - for tests and non critical things.
Focusing on your abandonment part etc etc:
- Its not really trying to achieve that, this is simply a more efficient model, with a good storage coupling (for chat).
- What is being achieved here is a cohesive solution for chat specific workloads and optimising for it.
One of the benefits is speed (throughput, latency) - and very cheap calls.
More so - under sql, and for developers who have different demands and mostly changing demands - the effort is quite complicated situation (I speak from my own experience)
The focus here is a simple downmost layer - that allow all the varieties of top layers & requirements to be achieved easily.
In my blog post - i share a variety of these.
Also product is not ready - but thought i share this idea - as i think it can be of benefit to most devs who are starting out with a product.
You technically dont even need progressdb to achieve it - You can do it with a backend based pebble or rocksdb database.
The database is completely opensource and is technically backed by Pebble golang - which is opensource.
There are no changes to the underlying core, i simply make it easy to use operationally, along with other chat centric demands in the AI age.
e.g Backups, Retention, Scaling it up etc.
So there is no "Vendor lock-in" - and i plan to opensource everything.
I dont even feel i own the idea - its pretty out there, am just bringing eyes towards it.
>For example, to build shared threads like OpenAI’s, you might end up creating a new collection, a caching layer, or an index just to connect messages that logically belong together.
Which messages belongs together depends on the context and when you discover that the fancy thread store you built your app around can't return a list of MY messages without a full sequential scan of every single thread you'll wish you could solve it with an index.
Hi - dont know how to answer you directly - but yes the store has an index already, at a timed sequence for example. And a key called thread:thread_meta.
This statement is in no way saying you dont need an index, its simply saying, this is the variety of ways people might end up choosing.
Which is just 1 example of the type of work - a thread model & compatible store can fix and fix more efficiently.
In no way fancy too - this is just redis with a disc and some steps imo.
The site is down. https://www.isitdownrightnow.com/progressdb.dev.html
I can access it - and dont see any resource outages etc.
Hmm - you can access it directly or its not showing when you visit?
Thanks & upscaled the vps a level up just in case.
It is now back up.