I recently upgraded two ~10 year old aging legacy applications at work. One was in Flask, and one in Django. This made me appreciate the "batteries included" philosophy of Django a lot more.
Even though the django legacy application was much larger, it had barely any extensions to "vanilla django". Comparably, the flask application had a dozen third-party flask-* dependencies that provided functionality like auth, permissions, and other features that Django has built-in. Many of these dependencies were archived/abandonware and hadn't been maintained in a decade.
When it came to upgrading the Django app, I had one giant release notes page to read. I didn't need to switch any packages, just make some pretty simple code changes for clearly documented deprecations. For the Flask app I had to read dozens of release notes pages, migrate to new maintained packages, and rework several untested features (see: legacy application).
In my mind, "batteries included" is an underrated philosophy of Djangoo. Also, it is now such a mature ecosystem it is unlikely there will be any radical breaking changes.
Perhaps there are some parallels to draw with newer trendy (but minimalistic) python frameworks like FastAPI.
If I were building a web application I wanted to last a decade or more, Django would be up there in tech choices - boring and sensible, but effective.
I haven't used Django in about 10 or 12 years but I cracked it open the day. It was cool to see all the things I loved are largely unchanged; I was able to step right back in.
The ecosystem has improved thought. django-ninja is great for API, django-coton brings component supports and you have better options than celery for qeueing.
There is buzz around the combination of Django and HTMX, worked on by the same people in one team, as a much simpler alternative to split frontend and backend teams with a REST API in between (and perhaps NextJS as well, etc).
Absolutely. For what it does, Django is pretty much the best full stack Python web framework there is. It's also a great way to rapidly develop (just sticking to synchronous, which Django is best at).
One can then later consider spinning certain logic off into a separate service (e.g. in Golang), if speed is a concern with Python.
I chose Django + htmx and a small amount of Alpine.js for a full-stack software project that is currently being launched. I had zero professional experience with Django (or Python really) before starting. I was able to develop the entire application on my own, in my spare time, and had time left over to also handle infrastructure and devops myself.
I prefer Python and it's web frameworks over Typescript/React because there is a lot more stability and lot less "framework-of-the-week"-itis to contend with. It's much easier to reason about Django code than any React project I've worked on professionally. IMO when you don't have a firehose of money aimed at you, then Python is the way to go
Yes, you can do the whole web app (or web site if you will) like that without any complicated dependencies. This will be great if you touch the project only now and again e.g. the typical side project that you want to still work in the year 2030 without major changes.
Yet the approach also scales up to enterprise grade project, leveraging DRF, Django-cotton and so on (and htmx).
Yes. Still one of the best batteries included web frameworks for creating anything that's more of a website (e.g. E-Commerce) than a web app (e.g. Photoshop). No, you don't need NextJs and friends for everything ;)
Is Groovy/Grails still popular? I also remember Groovy++ but I believe its features were incorporated into Groovy. But maybe these are already present in modern Java?
I just rolled a backend using FastAPI and SQLAlchemy and it made me miss Django.
Too much other stuff going on in this app to incorporate Django, but it's still way ahead of the curve compared to bringing together independent micro frameworks.
It's brave trying FastAPI if you haven't tried it before. Going async is going to be quite different and you need to be more careful when designing your API. Most people will never need it.
This is why most folks just needing a plain Python API without anything else, they usually go for Flask, which is vastly simpler. For a more complete web app or site, I would recommend Django.
The ORM is so so so much better designed that SQLAlchemy v2. Performing queries, joins, executing in transactions all feels clean and concise. The latter feels dated and I find it hard to believe there's not a widely accepted replacement yet.
In terms of views, route configuration and Django's class-based views are sorely missed when using FastAPI. The dependency pattern is janky and if you follow the recommended pattern of defining your routes in decorators it's not obvious where your URL structure is even coming from.
We could probably do with a "SQLAlchemy for Django users" article. SQLAlchemy is much more powerful and flexible than Django. After using SQLAlchemy it's hard to even consider an active record style ORM like Django an ORM at all. SQLAlchemy can truly map relational data onto objects and uses the unit of work pattern to coordinate updates. Django just feels like writing raw SQL but in nicer Python syntax. The details of relational models leak directly into the business logic and there isn't really much you can do about it. In short, SQLAlchemy is a different beast. If all you need is Django then you're probably only doing CRUD and you should just use Django.
Django definitely wins at CRUD because that's what it is. It calls itself a web framework but really it's a web-crud-app framework. Flask and Pyramid are web frameworks.
Dealing with the session object seems a small price to pay for the flexibility in architecture that it gets you.
My only criticism is that die-hard django devs constantly brush aside the admin and can't stop telling people not to use it. I think it's a huge mistake.
It's extremely well-designed and extensible, there is no reason to reinvent the wheel when so much time and effort has been put into it.
They will complain of things like "eventually you will have to start over with a custom solution anyway"... but whatever gripes they have, could just be put into improving the admin to make it better at whatever they're worried about.
Personally I've not run into something I couldn't make work in the admin without having to start over. My own usecases have been CRUD for backoffice users/management and I've had great success with that at several different companies over the last ~15 years.
People will say "it's only for admins you trust" yet it has very extensive permissions and form/model validation systems heavily used in the admin and elsewhere, and they are easily extensible.
I use the heck out of the admin. I wouldn't say I'm super experienced with Django, but my solution to users being overwhelmed by doing things there is to build a bit of extra UI just for the stuff they need to do, more Rails style. Meaning: I don't go into fighting too much with the admin when what it can do out of the box is not feasible for some. But even if the admin ends up being used only by devs and some trained folks, I think it has amazing utility.
LLMs are experts at Django, as there's 20 years of training data on it as well as just being written in the world's most popular language. LLMs can pump out full featured Django sites like anything.
I don't know why anyone would use any other framework.
We've used (and continue to use) Django for bespoke applications for a decade and a half now. It continues to be the most well-supported, well-governed, well-documented, batteries-included, extensible web framework of all the ones we've tried. Finding developers with experience using it (or upskilling them) is easy. As a choice of web technology, it's one of those that we've never regretted investing in.
For a complete solution requiring many traditional high-level components like templating, forms, etc, then yes, clearly Django. But for something looking more like a REST API, with auto-generated documentation, I would nowadays seriously consider FastAPI, which, when used with its typed Pydantic integration, provides a very powerful solution with very little code.
Works great, I've been using it in production for a few years. DRF was one of my least favorite bits of the Django world and Ninja has been an excellent alternative.
I still love Django for greenfield projects because it eliminates many decision points that take time and consideration but don't really add value to a pre-launch product.
Not in my org. Though we did choose it for _one_ new project recently, mostly because we re-used some code from another Django project we had, and we wanted to lean on some readily available functionality from jazzband libs.
We have a few FastAPI services, but are mostly moving away from Python for projects > 1kloc.
next.js + TS for front ends, C# for pure back-end services. Golang was a close second but we have some team experience with C# and like it's lack of ecosystem fragmentation, as that was another gripe we had with Python. The Dapper "micro-orm" is also refreshing after years of struggling to make Django's ORM do the right thing.
The threshold is arbitrary, and likely higher in reality. But we found that we want something with more sound type checking and mypy has lots of rough edges. The Python ecosystem has a lot of catching up to do here.
Idk if it's best practice, but I usually like to make apps similar to components, where I have an app for accounts which handles user accounts, and a files app which handles all the dimension and fact tables around user uploads, and a social app for social features, etc.
It makes it easy to compartmentalize the business logic in terms of module imports.
This sounds similar to a modular monolith design. But you have to be careful not to directly import things between apps and especially not to make foreign keys between the models of different apps. We ended up doing that and just wishing it was one big app.
Modular monolith is a good idea and if you want to do it in Django then make small apps that just expose services to each other (ie. high-level business functions). Then have a completely separate app just for the UI that uses those services.
Yeah. I was thinking a modular monolith since it's a django project, and I think that's django's sweet spot since it comes with so many things bundled.
For a true modular design I'd probably step away from django to a less comprehensive framework or just write in golang.
A composite is a structure of many pieces that work together.
Thus, a composite monolith is this arrangement of components in a way that they work together as a monolith. Separate modules, but working together as a single thing.
That's the same as what people are calling a modular monolith, no? The key thing with composition is the bits sit next to each other, there's no inheritance or dependencies between the components. In my idea you'd have a bunch of "pure service" apps that are focused on individual areas of the business or processes. Then you'd have one or more UI apps (either user-facing HTML or APIs) that compose those services to do what needs to be done, e.g. a "create order" endpoint might compose the create order service from fulfilment and the send notification service from comms to put an order on the fulfilment backlog and send notifications. Is this what you had in mind?
That sounds to me like good old "replaceable parts oriented programming".
Replaceable parts can have inheritance, but often what is good about them is the composability. Each part can connect to each other in different ways. We call these boundaries "connectors".
If it's the kind of project that is going to run against one PostgreSQL database then I'd probably start a new project with Django just for its database migration support. That doesn't mean everything in the project has to be Django.
Would love to hear an honest discussion of why Django and/or Python is a bad solution for any given problem. Is it because they are old technologies? Do they lack support for something in particular? Are they too expressive/not expressive enough?
- Imports are a mess
- No control of mutation in function signatures, and in general it's still a surprise when things mutate
- Slow
- Types and enums have room for improvement
You just need to work around those and get used to it. Then you can build nice Python projects that keep growing - but Django really helps you do that. Python however is always going to be "10-100x" slower than something like an API written in Go, Rust and so on. That's fine in most cases.
Thanks for the summary. Looking forward to the videos becoming available.
> I talked to this speaker afterward, and asked him how they did nested modals + updating widgets in a form after creating a new object in a nested modal. He showed me how he did it, I've been trying to figure this out for 8 months!
It's easy to get something quick working with HTMX and Django, but if you want robust UI tests that actually test what happens when users click stuff, don't you need to use something like Playwright? This can be pretty heavy, slow and flaky, compared to regular Django tests?
I find with HTMX, it can introduce a lot of edge cases to do with error handling, showing loading progress, and making sure the data on the current page is consistent when you're partially updating chunks of it. With the traditional clunky full-page-refresh Django way, you avoid a lot of this.
For the vast, vast, vast majority of people, if you don't have an obvious primary key, choosing UUIDv7 is going to be an absolute no-brainer choice that causes the least amount of grief.
Which of these is an amateur most likely to hit: crash caused by having too small a primary key and hitting the limit, slowdowns caused by having a primary key that is effectively unsortable (totally random), contention slowdowns caused by having a primary key that needs a lock (incrementing key), or slowdowns caused by having a key that is 16 bytes instead of 8?
Of all those issues, the slowdown from a 16 byte key is by far the least likely to be an issue. If you reach the point where that is an issue in your business, you've moved off of being a startup and you need to cough up real money and do real engineering on your database schemas.
The problem is that companies tend to only hire DB expertise when things are dire, and then, the dev teams inevitably are resistant to change.
You can monitor and predict the growth rate of a table; if you don’t know you’re going to hit the limit of an INT well in advance, you have no one to blame but yourself.
Re: auto-incrementing locks, I have never once observed that to be a source of contention. Most DBs are around 98/2% read/write. If you happen to have an extremely INSERT-heavy workload, then by all means, consider alternatives, like interleaved batches or whatever. It does not matter for most places.
I agree that UUIDv7 is miles better than v4, but you’re still storing far more data than is probably necessary. And re: 16 bytes, MySQL annoyingly doesn’t natively have a UUID type, and most people don’t seem to know about casting it to binary and storing it as BINARY(16), so instead you get a 36-byte PK. The worst.
> contention slowdowns caused by having a primary key that needs a lock (incrementing key)
This kind of problem only exists in unsophisticated databases like SQLite. Postgres reserves whole ranges of IDs at once so there is never any contention for the next ID in a serial sequence.
I think you’re thinking out the cache property of a sequence, but it defaults to 1 (not generating ranges at once). However, Postgres only needs a lightweight lock on the sequence object, since it’s separate from the table itself.
MySQL does need a special kind of table-level lock for its auto-incrementing values, but it has fairly sophisticated logic as of 8.0 as to when and how that lock is taken. IME, you’ll probably hit some other bottleneck before you experience auto-inc lock contention.
I was in the never-UUID camp, but have been converted. Of course depends on how much do you depend on your PKs for speed, but using UUIDs has a a great benefit in that you can create a unique key without a visit to the DB, and that can enormously simplify your app logic.
I’ve never understood this argument. In every RDBMS I’m aware of, you can either get the full row you just inserted sent back (RETURNING clause in Postgres, MariaDB, and new-ish versions of SQLite), and even in MySQL, you can access the last auto-incrementing id generated from the cursor used to run the query.
Now imagine that storing the complete model is the last thing you do in a business transaction. So the workflow is something like 'user enters some data, then over the course of the next minutes adds more data, the system contacts various remote services that too can take long time to respond, the user can even park the whole transaction for the day and restore it later', but you still want to have an unique ID identifying this dataset for logging etc. There is nothing you can insert at the start (it won't satisfy the constraints and is also completely useless). So you can either create a synthetic ID at the start but it won't be the real ID when you finally store the dataset. Or you can just generate an UUID anywhere anytime and it will be a real ID of the dataset forever.
So have a pending table with id, user_id, created_at, and index the latter two as a composite key. SELECT id FROM pending WHERE user_id = ? ORDER BY created_at DESC LIMIT 1.
Preferably delete the row once it's been permanently stored.
Keeping an actual transaction open for that long is asking for contention, and the idea of having data hanging around ephemerally in memory also seems like a terrible idea – what happens if the server fails, the pod dies, etc.?
> Keeping an actual transaction open for that long is asking for contention
Yes, which is why this is not an actual db transaction, it's a business transaction as mentioned.
> and the idea of having data hanging around ephemerally
The data is not ephemeral of course. But also the mapping between business transaction and model is not 1:1, so while there is some use for the transaction ID, it can't be used to identify a particular model.
> With UUIDs there is absolutely no need for that.
Except now (assuming it’s the PK) you’ve added the additional overhead of a UUID PK, which depending on the version and your RDBMS vendor, can be massive. If it’s a non-prime column, I have far fewer issues with them.
> The data is not ephemeral of course.
I may be misunderstanding, but if it isn’t persisted to disk (which generally means a DB), then it should not be seen as durable.
The whole content of the business transaction is persisted as a blob until it's "done" from the business perspective. After that, entities are saved(,updated,deleted) to their respective tables, and the blob is deleted. This gives the users great flexibility during their workflows.
Yes the overhead of UUIDs was something I mentioned already. For us it absolutely makes sense to use them, we don't anticipate to have hundreds of millions of records in our tables.
I also do that for convenience. It helps a lot in many cases. In other cases I might have tables that may grow into the millions of rows (or hundreds of millions), then I'd absolutely not use UUID PK's for those particular tables. And I'd also shard them across schemas or multiple DBs.
If you don't have a natural primary key (the usual use case for UUIDs in distributed systems such that you can have a unique value) how do you handle that with bigints? Do you just use a random value and hope for no collisions?
Wouldn't you just have an autoincrementing bigint as a surrogate key in your dimension table?
Or you could preload a table of autoincremented bigints and then atomically grab the next value from there where you need a surrogate key like in a distributed system with no natural pk.
Yes, if you have one database. For a distributed system though with many databases sharing data, I don't see a way around a UUID unless collisions (the random approach) are not costly.
Yes. This is the way so long as you can guarantee you wont grow past the bits.
Otherwise you can still use the pregenerated autoincrements. You just need to check out blocks of values for each node in your distributed system from the central source before you would need them:
N1 requests 100k values, N2 requests 100k values, etc. Then when you've allocated some amount, say 66%, request another chunk. That eay you have time to recover from a central manager going offline before it's critical.
I have no problem with using uuids but there are ways around it if you want to stick with integers.
But if you use sequential integers as primary key, you are leaking the cardinality of your table to your users / competitors / public, which can be problematic.
Twice now I was called to fix UUIDs making systems crawl to stop.
People underestimate how important efficient indexes are on relational databases because replacing autoincrement INTs with UUIDs works well enough for small databases, until it doesn't.
My gripe against UUIDs is not even performance. It's debugging.
Much easier to memorize and type user_id = 234111 than user_id = '019686ea-a139-76a5-9074-28de2c8d486d'
Is it really enormous? bigint vs UUID is similar to talking about self-hosting vs cloud to stakeholders. Which one has bigger risk of collision? Is the size difference material to the operations? Then go with the less risky one.
You shouldn't be using BIGINT for random identifiers so collision isn't a concern - this is just to future proof against hitting the 2^31 limit on a regular INT primary key.
I’ll go further; don’t automatically default to a BIGINT. Put some thought into your tables. Is it a table of users, where each has one row? You almost certainly won’t even need an INT, but you definitely won’t need a BIGINT. Is it a table of customer orders? You might need an INT, and you can monitor and even predict the growth rate. Did you hit 1 billion? Great, you have plenty of time for an online conversion to BIGINT, with a tool like gh-ost.
I work for a company that deals with very large numbers of users. We recently had a major project because our users table ran out of ints and had to be upgraded to bigint. At this scale that's harder than it sounds.
So your advice that you DEFINITELY won't need a BIGINT, well, that decision can come back to bite you if you're successful enough.
(You're probably thinking there's no way we have over 2 billion users and that's true, but it's also a bad assumption that one user row perfectly corresponds to one registered user. Assumptions like that can and do change.)
I’m not saying it’s not an undertaking if you’ve never done it, but there are plenty of tools for MySQL and Postgres (I assume others as well) to do zero-downtime online schema changes like that. If you’ve backed yourself into a corner by also nearly running out of disk space, then yes, you’ll have a large headache on your hands.
Also, protip for anyone using MySQL, you should take advantage of its UNSIGNED INT types. 2^32-1 is quite a bit; it’s also very handy for smaller lookup tables where you need a bit more than 2^7 (TINYINT).
> but it's also a bad assumption that one user row perfectly corresponds to one registered user. Assumptions like that can and do change.
There can be dupes and the like, yes, but if at some point the Customer table morphed into a Customer+CustomerAttribute table, for example, I’d argue you have a data modeling problem.
It looks like htmx is popular in the Django community. Is there any background story that made this? (Context: Just picked Django for a hobby project. Don't know much about Webdev trend beyond, like, what are talked about on the HN top page.)
The conference coordinators said they would be released in about a month, so I will update the post once they are released! I am really excited to watch them again. Amazingly informative stuff.
Pretty cool seeing how people still go for Django even with so many new frameworks, always makes me wanna go back to it when stuff gets messy tbh
Django is still great.
I recently upgraded two ~10 year old aging legacy applications at work. One was in Flask, and one in Django. This made me appreciate the "batteries included" philosophy of Django a lot more.
Even though the django legacy application was much larger, it had barely any extensions to "vanilla django". Comparably, the flask application had a dozen third-party flask-* dependencies that provided functionality like auth, permissions, and other features that Django has built-in. Many of these dependencies were archived/abandonware and hadn't been maintained in a decade.
When it came to upgrading the Django app, I had one giant release notes page to read. I didn't need to switch any packages, just make some pretty simple code changes for clearly documented deprecations. For the Flask app I had to read dozens of release notes pages, migrate to new maintained packages, and rework several untested features (see: legacy application).
In my mind, "batteries included" is an underrated philosophy of Djangoo. Also, it is now such a mature ecosystem it is unlikely there will be any radical breaking changes.
Perhaps there are some parallels to draw with newer trendy (but minimalistic) python frameworks like FastAPI.
If I were building a web application I wanted to last a decade or more, Django would be up there in tech choices - boring and sensible, but effective.
I haven't used Django in about 10 or 12 years but I cracked it open the day. It was cool to see all the things I loved are largely unchanged; I was able to step right back in.
The ecosystem has improved thought. django-ninja is great for API, django-coton brings component supports and you have better options than celery for qeueing.
I've stuck to DRF for all these years. But wouldn't mind looking at django-ninja. Is it better?
There is buzz around the combination of Django and HTMX, worked on by the same people in one team, as a much simpler alternative to split frontend and backend teams with a REST API in between (and perhaps NextJS as well, etc).
Are people choosing Django for new projects much these days?
Absolutely. For what it does, Django is pretty much the best full stack Python web framework there is. It's also a great way to rapidly develop (just sticking to synchronous, which Django is best at).
One can then later consider spinning certain logic off into a separate service (e.g. in Golang), if speed is a concern with Python.
I’m not sure there’s a better full stack platform in any other language really?
Yeah, you are right. Maybe Ruby on Rails is up there? But I don't really know Ruby, and I never got into that.
Someone just sent me a job opening for an engineer to work on a startup’s PHP backend and Python “everything else”. Absolutely insane.
Where? Two of the best languages. Sign me up.
I chose Django + htmx and a small amount of Alpine.js for a full-stack software project that is currently being launched. I had zero professional experience with Django (or Python really) before starting. I was able to develop the entire application on my own, in my spare time, and had time left over to also handle infrastructure and devops myself.
I prefer Python and it's web frameworks over Typescript/React because there is a lot more stability and lot less "framework-of-the-week"-itis to contend with. It's much easier to reason about Django code than any React project I've worked on professionally. IMO when you don't have a firehose of money aimed at you, then Python is the way to go
Yes, you can do the whole web app (or web site if you will) like that without any complicated dependencies. This will be great if you touch the project only now and again e.g. the typical side project that you want to still work in the year 2030 without major changes.
Yet the approach also scales up to enterprise grade project, leveraging DRF, Django-cotton and so on (and htmx).
Yes. Still one of the best batteries included web frameworks for creating anything that's more of a website (e.g. E-Commerce) than a web app (e.g. Photoshop). No, you don't need NextJs and friends for everything ;)
What would be the same that is for a statically typed language?
Play framework with Java or Scala is similar.
Is Groovy/Grails still popular? I also remember Groovy++ but I believe its features were incorporated into Groovy. But maybe these are already present in modern Java?
LoL nah
Not answering your question, but MyPy might be a compromise.
I just rolled a backend using FastAPI and SQLAlchemy and it made me miss Django.
Too much other stuff going on in this app to incorporate Django, but it's still way ahead of the curve compared to bringing together independent micro frameworks.
It's brave trying FastAPI if you haven't tried it before. Going async is going to be quite different and you need to be more careful when designing your API. Most people will never need it.
This is why most folks just needing a plain Python API without anything else, they usually go for Flask, which is vastly simpler. For a more complete web app or site, I would recommend Django.
Out of naive curiosity of considering your first stack vs. Django: What makes Django so way ahead of the curve?
The ORM is so so so much better designed that SQLAlchemy v2. Performing queries, joins, executing in transactions all feels clean and concise. The latter feels dated and I find it hard to believe there's not a widely accepted replacement yet.
In terms of views, route configuration and Django's class-based views are sorely missed when using FastAPI. The dependency pattern is janky and if you follow the recommended pattern of defining your routes in decorators it's not obvious where your URL structure is even coming from.
We could probably do with a "SQLAlchemy for Django users" article. SQLAlchemy is much more powerful and flexible than Django. After using SQLAlchemy it's hard to even consider an active record style ORM like Django an ORM at all. SQLAlchemy can truly map relational data onto objects and uses the unit of work pattern to coordinate updates. Django just feels like writing raw SQL but in nicer Python syntax. The details of relational models leak directly into the business logic and there isn't really much you can do about it. In short, SQLAlchemy is a different beast. If all you need is Django then you're probably only doing CRUD and you should just use Django.
The problem with sqla is that you need to deal with the session object all the time. Doing it right is not that easy, and never as concise.
SQLA is cleaner and more powerful, but when you just need CRUD, django wins.
Django definitely wins at CRUD because that's what it is. It calls itself a web framework but really it's a web-crud-app framework. Flask and Pyramid are web frameworks.
Dealing with the session object seems a small price to pay for the flexibility in architecture that it gets you.
Hmmm any specific syntax examples of pain points in Sqlalchemy? Having used both, they feel similar to me so I’d love your view!
`from sqlalchemy.orm import and_`
`options(selectinload(...))`
to name a couple goofy ones
Sqlalchemy definitely is more verbose.
django-ninja will give you a fastapi like experience without the hassle
All the time.
1. Very easy to find developers for. Python developers are everywhere, and even if they haven't worked with Django, it's incredibly easy to learn.
2. Simple stuff is ridiculously fast, thanks to the excellent ORM and (to my knowledge fairly unique) admin.
3. It changes surprisingly little over time, pretty easy to maintain.
My only criticism is that die-hard django devs constantly brush aside the admin and can't stop telling people not to use it. I think it's a huge mistake.
It's extremely well-designed and extensible, there is no reason to reinvent the wheel when so much time and effort has been put into it.
They will complain of things like "eventually you will have to start over with a custom solution anyway"... but whatever gripes they have, could just be put into improving the admin to make it better at whatever they're worried about.
Personally I've not run into something I couldn't make work in the admin without having to start over. My own usecases have been CRUD for backoffice users/management and I've had great success with that at several different companies over the last ~15 years.
People will say "it's only for admins you trust" yet it has very extensive permissions and form/model validation systems heavily used in the admin and elsewhere, and they are easily extensible.
I use the heck out of the admin. I wouldn't say I'm super experienced with Django, but my solution to users being overwhelmed by doing things there is to build a bit of extra UI just for the stuff they need to do, more Rails style. Meaning: I don't go into fighting too much with the admin when what it can do out of the box is not feasible for some. But even if the admin ends up being used only by devs and some trained folks, I think it has amazing utility.
Absolutely!
I've been saying the same thing for decades (checks calendar - almost literally!)
The admin is at least 75% of why I choose Django over another framework.
LLMs are experts at Django, as there's 20 years of training data on it as well as just being written in the world's most popular language. LLMs can pump out full featured Django sites like anything.
I don't know why anyone would use any other framework.
We've used (and continue to use) Django for bespoke applications for a decade and a half now. It continues to be the most well-supported, well-governed, well-documented, batteries-included, extensible web framework of all the ones we've tried. Finding developers with experience using it (or upskilling them) is easy. As a choice of web technology, it's one of those that we've never regretted investing in.
For a complete solution requiring many traditional high-level components like templating, forms, etc, then yes, clearly Django. But for something looking more like a REST API, with auto-generated documentation, I would nowadays seriously consider FastAPI, which, when used with its typed Pydantic integration, provides a very powerful solution with very little code.
Django Ninja?
Works great, I've been using it in production for a few years. DRF was one of my least favorite bits of the Django world and Ninja has been an excellent alternative.
I still love Django for greenfield projects because it eliminates many decision points that take time and consideration but don't really add value to a pre-launch product.
Not in my org. Though we did choose it for _one_ new project recently, mostly because we re-used some code from another Django project we had, and we wanted to lean on some readily available functionality from jazzband libs.
We have a few FastAPI services, but are mostly moving away from Python for projects > 1kloc.
What are you moving towards? Node/TS? Golang?
next.js + TS for front ends, C# for pure back-end services. Golang was a close second but we have some team experience with C# and like it's lack of ecosystem fragmentation, as that was another gripe we had with Python. The Dapper "micro-orm" is also refreshing after years of struggling to make Django's ORM do the right thing.
Why moving away from Python at that threshold?
The threshold is arbitrary, and likely higher in reality. But we found that we want something with more sound type checking and mypy has lots of rough edges. The Python ecosystem has a lot of catching up to do here.
You bet. Still the easiest (IMO) for websites, perhaps of any language.
It easy but having separate app spaces by default instead of just one like Laravel makes it slightly harder for just a website case.
You can use a single app, and it is probably the best way to go for the majority of projects - definitely the case for simple ones.
Concur. The multiple app paradigm doesn't fit any site I've built in Django. I make one called main.
Idk if it's best practice, but I usually like to make apps similar to components, where I have an app for accounts which handles user accounts, and a files app which handles all the dimension and fact tables around user uploads, and a social app for social features, etc.
It makes it easy to compartmentalize the business logic in terms of module imports.
This sounds similar to a modular monolith design. But you have to be careful not to directly import things between apps and especially not to make foreign keys between the models of different apps. We ended up doing that and just wishing it was one big app.
Modular monolith is a good idea and if you want to do it in Django then make small apps that just expose services to each other (ie. high-level business functions). Then have a completely separate app just for the UI that uses those services.
Yeah. I was thinking a modular monolith since it's a django project, and I think that's django's sweet spot since it comes with so many things bundled.
For a true modular design I'd probably step away from django to a less comprehensive framework or just write in golang.
Have you tried the monolithic composite approach instead?
I haven't heard of this, and my searches are failing for an exact hit. Do you have a link or can you give a brief explanation please?
It is quite vanguard technology.
A composite is a structure of many pieces that work together.
Thus, a composite monolith is this arrangement of components in a way that they work together as a monolith. Separate modules, but working together as a single thing.
That's the same as what people are calling a modular monolith, no? The key thing with composition is the bits sit next to each other, there's no inheritance or dependencies between the components. In my idea you'd have a bunch of "pure service" apps that are focused on individual areas of the business or processes. Then you'd have one or more UI apps (either user-facing HTML or APIs) that compose those services to do what needs to be done, e.g. a "create order" endpoint might compose the create order service from fulfilment and the send notification service from comms to put an order on the fulfilment backlog and send notifications. Is this what you had in mind?
That sounds to me like good old "replaceable parts oriented programming".
Replaceable parts can have inheritance, but often what is good about them is the composability. Each part can connect to each other in different ways. We call these boundaries "connectors".
It's quite a fascinating tech.
If it's the kind of project that is going to run against one PostgreSQL database then I'd probably start a new project with Django just for its database migration support. That doesn't mean everything in the project has to be Django.
Is pretty equivalent to alembic autogenerate, no?
Didn’t Meta build Threads.com on Django?
(Since Threads was based on the IG tech stack, and IG is a modified Django stack)
That's sort of like asking if people choose Python for new projects these days.
just did, and I really like it.
yes
Just like Python itself, unfortunately yes.
Would love to hear an honest discussion of why Django and/or Python is a bad solution for any given problem. Is it because they are old technologies? Do they lack support for something in particular? Are they too expressive/not expressive enough?
(Love django in spite of Python here)
- Imports are a mess - No control of mutation in function signatures, and in general it's still a surprise when things mutate - Slow - Types and enums have room for improvement
You just need to work around those and get used to it. Then you can build nice Python projects that keep growing - but Django really helps you do that. Python however is always going to be "10-100x" slower than something like an API written in Go, Rust and so on. That's fine in most cases.
Because assembly language or if you must go higher level, fortran exist and all the 10x coding intergalactic scalers say everything else is bad.
Thanks for the summary. Looking forward to the videos becoming available.
> I talked to this speaker afterward, and asked him how they did nested modals + updating widgets in a form after creating a new object in a nested modal. He showed me how he did it, I've been trying to figure this out for 8 months!
Do share!
It's easy to get something quick working with HTMX and Django, but if you want robust UI tests that actually test what happens when users click stuff, don't you need to use something like Playwright? This can be pretty heavy, slow and flaky, compared to regular Django tests?
I find with HTMX, it can introduce a lot of edge cases to do with error handling, showing loading progress, and making sure the data on the current page is consistent when you're partially updating chunks of it. With the traditional clunky full-page-refresh Django way, you avoid a lot of this.
> Always use a BigInt (64 bits) or UUID for primary keys.
Use bigint, never UUID. UUIDs are massive (2x a bigint) and now your DBMS has to copy that enormous value to every side of a relation.
It will bloat your table and indexes 2x for no good reason whatsoever.
Never use UUIDs as your primary keys.
> Never use UUIDs as your primary keys.
This seems like terrible advice.
For the vast, vast, vast majority of people, if you don't have an obvious primary key, choosing UUIDv7 is going to be an absolute no-brainer choice that causes the least amount of grief.
Which of these is an amateur most likely to hit: crash caused by having too small a primary key and hitting the limit, slowdowns caused by having a primary key that is effectively unsortable (totally random), contention slowdowns caused by having a primary key that needs a lock (incrementing key), or slowdowns caused by having a key that is 16 bytes instead of 8?
Of all those issues, the slowdown from a 16 byte key is by far the least likely to be an issue. If you reach the point where that is an issue in your business, you've moved off of being a startup and you need to cough up real money and do real engineering on your database schemas.
The problem is that companies tend to only hire DB expertise when things are dire, and then, the dev teams inevitably are resistant to change.
You can monitor and predict the growth rate of a table; if you don’t know you’re going to hit the limit of an INT well in advance, you have no one to blame but yourself.
Re: auto-incrementing locks, I have never once observed that to be a source of contention. Most DBs are around 98/2% read/write. If you happen to have an extremely INSERT-heavy workload, then by all means, consider alternatives, like interleaved batches or whatever. It does not matter for most places.
I agree that UUIDv7 is miles better than v4, but you’re still storing far more data than is probably necessary. And re: 16 bytes, MySQL annoyingly doesn’t natively have a UUID type, and most people don’t seem to know about casting it to binary and storing it as BINARY(16), so instead you get a 36-byte PK. The worst.
> contention slowdowns caused by having a primary key that needs a lock (incrementing key)
This kind of problem only exists in unsophisticated databases like SQLite. Postgres reserves whole ranges of IDs at once so there is never any contention for the next ID in a serial sequence.
I think you’re thinking out the cache property of a sequence, but it defaults to 1 (not generating ranges at once). However, Postgres only needs a lightweight lock on the sequence object, since it’s separate from the table itself.
MySQL does need a special kind of table-level lock for its auto-incrementing values, but it has fairly sophisticated logic as of 8.0 as to when and how that lock is taken. IME, you’ll probably hit some other bottleneck before you experience auto-inc lock contention.
I was in the never-UUID camp, but have been converted. Of course depends on how much do you depend on your PKs for speed, but using UUIDs has a a great benefit in that you can create a unique key without a visit to the DB, and that can enormously simplify your app logic.
I’ve never understood this argument. In every RDBMS I’m aware of, you can either get the full row you just inserted sent back (RETURNING clause in Postgres, MariaDB, and new-ish versions of SQLite), and even in MySQL, you can access the last auto-incrementing id generated from the cursor used to run the query.
Now imagine that storing the complete model is the last thing you do in a business transaction. So the workflow is something like 'user enters some data, then over the course of the next minutes adds more data, the system contacts various remote services that too can take long time to respond, the user can even park the whole transaction for the day and restore it later', but you still want to have an unique ID identifying this dataset for logging etc. There is nothing you can insert at the start (it won't satisfy the constraints and is also completely useless). So you can either create a synthetic ID at the start but it won't be the real ID when you finally store the dataset. Or you can just generate an UUID anywhere anytime and it will be a real ID of the dataset forever.
So have a pending table with id, user_id, created_at, and index the latter two as a composite key. SELECT id FROM pending WHERE user_id = ? ORDER BY created_at DESC LIMIT 1.
Preferably delete the row once it's been permanently stored.
Keeping an actual transaction open for that long is asking for contention, and the idea of having data hanging around ephemerally in memory also seems like a terrible idea – what happens if the server fails, the pod dies, etc.?
> So have a pending table
With UUIDs there is absolutely no need for that.
> Keeping an actual transaction open for that long is asking for contention
Yes, which is why this is not an actual db transaction, it's a business transaction as mentioned.
> and the idea of having data hanging around ephemerally
The data is not ephemeral of course. But also the mapping between business transaction and model is not 1:1, so while there is some use for the transaction ID, it can't be used to identify a particular model.
> With UUIDs there is absolutely no need for that.
Except now (assuming it’s the PK) you’ve added the additional overhead of a UUID PK, which depending on the version and your RDBMS vendor, can be massive. If it’s a non-prime column, I have far fewer issues with them.
> The data is not ephemeral of course.
I may be misunderstanding, but if it isn’t persisted to disk (which generally means a DB), then it should not be seen as durable.
The whole content of the business transaction is persisted as a blob until it's "done" from the business perspective. After that, entities are saved(,updated,deleted) to their respective tables, and the blob is deleted. This gives the users great flexibility during their workflows.
Yes the overhead of UUIDs was something I mentioned already. For us it absolutely makes sense to use them, we don't anticipate to have hundreds of millions of records in our tables.
I also do that for convenience. It helps a lot in many cases. In other cases I might have tables that may grow into the millions of rows (or hundreds of millions), then I'd absolutely not use UUID PK's for those particular tables. And I'd also shard them across schemas or multiple DBs.
>Use bigint, never UUID. UUIDs are massive (2x a bigint) and now your DBMS has to copy that enormous value to every side of a relation.
"enormous value" = 128 bits (compared to 64 bits)
In the worst case this causes your m2m table to double, but I doubt this has a significant impact on the overall size of the DB.
The concern isn’t the sign of the db on disc but doubling the size of all the indexes in memory
Or if it’s MySQL, the PK is implicitly copied into every secondary index. Adds up quickly.
Wow did not know that. MySQL has tons of hidden behavior.
So does Postgres to an extent, but in general, MySQL has more hidden edge cases, and Postgres has more hidden maintenance requirements.
When you have a few million rows, no. When you have hundreds of millions or billions of rows, yes, it matters very much.
In many-to-many tables, the per-row overhead of the DBM usually weighs much more than the actual column data.
If you don't have a natural primary key (the usual use case for UUIDs in distributed systems such that you can have a unique value) how do you handle that with bigints? Do you just use a random value and hope for no collisions?
You use a regular bigint/bigserial for internal table relations and a UUID as an application-level identifier and natural key.
Wouldn't you just have an autoincrementing bigint as a surrogate key in your dimension table?
Or you could preload a table of autoincremented bigints and then atomically grab the next value from there where you need a surrogate key like in a distributed system with no natural pk.
Yes, if you have one database. For a distributed system though with many databases sharing data, I don't see a way around a UUID unless collisions (the random approach) are not costly.
Peel off a few bits at one end, and assign a value per instance.
Yes. This is the way so long as you can guarantee you wont grow past the bits.
Otherwise you can still use the pregenerated autoincrements. You just need to check out blocks of values for each node in your distributed system from the central source before you would need them:
N1 requests 100k values, N2 requests 100k values, etc. Then when you've allocated some amount, say 66%, request another chunk. That eay you have time to recover from a central manager going offline before it's critical.
I have no problem with using uuids but there are ways around it if you want to stick with integers.
And assuming we're not talking v7 UUIDs.. your indexes are gonna have objects you might commonly fetch together randomly spread everywhere.
But if you use sequential integers as primary key, you are leaking the cardinality of your table to your users / competitors / public, which can be problematic.
I made a ton of money because of this mistake.
Twice now I was called to fix UUIDs making systems crawl to stop.
People underestimate how important efficient indexes are on relational databases because replacing autoincrement INTs with UUIDs works well enough for small databases, until it doesn't.
My gripe against UUIDs is not even performance. It's debugging.
Much easier to memorize and type user_id = 234111 than user_id = '019686ea-a139-76a5-9074-28de2c8d486d'
Is it really enormous? bigint vs UUID is similar to talking about self-hosting vs cloud to stakeholders. Which one has bigger risk of collision? Is the size difference material to the operations? Then go with the less risky one.
You shouldn't be using BIGINT for random identifiers so collision isn't a concern - this is just to future proof against hitting the 2^31 limit on a regular INT primary key.
Wasn't UUIDs default go to types for primary keys in the .NET/SQL Server world even 20 years ago?
I don't think so.
Even Microsoft default sample database schema uses INT ids.
I’ll go further; don’t automatically default to a BIGINT. Put some thought into your tables. Is it a table of users, where each has one row? You almost certainly won’t even need an INT, but you definitely won’t need a BIGINT. Is it a table of customer orders? You might need an INT, and you can monitor and even predict the growth rate. Did you hit 1 billion? Great, you have plenty of time for an online conversion to BIGINT, with a tool like gh-ost.
I work for a company that deals with very large numbers of users. We recently had a major project because our users table ran out of ints and had to be upgraded to bigint. At this scale that's harder than it sounds.
So your advice that you DEFINITELY won't need a BIGINT, well, that decision can come back to bite you if you're successful enough.
(You're probably thinking there's no way we have over 2 billion users and that's true, but it's also a bad assumption that one user row perfectly corresponds to one registered user. Assumptions like that can and do change.)
I’m not saying it’s not an undertaking if you’ve never done it, but there are plenty of tools for MySQL and Postgres (I assume others as well) to do zero-downtime online schema changes like that. If you’ve backed yourself into a corner by also nearly running out of disk space, then yes, you’ll have a large headache on your hands.
Also, protip for anyone using MySQL, you should take advantage of its UNSIGNED INT types. 2^32-1 is quite a bit; it’s also very handy for smaller lookup tables where you need a bit more than 2^7 (TINYINT).
> but it's also a bad assumption that one user row perfectly corresponds to one registered user. Assumptions like that can and do change.
There can be dupes and the like, yes, but if at some point the Customer table morphed into a Customer+CustomerAttribute table, for example, I’d argue you have a data modeling problem.
Gosh this debate again.
I’ll be 110 years old telling my great-grandchildren about how we used integers for primary keys, until reason arrived and we started using uuids.
And they’ll be like, “you weren’t one of those anti-vaxxers were you?”
It looks like htmx is popular in the Django community. Is there any background story that made this? (Context: Just picked Django for a hobby project. Don't know much about Webdev trend beyond, like, what are talked about on the HN top page.)
Server side template rendering is popular already and well supported in Django ecosystem
Some of the talks look really interesting — are there any YouTube videos linked? I couldn’t find those.
The conference coordinators said they would be released in about a month, so I will update the post once they are released! I am really excited to watch them again. Amazingly informative stuff.
[dead]