One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
This is an incomplete and quite superficial view of what is going on out there, in my opinion. I've worked on plenty of projects where the assumption was that since the round-trip to the server is going to take almost 100ms that'll dwarf anything that's going to happen on the server itself, justifying poor choices that lead to potentially adding a whopping 100ms onto that number. These numbers only get larger with a larger perceived "Nothing we can do about it" budget as well, programmers often feel justified in doing just about anything once round-trip time grows, not understanding that they're just adding to an already existing problem.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
I don't think it's superficial, but two problems adding up. Previous poster is talking about general latency issues because everything is networked and potentially quite far away.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
I have a new laptop with a rtx 5090. Opening any GL context takes more than half a second. There's tons of things that can be optimized and are pretty far from web.
I assume you're running proprietary drivers? Because I've never experienced anything like that on mesa. Launching an app that opens a window with a gl or vk context is so fast on my almost 10 year old hardware that it's nearly imperceptible.
There must be some quirk of whatever combination of software packages are installed there, but the proprietary drivers are not the culprit, at least not alone (i.e. there could be some interaction with other software packages with which I have little experience, like Gnome).
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
So the usual self inflicted misconfiguration then.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
No idea about your setup but that's probably a fixable driver issue. My desktop with an RTX 40-series GPU takes _maybe_ 4 frames to create an OpenGL context.
I agree and see this as a side effect of a subtler thing. As people shall be replacable, software is designed and subtly mimics the organization's communication patterns. Features are cut up into the tiniest pieces with clear separation from the start (at least its claimed), and over time whatever change or feature seems overly complicated, won't be done or won't be done in a sane manner because its uneconomical.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn - you stick your hand in there and it’ll chop it off, the end. You don’t think "oh, the lawnmower hates me" – lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle."
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
I could go on about this all day. Best meme is when you estimate stories for storypoints and then they haggle with you without changing the content of the story. Meanwhile points mean actual time. Then on another side if you haggled down points, you then have more slots for more points. So you end up getting assigned 3x work for the same time. The blame game starts when the sprints elapse lmao
The only ways out of these situations are: a manager that understands what you’re doing; knowing that these “story points” are just indicative and there’s no broken incentive towards gaming them.
Yes. Apple Music is the most egregious example of this. It could be ridiculously fast on your pocket supercomputer but the moment a web request gets fired off from stumbling blindly across the field-of-dung user interface, bam, you’re done. Especially if your network connection isn’t great at that time.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
Sonos’ app is another example of this. As a very brief tl;dr if anyone isn’t aware of it, Sonos is a wireless speaker company that can group speakers in different rooms into zones, so you can have different music playing in different rooms, or all the same, or at different volumes, etc. The quality isn’t going to blow away audiophiles, but IMO they’re legitimately great.
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
This is kind of a shallow assessment of "slowness". Slowness is a feeling, not a fact. Network is slow as a rule relative to other parts of the stack, but it is not usually what contributes to the feeling that your software is slow. It takes a good amount of incompetence and arrogance to cultivate that particular experience.
Fair, it's shallow because I was succinct however, I do understand the problem space more in depth than this.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
True, dev environments are fast. One dev implements a wrapper with roundtrips, another integrates it into a UI and no-one stops to think if it'll have terrible lag in practise. They don't notice so even if there's a ticket it'll starve and end up WONTFIX.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
The UI threading model is usually not the problem.
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
Networks are much faster than you think, it's networked software that tends to be slow. 10 GbE is now table stakes, you should fire any vendor who can't offer it. I certainly don't require your internet connection to be 10Gbps, or all your desktop machines, but your internal server network should be if you're building a new one in 2026, because there's no excuse not to any more.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
The majority of companies are building their stuff entirely in the cloud, where network speed scales with the instance size. I have had to explain this to multiple engineers at multiple companies, who are surprised to learn that network bandwidth isn’t unlimited.
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
It isn’t, but at the same time, smart hackers were working with highly constrained PC hardware in the 1980s and early 1990s and were cranking surprisingly good performance out of it. Folklore.org has plenty of stories about it, and John Carmack’s early career history is very impressive. We mustn’t forget the demo scene hackers either.
Recently I tried an approach of "just write the f*** code" instead of using infinite abstractions on a new UI side project. So for instance when you scroll, it shifts the pixels and just redraws the new exposed area. This is how stuff worked in the 90s. And it's blazing fast and uses very little memory. It's easy to mess up redraw code like that - in my case, when the window goes past the screen border and the pixels to copy aren't there. That's a bug we also had a lot of in the 90s.
That's why phones and windows use animations. You can also use intersitials related to the product you sell. Users are usually fine seeing many changes on the screen quickly because it gives the impression that stuff is happening on the background. For example in the interstitial, use an animation that takes up a small portion of the screen and not just a simple spinner or loading icon. Something more complicated with 2 or more things moving or changing at once.
If your app absolutely must rely on the cloud for every one of its interactions, then fine. If not, you're just applying band-aids to a problem of your own making. Many apps could easily be local only, or local first. If you're not constantly accessing the network for information which could be stored locally, then you don't need to hide your app's slowness behind animations.
Canva replaced PowerPoint. Canva is cloud based and PowerPoint is not. There's so many apps that are cloud only so that corporate it no longer has to manage installations and users don't need to ask IT for permission anymore. I think SaaS doesn't really work without the cloud, you could technically do what adobe does but why bother with app distribution and windows' quirks. Cloud based web apps are write once, run anywhere come true with no installation required. Cloud based is more convenient for the user and the developer, at the cost of app runtime speed
I like this example. A lot of people would prefer to use PowerPoint because they can still use their files and templates they made last year even if Microsoft doubles the price of Powerpoint, removes features, discontinues the product, or goes bankrupt.
Just nerds I think. I'm not seeing any evidence that people who want freedom from the cloud make up any sizeable market segment. Most corporations actually prefer the opposite - they prefer a monthly fee and continuous silent updates.
My bad, I mean the interstitial. Like it should stay for at least a second to not make it jarring. People believe computers need to think so you can't make things too fast either. Not the interstitial and not the app either, to the point you sometimes have to deliberately slow down the app, add latency to make people trust it because it "gives the computer time to think"
Power users absolutely hate this mindset. It’s resulted in Apple animations taking seconds for something that was done before the animation even started.
I don't doubt that was an original justification, but most of what I see are not for this purpose. Most of the time they're just adding unnecessary delay and CPU cycles.
I think a lot of the time when people say the network is slow. They really mean their backend is slow.
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
I don't know how hard it is. But I can certainly say there is no business inscentive for it.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
> With a fast backend ~1-5ms response times (not even that fast).
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
More to the point it's that the server has to retrieve and massage data from several docker services to retrieve the full context needed to process the request
This is why my most recent website does its server-side rendering on the client. I'm not even being that sarcastic. We stream a subset of the user's data (1 mb at most) in the background and have a WASM client that has the exact same server views. On SPA events, the WASM blob intercepts a lot of requests and can instantly render. Makes a laggy connection feel pretty quick.
Are you counting mainframes with dumb terminals then smart terminals then PCs, early SSR websites then web 2.0, the multiseat/thinclient craze of the 2000s then computers becoming cheap enough to stick them to the back of monitors?
To dogfood this, rent a VPS in Australia and put a test environment there. Should be about 200ms ping or a bit higher. How it runs for you is how Australian users are seeing your site, even if their last mile connection is fiber.
> If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
Yes, but my point is that it's an awfully shitty fallback if you need to wait 30 seconds for something to time out first.
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
There are CDNs everywhere, true. It is not true that everyone deploys to them all, though. Also it dodges that the networks available to everyone are still not equal.
I think that "almost everyone" has a multi region cdn, but fewer have multi region application deployment, or cdn workers handling a significant portion of the application logic. My experience here may be incorrect or not generalizable, but I've rarely seen web apps that are slow due to latency loading static resources, but I've often seen slowness from high latency of the API calls and due to large static payloads.
That only helps the assets, does nothing when they run their actual backend servers in one of the US aws regions and your traffic has to traverse the planet anyway.
UI blocking can make sense in some situations; otherwise things "happen" suddenly that are unexpected.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
Shotout to PowerSync for enabling companies to go in the opposite direction and build offline-first apps. My company is a (production) customer, and we recommend it.
This is true, but it is an entirely different problem in a different place to what the article is talking about.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks, using a markov chain, and conditioned on those pages being low bandwidth to pre-load.
So Next.JS actually pre-fetches links when they move into the viewport or you hover over it. It's interesting, but then you get wasted battery on mobile while on bad networks. The world is full of tradeoffs. Tech workers tend to want to consume more battery and data to be faster. Other people want to do less work.
segor.de goes one better and just downloads the entire catalogue when you first open it. About 2MB decompressed. Clicking and even searching is instant because it's all fully client side until you place an order.
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
(Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.)
> pre-caches frequent button clicks, using a markov chain
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
The problem isn't in preloading, it's in how much data needs to be sent while quite probably most of the data could be either fetched on startup in an efficient format and rendered natively, or is completely unnecessary in the first place (telemetry, ads).
If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
>but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
And the first thing webdevs did once this became widely available, is change their apps to cache-bust their code; between that, and the short release periods in webshit ecosystem in general, and security and privacy considerations messing up things as usual, the promise of users mostly hitting just local cache with any marginal request, never materialized.
> If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
This exist but the downside is that it uses much more of your bandwidth and client resources (probably not matter in many cases but it does if on a phone in a country with bad connection) and your server resources (if not mostly static content)
Oh god, don't give them ideas. All ML is in-cloud AI now. I dread the day everything around my mouse movements needs to get tokenized and vibed into the ClosedAI cloud before my buttons start working again.
there is different scales at which software is slow. this os one and definitely a pain in the ass. everything being online for no good reason other than to harvest user data. which it really turns out to be every time. (for good or bad purpose).
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
I’ve been playing with this with software that needs to work with agents and also without internet at all (we’re serving construction projects that have limited access)
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
Surely the biggest cause of slowness is doing more stuff.
We don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
A browser isn't really a program any more but a platform for running other programs. Like how javaw.exe is really Minecraft, firefox.exe is really YouTube. Go to about:processes to find more detail.
The point is, if I want to edit a text file, I don't need to use eclipse. I can use something that hasn't added loads of features. I don't need to use whatever Adobe product, I can use some paint app.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
No there's a simple reasons actually: we have surrendered entirely to web based apps & the resulting ad bloat.
You used to buy a computer, install an OS, buy apps and run them. Neither the OS nor the apps didn't suddenly degraded themselves with "free update" that were developed targeting newer & newer machines. You bought a new computer when new software you wanted to buy wouldn't run fast enough on it, OR the computer died.
Nowadays, you buy a new computer when the same software you've been running for years or the OS itself that demands to be updated "free" makes your machine intolerably slow.
Mine is for Java and is intended to be production grade. The first goal is to guarantee linear-time behavior to prevent ReDoS attacks. My collaborator and I have recently been optimizing it to try to surpass native RE2 in performance.
It turns out optimizations are incredibly well suited for an agentic loop. You've got concrete acceptance criteria (must show a meaninging improvement on a benchmark case, must pass tests). The agent is really, really good at using tools like a profiler and disassembler, better than I am (and I've been doing this for 20 years). It also papers over things that would take me a while to learn, like how the in-incubation Vector (SIMD) API works in Java. I understand the concept but it would take me a while to understand Java's implementation. The agent can just read the docs and go.
The key is creating a good benchmark suite and ensuring the agent doesn't ship optimizations that are too narrow or too focused on the benchmark cases. You also need a really strong test suite to make sure you're not regressing correctness. SafeRE has billions of tests; a subset of several million run on CI, and the others run on-demand.
One little thing I spotted is you use Boyer Moore Horspool for fast literal search. This is actually not linear in the worst case, although it is almost always sublinear. Worst case would be a literal composed of the same character searching a text of the same character, where it becomes quadratic.
You can actually search strings with character classes using Horspool if you want to, and I have some enhancements to basic Horspool which could maybe help. My library, byteseek [1], implements these.
I also have a much faster algorithm, HashChain [2] which also has a guaranteed linear time version. This was published in the Symposium for Experimental Algorithmics in 2024.
Reading a bit closer, you seem to track how much work is being done in Horspool on each character comparison, and then fall back to the linear KMP if the work budget is exhausted.
This will first massively slow down the Horspool scan, and then once you have done all that work, you rescan it all from the start with KMP if it is doing too much.
One little fix might be to only add to the work counter and compare it outside of the main character comparison loop.
But it would be better to use the linear version of Hashchain. It also uses KMP to make it linear, but it is fully integrated and you would not need to track the work or restart scanning at all. And its a lot faster than Horspool anyway!
Not OP, but I went through this last week. I (ok, codex) optimized a hot path in some Java code from ~350ms to ~60ms, which made a substantial difference in "is this whole business going to work".
My Java profiling knowledge is... let's call it "antique". I was really not looking forward to ramping back up for this work. Turns out, I didn't have to do any of it. The LLM chose the tools (flight recorder) and even built a JMH (also new to me) harness to experiment with different algorithms.
About half of the optimizations were things that I would have figured out on my own; the other half were definitely "wow" moments.
The whole thing was done in a couple hours, with just a few back-and-forths. Sans AI, it would have taken a week, with nowhere near the same gain. I'm impressed.
"Figure out how to make this process fast" is really a perfect activity for LLMs. And the prompt doesn't really have to be much more sophisticated than that.
JMH as the framework to write microbenchmarks. It takes care of dealing with JIT warmup, etc. It's the standard way to write rigorous Java microbenchmarks.
async-profiler (https://github.com/async-profiler/async-profiler) for profiling. Java has a problem where many profilers are based on safepoints, which are biased toward particular program points. async-profiler is not biased in this way.
Java Flight Recorder for memory allocation data.
One thing I've observed in all of this is that it's really useful to have expertise in the programming language and ecosystem you're writing in, otherwise it's all Greek to you and you can't really guide the agent to do the right thing. I have opinions about e.g. profilers and I can point the agent to one that I think is more accurate than other options.
This, 100%. I’ve rewritten some hydrology code in rust using codex sol to first a) profile and create comprehensive tests of the python, b) create full benchmark suites, c) create full scientific benchmark suites, then d) port to rust using a few different techniques.
It works, it’s at least 5x faster, sometimes much more, and memory use is like 10x less and even less in cases where lots of map tiles are involved.
This shit rules.
What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
> What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
Why you even start with Python at this point? Just write the Rust version straight up instead of porting things?
Personally I used to use dynamic languages for most things, because development and maintenance is so much faster and easier, particularly for larger projects (granted you know how to work with those sort of languages), but now when the LLM writes most of the code, I'm able to work as fast with Rust as with I used to be able to do with Clojure or other dynamic languages.
It's becoming increasingly evident this is not how things are going to shake out. Even without frontier models, running Qwen 3.8 27B has demonstrated for me and others a "good enough" competency at general programming. Additionally, large open weight models have proven themselves as viable alternatives and we're still in the early years of dedicated hardware
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
> - The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
That's fair for a well-scoped subroutine: what I meant is that if you ask an agent to write a compiler and let it rip for a few days, you are going to be spending a few more days correcting the default behaviors in the distribution, which often do not tend towards hardware-oriented design.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
Yes, say "Build a compiler" will require you to clean up stuff if you leave the agent for days, but not because of the LLM or the quality of the tool, but because you hardly specified anything, so of course it's gonna make assumptions you need to correct.
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
I don't know if it's the first and it certainly isn't state of the art at this point but I think karpathy/autoresearch is a quintessential starting point.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
> It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why.
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
yep, i think the real LLM superpower is knowing that something has been done before and having access to the code that did it. so much of even novel software includes bits and pieces that have well-optimised existing solutions, and the bot knows those solutions a lot better than i do, and can even pattern match them from the general shape of the problem.
Yes. It’s also excellent at reading large codebases and putting together a picture of what’s going on. I’ve been using it a lot lately to brief me on projects and design decisions. “Look at these two projects. They both solve task X. Write a report about their similarities and differences, and the tradeoffs as a result.” And then I ask followup questions. Saves a ton of time.
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
Reasons this doesn't follow:
(1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising)
(2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
"LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly."
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
I think you’re talking somewhat at cross purposes to the original article.
The points I take away are:
- Good optimization is difficult and slow work, hence expensive, but LLMs can do it so we should be able to afford it more often now.
- There’s always a risk of over-fitting to your specific problem, but if everyone is now making bespoke optimizations maybe that isn’t actually a problem.
You said:
LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
There’s probably something in that, but it can be mitigated by testing against a local benchmark. LLMs are good at iterating tirelessly and finding incremental improvements. And as noted above, it doesn’t necessarily matter if your benchmark isn’t fully general.
Just admit that you were wrong instead of this doubling down nonsense. LLMs are amazing at optimizing memory utilization. They have no problem obsessing over fitting as much data as possible onto a single cache line and micro benchmarking cache hits.
This seems like way too caustic of a reaction, OP is correct.
If you actually know what you're doing in $language, and you know how $language wants to emit the assembly or whatever, there's no huge advantage to just programming directly in assembly.
Seems like author’s main focus recently is AI and agents unsurprisingly, hence the suspicion. But it seems like he has a backgrounded in relevant fields in the past.
I think it's more about not doing unnecessary things. Like a like a saying I heard somewhere "a clever person solves a problem, a wise person avoids it". Not every good idea _needs_ to become a feature. And if you think it's that good, give users the option to turn it on/off and track that as a metric.
My contradictory proof: I have been working on an old service with tons of performance issues, from server memory bloat, client graph rendering, excessive network requests, excessive repeat rendering, memory leaks, resource leaks, etc.
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.
OpenAI has 4,500 employees and a post-money valuation of $852 billion at its last funding round. You'd think they could get someone who specialises in writing MacOS apps to write their MacOS app.
Hopefully at some point gravity pulls them back down and they reach the maturity and humility to realize that LLMs are just tools and even though they're good for many things, you can't do absolutely everything with them.
I think the "maturity and humility" phase of the LLM hype cycle is probably still 2-3 years into the future.
And that phase probably comes immediately after the "LLMs overinvestments have caused a massive global recession".
I think the software out of the big AI labs: OpenAI's ChatGPT MacOSX app, Anthropic Claude Code - is a sign of the future to come.
Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).
They absolutely do not have AI-heads with PhDs writing the apps and general infra. I've met a guy that worked at Anthropic, and he was "just an engineer", so exactly that: a top guy, but not specifically AI focused, doing regular stuff there but with infinite access to the top of the crop AI.
Now look at what software they make. It speaks for itself.
I've been using computers for 4 decades. They have gotten no faster. The nuclear plant computer system we built in 1989 had to present selected screens in 1 second. I don't think any apps I use today can do that.
It’s all incentives. I’ve worked on web projects where the people in charge cared about performance. It’s easy to get sub second speeds if you start with that goal.
I’ve also worked on projects where the people in charge added 1mb client-side mapping libraries to render a static map and ignored my push back. Those website were slow
This. The author talks about the fact that now they can try far more experimental optimizations than they could before. But they already had an architecture with efficiency in mind, and were trying optimizations, to begin with. The kind of company that ships web UIs that take 5-10s to do some action 1) don't care or aren't capable of good architecture 2) don't care or aren't capable of doing the simplest low-hanging optimizations.
At some point, probably in the 1980s, people probably decided that computers could update UI fast enough, so any additional compute power or speed has been used for other things like making it prettier or reducing development effort or time, and now ai
I've heard it before and believe it that one reason is because many of the developers are working on new maxed-out machines and network connections both at work and home, so they don't notice problems for older or cheaper ones and/or can't justify it to management.
In my case neither Andy nor Bill was involved. For that project we used Sun 3 workstations which were powered by Motorola 68020 I believe. So a more fair comparison would be that 1988 workstation against a modern Unix workstation. I bet the results would still be that a modern workstation would struggle to update the display in one second.
Well I don't know about specific examples, but it's a phenomenon that has been observed in many areas over decades, thus the funny saying.
After crossing 40 years of age, and working for a while now, I believe it's also because of politics. You might think the goal is to deliver the best possible product, but territory grabs within companies are important, and done by people that don't have enough skill other than territory grabs. E.g. look at Trump and his behavior. No skills other than having his way, and then he gets to decide.
Of the technical reason, for sure we underestimate how much faster technology gets. Fred Brooks had this example in his Mythical Man Month book, how the os/360 got the option of a disk drive instead of mag tape, but the result was worse because everyone assumed it was much faster than it really was.
This was for https://non.io, which is something I had purposefully written to be as fast as possible (hand wrote all the comopnents, didnt even use react).
I've been considering creating a skill / utility to do this based on learnings from the speedups - would others find this kind of thing useful?
Why is it so slow? Like clicking around this is a very simple site, it seems like the fade in and fade out, besides being jarring and annoying, is just adding load time.
An annoyance: clicking an image transitions the tiny image to a bigger size, then it gets replaced with a larger image file. The result is: the image slides bigger (but blurred), then immediately disappears, and reloads slowly from the top down. It's an annoying flash, and the slide to a bigger size was a waste of time.
That seriously needs some optimising. For example: on click, could the bigger image be inserted behind the small one so it's hidden; on load of the bigger image, hide the smaller image; then do the slide-bigger transition with both images together?
The problem is that there’s a tradeoff between MANY objectives, not just performance
You want code that’s fast, correct, stable, memory efficient, secure, usable, and more
The reason code might be slow is about product tradeoffs. It’s certainly easy to have an LLM optimize performance only, but what do you sacrifice? Is that an acceptable tradeoff?
Maybe slow code is a good idea if it’s the only way to get correct and secure code?
I understand now that most software is slow because of co-tenancy reasons requiring controlling resources or simply because they're safely insulated from competition. e.g. GitHub is the former: you can give yourself a git host and CI/CD system that is much higher quality by yourself since you're probably not using its social features. I think things like Apple's five-finger inward gesture are the latter. Once you could do it and start typing but nowadays it needs to render the animation etc. before keystrokes register. This software is slow because you cannot replace it in MacOS.
But all these things will change in time. Hell is other people's software.
> Apple's five-finger inward gesture are the opposite. Once you could do it and start typing but nowadays it needs to render the animation
There is no technical reason that animation should take that long.
Someone said, "good enough" and let it be visibly slow. The reasons for that could be anything, including non-performant code in dependencies, written by other people. But it just does not take much computing power by today's standards to composite code-generated animation.
Somehow, despite year after year of percentage-speed hardware improvements, there are cultural and structural reasons people ship code visibly slower than it needs to be. And faster compute appears to be irrelevant.
Perhaps the real tragedy is not the animation. It is the fact that previously keystrokes would register even if the animation had not completed and now they do not. Interaction once you've entered a new mode should be instantaneous. Instead we are faced with inputs going neither to the old app nor to the new app. In the prior world, I could do the gesture and type the letters and hit Enter and the animation would not interrupt me from getting to my destination. Now I simply <backspace> <backspace> <backspace> and suffer the ignominy of a software system that does not care about my niche needs and is nonetheless beyond my present ability to modify while retaining all else.
Every visible UI animation is visibly slow. I've used MacOS computers before, and despite having hardware that benchmarks faster, they all feel slower than my old Linux desktop. The biggest reason for this is the animations.
Making animations faster is one of the main reasons why, despite all the glass effect rendering, the 26 Apple OSes can feel faster than their previous versions. (Not always, of course.) They simply shortened the animations. The 27 versions do the same thing. Of course, this doesn't help all interactions, but it definitely improves the first impression.
Next time you ask your AI of choice to build an app, please tell them to use plain language without frameworks at all and keep modules to a minimum (ie. node express, ejs and pg, nothing else) and you will see a huuuuge difference
I like to use frameworks to steer the agent. I constrain it different primitives for different tasks in order get the shape I want without explicitly prompting too much. You can think of it as API driven development
If this is true, the frameworks will either move to the specs or, worse, remain in the models.
You can't exhaustively write down all the possible ways for software to execute, especially since many programs run forever (e.g. servers, GUIs etc)
So when you say in your spec "when the user clicks here it shows this stuff" there is the tacit expectation that this will always happen and isn't dependent on the entire execution path up to this point. You also wouldn't expect a server to just finish execution after processing a request because you forgot to write "finally, cede control back to the main loop" at the end.
These assumptions are the framework. Almost everything you read is likely written in a framework, unless you limit yourself to non-technical writing for younger readers.
You could try to formalise the framework in your specs. But you'll then end up copy and pasting these paragraphs every time you want another server program. These context frameworks will probably get names, just like today.
Or you could skip that entirely and pretend you're not using a framework, but you are. It's just that the most has already learned tons of frameworks. What worries me here is stagnation. There's no way for the model to learn new frameworks. But this might be a short term effect. Maybe software in the next few years will look quite odd to those in the future as we keep writing it in the language of frameworks we already know. Maybe they'll look back at these relics and ask "what is a view, and why did the model understand that?"
Yep, I feel like there's less of a reason to use frameworks now that you can just have it bash out only the needed stuff on its own. I've been able to make some really really tiny C tools and I'm absolutely loving it.
I somehow managed to avoid the entire SPA scene. I feel so vindicated because I am absolutely zooming with my tried and true Vanilla JS these days. I have tried React, Vue, Angular, and ever just plain TS, and I hate them all.
Basically you are telling the agent to inline those frameworks. It has access to the frameworks, their functionality, and how they implemented things, combined with countless examples on how to use those frameworks.
If only we had a sufficiently smart™ compiler, it could do the inkling, dead code elimination, and WPO across the application.
(Oh we do have that, as long as you don’t write in JavaScript…)
Fun blender fact: every .blend file contains a serialisation of the complete data model of the blender version that wrote it. That's why they have a minimum file size of about 280kB.
You could compare it to Windows vs. Linux in a sense. 3ds Max has decades of legacy plugin/SDK compatibility baggage, lots of commercial and contractual obligations with business/enterprise customers... and Blender has none of those things weighing it down.
Ha, I'm always struggling with UI performance in Blender. Undo takes ages and freezes up everything, modifiers start lagging mouse movement as soon as you have more than a handful of polygons. I shudder to imagine what Max must be like.
Though I must admit you can have an absolute shit ton of ridiculously complex objects in a scene, and it will run smoothly if none of them is in edit mode which is at least fairly impressive.
Giving an LLM a program that does a well defined thing correctly and asking it to make a faster version and a crapload of tests to show that it performs exactly the same task is a great way to burn through a bunch of tokens and getting a faster program in return.
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I have another where I'm testing how well it can iteratively rewrite the same program without regressions and successively stricter resource conditions. Instead of progressively adding lines of code to make an over engineered behemoth, starting over with the previous version as a template of functionality lets it consider the problem increasingly holistically
I would be genuinely interested to see how it evolves the same program over a large number of generations. Given a whole lot of software is more a collection of programs rather than a singular "program", I have to imagine that the outcomes would be vastly different in shape when dealing with smaller targeted, articulable-in-a-few-paragraphs programs vs many-years-of-business-domain layered programs.
> Completely agree with your closing point. Dynamic custom software, fitted to a particular workload rather than a class of workloads, seems like a very likely outcome.
I disagree with the premise that this is the desired outcome. If every piece of software is bespoke and everyone’s instance of it works slightly differently, then it’s impossible to get support or a shared knowledge of how it works. There’s no “just share the excel file”, there’s no “press the this button on the left”, there’s no “oh I use program X to solve Y” (instead you have to know what you need so the custom software can solve it, but my time in startups taught me that most users don’t know what they want or need).
Much like security, optimization is now a function of token spend—which in some ways means it's a function of care. The reason why software may continue to be slower or less secure than it could be is simply that no one cares enough to invest the time and money in improving it.
Off-topic, but Dan, in the name of all that is holy, throw a `max-width: 40em;` on your blog!
That said, performance and security considerations can be automated by a reusable harness and I think within six months it will just be built in to things like claude code and codex the same way you might tack on a performance/security section to your project's AGENTS.md.
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.
What makes large things slow is not the individual pieces of code but the way the architecture dynamically reacts to changes in the load. Sync vs async calls, buffers, parallel vs sequential processing. This includes optimizations made: do we want to wait until the call returns or do we proceed anyway and deal with the negative response later? Most developers can’t coherently (ie, with math, graphs and figures) explain, how a connection pool prevents undesirable consequences of brief spikes in incoming requests. And I’m pretty sure AI can’t either.
Most of the time, it's the network related stuff, requests that are slow.
But with AI, devs nowadays are chasing the aesthetic vibe, cool animations, forward-backward video on the background relative to the scrolls... Which worsen the performance drastically just for a cool landing page. I mean that could be cool for landing page only but some people put all those animations in the product itself too, which is a pain to use.
Good read. I liked the point that even small performance improvements can be useful if they don’t take much time to find and test. But measuring the actual impact is still important.
And now the other aspect of the trade off is token budget allocation.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
Yes I do understand what author wrote as I read the whole thing, but I argue that title is not universally true.
Like we have OLAP and OLTP, there are still batch jobs that need to happen before you can see your dashboard, we have multiple users using the same resources, there are still DB Locks when writing needed, there are "first load" issues
— heck now we have to compete for resources with even more bots than ever.
Transferring 10GB or 1TB of data still not going to be instant laws of physics still apply.
I was very much on-the-fence and somewhat sceptical with respect to AI-assisted software development and optimisation, until this recent article pushed me strongly into the "believer" side: https://news.ycombinator.com/item?id=49226923
I regularly support Windows users and when I do, I cringe at just how awful the experience is! (My daily driver for the last decade have been various flavours of linux)
High end Windows 10/11 machines (like my Alienware I9 w/ 64GB mem + RTX4090) seem a bit sluggish. Low end machines with Windows are absolutely painful to use!
I've install debian onto my fairly ancient Lenovo X201 Tablet and it is exceptionally snappy.
Just an aside. mawk 2.0 (high performance awk) has been out for about a decade. It's about 2x the performance of its predecessor mawk 1.3.4, and 4x performance of GNU awk. Practically no distro uses it, because the author Mike Brennan is not the "I maintain this code in perpetuity" guy and the mawk maintainer Thomas E. Dickey doesn't care move on from from 1.3.4.
The point is: Code needs to be maintained if it is to be relied on. I haven't heard good things about LLM's on that front.
I was thinking about this the other day, but along a slightly different path. I realised there is no need to use web technology to provide native experiences anymore. Bundling a web browser was always a way to ship native apps to multiple platforms with minimal effort, but these days prompting up an actual native client for each platform is already accessible even to small shops!
That's probably one of the biggest sources of software slowness, and we can just stop using it now, with almost no trade-off. Neat!
I can't tell if you are saying the market doesn't care if your software is slow, or quoting someone else. Either way it's objectively false and that has been known for decades. Making users wait is a surefire way to make them engage with your product less, and the more you make them wait the less they will use it.
If your boss doesn't care about your software being slow he's incompetent.
The market cares. In my experience it’s harder to make your boss care.
It’s often been hard for me to make coworkers care as well which is even more frustrating.
Unless your job really prioritizes performance and builds in benchmarking people often focus on finishing a feature as fast as possible and ignore performance (and accessibility, bugs, UI polish, etc.)
Do you think most websites are hitting 100ms? You are picking silly numbers. The real world case is more like 500ms vs 2.5s. The pingdom numbers suggest that is more like 5%. It's very common for pages to take over 5s to load. That is a huge bounce multiplier. I just did a search for "sony lens" on fb marketplace, and it took about 2s to load the page. This is a very very optimized page, it got some results at the top within 500ms, which was probably enough to fill above the fold on mobile (and thus seem like it was finished). however the broader search to fill the results took about 2s. When I worked at eBay searches took 2-3s routinely if they weren't warm. Getting to 100ms for a lot of very important ecommerce pages, which are critical in terms of bounce rate sensitivity, is a real engineering challenge that millions of dollars are spent on. Imagine if eBay could get 3% lower bounce on their search pages, delivering that would make multiple careers.
Jevon's paradox implies we're getting more fast software, and more slow software. Sturgeon’s law implies the ratio will remain the same. 90% of what ships will be slow, buggy, slop. Just as it was before LLMs.
I wonder if AI has changed the 90% rate. From my perspective it’s already increased the ratio of what I’d consider bad.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
I think this is the most accurate take on this thread. I've written super optimised code in assembly and SIMD. But only for contests never at work, because I've never had an employer that cared to optimise things that far, they just wanted cheap software that worked just good enough to get the job done. Even in places where they actually cared about quality would usually prefer adding a couple features over optimising things that much.
This makes sense for a verifiable task like a regex engine - but those are clearly defined and I doubt an LLM will have a meaningful speed increase over current human regex engines.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
Yes, simply stop using JavaScript and give me static webpages. Done. I don't give a fuck if your website have a nice effect that follows my cursor, I just think about the amount of energy, effort and time wasted on making this demo.
Sites can be fast with PHP and JavaScript. The slowness you see is mostly coming from all the frameworks too many developers "include" willy nilly with code that can only be described as bloat because 99% of it is completely unused, yet being loaded and computed by the browser on every load. That's what you're experiencing on too many websites.
Funnily enough, since most JS frameworks are utter junk, developers are better off instructing LLMs to write pure JS instead.
Fun story, I was writing some 3D graphics animations for a website, and the only dependency I allowed was three.js. It ran well, but the site was loading slightly too slow so I told the LLM to optimize. It essentially hit a hard wall and told me something along the lines of "I can't optimize further, three.js is the bottleneck due to blablabla, but you don't need it. If you want I can rewrite it with pure WebGL". I said sure and load times were cut by like 90% and framerate went up too. Staggering.
This needs to be qualified with "to the degree that you have a specification of what that software should do." The better the spec, the more leeway you can give the optimizer. A very thorough spec lets you give the LLM total free rein to run optimization passes over your codebase.
> performance work that used to require a person or team that had a rare set of skills can be done by anyone who can type a few sentences
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
The essence of the problem is that the LLM does exactly what you tell it. In the hands of a skilled engineer who understands the project, this is a superpower. In the hands of a junior, this is dangerous... Because it will do things that are insecure; it doesn't push back on a user request; it just goes ahead and tries to make it work, even if it's a bad idea and the problem is best solved in a different way.
> This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
I think for well defined problems, it does a really good job, but if you're starting from scratch and don't know what to ask, it can take you on a wild goose chase in an extremely sub-optimal direction and it never backtracks from that bad design.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
Sure there is. Nobody cares about quality, mvp and get to market first is what counts. AI means there is more reason for software to be slow, because it creates code more quickly and with poorer quality.
Funny they mention demoscene as an example. I've tried to get various LLMs to explain or modify my demos and they utterly fail. At the extreme end this style of coding requires you to hold all understanding of what is happening end to end in your head, this x-ray vision allows you to reason and think about fundamentals without getting held back by abstractions.
LLMs operate in abstractions. "Reasoning" is bolted on.
Incentive has shifted. It’s now more important to worry about “developer productivity” and churning out as much code as quickly as possible than it is to spend time optimizing that code.
I recently built a piece of code which downloads a bulk set of data, indexes it for search, and then serves a pretty web UI on top of this with the help of some AIs. Normally I would have reached for sstables, sqlite, etc. This time, because the lookup patterns actually would not have been too efficient on sstables and SQLite would have been overkill, I had an agent take the data structures, pack the text effectively, and build a prefix tree for fast auto completion from the search bar. It was great. I could have done this all before but I wouldn't have. I would have felt sqlite was fast enough. The resulting web server is significantly faster feeling (because the optimized lookup speeds) than an sqlite implementation would feel like.
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
"There's no reason" being very different than "we can now do the thing easier". There are still tons of reasons for software to be slow, the biggest of which is priorities.
If you want your software to be fast, stop writing it in Python and TypeScript and instead write it in Go, Rust, C++ or C. But nobody's going to do that, because humans are emotional creatures who get attached to things (like programming languages). Or if you want your software to be fast, stop spending your time and money (tokens) on features, and start spending it on profiling log replays to find a bottleneck causing a 50ms delay for the 95th percentile. But nobody's going to do that either, because how slow is too slow? Answer: when it's slow enough to scare the shareholders or annoy the developer on their high-powered laptop.
If you want faster/more efficient software, force it to run on a 100MHz CPU with 512KB RAM and a 56k modem. You'll definitely prioritize speed then.
>LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
Well, at least with the RAM crunch that the AI boom has caused, I hope there will be some sort of pressure to write efficient software, otherwise device manufacturers are quite fucked.
Yeah, I've been thinking this as well. I have some optimism, but not with high confidence. The exponentially increasing power of computing hardware up until this point is often cited as the reason performance optimization has been sidelined in the software industry. Now that there's a definite hiccup in that trend, I'm hoping programmers will remember that software actually can be fast and memory-efficient, and that poor design choices that lead to bad performance are exactly that, a choice.
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
I've been hoping for this for my entire software engineering career but it never came.
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations. And the tech monopolies they work for never seem to run out of money no matter how much enshittification occurs as a result.
Same perverse incentive for open source projects; engineers who lead projects that are full of bugs can charge more consulting fees and sell more enterprise licenses to work around the issues.
And engineers are often forced to use shitty tools by their employers so we're always stuck between a rock and a hard place with limited control over our output... Which we are blamed for when things go wrong but never given credit for when we pull heroic feats because those achievements are often overshadowed by missed deadlines resulting from being forced to use shitty software.
Adoption and quality are orthogonal, unfortunately.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who care about efficiency have no voice in this market.
Yeah.... maybe. I think AI is still too prone to go down rabbit holes and not understand the whole picture well enough to come up with good optimisations. It's probably good at micro-optimisation, e.g. pre-allocating arrays. But is it going to realise that you don't need that array in the first place? I dunno.
On the other hand it does mean there's no excuse for writing your program using Python or Electron or whatever, which will have a big effect on performance.
> everything else inside an Electron app can be identically fast to a native application.
HTML layout and rendering is almost never as fast as GUIs like Qt. Also Electron apps usually use JavaScript which is slower than "fast" languages like C++/Rust/Kotlin/etc.
The reason people use Electron is because it's easy, especially if you have to have a website anyway. Slack is a great example of this. They already have a full website so it's kind of silly to redo the whole thing...
The reason for using Electron is to write once and run everywhere right?
I get that LLMs would make it faster to write distinct copies for different operating systems and maintain them both but thats still more costly than a single system.
Sorry if I’m misunderstanding the goal of Electron but it still seems cheaper and easier (although admittedly worse than multiple native apps)
> There's no reason for software to be slow anymore
There has never been. We are running supercomputers. And somehow every app is a dog-slow barely moving monstrosity incapable of showing a page of text without stuttering.
All we had to do was stop approving requests for developers to have 64GB of ram as a daily default.
They could have worked in 8GB of RAM but chose not to, and now the rest of us suffer as a result because we couldn’t afford $5000 dev laptops as daily drivers.
Every post by the this person, beats the record for most unreadable, most difficult to parse, smallest font, mangled and aggregated paragraphs.
Make it as harder as possible to communicate your message...I am not sure if this is supposed to pass for minimalism...but looks more lie readability hostile snobbery. Good content does not excuse contempt for the person trying to read it.
If your argument is worth attention, presenting it legibly is basic respect for the reader....
LLMs have essentially made any language above the C++/Rust systems level tier obsolete. The only reason you picked a language like C# or Python is often out of convenience of having a broad ecosystem or due to the lower level languages being too hard to master, with too many footguns. The tradeoff being is that you got a less performant implement, while being easier to write. Nowadays, those arguments are dead.
LLMs can now write perfect code in low-level languages, that is frankly _safer_ and _exponentially_ faster than implementations in high-level languages.
Yeah, let LLM write C++ when your team have cannot comfortably write C++ in the first place. This is surely a frankly safter way than human-written C# and Python.
Yes indeed, that will be a much better road to a durable, maintainable, secure and agile product than focusing on product goals in a higher level language with a framework.
When will we start using LLMs to write my web backend directly in UEFI and assembler? (Although uefi is an OS, so maybe more fair to exit the boot services)
I think this oversimplifies the problem of dealing with performance issues. In my experience, there are three aspects that contribute to the software engineering cost of performance optimizations:
1. Identifying the cause of poor performance.
2. Implementation.
3. Architectural impact (performance is a classic case of a cross-cutting concern)
(I am specifically leaving out the case of realtime systems, hard or soft, where additional factors come into play.)
But the article seems to focus entirely on the second aspect, while largely ignoring the other two.
Most performance bugs are not difficult to fix [1, 2], but can sometimes be hard to identify. Implementation effort is not the driving limitation in those cases.
Conversely, other performance improvements may affect the overall design, e.g. 27% of all bugs identified in [3].
Having an obvious, self-contained optimization target with a benchmark where algorithmic optimization within a module is also the key problem seems to be the exception, not the rule.
Also, not all performance issues are created equal. Many have trivial cost.
In contrast, some of the most challenging performance issues are the ones that affect the design and architecture of the system. After all, the hard part of software engineering is not writing a small, self-contained application. It's managing system complexity, while maintaining (in the words of Fred Brooks) conceptual integrity. Fixing performance issues is at least in this regard not fundamentally different from fixing other software defects.
Unfortunately, this is an area that is also full of trade-offs, such as performance vs. architectural simplicity, or performance in one part of the system vs. performance in another part, all of which requires judgement.
For example, you may need to bypass an abstraction boundary or reorganize abstraction boundaries to improve performance. Or you may have to special-case something while keeping duplicated code at a minimum and easy to maintain.
This is not to say that agents cannot help here, too. In fact, agents can be very helpful at e.g. identifying bottlenecks that are not directly visible in a profiler or can be used quickly do comparative evaluations of the various options for an architectural change. But solving these issues is not, like with the regex example in the article, about hillclimbing towards better performance, but involves a combination of puzzle-solving and design skill, IMHO.
And finally, even a self-contained algorithmic improvement may come with an increased maintenance burden, especially around edge cases and through increased code complexity.
[2] Selakovic, Marija & Pradel, Michael. (2016). Performance issues and optimizations in JavaScript: an empirical study. 61-72. 10.1145/2884781.2884829.
[3] Zhao, Yutong & Xiao, Lu & Bondi, André & Chen, Bihuan & Liu, Yang. (2023). A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source Projects. IEEE Transactions on Software Engineering. 49. 924-946. 10.1109/TSE.2022.3167628.
There is no reason for it to be non-extensible either and depending on the use cases people might want to trade speed/optimization for more runtime customizations which will inherently be slower than pre-baked SIMD regex and whatnot.
It's all about the test suite. The test suite becomes an executable specification, and the better the spec, the better the results you can get from AI.
The very first thing many people did with AI is start calling the automated tests it writes good enough to capture the desired behavior. Writing tests isn't a whole lot of fun for most, so now the thing that it is supposed to help ensure we're not evolving our software into piles of trash is, in fact, one of the most neglected parts of vibed out codebases.
No it really is about the test suite, and provably so. As another poster pointed out, speed is a superoptimization problem and the test suite provides the constraints. If the constraints are appropriately set, even a naive genetic algorithm will eventually improve the outcome over time, provided suitable mixing of the proposal scheme. LLMs provide measurably better proposals than naive approaches, so the entire chain is sound. The issue really is an inability to set appropriate constraints on what the user is looking for, and poor quantification of the multiple objectives one should try to balance in practice. What's great is that's a human problem. Diverting that to the models is obviously a disaster.
I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.
I hope you understand the context in which that was said. The point of that statement is that the only way to rigorously verify correctness of a program is by using formal methods. Those are often too difficult to achieve by humans, which is why there is an entire program of developing autoformalization using LLMs. You are supporting what I have said.
In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.
The test suite is the same thing as the code, just approached from the other side.
i want the code to do this --> write the code to do this
i want the code to do this --> write the tests to specify this
the hard part is not the code or the tests. it's knowing what 'this' is supposed to be, exactly.
if you have the code, it is trivial to write the tests that will perfectly test the code.
if you have the tests, it is trivial to write the code that will perfectly pass the tests.
however, none of this has anything to do with whether the code or the tests are good, and having a bad version of one will not help you write a good version of the other.
> Only the small and unimpressive programs can be checked exhaustively.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
Not true at all! Most of the HTTP APIs, and a good chunk of the webapps, that I've worked on can be defined as a combination of an API spec that carves out valid and invalid behaviors, and a set of behavioral tests for the workflows that the client users care about. Working from a codebase which is generated from a spec document (e.g. OpenAPI or gRPC) and use of tools like https://pkg.go.dev/net/http/httptest and https://bun.com/docs/test/dom makes this a pretty achievable goal in practice.
It's part of it but you need good systems thinking in order to write good tests. The kind of thinking which requires years of software engineering experience to acquire.
Also the architecture really matters now because a poor architecture will lead to more weird edge cases and require more exotic tests to plug all the holes.
It doesn't sound very intelligent to just brute force iterate until you pass all the tests. This sounds like what Rich Hickey called "bashing into the guard rails". Even if we assume the tests suite is perfect and will guide the model to a correct solution, maybe this is one case we really should just say "LLM" rather than "AI".
Regarding speed - now it takes much less learning to use a fast web framework(like Meteor), this means web based software will be more performant.
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
I cannot imagine how nitpicky people are. AI went from 'good autocomplete' to 'I just tell it to do an entire thing and it does it and it works the first try'.
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
I feel like I’m reading a post from 2025 instead of one written three minutes ago.
People get entrenched in closedminded thinking. My guess is that this has been your canned spasm for the past year. Repeating these things won’t make them true or protect your job.
You protect your job by learning how to use new tools as the technology advances.
This sort of lazy thinking isn’t acceptable anymore. Whether this is a reflexive Luddite response or a reflection of your work ethic, neither is a good look.
Very much not a skill issue; even with the best spec, guardrails and evals, frontier models will just do the hell they want.
> one-shot secure, performant code.
I call it reverse skill issue; it feels secure and performant because you can't verify properly, it feels right but it's actually broken in many nuanced ways. You can measure performance and produce a broken measurement as well. It's not as simple as you make it
Color me sceptical but I’d say that’s a bold claim.
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
So long as the way to improve software is "just stack another VM on it", software will continue to get slower.
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
People these days are saying “don’t look at the code”.
People are shipping all sorts of weird architectures, non-performance code, etc.
I wish that the ram scarcity would drive more performant software however I just think that the way people are shipping software currently will not lead to this
I agree, but it won’t be because we can’t do it. It will be easier than ever to achieve, but of course money for the tokens to do it will go toward some other aspect of the business. That the products get worse seems to be the rule.
you are probably right as far as desktop stuff is concerned.
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
For the same reasons, too. Software is buggy and slow because you impact the bottom line when you make it better. Same thing with LLMs and token expenses.
For Desktop use, 100% of electron and python slowness will be gone in a couple of years, at least for new projects. If you're using Rust (or Mojo or C/C++), you'll have to try hard (almost a deliberate effort) to make software slow.
I agree with the headline, but I don't think LLMs are a reliable way of optimizing software.
First of all, coding speed is not the main bottleneck. I've been working on my own JavaScript framework. Most of my work is talking to people about challenges they have with web development, doing research on existing tools, and thinking about features to add.
Coding more slowly without LLM usage is a benefit because it helps me understand every line of code. The slower pace also helps me think about making things intuitive and reflect on my experiences with React.
While my framework is outperforming React, that isn't particularly impressive. On the other hand, it demonstrates how ignoring LLMs improves quality. Also, my monthly token spend is $0.
This doesn’t seem to mention that having AI aggressively optimize software will result in code that is too clever / complex to be maintained by the original human author. For something with a very stable API and can be tested to death like a regex engine it’s likely
workable. Other things, not so much. We have enough trouble maintaining AI generated code _without_ aggressive optimizations as is!
Its more about discipline because you now have so much more option for that power, not just optimization. The engineer or the manager can choose more tech debt, then LLM's power is neutered because the complexity to optimize has increased by that much, and somehow evens out, and you are back to square one.
Feel like this is the equivalent of a traffic engineer standing at the grand opening of the 5th lane for the highway saying there’s no reason for traffic to be slow anymore.
That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes), even if that code is “high quality”.
Sorry I can't resist the pedantry but a traffic engineer knows better than anyone that adding a 5th lane (often even a 4th) won't improve traffic except in fairly specific circumstances. :D
That cannot be an emperical statement. Traffic engineers have absolutely wrecked the american city, largely be ause they didnt know the impact of adding lanes.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
Who's to say they aren't the same as the developers implementing all the bad stuff at <pick your favorite hated tech company>? Listen to what they say, and plenty know it's bad, but they shrug their shoulders because that's what the bosses want and the pay is good.
I've spoken with traffic engineers before, and the consensus I've gotten is that, excusing a few bad apples, much of the problems originate with popular or political interference.
To do the best they can do within the confines of what's politically acceptable. There's an intersection near me that is just horrendous, but the engineer I talked to, who is now retired, said it was his mentor's task to fix it some 50 years ago. Actually fixing it, though, would require removing parking spaces, and the mayors have routinely vetoed that idea after they received pushback.
The people are to blame. You get what you vote for.
I mean that was kind of my point, but judging by the responses I may have been too snarky.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
This isn't categorically true either, the real world is delightfully complicated, far more so than you imagine. The way cars interact within a road, as they enter an exit the road, or how they choose their route can all create conditions where adding a lane only makes traffic worse and can reduce both speed and volume during peak traffic hours.
I'm really bothered by the mindset of the people against this. People act like induced demand is evil.
In reality, if more people were able to go wherever they wanted to go, that's actually a good thing regardless of if the existing people didn't get to go somewhere faster.
Locally. Under many circumstances. But always? That's not a universal law, as it's a part of a larger system that may perform more poorly if this leg provides more traffic.
I agree with traffic example, however, I don’t think it quite applies to software in the same way. Imagine a world where for some reason our current quality of hardware is locked in place; they can never get better or faster in any way shape or form. Then the big tech race will be optimizing software. I personally think that there would be a 10x to even 100x increase in software speed that is possible through many optimizations. The current software incentive is geared more towards adding more features rather than optimizing performance since software engineers know they can always know hardware will keep getting better which will keep their
slower, less optimize software at similar speeds. This isn’t a critique of software engineers, simply a statement of the incentives under which they operate.
The title is a disservice to the author’s point of view, then, because the article also reads as a “look what agents can do to optimize a toy problem” blog post.
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
The HN crowd understands that, depending on bottleneck, adding a fifth network connection (lane) between 2 nodes may not necessary decrease the latency (travel time).
you dont mesaure the quality of a plane by how much it weighs. but yea i think everything is rot to the core. app to os if just wackness that nobody cared about.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
Because it's not built with client side optimization in mind any more. It's built with circular investment in mind.
Big tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
The private equity and venture capitalist playbook has always been enshittification: you either take an existing high quality product or innovate one subsidized with investment capital, then once it’s time to flip the business you 10x the price and cut costs (which usually cuts quality).
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
There are so many unwarranted ideas here... I stopped reading half-way because of it.
Eg. JIT compilers are rare because they are difficult to write. JIT compilers come with tradeoffs, they aren't an absolute boon. JIT-compiled code isn't good for short-running scripts, for example. Even normally JIT-compiled languages try to special-case parts of the code that would not benefit from compilation and interpret them instead. Another huge downside of JIT-compiled code is debugging it. JIT compilation is better for languages that don't deal with memory allocation directly, and, in general, expose fewer low-level primitives to the language user. It would be a fool's errand to make a JIT-compiled C... I bet someone tried it, but, seriously...
Another one: a solution to slow search is building an index... Boy do I have bad news for you... Even a beginner DBA knows that an index can be a curse or a blessing. There's no one size fits all solution to this.
And the author continues to misrepresent similar problems only to make his solution make sense.
The software model has to however be easy for agents to debug in a loop, then it works very well. I recently got Fable to take a desktop app (AzWriter[1] - screenshot Mac: https://imgur.com/31DBG04 + Linux https://imgur.com/1IavBvS) from about 150 Mb -> 80 - 90 Mb on a reasonably complex UI (around 40 pages of text, paginated, etc.), even beating KWrite (160Mb even on KDE, even though KWrite doesn't do pagination or complex text layout, was a surprise to me).
What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).
So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).
But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.
Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.
Eh, not sure. Very few humans do software optimization productively, so I'm not optimistic about machines built by humans, either. Every encounter I've had with agentic optimization involved trying to apply a bunch of myths to an existing code base, in a way that seemed cool (unrolling loops, eliminating apparent branches, SIMD) but which was all pointless because the only credible optimizations were going to come from doing fewer loads and taking up fewer itlb slots and stuff like that.
Data point of one, but after a few months of uselessness, I have managed to get some pretty serious, measurable performance improvements with AI optimizations -- order-of-magnitude speedups of business critical processes which took days as well as significant latency reductions.
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
Software speed is actually a binary thing. If it's as fast as the user then it's good. If it's not then it's bad. Think of it like a car - in theory different cars have different accelerations and vmaxes, but in practice if you can reach max allowed speed in your country (except Germany) and overtake all other cars you want to overtake, then you're happy; if you cannot then you're not.
Here in hackernews we have a bubble of people who are very smart so they process information at high speed in general, and have lots of experience with computers, so they process computer-related information with extra speed. We're like race drivers, squeezing every single bit of performance from the car, while majority of use cases for a car is being stuck in traffic on your way to work. Those people simply don't care about performance above 150km/h because they'll never reach such speeds so putting expensive, high-performance engines is a waste of resources.
The point I'm trying to make is that if software is fast enough for average user, and most users are fucking slow, there's no business need to further optimize it. You might keep arguing "but but but back in my days apps were instant and now they suck" but users simply do not care. This is why companies put shitty, time-wasting animations even if nothing is being done in the background.
Ironically a perfect thing to iterate on with your favorite coding agent. "make this page look better, present me 2 versions, I'll pick my favorite, and we'll iterate until I say stop"
Counterpoint - I like Dan's site. I think it fits his vibe and priorities, and I don't find it unreadable. I can always pick my browser width and font size with a couple keyboard touches, and I never have to fight his idea of how I should want to read it.
What exactly do you mean by "make it approachable"? Is the content-to-fluff ratio too high for you? This page is exactly what I want the majority of websites to look like: no fluff, no idiotic "modern design" trendchasing bullshit, just pure and simple content.
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
I don’t know why you got downvoted but you have a point. It looks as if OP doesn’t write for readership and his posts contents have similar characteristics of reckless abandon for conventional norms for a tech post. Too many cross references and link-backs to his own other articles that makes it really hard to follow the main thread of the essay.
His site has always looked like this, i'm sure of it. This is what sites look like if you just write them as text. Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
I'm almost certain he used to have a reasonably styled website. I just tried to look it up but it looks like he's excluded the website from WayBackMachine.
> Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
lol at the idea that Dan Luu can't add css to his website by copypasting some links into his html. He could make the site readable in about 3 mins and never have to worry about it again.
BTW this is just the first such library I found. There are lots of 'em, and all very well documented, requiring less knowledge to use than what it takes to publish a blog... which he's mastered.
Honestly I prefer the minimal style because I add my own to many pages as it is, or use the built-in reader or a reader extension. People have much different opinions on what's readable.
The downvotes may be related to community enforcing what's in the HN guidelines:
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
I had an Odin code base with raylib doing cpu bound stuff. On Catchy OS it tanked my frame rate from 120 to 90 fps (intentionally, seeing what I can get away with). Same build, on Windows 11, 40fps.
One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
This is an incomplete and quite superficial view of what is going on out there, in my opinion. I've worked on plenty of projects where the assumption was that since the round-trip to the server is going to take almost 100ms that'll dwarf anything that's going to happen on the server itself, justifying poor choices that lead to potentially adding a whopping 100ms onto that number. These numbers only get larger with a larger perceived "Nothing we can do about it" budget as well, programmers often feel justified in doing just about anything once round-trip time grows, not understanding that they're just adding to an already existing problem.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
I don't think it's superficial, but two problems adding up. Previous poster is talking about general latency issues because everything is networked and potentially quite far away.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
I have a new laptop with a rtx 5090. Opening any GL context takes more than half a second. There's tons of things that can be optimized and are pretty far from web.
I assume you're running proprietary drivers? Because I've never experienced anything like that on mesa. Launching an app that opens a window with a gl or vk context is so fast on my almost 10 year old hardware that it's nearly imperceptible.
There must be some quirk of whatever combination of software packages are installed there, but the proprietary drivers are not the culprit, at least not alone (i.e. there could be some interaction with other software packages with which I have little experience, like Gnome).
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
So the usual self inflicted misconfiguration then.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
We can only hope that the company building the hardware on which this new age of AI is based on soon starts "vibe-optimising" their own driver stack.
They don't want other companies to train on their IP and regurgitate it to random people.
You should profile that, it is probably hitting the registry, the disk and maybe the network.
Try turning off wifi and see if it improves.
i'm on linux
No idea about your setup but that's probably a fixable driver issue. My desktop with an RTX 40-series GPU takes _maybe_ 4 frames to create an OpenGL context.
There’s no reason for (most) software to just be hosted in the US anymore.
We as an industry should use AI to enable a standard of software quality that was previously uneconomical.
I agree and see this as a side effect of a subtler thing. As people shall be replacable, software is designed and subtly mimics the organization's communication patterns. Features are cut up into the tiniest pieces with clear separation from the start (at least its claimed), and over time whatever change or feature seems overly complicated, won't be done or won't be done in a sane manner because its uneconomical.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
Best example in a monolith case is oracle...
Oracle:
— Bryan Cantrilli despise people like you who not only insist on beating a dead horse but insist on doing so with such an insipid quote.
you are the fucking lawn mower
> ticket driven development
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
I could go on about this all day. Best meme is when you estimate stories for storypoints and then they haggle with you without changing the content of the story. Meanwhile points mean actual time. Then on another side if you haggled down points, you then have more slots for more points. So you end up getting assigned 3x work for the same time. The blame game starts when the sprints elapse lmao
The only ways out of these situations are: a manager that understands what you’re doing; knowing that these “story points” are just indicative and there’s no broken incentive towards gaming them.
Yes. Apple Music is the most egregious example of this. It could be ridiculously fast on your pocket supercomputer but the moment a web request gets fired off from stumbling blindly across the field-of-dung user interface, bam, you’re done. Especially if your network connection isn’t great at that time.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
Sonos’ app is another example of this. As a very brief tl;dr if anyone isn’t aware of it, Sonos is a wireless speaker company that can group speakers in different rooms into zones, so you can have different music playing in different rooms, or all the same, or at different volumes, etc. The quality isn’t going to blow away audiophiles, but IMO they’re legitimately great.
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
This is kind of a shallow assessment of "slowness". Slowness is a feeling, not a fact. Network is slow as a rule relative to other parts of the stack, but it is not usually what contributes to the feeling that your software is slow. It takes a good amount of incompetence and arrogance to cultivate that particular experience.
Fair, it's shallow because I was succinct however, I do understand the problem space more in depth than this.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
True, dev environments are fast. One dev implements a wrapper with roundtrips, another integrates it into a UI and no-one stops to think if it'll have terrible lag in practise. They don't notice so even if there's a ticket it'll starve and end up WONTFIX.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
The UI threading model is usually not the problem.
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
Networks are much faster than you think, it's networked software that tends to be slow. 10 GbE is now table stakes, you should fire any vendor who can't offer it. I certainly don't require your internet connection to be 10Gbps, or all your desktop machines, but your internal server network should be if you're building a new one in 2026, because there's no excuse not to any more.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
The majority of companies are building their stuff entirely in the cloud, where network speed scales with the instance size. I have had to explain this to multiple engineers at multiple companies, who are surprised to learn that network bandwidth isn’t unlimited.
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
One of my all-time favorite software quotes is, "The fastest request is no request at all."
After traveling around in places with very poor wifi/phone data speeds. I couldn't agree more with you.
It's not very hard to engineer software with these two constraints at the same time:
* Must feel very responsive * Network requests can take up to 500ms end to end
It isn’t, but at the same time, smart hackers were working with highly constrained PC hardware in the 1980s and early 1990s and were cranking surprisingly good performance out of it. Folklore.org has plenty of stories about it, and John Carmack’s early career history is very impressive. We mustn’t forget the demo scene hackers either.
Recently I tried an approach of "just write the f*** code" instead of using infinite abstractions on a new UI side project. So for instance when you scroll, it shifts the pixels and just redraws the new exposed area. This is how stuff worked in the 90s. And it's blazing fast and uses very little memory. It's easy to mess up redraw code like that - in my case, when the window goes past the screen border and the pixels to copy aren't there. That's a bug we also had a lot of in the 90s.
That's why phones and windows use animations. You can also use intersitials related to the product you sell. Users are usually fine seeing many changes on the screen quickly because it gives the impression that stuff is happening on the background. For example in the interstitial, use an animation that takes up a small portion of the screen and not just a simple spinner or loading icon. Something more complicated with 2 or more things moving or changing at once.
If your app absolutely must rely on the cloud for every one of its interactions, then fine. If not, you're just applying band-aids to a problem of your own making. Many apps could easily be local only, or local first. If you're not constantly accessing the network for information which could be stored locally, then you don't need to hide your app's slowness behind animations.
Canva replaced PowerPoint. Canva is cloud based and PowerPoint is not. There's so many apps that are cloud only so that corporate it no longer has to manage installations and users don't need to ask IT for permission anymore. I think SaaS doesn't really work without the cloud, you could technically do what adobe does but why bother with app distribution and windows' quirks. Cloud based web apps are write once, run anywhere come true with no installation required. Cloud based is more convenient for the user and the developer, at the cost of app runtime speed
I like this example. A lot of people would prefer to use PowerPoint because they can still use their files and templates they made last year even if Microsoft doubles the price of Powerpoint, removes features, discontinues the product, or goes bankrupt.
Just nerds I think. I'm not seeing any evidence that people who want freedom from the cloud make up any sizeable market segment. Most corporations actually prefer the opposite - they prefer a monthly fee and continuous silent updates.
Difficulty - Microsoft retroactively cancelling lifetime licenses.
Let them sue you if they want to.
> Users are usually fine seeing many changes on the screen quickly because it gives the impression that stuff is happening on the background
This gave me a chuckle because I personally hate things like watching the browser jump through 50+ redirects when logging into a website.
My bad, I mean the interstitial. Like it should stay for at least a second to not make it jarring. People believe computers need to think so you can't make things too fast either. Not the interstitial and not the app either, to the point you sometimes have to deliberately slow down the app, add latency to make people trust it because it "gives the computer time to think"
Sometimes. Not by default.
Power users absolutely hate this mindset. It’s resulted in Apple animations taking seconds for something that was done before the animation even started.
I don't doubt that was an original justification, but most of what I see are not for this purpose. Most of the time they're just adding unnecessary delay and CPU cycles.
I absolutely hate this shit, because my goal is actually doing something in a reasonable amount of time.
In my experience the main driver of latency is not ping time, but how long the server takes to process the request.
I think a lot of the time when people say the network is slow. They really mean their backend is slow.
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
1-5ms response time is clearly hard for most real world endpoints.
I don't know how hard it is. But I can certainly say there is no business inscentive for it.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
> With a fast backend ~1-5ms response times (not even that fast).
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
But why is that?
Reddit's reaponses are rarely above 400ms. And yet their frontend routinely takes several seconds to render that response
More to the point it's that the server has to retrieve and massage data from several docker services to retrieve the full context needed to process the request
A lot of that is ultimately ping time, too though. Like making a naive number of round trips to a database not on the same machine.
How long the server takes to process the 42 requests ahead of you in line, or possibly how long the 318 poorly architected microservices take.
This is why my most recent website does its server-side rendering on the client. I'm not even being that sarcastic. We stream a subset of the user's data (1 mb at most) in the background and have a WASM client that has the exact same server views. On SPA events, the WASM blob intercepts a lot of requests and can instantly render. Makes a laggy connection feel pretty quick.
Why aren't those views just part of your SPA in the first place?
>SSR but in the browser
We've come full circle.
I think this is our third lap.
Are you counting mainframes with dumb terminals then smart terminals then PCs, early SSR websites then web 2.0, the multiseat/thinclient craze of the 2000s then computers becoming cheap enough to stick them to the back of monitors?
To dogfood this, rent a VPS in Australia and put a test environment there. Should be about 200ms ping or a bit higher. How it runs for you is how Australian users are seeing your site, even if their last mile connection is fiber.
> If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
I wish Atlassian listened to you.
It's particularly egregious to see the number of apps that will slow to a crawl even for things that works offline if the network is down or slow.
Being offline available/fallback and offline first are two different things
Yes, but my point is that it's an awfully shitty fallback if you need to wait 30 seconds for something to time out first.
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
Don’t know about that, pretty much everything is hosted on Cloudflare, Azure or S3 these days and all of them have at least one CDN on each continent.
There are CDNs everywhere, true. It is not true that everyone deploys to them all, though. Also it dodges that the networks available to everyone are still not equal.
I think that "almost everyone" has a multi region cdn, but fewer have multi region application deployment, or cdn workers handling a significant portion of the application logic. My experience here may be incorrect or not generalizable, but I've rarely seen web apps that are slow due to latency loading static resources, but I've often seen slowness from high latency of the API calls and due to large static payloads.
That only helps the assets, does nothing when they run their actual backend servers in one of the US aws regions and your traffic has to traverse the planet anyway.
UI blocking can make sense in some situations; otherwise things "happen" suddenly that are unexpected.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
Shotout to PowerSync for enabling companies to go in the opposite direction and build offline-first apps. My company is a (production) customer, and we recommend it.
Yup, this is the solution for a huge class of "slow" apps. More alternatives like PowerSync here: https://zero.rocicorp.dev/docs/when-to-use#alternatives
\m/ d-_-b \m/
This is true, but it is an entirely different problem in a different place to what the article is talking about.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks, using a markov chain, and conditioned on those pages being low bandwidth to pre-load.
So Next.JS actually pre-fetches links when they move into the viewport or you hover over it. It's interesting, but then you get wasted battery on mobile while on bad networks. The world is full of tradeoffs. Tech workers tend to want to consume more battery and data to be faster. Other people want to do less work.
https://nextjs.org/docs/app/guides/prefetching#hover-trigger...
In my view, websites should not take seconds to load with gigabit fiber. Whatever happened to "mobile first"?
McMaster Carr website prefetches _all_ links upon hover. Saves you like half a second in many cases.
segor.de goes one better and just downloads the entire catalogue when you first open it. About 2MB decompressed. Clicking and even searching is instant because it's all fully client side until you place an order.
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
(Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.)
> pre-caches frequent button clicks, using a markov chain
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
The problem isn't in preloading, it's in how much data needs to be sent while quite probably most of the data could be either fetched on startup in an efficient format and rendered natively, or is completely unnecessary in the first place (telemetry, ads).
If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
We have that, it's called an <a> tag.
>but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
And the first thing webdevs did once this became widely available, is change their apps to cache-bust their code; between that, and the short release periods in webshit ecosystem in general, and security and privacy considerations messing up things as usual, the promise of users mostly hitting just local cache with any marginal request, never materialized.
> If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
Welcome to Next.js
Can next.js give me a page's links without requiring that I execute any code that I didn't have prior to visiting that page?
The .js part makes me think not.
Speculative Rules API
This exist but the downside is that it uses much more of your bandwidth and client resources (probably not matter in many cases but it does if on a phone in a country with bad connection) and your server resources (if not mostly static content)
Oh god, don't give them ideas. All ML is in-cloud AI now. I dread the day everything around my mouse movements needs to get tokenized and vibed into the ClosedAI cloud before my buttons start working again.
I would assume a big part of software have plenty of lower hanging fruits for speedup and don't even profile to find where the bottlenecks are
Heh, this is how browser accelerators from the dial up era worked
there is different scales at which software is slow. this os one and definitely a pain in the ass. everything being online for no good reason other than to harvest user data. which it really turns out to be every time. (for good or bad purpose).
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
I’ve been playing with this with software that needs to work with agents and also without internet at all (we’re serving construction projects that have limited access)
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
Surely the biggest cause of slowness is doing more stuff.
We don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
A browser isn't really a program any more but a platform for running other programs. Like how javaw.exe is really Minecraft, firefox.exe is really YouTube. Go to about:processes to find more detail.
The point is, if I want to edit a text file, I don't need to use eclipse. I can use something that hasn't added loads of features. I don't need to use whatever Adobe product, I can use some paint app.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
Incredible that the top reply is a cop out.
No there's a simple reasons actually: we have surrendered entirely to web based apps & the resulting ad bloat.
You used to buy a computer, install an OS, buy apps and run them. Neither the OS nor the apps didn't suddenly degraded themselves with "free update" that were developed targeting newer & newer machines. You bought a new computer when new software you wanted to buy wouldn't run fast enough on it, OR the computer died.
Nowadays, you buy a new computer when the same software you've been running for years or the OS itself that demands to be updated "free" makes your machine intolerably slow.
Funny how that all worked out.
I've been working on a similar agentically engineered regex project called SafeRE:
https://github.com/eaftan/safere
https://eaftan.github.io/safere-intro/
Mine is for Java and is intended to be production grade. The first goal is to guarantee linear-time behavior to prevent ReDoS attacks. My collaborator and I have recently been optimizing it to try to surpass native RE2 in performance.
It turns out optimizations are incredibly well suited for an agentic loop. You've got concrete acceptance criteria (must show a meaninging improvement on a benchmark case, must pass tests). The agent is really, really good at using tools like a profiler and disassembler, better than I am (and I've been doing this for 20 years). It also papers over things that would take me a while to learn, like how the in-incubation Vector (SIMD) API works in Java. I understand the concept but it would take me a while to understand Java's implementation. The agent can just read the docs and go.
The key is creating a good benchmark suite and ensuring the agent doesn't ship optimizations that are too narrow or too focused on the benchmark cases. You also need a really strong test suite to make sure you're not regressing correctness. SafeRE has billions of tests; a subset of several million run on CI, and the others run on-demand.
Looks nice!
One little thing I spotted is you use Boyer Moore Horspool for fast literal search. This is actually not linear in the worst case, although it is almost always sublinear. Worst case would be a literal composed of the same character searching a text of the same character, where it becomes quadratic.
You can actually search strings with character classes using Horspool if you want to, and I have some enhancements to basic Horspool which could maybe help. My library, byteseek [1], implements these.
I also have a much faster algorithm, HashChain [2] which also has a guaranteed linear time version. This was published in the Symposium for Experimental Algorithmics in 2024.
[1] https://github.com/nishihatapalmer/byteseek
[2] https://github.com/nishihatapalmer/HashChain
Reading a bit closer, you seem to track how much work is being done in Horspool on each character comparison, and then fall back to the linear KMP if the work budget is exhausted.
This will first massively slow down the Horspool scan, and then once you have done all that work, you rescan it all from the start with KMP if it is doing too much.
One little fix might be to only add to the work counter and compare it outside of the main character comparison loop.
But it would be better to use the linear version of Hashchain. It also uses KMP to make it linear, but it is fully integrated and you would not need to track the work or restart scanning at all. And its a lot faster than Horspool anyway!
This sounds very interesting. Which JVM profiler do you use?
Not OP, but I went through this last week. I (ok, codex) optimized a hot path in some Java code from ~350ms to ~60ms, which made a substantial difference in "is this whole business going to work".
My Java profiling knowledge is... let's call it "antique". I was really not looking forward to ramping back up for this work. Turns out, I didn't have to do any of it. The LLM chose the tools (flight recorder) and even built a JMH (also new to me) harness to experiment with different algorithms.
About half of the optimizations were things that I would have figured out on my own; the other half were definitely "wow" moments.
The whole thing was done in a couple hours, with just a few back-and-forths. Sans AI, it would have taken a week, with nowhere near the same gain. I'm impressed.
"Figure out how to make this process fast" is really a perfect activity for LLMs. And the prompt doesn't really have to be much more sophisticated than that.
> the other half were definitely "wow" moments.
do you have some samples? It would be interesting to learn what it might be.
I've been using:
JMH as the framework to write microbenchmarks. It takes care of dealing with JIT warmup, etc. It's the standard way to write rigorous Java microbenchmarks.
async-profiler (https://github.com/async-profiler/async-profiler) for profiling. Java has a problem where many profilers are based on safepoints, which are biased toward particular program points. async-profiler is not biased in this way.
Java Flight Recorder for memory allocation data.
One thing I've observed in all of this is that it's really useful to have expertise in the programming language and ecosystem you're writing in, otherwise it's all Greek to you and you can't really guide the agent to do the right thing. I have opinions about e.g. profilers and I can point the agent to one that I think is more accurate than other options.
This, 100%. I’ve rewritten some hydrology code in rust using codex sol to first a) profile and create comprehensive tests of the python, b) create full benchmark suites, c) create full scientific benchmark suites, then d) port to rust using a few different techniques.
It works, it’s at least 5x faster, sometimes much more, and memory use is like 10x less and even less in cases where lots of map tiles are involved.
This shit rules.
What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
> What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
Why you even start with Python at this point? Just write the Rust version straight up instead of porting things?
Personally I used to use dynamic languages for most things, because development and maintenance is so much faster and easier, particularly for larger projects (granted you know how to work with those sort of languages), but now when the LLM writes most of the code, I'm able to work as fast with Rust as with I used to be able to do with Clojure or other dynamic languages.
Because it’s a port of a scientific process and that’s what the scientists work in.
What happens when LLMs will become either too expensive or unavailable?
It's becoming increasingly evident this is not how things are going to shake out. Even without frontier models, running Qwen 3.8 27B has demonstrated for me and others a "good enough" competency at general programming. Additionally, large open weight models have proven themselves as viable alternatives and we're still in the early years of dedicated hardware
Here's this boiled down:
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
> - The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
That's fair for a well-scoped subroutine: what I meant is that if you ask an agent to write a compiler and let it rip for a few days, you are going to be spending a few more days correcting the default behaviors in the distribution, which often do not tend towards hardware-oriented design.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
Yes, say "Build a compiler" will require you to clean up stuff if you leave the agent for days, but not because of the LLM or the quality of the tool, but because you hardly specified anything, so of course it's gonna make assumptions you need to correct.
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
You're making assumptions; OP made no mention of how detailed their spec was.
Agreed, IME Fable can churn out decent SIMD kernels optimized for whatever tradeoffs you give it.
> iykyk
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
Small OT:
> Maybe I should document what I did!
Please do!
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
Thanks! If I can ask - where did you get stuck?
The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
True! How do you set up auto research loops? Are there any special tricks to it?
I don't know if it's the first and it certainly isn't state of the art at this point but I think karpathy/autoresearch is a quintessential starting point.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
Hi Joseph,
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
> It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why.
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
Like Aladdin's jinni is a slave of the lamp, LLM is a slave of its context window :)
yep, i think the real LLM superpower is knowing that something has been done before and having access to the code that did it. so much of even novel software includes bits and pieces that have well-optimised existing solutions, and the bot knows those solutions a lot better than i do, and can even pattern match them from the general shape of the problem.
Yes. It’s also excellent at reading large codebases and putting together a picture of what’s going on. I’ve been using it a lot lately to brief me on projects and design decisions. “Look at these two projects. They both solve task X. Write a report about their similarities and differences, and the tradeoffs as a result.” And then I ask followup questions. Saves a ton of time.
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
Reasons this doesn't follow: (1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising) (2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
Both of those things are indicators of deficiencies in the testing process.
"LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly."
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
I think you’re talking somewhat at cross purposes to the original article.
The points I take away are:
- Good optimization is difficult and slow work, hence expensive, but LLMs can do it so we should be able to afford it more often now.
- There’s always a risk of over-fitting to your specific problem, but if everyone is now making bespoke optimizations maybe that isn’t actually a problem.
You said:
LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
There’s probably something in that, but it can be mitigated by testing against a local benchmark. LLMs are good at iterating tirelessly and finding incremental improvements. And as noted above, it doesn’t necessarily matter if your benchmark isn’t fully general.
> This person doesn't understand how to make efficient code
The author is one of the most knowledgeable people about performance there is
The author didn't write the quote.
Just admit that you were wrong instead of this doubling down nonsense. LLMs are amazing at optimizing memory utilization. They have no problem obsessing over fitting as much data as possible onto a single cache line and micro benchmarking cache hits.
This seems like way too caustic of a reaction, OP is correct.
If you actually know what you're doing in $language, and you know how $language wants to emit the assembly or whatever, there's no huge advantage to just programming directly in assembly.
Seems like author’s main focus recently is AI and agents unsurprisingly, hence the suspicion. But it seems like he has a backgrounded in relevant fields in the past.
I think it's more about not doing unnecessary things. Like a like a saying I heard somewhere "a clever person solves a problem, a wise person avoids it". Not every good idea _needs_ to become a feature. And if you think it's that good, give users the option to turn it on/off and track that as a metric.
My contradictory proof: I have been working on an old service with tons of performance issues, from server memory bloat, client graph rendering, excessive network requests, excessive repeat rendering, memory leaks, resource leaks, etc.
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
I remember the co-founder of Anduril Industries being the author of this tweet!
The author was paraphrasing a tweet written by somebody else:
> The other day, I saw a viral tweet saying [...]
Also, LLMs are terrible at optimizing memory utilization.
I've already posted this elsewhere, but here it is again: https://news.ycombinator.com/item?id=49226923
A vibe-coded OS that runs on an 8088 with 256KB of RAM.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
"It's not the tool, it's how you use it..."
ChatGPT MacOSX is the only software that regularly crashes on my machine when its memory consumption for no apparent reason spins up towards 50 GB.
And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.
Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.
OpenAI has 4,500 employees and a post-money valuation of $852 billion at its last funding round. You'd think they could get someone who specialises in writing MacOS apps to write their MacOS app.
They can probably hire them, and they'd get PIP'd for not jumping when management says jump.
Toxic corporate culture can go a long way to self-sabotage.
No. They want the AI to do it.
The genius play is to have an OSX expert write it with AI assistance, then let the press release say AI did it
Hopefully at some point gravity pulls them back down and they reach the maturity and humility to realize that LLMs are just tools and even though they're good for many things, you can't do absolutely everything with them.
I think the "maturity and humility" phase of the LLM hype cycle is probably still 2-3 years into the future.
And that phase probably comes immediately after the "LLMs overinvestments have caused a massive global recession".
So have the AI build a hybrid Crux, Windows and web app with shared Rust base.
I think the software out of the big AI labs: OpenAI's ChatGPT MacOSX app, Anthropic Claude Code - is a sign of the future to come.
Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).
They absolutely do not have AI-heads with PhDs writing the apps and general infra. I've met a guy that worked at Anthropic, and he was "just an engineer", so exactly that: a top guy, but not specifically AI focused, doing regular stuff there but with infinite access to the top of the crop AI.
Now look at what software they make. It speaks for itself.
Coding is a solved problem. They simply don't care if the app is full of bugs. You are using it anyway, right?
I've been using computers for 4 decades. They have gotten no faster. The nuclear plant computer system we built in 1989 had to present selected screens in 1 second. I don't think any apps I use today can do that.
It’s all incentives. I’ve worked on web projects where the people in charge cared about performance. It’s easy to get sub second speeds if you start with that goal.
I’ve also worked on projects where the people in charge added 1mb client-side mapping libraries to render a static map and ignored my push back. Those website were slow
This. The author talks about the fact that now they can try far more experimental optimizations than they could before. But they already had an architecture with efficiency in mind, and were trying optimizations, to begin with. The kind of company that ships web UIs that take 5-10s to do some action 1) don't care or aren't capable of good architecture 2) don't care or aren't capable of doing the simplest low-hanging optimizations.
At some point, probably in the 1980s, people probably decided that computers could update UI fast enough, so any additional compute power or speed has been used for other things like making it prettier or reducing development effort or time, and now ai
The problem is that the UI speed largely was not kept at that level but considerably slowed down again despite exponentially growing compute power.
I've heard it before and believe it that one reason is because many of the developers are working on new maxed-out machines and network connections both at work and home, so they don't notice problems for older or cheaper ones and/or can't justify it to management.
What Andy giveth, Bill taketh away.
In my case neither Andy nor Bill was involved. For that project we used Sun 3 workstations which were powered by Motorola 68020 I believe. So a more fair comparison would be that 1988 workstation against a modern Unix workstation. I bet the results would still be that a modern workstation would struggle to update the display in one second.
Well I don't know about specific examples, but it's a phenomenon that has been observed in many areas over decades, thus the funny saying.
After crossing 40 years of age, and working for a while now, I believe it's also because of politics. You might think the goal is to deliver the best possible product, but territory grabs within companies are important, and done by people that don't have enough skill other than territory grabs. E.g. look at Trump and his behavior. No skills other than having his way, and then he gets to decide.
Of the technical reason, for sure we underestimate how much faster technology gets. Fred Brooks had this example in his Mythical Man Month book, how the os/360 got the option of a disk drive instead of mag tape, but the result was worse because everyone assumed it was much faster than it really was.
This speaks to me. I've been running an autoresearch loop the past couple of days to improve the load time of my various projects' frontends.
I've been really, really impressed with how effective this is. I went from a 4s load on simulated slow 4g to ~750ms: https://image.non.io/speedup-graphs.webp
Side by side vid of the results: https://video.non.io/speedups.mp4
This was for https://non.io, which is something I had purposefully written to be as fast as possible (hand wrote all the comopnents, didnt even use react).
I've been considering creating a skill / utility to do this based on learnings from the speedups - would others find this kind of thing useful?
It takes about 6 seconds to get the site fully rendered on MacBook Air 2017 in Chile. HN is slightly above 1 second in comparison.
> This was for https://non.io
Maybe hugged but feels really sluggish to me for it is.
Never visited before and the site was HN load speed. Very fast.
These are the stats I'm seeing: https://image.non.io/7a8adcb1-2a17-4e2c-b213-d0c3e173a1d5.we...
Should note the server is on USW and I don't have edge servers for it at the moment.
>something I had purposefully written to be as fast as possible
and then you threw it all away by adding transition animation
Why is it so slow? Like clicking around this is a very simple site, it seems like the fade in and fade out, besides being jarring and annoying, is just adding load time.
An annoyance: clicking an image transitions the tiny image to a bigger size, then it gets replaced with a larger image file. The result is: the image slides bigger (but blurred), then immediately disappears, and reloads slowly from the top down. It's an annoying flash, and the slide to a bigger size was a waste of time.
That seriously needs some optimising. For example: on click, could the bigger image be inserted behind the small one so it's hidden; on load of the bigger image, hide the smaller image; then do the slide-bigger transition with both images together?
Autoresearch is crazy good! Closest thing to “Make this app fast!” we have now.
The problem is that there’s a tradeoff between MANY objectives, not just performance
You want code that’s fast, correct, stable, memory efficient, secure, usable, and more
The reason code might be slow is about product tradeoffs. It’s certainly easy to have an LLM optimize performance only, but what do you sacrifice? Is that an acceptable tradeoff?
Maybe slow code is a good idea if it’s the only way to get correct and secure code?
I understand now that most software is slow because of co-tenancy reasons requiring controlling resources or simply because they're safely insulated from competition. e.g. GitHub is the former: you can give yourself a git host and CI/CD system that is much higher quality by yourself since you're probably not using its social features. I think things like Apple's five-finger inward gesture are the latter. Once you could do it and start typing but nowadays it needs to render the animation etc. before keystrokes register. This software is slow because you cannot replace it in MacOS.
But all these things will change in time. Hell is other people's software.
> Apple's five-finger inward gesture are the opposite. Once you could do it and start typing but nowadays it needs to render the animation
There is no technical reason that animation should take that long.
Someone said, "good enough" and let it be visibly slow. The reasons for that could be anything, including non-performant code in dependencies, written by other people. But it just does not take much computing power by today's standards to composite code-generated animation.
Somehow, despite year after year of percentage-speed hardware improvements, there are cultural and structural reasons people ship code visibly slower than it needs to be. And faster compute appears to be irrelevant.
Perhaps the real tragedy is not the animation. It is the fact that previously keystrokes would register even if the animation had not completed and now they do not. Interaction once you've entered a new mode should be instantaneous. Instead we are faced with inputs going neither to the old app nor to the new app. In the prior world, I could do the gesture and type the letters and hit Enter and the animation would not interrupt me from getting to my destination. Now I simply <backspace> <backspace> <backspace> and suffer the ignominy of a software system that does not care about my niche needs and is nonetheless beyond my present ability to modify while retaining all else.
> There is no technical reason that animation should take that long.
The technical reason is you will not see all the animated beauty if it's fast, so animation forces feeds min delays for states transition
Every visible UI animation is visibly slow. I've used MacOS computers before, and despite having hardware that benchmarks faster, they all feel slower than my old Linux desktop. The biggest reason for this is the animations.
Windows used to have a checkbox somewhere to globally turn animations on or off.
Making animations faster is one of the main reasons why, despite all the glass effect rendering, the 26 Apple OSes can feel faster than their previous versions. (Not always, of course.) They simply shortened the animations. The 27 versions do the same thing. Of course, this doesn't help all interactions, but it definitely improves the first impression.
Next time you ask your AI of choice to build an app, please tell them to use plain language without frameworks at all and keep modules to a minimum (ie. node express, ejs and pg, nothing else) and you will see a huuuuge difference
And just use C.
I had the same thought a few days ago. Coding agents have rendered frameworks obsolete. Apparently many people disagree though.
I like to use frameworks to steer the agent. I constrain it different primitives for different tasks in order get the shape I want without explicitly prompting too much. You can think of it as API driven development
If this is true, the frameworks will either move to the specs or, worse, remain in the models.
You can't exhaustively write down all the possible ways for software to execute, especially since many programs run forever (e.g. servers, GUIs etc)
So when you say in your spec "when the user clicks here it shows this stuff" there is the tacit expectation that this will always happen and isn't dependent on the entire execution path up to this point. You also wouldn't expect a server to just finish execution after processing a request because you forgot to write "finally, cede control back to the main loop" at the end.
These assumptions are the framework. Almost everything you read is likely written in a framework, unless you limit yourself to non-technical writing for younger readers.
You could try to formalise the framework in your specs. But you'll then end up copy and pasting these paragraphs every time you want another server program. These context frameworks will probably get names, just like today.
Or you could skip that entirely and pretend you're not using a framework, but you are. It's just that the most has already learned tons of frameworks. What worries me here is stagnation. There's no way for the model to learn new frameworks. But this might be a short term effect. Maybe software in the next few years will look quite odd to those in the future as we keep writing it in the language of frameworks we already know. Maybe they'll look back at these relics and ask "what is a view, and why did the model understand that?"
Yep, I feel like there's less of a reason to use frameworks now that you can just have it bash out only the needed stuff on its own. I've been able to make some really really tiny C tools and I'm absolutely loving it.
Good point.
Maybe if this philosophy / approach has a catchy name it might be more widely adopted and we all can have better more performant web and apps.
My experiences might mirror yours to some degree.
I somehow managed to avoid the entire SPA scene. I feel so vindicated because I am absolutely zooming with my tried and true Vanilla JS these days. I have tried React, Vue, Angular, and ever just plain TS, and I hate them all.
Basically you are telling the agent to inline those frameworks. It has access to the frameworks, their functionality, and how they implemented things, combined with countless examples on how to use those frameworks.
If only we had a sufficiently smart™ compiler, it could do the inkling, dead code elimination, and WPO across the application. (Oh we do have that, as long as you don’t write in JavaScript…)
What's the stance on Astro?
That still overkill?
I find it decently fast out the box and possible to manage as it grows.
> my preferred frameworks can’t possibly be slow and yours can’t possibly be fast
For me the comparison has always been 3DsMax vs Blender. Same kind of software, same kind of features, but Blender is so much faster.
Architectural decisions have always been important.
Fun blender fact: every .blend file contains a serialisation of the complete data model of the blender version that wrote it. That's why they have a minimum file size of about 280kB.
You could compare it to Windows vs. Linux in a sense. 3ds Max has decades of legacy plugin/SDK compatibility baggage, lots of commercial and contractual obligations with business/enterprise customers... and Blender has none of those things weighing it down.
Well, I have been using 3DsMax and Blender from the early versions and it's always been the case that Blender is way faster.
Legacy baggage has nothing to do with it, in my opinion.
Ha, I'm always struggling with UI performance in Blender. Undo takes ages and freezes up everything, modifiers start lagging mouse movement as soon as you have more than a handful of polygons. I shudder to imagine what Max must be like.
Though I must admit you can have an absolute shit ton of ridiculously complex objects in a scene, and it will run smoothly if none of them is in edit mode which is at least fairly impressive.
Giving an LLM a program that does a well defined thing correctly and asking it to make a faster version and a crapload of tests to show that it performs exactly the same task is a great way to burn through a bunch of tokens and getting a faster program in return.
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I have another where I'm testing how well it can iteratively rewrite the same program without regressions and successively stricter resource conditions. Instead of progressively adding lines of code to make an over engineered behemoth, starting over with the previous version as a template of functionality lets it consider the problem increasingly holistically
I would be genuinely interested to see how it evolves the same program over a large number of generations. Given a whole lot of software is more a collection of programs rather than a singular "program", I have to imagine that the outcomes would be vastly different in shape when dealing with smaller targeted, articulable-in-a-few-paragraphs programs vs many-years-of-business-domain layered programs.
> Completely agree with your closing point. Dynamic custom software, fitted to a particular workload rather than a class of workloads, seems like a very likely outcome.
I disagree with the premise that this is the desired outcome. If every piece of software is bespoke and everyone’s instance of it works slightly differently, then it’s impossible to get support or a shared knowledge of how it works. There’s no “just share the excel file”, there’s no “press the this button on the left”, there’s no “oh I use program X to solve Y” (instead you have to know what you need so the custom software can solve it, but my time in startups taught me that most users don’t know what they want or need).
The article links to "You can just choose how many bugs you want now" https://nolanlawson.com/2026/08/16/you-can-just-choose-how-m...
So going on a tangent: an additional mechanism you have that becomes feasible with LLM assistance today is proving your code correct.
Much like security, optimization is now a function of token spend—which in some ways means it's a function of care. The reason why software may continue to be slower or less secure than it could be is simply that no one cares enough to invest the time and money in improving it.
Off-topic, but Dan, in the name of all that is holy, throw a `max-width: 40em;` on your blog!
That said, performance and security considerations can be automated by a reusable harness and I think within six months it will just be built in to things like claude code and codex the same way you might tack on a performance/security section to your project's AGENTS.md.
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.
If i had to complain about something, it's the font that's too small.
I paid for a wide screen monitor, let me use the whole width of the screen.
Reader mode (built-in to both Chrome and Firefox) does the trick.
Then there was no reason it took almost a year for Anthropic to find a workaround for a bit of screen flickering, right?
It's because they did it with ai
What makes large things slow is not the individual pieces of code but the way the architecture dynamically reacts to changes in the load. Sync vs async calls, buffers, parallel vs sequential processing. This includes optimizations made: do we want to wait until the call returns or do we proceed anyway and deal with the negative response later? Most developers can’t coherently (ie, with math, graphs and figures) explain, how a connection pool prevents undesirable consequences of brief spikes in incoming requests. And I’m pretty sure AI can’t either.
Can you explain that? I don't think you can explain it either.
It can in general: https://chatgpt.com/share/6a893693-51a8-83ed-86f9-66a1f83cb1...
Whether it can apply it directly from the codebase, I’m unsure and depends on many factors.
Most of the time, it's the network related stuff, requests that are slow. But with AI, devs nowadays are chasing the aesthetic vibe, cool animations, forward-backward video on the background relative to the scrolls... Which worsen the performance drastically just for a cool landing page. I mean that could be cool for landing page only but some people put all those animations in the product itself too, which is a pain to use.
There's no reason for sites to look super ugly, full width and unreadable anymore.
Good read. I liked the point that even small performance improvements can be useful if they don’t take much time to find and test. But measuring the actual impact is still important.
How will the user know the computer worked 'HARD', if it is not slow?
Do you not add sleep to your programs?
As long as there’s a trade off between experimentation and performance, software will always be slightly too slow.
And now the other aspect of the trade off is token budget allocation.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
Yes I do understand what author wrote as I read the whole thing, but I argue that title is not universally true.
Like we have OLAP and OLTP, there are still batch jobs that need to happen before you can see your dashboard, we have multiple users using the same resources, there are still DB Locks when writing needed, there are "first load" issues
— heck now we have to compete for resources with even more bots than ever.
Transferring 10GB or 1TB of data still not going to be instant laws of physics still apply.
I was very much on-the-fence and somewhat sceptical with respect to AI-assisted software development and optimisation, until this recent article pushed me strongly into the "believer" side: https://news.ycombinator.com/item?id=49226923
In my experience AI coding is often about making lots of additive changes without a lot of (or any) thinking in advance.
Not sure how either of those things are conducive to performance
I regularly support Windows users and when I do, I cringe at just how awful the experience is! (My daily driver for the last decade have been various flavours of linux)
High end Windows 10/11 machines (like my Alienware I9 w/ 64GB mem + RTX4090) seem a bit sluggish. Low end machines with Windows are absolutely painful to use!
I've install debian onto my fairly ancient Lenovo X201 Tablet and it is exceptionally snappy.
The linked article is such a gem: You can choose how many bugs you want now https://nolanlawson.com/2026/08/16/you-can-just-choose-how-m...
Love the concept of agents adding epicycles to fix bugs. It maps closely to my experience.
Just an aside. mawk 2.0 (high performance awk) has been out for about a decade. It's about 2x the performance of its predecessor mawk 1.3.4, and 4x performance of GNU awk. Practically no distro uses it, because the author Mike Brennan is not the "I maintain this code in perpetuity" guy and the mawk maintainer Thomas E. Dickey doesn't care move on from from 1.3.4.
The point is: Code needs to be maintained if it is to be relied on. I haven't heard good things about LLM's on that front.
I was thinking about this the other day, but along a slightly different path. I realised there is no need to use web technology to provide native experiences anymore. Bundling a web browser was always a way to ship native apps to multiple platforms with minimal effort, but these days prompting up an actual native client for each platform is already accessible even to small shops!
That's probably one of the biggest sources of software slowness, and we can just stop using it now, with almost no trade-off. Neat!
I gave Fable 5 access to the database and the browser and asked it to optimize for performance.
Woke up today with 2x improvement and fully tested.
The code optimized was mostly written by humans over the years. That code was too cumbersome to optimize by humans so we never touched it.
>you can choose how many bugs you want now,
You are a rust expert. Make only 5 mistakes.
:) Interesting quote though - I guess one does get to decide how much testing/debugging effort one wants now
What about these reasons?
- Your boss/most of the market doesn't care if your software is slightly slow
- Your boss/the market wants new features
It's easier now to slip in fun optimization features when no one's looking, but the incentives are still off.
I can't tell if you are saying the market doesn't care if your software is slow, or quoting someone else. Either way it's objectively false and that has been known for decades. Making users wait is a surefire way to make them engage with your product less, and the more you make them wait the less they will use it.
If your boss doesn't care about your software being slow he's incompetent.
https://services.google.com/fh/files/blogs/google_delayexp.p...
https://business.google.com/ca-en/think/marketing-strategies...
https://medium.com/ft-product-technology/a-faster-ft-com-10e...
https://www.pingdom.com/blog/page-load-time-really-affect-bo...
The market cares. In my experience it’s harder to make your boss care.
It’s often been hard for me to make coworkers care as well which is even more frustrating.
Unless your job really prioritizes performance and builds in benchmarking people often focus on finishing a feature as fast as possible and ignore performance (and accessibility, bugs, UI polish, etc.)
Experiments demonstrate that increasing web search latency 100 to 400 ms reduces the daily number of searches per user by 0.2% to 0.6%
It obviously depends a lot on what you do, but often that doesn't matter.
Do you think most websites are hitting 100ms? You are picking silly numbers. The real world case is more like 500ms vs 2.5s. The pingdom numbers suggest that is more like 5%. It's very common for pages to take over 5s to load. That is a huge bounce multiplier. I just did a search for "sony lens" on fb marketplace, and it took about 2s to load the page. This is a very very optimized page, it got some results at the top within 500ms, which was probably enough to fill above the fold on mobile (and thus seem like it was finished). however the broader search to fill the results took about 2s. When I worked at eBay searches took 2-3s routinely if they weren't warm. Getting to 100ms for a lot of very important ecommerce pages, which are critical in terms of bounce rate sensitivity, is a real engineering challenge that millions of dollars are spent on. Imagine if eBay could get 3% lower bounce on their search pages, delivering that would make multiple careers.
There's also no reason for body text spanning the whole width of my 24 inch display.
Jevon's paradox implies we're getting more fast software, and more slow software. Sturgeon’s law implies the ratio will remain the same. 90% of what ships will be slow, buggy, slop. Just as it was before LLMs.
I wonder if AI has changed the 90% rate. From my perspective it’s already increased the ratio of what I’d consider bad.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
I think this is the most accurate take on this thread. I've written super optimised code in assembly and SIMD. But only for contests never at work, because I've never had an employer that cared to optimise things that far, they just wanted cheap software that worked just good enough to get the job done. Even in places where they actually cared about quality would usually prefer adding a couple features over optimising things that much.
There never was. We had software that felt snappy on 90's hardware
It's all about machine thinking. E.g. when user scrolls, copy the pixels, don't redraw the whole page.
Slow software is solved problem. Just give all developers Celeron J3455.
This makes sense for a verifiable task like a regex engine - but those are clearly defined and I doubt an LLM will have a meaningful speed increase over current human regex engines.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
Not paying for premium? That's still a good reason. Even solo SaaS builders have to pay the rent.
Dan Luu might benefit from having an LLM optimize his website for readability.
Yes, simply stop using JavaScript and give me static webpages. Done. I don't give a fuck if your website have a nice effect that follows my cursor, I just think about the amount of energy, effort and time wasted on making this demo.
See https://deepseek.com/harness/en/
Is anyone really impressed by this gimmick anymore? Just give me a blank HTML with <table>. Its fine. I dont think anyone care.
Sites can be fast with PHP and JavaScript. The slowness you see is mostly coming from all the frameworks too many developers "include" willy nilly with code that can only be described as bloat because 99% of it is completely unused, yet being loaded and computed by the browser on every load. That's what you're experiencing on too many websites.
Funnily enough, since most JS frameworks are utter junk, developers are better off instructing LLMs to write pure JS instead.
Fun story, I was writing some 3D graphics animations for a website, and the only dependency I allowed was three.js. It ran well, but the site was loading slightly too slow so I told the LLM to optimize. It essentially hit a hard wall and told me something along the lines of "I can't optimize further, three.js is the bottleneck due to blablabla, but you don't need it. If you want I can rewrite it with pure WebGL". I said sure and load times were cut by like 90% and framerate went up too. Staggering.
This needs to be qualified with "to the degree that you have a specification of what that software should do." The better the spec, the more leeway you can give the optimizer. A very thorough spec lets you give the LLM total free rein to run optimization passes over your codebase.
> performance work that used to require a person or team that had a rare set of skills can be done by anyone who can type a few sentences
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
The essence of the problem is that the LLM does exactly what you tell it. In the hands of a skilled engineer who understands the project, this is a superpower. In the hands of a junior, this is dangerous... Because it will do things that are insecure; it doesn't push back on a user request; it just goes ahead and tries to make it work, even if it's a bad idea and the problem is best solved in a different way.
> This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
I think for well defined problems, it does a really good job, but if you're starting from scratch and don't know what to ask, it can take you on a wild goose chase in an extremely sub-optimal direction and it never backtracks from that bad design.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
Sure there is. Nobody cares about quality, mvp and get to market first is what counts. AI means there is more reason for software to be slow, because it creates code more quickly and with poorer quality.
Great news, let's just wait an eternity for this myth to morph into reality!
With RAM becoming expensive, developers will have no choice but to make memory efficient software.
Funny they mention demoscene as an example. I've tried to get various LLMs to explain or modify my demos and they utterly fail. At the extreme end this style of coding requires you to hold all understanding of what is happening end to end in your head, this x-ray vision allows you to reason and think about fundamentals without getting held back by abstractions.
LLMs operate in abstractions. "Reasoning" is bolted on.
Incentive has shifted. It’s now more important to worry about “developer productivity” and churning out as much code as quickly as possible than it is to spend time optimizing that code.
I recently built a piece of code which downloads a bulk set of data, indexes it for search, and then serves a pretty web UI on top of this with the help of some AIs. Normally I would have reached for sstables, sqlite, etc. This time, because the lookup patterns actually would not have been too efficient on sstables and SQLite would have been overkill, I had an agent take the data structures, pack the text effectively, and build a prefix tree for fast auto completion from the search bar. It was great. I could have done this all before but I wouldn't have. I would have felt sqlite was fast enough. The resulting web server is significantly faster feeling (because the optimized lookup speeds) than an sqlite implementation would feel like.
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
i can pore all my time, agents running profiling, workloads and so but no one gives a damn. tried the whole year, still do for my personal projects
"There's no reason" being very different than "we can now do the thing easier". There are still tons of reasons for software to be slow, the biggest of which is priorities.
If you want your software to be fast, stop writing it in Python and TypeScript and instead write it in Go, Rust, C++ or C. But nobody's going to do that, because humans are emotional creatures who get attached to things (like programming languages). Or if you want your software to be fast, stop spending your time and money (tokens) on features, and start spending it on profiling log replays to find a bottleneck causing a 50ms delay for the 95th percentile. But nobody's going to do that either, because how slow is too slow? Answer: when it's slow enough to scare the shareholders or annoy the developer on their high-powered laptop.
If you want faster/more efficient software, force it to run on a 100MHz CPU with 512KB RAM and a 56k modem. You'll definitely prioritize speed then.
>LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
No reason web pages need to be ugly anymore.
Eppur si muove
Well, at least with the RAM crunch that the AI boom has caused, I hope there will be some sort of pressure to write efficient software, otherwise device manufacturers are quite fucked.
Yeah, I've been thinking this as well. I have some optimism, but not with high confidence. The exponentially increasing power of computing hardware up until this point is often cited as the reason performance optimization has been sidelined in the software industry. Now that there's a definite hiccup in that trend, I'm hoping programmers will remember that software actually can be fast and memory-efficient, and that poor design choices that lead to bad performance are exactly that, a choice.
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
Yah, maybe $1k/dimm on 32 dimm system is a slight price bump.
I've been hoping for this for my entire software engineering career but it never came.
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations. And the tech monopolies they work for never seem to run out of money no matter how much enshittification occurs as a result.
Same perverse incentive for open source projects; engineers who lead projects that are full of bugs can charge more consulting fees and sell more enterprise licenses to work around the issues.
And engineers are often forced to use shitty tools by their employers so we're always stuck between a rock and a hard place with limited control over our output... Which we are blamed for when things go wrong but never given credit for when we pull heroic feats because those achievements are often overshadowed by missed deadlines resulting from being forced to use shitty software.
Adoption and quality are orthogonal, unfortunately.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who care about efficiency have no voice in this market.
Yeah.... maybe. I think AI is still too prone to go down rabbit holes and not understand the whole picture well enough to come up with good optimisations. It's probably good at micro-optimisation, e.g. pre-allocating arrays. But is it going to realise that you don't need that array in the first place? I dunno.
On the other hand it does mean there's no excuse for writing your program using Python or Electron or whatever, which will have a big effect on performance.
Are you suggesting Electron is slow? What evidence do you have? There may be a +200ms start up time compared to a native application; is that so bad?
Besides a bootup that is no longer than your browser's, everything else inside an Electron app can be identically fast to a native application.
Please be a better filter - do not spread bad memes.
> everything else inside an Electron app can be identically fast to a native application.
HTML layout and rendering is almost never as fast as GUIs like Qt. Also Electron apps usually use JavaScript which is slower than "fast" languages like C++/Rust/Kotlin/etc.
The reason people use Electron is because it's easy, especially if you have to have a website anyway. Slack is a great example of this. They already have a full website so it's kind of silly to redo the whole thing...
Unless AI makes it easy. We'll see I guess.
The reason for using Electron is to write once and run everywhere right?
I get that LLMs would make it faster to write distinct copies for different operating systems and maintain them both but thats still more costly than a single system.
Sorry if I’m misunderstanding the goal of Electron but it still seems cheaper and easier (although admittedly worse than multiple native apps)
Yeah that's my point. Maybe AI will lower that cost to the point where it makes sense to write e.g. a native Slack app. I dunno, we'll see.
> There's no reason for software to be slow anymore
There has never been. We are running supercomputers. And somehow every app is a dog-slow barely moving monstrosity incapable of showing a page of text without stuttering.
All we had to do was stop approving requests for developers to have 64GB of ram as a daily default.
They could have worked in 8GB of RAM but chose not to, and now the rest of us suffer as a result because we couldn’t afford $5000 dev laptops as daily drivers.
Every post by the this person, beats the record for most unreadable, most difficult to parse, smallest font, mangled and aggregated paragraphs.
Make it as harder as possible to communicate your message...I am not sure if this is supposed to pass for minimalism...but looks more lie readability hostile snobbery. Good content does not excuse contempt for the person trying to read it.
If your argument is worth attention, presenting it legibly is basic respect for the reader....
LLMs have essentially made any language above the C++/Rust systems level tier obsolete. The only reason you picked a language like C# or Python is often out of convenience of having a broad ecosystem or due to the lower level languages being too hard to master, with too many footguns. The tradeoff being is that you got a less performant implement, while being easier to write. Nowadays, those arguments are dead.
LLMs can now write perfect code in low-level languages, that is frankly _safer_ and _exponentially_ faster than implementations in high-level languages.
Yeah, let LLM write C++ when your team have cannot comfortably write C++ in the first place. This is surely a frankly safter way than human-written C# and Python.
Yes indeed, that will be a much better road to a durable, maintainable, secure and agile product than focusing on product goals in a higher level language with a framework.
When will we start using LLMs to write my web backend directly in UEFI and assembler? (Although uefi is an OS, so maybe more fair to exit the boot services)
I think this oversimplifies the problem of dealing with performance issues. In my experience, there are three aspects that contribute to the software engineering cost of performance optimizations:
1. Identifying the cause of poor performance. 2. Implementation. 3. Architectural impact (performance is a classic case of a cross-cutting concern)
(I am specifically leaving out the case of realtime systems, hard or soft, where additional factors come into play.)
But the article seems to focus entirely on the second aspect, while largely ignoring the other two.
Most performance bugs are not difficult to fix [1, 2], but can sometimes be hard to identify. Implementation effort is not the driving limitation in those cases.
Conversely, other performance improvements may affect the overall design, e.g. 27% of all bugs identified in [3].
Having an obvious, self-contained optimization target with a benchmark where algorithmic optimization within a module is also the key problem seems to be the exception, not the rule.
Also, not all performance issues are created equal. Many have trivial cost.
In contrast, some of the most challenging performance issues are the ones that affect the design and architecture of the system. After all, the hard part of software engineering is not writing a small, self-contained application. It's managing system complexity, while maintaining (in the words of Fred Brooks) conceptual integrity. Fixing performance issues is at least in this regard not fundamentally different from fixing other software defects.
Unfortunately, this is an area that is also full of trade-offs, such as performance vs. architectural simplicity, or performance in one part of the system vs. performance in another part, all of which requires judgement.
For example, you may need to bypass an abstraction boundary or reorganize abstraction boundaries to improve performance. Or you may have to special-case something while keeping duplicated code at a minimum and easy to maintain.
This is not to say that agents cannot help here, too. In fact, agents can be very helpful at e.g. identifying bottlenecks that are not directly visible in a profiler or can be used quickly do comparative evaluations of the various options for an architectural change. But solving these issues is not, like with the regex example in the article, about hillclimbing towards better performance, but involves a combination of puzzle-solving and design skill, IMHO.
And finally, even a self-contained algorithmic improvement may come with an increased maintenance burden, especially around edge cases and through increased code complexity.
[1] Jin, Guoliang & Song, Linhai & Shi, Xiaoming & Scherpelz, Joel & Lu, Shan. (2012). Understanding and Detecting Real-World Performance Bugs. Sigplan Notices - SIGPLAN. 47. 10.1145/2345156.2254075.
[2] Selakovic, Marija & Pradel, Michael. (2016). Performance issues and optimizations in JavaScript: an empirical study. 61-72. 10.1145/2884781.2884829.
[3] Zhao, Yutong & Xiao, Lu & Bondi, André & Chen, Bihuan & Liu, Yang. (2023). A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source Projects. IEEE Transactions on Software Engineering. 49. 924-946. 10.1109/TSE.2022.3167628.
There is no reason for it to be non-extensible either and depending on the use cases people might want to trade speed/optimization for more runtime customizations which will inherently be slower than pre-baked SIMD regex and whatnot.
One article said secure software was here because of AI, this one says it can now be performant.
Yet when I ask for code it writes, by default, both slow and insecure code that mostly works. Kinda.
As I try to get AI to rewrite it into more secure, less bloated and optimized code is when it starts to randomly crash.
Then I read articles about how AI is "moving too fast" and cry.
It's all about the test suite. The test suite becomes an executable specification, and the better the spec, the better the results you can get from AI.
The very first thing many people did with AI is start calling the automated tests it writes good enough to capture the desired behavior. Writing tests isn't a whole lot of fun for most, so now the thing that it is supposed to help ensure we're not evolving our software into piles of trash is, in fact, one of the most neglected parts of vibed out codebases.
It really isn't.
It's s about subpar models trained on subpar data doing subpar work.
The only reason why anyone takes it seriously is that we've had a glut of subpar developers for 30 years so it all balances out in the end.
No it really is about the test suite, and provably so. As another poster pointed out, speed is a superoptimization problem and the test suite provides the constraints. If the constraints are appropriately set, even a naive genetic algorithm will eventually improve the outcome over time, provided suitable mixing of the proposal scheme. LLMs provide measurably better proposals than naive approaches, so the entire chain is sound. The issue really is an inability to set appropriate constraints on what the user is looking for, and poor quantification of the multiple objectives one should try to balance in practice. What's great is that's a human problem. Diverting that to the models is obviously a disaster.
I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.
Tests do not prove the absence of bugs.
I hope you understand the context in which that was said. The point of that statement is that the only way to rigorously verify correctness of a program is by using formal methods. Those are often too difficult to achieve by humans, which is why there is an entire program of developing autoformalization using LLMs. You are supporting what I have said.
In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.
Llms are great at converting natural language to a spec because that's a translation from one language to another.
Only a moron would then use them to code up that spec.
The test suite is the same thing as the code, just approached from the other side.
i want the code to do this --> write the code to do this
i want the code to do this --> write the tests to specify this
the hard part is not the code or the tests. it's knowing what 'this' is supposed to be, exactly.
if you have the code, it is trivial to write the tests that will perfectly test the code. if you have the tests, it is trivial to write the code that will perfectly pass the tests.
however, none of this has anything to do with whether the code or the tests are good, and having a bad version of one will not help you write a good version of the other.
How do you write a spec for correctness? Only the small and unimpressive programs can be checked exhaustively.
> Only the small and unimpressive programs can be checked exhaustively.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
You can formally prove the correctness of even massive programs.
Not true at all! Most of the HTTP APIs, and a good chunk of the webapps, that I've worked on can be defined as a combination of an API spec that carves out valid and invalid behaviors, and a set of behavioral tests for the workflows that the client users care about. Working from a codebase which is generated from a spec document (e.g. OpenAPI or gRPC) and use of tools like https://pkg.go.dev/net/http/httptest and https://bun.com/docs/test/dom makes this a pretty achievable goal in practice.
It's part of it but you need good systems thinking in order to write good tests. The kind of thinking which requires years of software engineering experience to acquire.
Also the architecture really matters now because a poor architecture will lead to more weird edge cases and require more exotic tests to plug all the holes.
It's all now about money extraction from the areas dependant on software, not the sofrware itself.
It doesn't sound very intelligent to just brute force iterate until you pass all the tests. This sounds like what Rich Hickey called "bashing into the guard rails". Even if we assume the tests suite is perfect and will guide the model to a correct solution, maybe this is one case we really should just say "LLM" rather than "AI".
If you have such a test suite then you really don't need AI to write the code for you.
Writing a benchmark test is about 100x to 1000x easier than writing optimizations. A benchmark test can be as simple as:
Actually optimizing the runCodeUnderBenchmark() function is far more difficult.Aha, the problem is you're asking for code that works.
Regarding speed - now it takes much less learning to use a fast web framework(like Meteor), this means web based software will be more performant.
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
I cannot imagine how nitpicky people are. AI went from 'good autocomplete' to 'I just tell it to do an entire thing and it does it and it works the first try'.
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
The goal posts always move
I feel like I’m reading a post from 2025 instead of one written three minutes ago.
People get entrenched in closedminded thinking. My guess is that this has been your canned spasm for the past year. Repeating these things won’t make them true or protect your job.
You protect your job by learning how to use new tools as the technology advances.
This sort of lazy thinking isn’t acceptable anymore. Whether this is a reflexive Luddite response or a reflection of your work ethic, neither is a good look.
> Yet when I ask for code
Engineers don't ask a tool to write code.
> Repeating these things won’t make them true or protect your job.
Most of these things are just true as of today.
> You protect your job by learning how to use new tools as the technology advances.
Sure and the more you learn the more you see how hyped they were and reality untangle
> Engineers don't ask a tool to write code.
And what they ask? They create loops or graphs? These tools ultimately vomit code anyways
Skill issue.
The fact is that with the right spec, agentic guardrails, and evals, frontier models can now one-shot secure, performant code.
This is the job of software engineering now. Learn to properly use the tools for the job.
Very much not a skill issue; even with the best spec, guardrails and evals, frontier models will just do the hell they want.
> one-shot secure, performant code.
I call it reverse skill issue; it feels secure and performant because you can't verify properly, it feels right but it's actually broken in many nuanced ways. You can measure performance and produce a broken measurement as well. It's not as simple as you make it
Color me sceptical but I’d say that’s a bold claim.
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
100% sure software in the coming years will just feel slower and slower.
So long as the way to improve software is "just stack another VM on it", software will continue to get slower.
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
It will be the equivalent of that Excel =COPILOT(“sum the numbers above”) meme.
aka electron
I’m a lot more optimistic now that RAM is scarce. A lot of focus will be on maximizing software performance.
See: iOS 27. It’s faster than the previous version, even on very old phones.
I also think AI will contribute to removing a lot of the tedium surrounding optimization.
People these days are saying “don’t look at the code”.
People are shipping all sorts of weird architectures, non-performance code, etc.
I wish that the ram scarcity would drive more performant software however I just think that the way people are shipping software currently will not lead to this
> People are shipping all sorts of weird architectures, non-performance code, etc.
And they weren’t before AI?
Is iOS 27 particularly impressive, or was iOS 26 a shitheap? Maybe a bit of both.
It fucking better be faster, iOS 26 basically bricked my iPhone SE, which had never been the slightest bit laggy or glitchy before that cursed update.
I agree, but it won’t be because we can’t do it. It will be easier than ever to achieve, but of course money for the tokens to do it will go toward some other aspect of the business. That the products get worse seems to be the rule.
you are probably right as far as desktop stuff is concerned.
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
For the same reasons, too. Software is buggy and slow because you impact the bottom line when you make it better. Same thing with LLMs and token expenses.
For Desktop use, 100% of electron and python slowness will be gone in a couple of years, at least for new projects. If you're using Rust (or Mojo or C/C++), you'll have to try hard (almost a deliberate effort) to make software slow.
I agree with the headline, but I don't think LLMs are a reliable way of optimizing software.
First of all, coding speed is not the main bottleneck. I've been working on my own JavaScript framework. Most of my work is talking to people about challenges they have with web development, doing research on existing tools, and thinking about features to add.
Coding more slowly without LLM usage is a benefit because it helps me understand every line of code. The slower pace also helps me think about making things intuitive and reflect on my experiences with React.
While my framework is outperforming React, that isn't particularly impressive. On the other hand, it demonstrates how ignoring LLMs improves quality. Also, my monthly token spend is $0.
This doesn’t seem to mention that having AI aggressively optimize software will result in code that is too clever / complex to be maintained by the original human author. For something with a very stable API and can be tested to death like a regex engine it’s likely workable. Other things, not so much. We have enough trouble maintaining AI generated code _without_ aggressive optimizations as is!
I feel so nostalgic viewing this website. I love the simple old school design. we need to bring it back
Its more about discipline because you now have so much more option for that power, not just optimization. The engineer or the manager can choose more tech debt, then LLM's power is neutered because the complexity to optimize has increased by that much, and somehow evens out, and you are back to square one.
Feel like this is the equivalent of a traffic engineer standing at the grand opening of the 5th lane for the highway saying there’s no reason for traffic to be slow anymore.
That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes), even if that code is “high quality”.
Sorry I can't resist the pedantry but a traffic engineer knows better than anyone that adding a 5th lane (often even a 4th) won't improve traffic except in fairly specific circumstances. :D
That cannot be an emperical statement. Traffic engineers have absolutely wrecked the american city, largely be ause they didnt know the impact of adding lanes.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
Who's to say they aren't the same as the developers implementing all the bad stuff at <pick your favorite hated tech company>? Listen to what they say, and plenty know it's bad, but they shrug their shoulders because that's what the bosses want and the pay is good.
I've spoken with traffic engineers before, and the consensus I've gotten is that, excusing a few bad apples, much of the problems originate with popular or political interference.
I believe it, but what's the point of having them then.
To do the best they can do within the confines of what's politically acceptable. There's an intersection near me that is just horrendous, but the engineer I talked to, who is now retired, said it was his mentor's task to fix it some 50 years ago. Actually fixing it, though, would require removing parking spaces, and the mayors have routinely vetoed that idea after they received pushback.
The people are to blame. You get what you vote for.
I mean that was kind of my point, but judging by the responses I may have been too snarky.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
But it will definitely improve, either speed or volume.
This isn't categorically true either, the real world is delightfully complicated, far more so than you imagine. The way cars interact within a road, as they enter an exit the road, or how they choose their route can all create conditions where adding a lane only makes traffic worse and can reduce both speed and volume during peak traffic hours.
I'm really bothered by the mindset of the people against this. People act like induced demand is evil.
In reality, if more people were able to go wherever they wanted to go, that's actually a good thing regardless of if the existing people didn't get to go somewhere faster.
Most people use roads to get to work. It's definitely not where they want to go.
Locally. Under many circumstances. But always? That's not a universal law, as it's a part of a larger system that may perform more poorly if this leg provides more traffic.
Check out the Downs-Thomson paradox.
I agree with traffic example, however, I don’t think it quite applies to software in the same way. Imagine a world where for some reason our current quality of hardware is locked in place; they can never get better or faster in any way shape or form. Then the big tech race will be optimizing software. I personally think that there would be a 10x to even 100x increase in software speed that is possible through many optimizations. The current software incentive is geared more towards adding more features rather than optimizing performance since software engineers know they can always know hardware will keep getting better which will keep their slower, less optimize software at similar speeds. This isn’t a critique of software engineers, simply a statement of the incentives under which they operate.
The author clearly understood this part of the topic and agrees. There are multiple reasons that software is slow.
As you imply, some of them are not because spending time on optimization seems relatively expensive compared to value. But some of them are.
The title is a disservice to the author’s point of view, then, because the article also reads as a “look what agents can do to optimize a toy problem” blog post.
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
I get the paradox for adding a few lanes…
Imagine traffic not getting any better after adding 1000 lanes in a medium sized town… not just to one area but to every single damn road.
Yet we’ve done this with RAM, CPU, and storage…
The HN crowd understands that, depending on bottleneck, adding a fifth network connection (lane) between 2 nodes may not necessary decrease the latency (travel time).
you dont mesaure the quality of a plane by how much it weighs. but yea i think everything is rot to the core. app to os if just wackness that nobody cared about.
Still a struggle to get coworkers to write docs or tests despite it being effortless.
At the end of the day its a mindset.
What's the public transport of software development? Where can I get good train service here?
> What's the public transport of software development?
Understanding.
just¹ prompt like the lotus philosophy:
Simplify, then add lightness.
[1] hn favorite word
The vast majority of users - weighting by the profits they generate for software companies - obviously don't care about "slow".
And Munger's Law never cares if you hate its consequences.
But software seems to be getting slower and less user friendly by the hour.
Both can be true.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
Because it's not built with client side optimization in mind any more. It's built with circular investment in mind.
Big tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
Isn't this just enshittification? Where money is the driver, enshittification is just bound to happen.
If that euphemism makes sense of it for you, sure.
The private equity and venture capitalist playbook has always been enshittification: you either take an existing high quality product or innovate one subsidized with investment capital, then once it’s time to flip the business you 10x the price and cut costs (which usually cuts quality).
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
who is Jamie Brandon
It is https://www.scattered-thoughts.net/
we should just have the LLM rewrite everything in Rust :D
problem solved
There are so many unwarranted ideas here... I stopped reading half-way because of it.
Eg. JIT compilers are rare because they are difficult to write. JIT compilers come with tradeoffs, they aren't an absolute boon. JIT-compiled code isn't good for short-running scripts, for example. Even normally JIT-compiled languages try to special-case parts of the code that would not benefit from compilation and interpret them instead. Another huge downside of JIT-compiled code is debugging it. JIT compilation is better for languages that don't deal with memory allocation directly, and, in general, expose fewer low-level primitives to the language user. It would be a fool's errand to make a JIT-compiled C... I bet someone tried it, but, seriously...
Another one: a solution to slow search is building an index... Boy do I have bad news for you... Even a beginner DBA knows that an index can be a curse or a blessing. There's no one size fits all solution to this.
And the author continues to misrepresent similar problems only to make his solution make sense.
The software model has to however be easy for agents to debug in a loop, then it works very well. I recently got Fable to take a desktop app (AzWriter[1] - screenshot Mac: https://imgur.com/31DBG04 + Linux https://imgur.com/1IavBvS) from about 150 Mb -> 80 - 90 Mb on a reasonably complex UI (around 40 pages of text, paginated, etc.), even beating KWrite (160Mb even on KDE, even though KWrite doesn't do pagination or complex text layout, was a surprise to me).
What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).
So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).
But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.
Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.
[1] https://azul.rs/ui/release/0.2.0#demos
[2] https://github.com/fschutt/azul/blob/master/e2e/css-anim-per...
[3] https://github.com/fschutt/azul/blob/master/layout/benches/f...
Note: UI Toolkit is still very WIP, docs + code are still slop, etc. - working on it. But I just wanted to add this.
What if we need pretty, rounded, softly opaque corners on all our transparent windows hovering above a gently shadowed background?
The demoscene could do that, and much more, on 90s hardware.
apple might sue you for infringing on their primary money maker
- Computer, enhance.
Eh, not sure. Very few humans do software optimization productively, so I'm not optimistic about machines built by humans, either. Every encounter I've had with agentic optimization involved trying to apply a bunch of myths to an existing code base, in a way that seemed cool (unrolling loops, eliminating apparent branches, SIMD) but which was all pointless because the only credible optimizations were going to come from doing fewer loads and taking up fewer itlb slots and stuff like that.
Data point of one, but after a few months of uselessness, I have managed to get some pretty serious, measurable performance improvements with AI optimizations -- order-of-magnitude speedups of business critical processes which took days as well as significant latency reductions.
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
What I want is the PMU in the loop.
Software speed is actually a binary thing. If it's as fast as the user then it's good. If it's not then it's bad. Think of it like a car - in theory different cars have different accelerations and vmaxes, but in practice if you can reach max allowed speed in your country (except Germany) and overtake all other cars you want to overtake, then you're happy; if you cannot then you're not.
Here in hackernews we have a bubble of people who are very smart so they process information at high speed in general, and have lots of experience with computers, so they process computer-related information with extra speed. We're like race drivers, squeezing every single bit of performance from the car, while majority of use cases for a car is being stuck in traffic on your way to work. Those people simply don't care about performance above 150km/h because they'll never reach such speeds so putting expensive, high-performance engines is a waste of resources.
The point I'm trying to make is that if software is fast enough for average user, and most users are fucking slow, there's no business need to further optimize it. You might keep arguing "but but but back in my days apps were instant and now they suck" but users simply do not care. This is why companies put shitty, time-wasting animations even if nothing is being done in the background.
Wow, my browser's Reader Mode saved my bacon on this one.
Otherwise, I would have left immediately.
I'm all for speedy, simple, plaintext websites, but it is a negligible amount of work in 2026 to throw some barebones CSS in and make it approachable.
Ironically a perfect thing to iterate on with your favorite coding agent. "make this page look better, present me 2 versions, I'll pick my favorite, and we'll iterate until I say stop"
Counterpoint - I like Dan's site. I think it fits his vibe and priorities, and I don't find it unreadable. I can always pick my browser width and font size with a couple keyboard touches, and I never have to fight his idea of how I should want to read it.
What exactly do you mean by "make it approachable"? Is the content-to-fluff ratio too high for you? This page is exactly what I want the majority of websites to look like: no fluff, no idiotic "modern design" trendchasing bullshit, just pure and simple content.
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
I don’t know why you got downvoted but you have a point. It looks as if OP doesn’t write for readership and his posts contents have similar characteristics of reckless abandon for conventional norms for a tech post. Too many cross references and link-backs to his own other articles that makes it really hard to follow the main thread of the essay.
Dan Luu is a bit famous around HN.
I think he might subscribe to the philosophy of everyone should style websites however they like and not be prescribed by the author.
Nah, he's just trying to be different. It's vanity.
He's had a usable website in the past. This one is explicitly made to be shitty. "Look at me, I'm such a geek" energy.
His site has always looked like this, i'm sure of it. This is what sites look like if you just write them as text. Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
I'm almost certain he used to have a reasonably styled website. I just tried to look it up but it looks like he's excluded the website from WayBackMachine.
> Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
lol at the idea that Dan Luu can't add css to his website by copypasting some links into his html. He could make the site readable in about 3 mins and never have to worry about it again.
BTW this is just the first such library I found. There are lots of 'em, and all very well documented, requiring less knowledge to use than what it takes to publish a blog... which he's mastered.Honestly I prefer the minimal style because I add my own to many pages as it is, or use the built-in reader or a reader extension. People have much different opinions on what's readable.
The downvotes may be related to community enforcing what's in the HN guidelines:
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
https://news.ycombinator.com/newsguidelines.html
"There's no reason for websites to be ugly anymore"
You must not look at many academic websites.
Software will get slower as we do things beyond the current hardware capabilities which is always the case as hardware improves.
We have yet to reach the limit of the radius that we already have
https://www.dgtlgrove.com/p/near-the-zero-point
I had an Odin code base with raylib doing cpu bound stuff. On Catchy OS it tanked my frame rate from 120 to 90 fps (intentionally, seeing what I can get away with). Same build, on Windows 11, 40fps.
the OS makes a huge baseline difference