One of my first jobs, was as a maintenance engineer, on a 100KLoC+ codebase of 1979s-era FORTRAN IV.
No comments.
No subroutines (what we now call “functions”).
No variable name longer than 4 characters.
Fun. The most effective debug tool, was a Ouija board. It made me an expert WAGger.
It was the main reason that I am so anal about code Quality, these days. I never want to subject anyone else to that.
BTW: with today’s LLMs, there’s really no excuse for badly-documented, or badly-formatted code. You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
I'd phrase it differently; we're now able to accumulate technical debt faster than ever, without even building the institutional knowable needed to keep it sane. While the models writes novels about what it's doing that no human or LLM will find any use for.
But at the same time; if the LLM makes coding 5-10x faster, there's plenty of left-over time we can now spend doing things properly. Document, test, plan, refactor, lint, use CI tooling. There is no excuse now that LLMs reduce the pain threshold for all of them.
> we're now able to accumulate technical debt faster than ever
LLM's just enable you to speed run your way into a legacy code base.
> without even building the institutional knowable needed to keep it sane
Does feature XXX move the needle? Did you gain more business or retain existing business because a feature exits? Has AI tooling helped your product team move the needle? No? Why not?
How easy is it to remove the feature that NO ONE uses is a question no one is asking. How many people got promoted for "removing the most garbage" from the system?
We, as an industry, might need to have a candid conversation about what we're doing and how we do it.
How do you properly measure this? Do you setup a database that monitors when every feature is used? Is the feature only used once a year for some reason? Is the feature only used on some rare data that is uncommon but still can occur?
And that really applies to something fully in your control. In systems controlled by customers its far harder.
The same way you measure utilization and cost at a customer level.
I can name at lest three companies where their biggest clients are also the bulk of their costs - they lift revenue but drag the margins. I can name another couple who spent years marking their products entirely wrong because they simply had no clue how their product was really being used.
> Is the feature only used on some rare data that is uncommon but still can occur?
This is where the competence of your product team comes into play. We're building features faster with AI but none of it is moving the needle. That has little to nothing to do with code quality, and everything to do with product teams.
And yet technical debt is still much, much worse than it used to be thanks to LLMs.
As a technological innovation theyre a bit like amphetamines for code. Sure, maybe some people are using the "tankers chocolate" their employer handed them responsibly...I guess.
There are certainly a lot of people online who have gone all in who swear blind theyre not abusing them.
I've seen "clean" codebases that conform to "best practices" which are even less decipherable than what you describe.
100KLoC sounds like paradise compared to the latest codebase I touched. Having the signal to noise ratio fluctuate wildly at every member & file is highly distracting. When the information is dense and consistent, you can drop into a flow state more easily.
Four character variable names might sound awful but they can have an advantage. It's a form of compression once you are adapted to it. It forces you to keep things simple. When we can write an entire novel for a variable name, we may be tempted to inflate the scope of a solution.
No comments is universally a feature. If I want justification for a section of code, I am going to check git blame, PRs, linked issues, email, project management system, etc. The only code comments I value less than those written by humans are those written by LLMs. It is beyond pointless to shit up a codebase with this stuff. You could just ask the LLM to give you a live interpretation of the current state of the code instead of risking something falling out of sync.
Eh. I've found a "happy medium." I generally restrict comments to headerdocs at the function and property level. I use descriptive names, and keep the functions relatively small and task-oriented, so they do a fairly decent job of documenting themselves.
Every now and then, I'll insert a "why" comment, if I feel it's necessary.
Greetings fellow traveler, I did the same. FORTRAN IV, I was tasked with finding a way to introduce subroutines. But it was a tangled mess of gotos with loops inside loops and overlapping loops and spectacular jumps in and out. It was an obscure undocumented algorithm for calculating the thermodynamic properties of turbine stages or some such.
Running code through a deterministic formatter definitely helps a lot, but I've generally found LLMs' comments to add no-to-negative value. They just say what code does, typically very verbosely that just inflates files more while also just stating the most obvious aspects of the code and not adding any meaning behind it because they obviously don't know that. Not to the mention all the cases where they make mistakes and then you have code you don't intuitively understand and also have comments telling you it does something it doesn't.
Or worse, the comments are a tutorial through the changes for your review, referring to something that now doesn't exist because of the changes you're making. Less than informative, literally backwards facing.
The worst thing about Ouija board debugging is the latency and bitrate. Honestly, I can tolerate a puzzle project. It’s almost satisfying. But sitting there and waiting for the response is agonizing.
Perhaps not the most reasonable thing ever to rely on an LLM for formatting given that there exist good old zero-token-using formatters for pretty much any language under the sky.
> BTW: with today’s LLMs, there’s really no excuse for badly-documented, or badly-formatted code. You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
And how do you know that the documentation is correct? Because if it's not, it's worse than being absent. The verification work sounds close to as hard as writing it in the first place.
This is what I never grasp when people suggest LLMs for anything precise (outside of cases where the LLM output is in a machine-verifiable language).
This has been a nice thing about using doctests in elixir. doc examples are compiled and run so they can’t drift. The prose around the examples can still drift of course
> You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
Yes, you can tell the LLM to format and document it. You can also tell an effigy of Richard Nixon, or write it on a piece of paper and burn it. Of course you can do such things, but the important question is what that gains you.
Yesterday I vibe slop coded something, being very lazy about it, it being a throwaway experiment. Gemini wrote this for me:
Note the function name "loadFolder", and the short distance between the definition and where it gets called... So after a bunch of other changes, one change it made completely broke everything. I didn't check any of the code, but just described the symptoms etc.
first fix attempt:
// Call your backend loader safely
if (typeof loadFolderFiles === "function") {
loadFolderFiles(nodePath);
} else if (typeof loadFiles === "function") {
loadFiles(nodePath);
}
second fix attempt:
// Call your app's existing folder loader
if (typeof loadFolderFiles === "function") {
loadFolderFiles(nodePath);
} else if (typeof loadFiles === "function") {
loadFiles(nodePath);
}
> Most likely, your original code either passed childNode to a function like selectFolder(node) or sent a specific fetch() request. Here is the updated renderTree function [..]
if (typeof selectFolder === "function") {
selectFolder(childNode);
} else if (typeof onFolderSelect === "function") {
onFolderSelect(nodePath);
} else if (typeof loadFolderFiles === "function") {
loadFolderFiles(nodePath);
} else if (typeof loadFiles === "function") {
loadFiles(nodePath);
} else {
// Direct API fetch fallback if your backend uses a standard endpoint
fetch(`/api/files?path=${encodeURIComponent(nodePath)}`)
.then(res => res.json())
.then(data => {
if (typeof renderFileList === "function") renderFileList(data);
})
.catch(err => console.error("Error fetching folder files:", err));
}
I can only imagine what is going on out there right now, but I fully assume most of it is not very good, lots of it horrifying crap that technically, kinda works.
The problem with using LLMs to write Javascript is that they’re trained on Javascript written by humans. Which is not the training corpus I’d choose to teach a neural net to write code.
I laughed. Point noted. And yet, what do you have that's better?
We could train it on a carefully-curated best-practices corpus, but that's likely to be considerably smaller. It might not have the breadth of coverage that one would wish for a training corpus.
I’m saying that Javascript is particularly problematic here. The comparisons to “function” in the examples above are something you’d be unlikely to see in most other languages.
Using Javascript is like prompting the LLM “You are a marginally competent developer…” Not that there are no good Javascript developers, but the average quality of JS code tends to be lower, encouraged by the language’s many misfeatures.
I’ve seen an LLM fix an import error in Python by wrapping it in an exception and trying a different import if it fails. Again, this is a function of the language’s weaknesses. In languages with good module systems, an LLM couldn’t do this because it’s typically not an available feature.
> Technical debt has no bankruptcy, no clean reset
There definitely is a technical debt bankruptcy option.
You can stop using a particular piece of code or technology, e.g. by replacing it (with new code, or a third-party/vendor solution), or re-architecting a system or business process so that the code's function is no longer needed.
This is exactly what the technical debt metaphor means. Short-lived systems can accumulate a lot of technical debt without as much concern, because you're planning to declare bankruptcy (deprecate and decommission) the code soon anyways; long-lived systems must plan to pay off their technical debt on the usual installment plan.
Yes, a rewrite or replacement is the software analogue of the debt metaphor.
Section 3 of the article elaborates more on the technical debt statement in the TL;DR, but the gist of the argument there is that for a lot of systems, the rewrite is not really an option. Pretending it is an option is a partial contributor to the attitudes that make the debt spiral (or so I argue).
Yes, sometimes a rewrite is possible but the success stories are uncommon and are far outnumbered by failures.
Insightful article. The author makes some great points about the dynamics of technical debt in organizations.
I think a sinking ship is a reasonably good metaphor though. A sunken ship is a horrible, terrible outcome.
In the metaphor, a codebase is sunken when it’s unviable to continue using it. The organization stops working on it and no longer runs it. At this point, the code is completely worthless to the business, like a ship at the bottom of the ocean. True, you could imagine the code getting worse theoretically. But in reality, it will just sit there and rot.[0] The business might attempt a rewrite or just discontinue the product.
It’s possible, but not a given, that a business sinks with one of its codebases. This can happen with a ship too, if a business relies heavily on it.
1. LLM can fix this in my experience. an LLM, especially the more modern ones, have way more short term memory than most humans (or at least way more than me). They can dig through this kind of code and figure out all the edge cases, write test, suggest various paths to make things better and then execute on those paths. On request they will happily setup dev systems, staging systems, whatever it takes to make the transition safe. At least that's my experience. They can dig much further than I ever would.
2. Short of and maybe separate to the LLM fix, this pattern of technical debt I think is nearly inevitable, at least with humans. In a perfect world, every human and every reviewer knows exactly what architecture to write and what tests to convey all of the rules and assumptions because no matter what, people are going to leave. I've never seen that code base though. So the rules and assumptions are at best half written down, maybe in some comments or docs, comments or docs that the next person to edit that part of the code base may or may not see. And so it goes.
I work on a code base that runs on Windows, Mac, Linux, Android, iOS. Those OSes change over time, their requirements change, their APIs change, the world changes and new APIs are needed for new things people do, and our original choices for cross platform solutions, no longer fit perfectly. We need to keep moving and shipping and we can't just stop the world and re-architect. Further, like the OP, not everything is written down and communicating across hundreds of programmers is hard. So yes, not every decision is perfect. It never will be. Fortunately we work to fix these things and pay down our technical debt but it can take 5 to 10 years before we can finally remove some old path while we wait on usage numbers for old OSes to go down far enough that we can remove those paths and switch everyone over to the new. And then the cycle repeats.
You are the very first person I encountered who claim LLM can fix technical debt. Usually I seem comments and articles saying LLM can only produce it. And I can agree with those articles and comments
Here's the result of a prompt that specifically looks for correctness and simplifications to make "by construction" before I added "Design bar: ensure correctness by construction rather than by convention" to AGENTS.md:
I did. Just about all of the findings were re-verified to be good and then implemented by a rigorous pipeline of clankers, and the project was improved.
If it weren't for me insisting that I manually sign off on the solution of every finding, then it would have been fully automated too.
There is a lot of noise out there from anti-AI types who never seriously tried to clean up messes with AI. If you just ask for new features and never look at the code, coding agents will make a mess. If you often ask questions about how to clean up messy code, coding agents are great help for that, too. But you have to ask.
Agents (and humans) don't know what cleanups are safe or a good idea. They can get stuck in loops when different goals (conciseness, performance, etc.) clash. Their "improvements" can break things unintentionally. Targeted improvements can be good. Giant lists of "fix everything" are bad and will break things.
Tech debt comes with insufficient tests, so you won't know what you've broken until too late in many cases.
Yeah that's why step 0 is to tell the LLM to write a test suite for the thing you want to rewrite. It'll do that, but it does require adult supervision. Once you have that test suite, you can send the LLM into an inner dev loop. Like human programmers, LLMs are great when they have a fast inner loop.
The fact that LLMs exist doesn't mean you can just stop thinking. It mean the things you have to think about will be different. You have to treat them as savants with absolutely no ambition.
There’s a trap here though: the LLMs if you let them will spend all their time maintaining that test suite, they can become quite rigid in the way they design them. So it’ll write some code, break a test, which breaks a contract so then it updates the test and the contract but updating the test also broke another contract… soon enough you’re spending all your tokens fixing contracts instead of writing code.
In a sense the LLM having a fast inner loop is its blessing and curse. A blessing because it gets feedback quickly, but a curse because it becomes naval gazing and cannot see the forest for the trees.
At least this is my experience with Sol maby other models behave differently.
Yes like any tool, LLMs can be used poorly. But they do enable large-scale rewrites or poor-quality code because they are capable of understanding it and enumerating its behaviors in a matter of minutes or hours. It's the kind of understanding that would take a human months or years to develop.
Add to that their ability to basically set up what is essentially a perfect testing environment when asked, and you've got a feedback loop that lets you just step back while the agent cranks out a new implementation in a memory-safe language with a full test suite and bug-for-bug compatibility. I'm not kidding or exaggerating. This stuff is possible now, people just need to look past their anxieties about being replaced.
I’ll be the second then. I’ve found it’s very good at creating debt but equally good at resolving it. The thing is the default mode is creating debt so you have to intentionally pay it down.
Refactors that used to take me a month now take a week, it’s very handy. You can instruct them to move functions around, change interfaces, add or remove abstractions, remove redundant authorities, untangle spaghetti, rename identifiers across a codebase, and it will return very good results.
> an LLM, especially the more modern ones, have way more short term memory than most humans
But it's not unlimited. So they can write code that they "understand" within the limits of their working context. When it grows beyond that, they can't cope with it in its entirety. And a human, at that point, has no prayer.
For 1, effectiveness still depends on the size and complexity of the codebase.
From my experience, as complexity and size grow, each prompt takes longer, does less, and is prone to more mistakes and disruptions to other parts of the codebase.
The code base I work on is > 30 million lines (excluding comments and blank lines) and most (but not all) of the people on the team are finding it pretty effective. That doesn't mean it wouldn't be more effective on a smaller code base.
I worked one place where I had a team of 4 offshore devs replace my two good people. Three months of some of the worst quality work I’ve ever seen. And I was responsible for it on paper.
Decided fuck it, promoted one of them to manager and gave him a project to finish in 3 months, leading the other 3.
In that three months I redid that entire project, did their current project and a week of another one. All in a 25 hour week calendar.
The worst experience I had was with a telecom company. They had an internal web app that allowed them to upload a gigantic CSV file with millions of records from new phone activations. The web app took 59 minutes to process it!
The culprit: a single 30K .js file with one function, more and more recursive functions, and deeply nested if/else statements, up to 10 levels.
We spent one month undoing that Gordian knot and got the same process down to less than a minute. With AI, we probably would have done it in less than a day, and with a better programming language, probably even faster.
If it only took 59 minutes to process millions of records, it was nowhere near as bad as it could have been. At around one millisecond per record, it sounds like you probably don’t even have any accidental quadratic time complexity on the number of records! What are you complaining about?
As I read this article, as a non-coder (for a living) I noticed that what is being assumed is that the code produces a desired result. To that I will agree. You can really produce some bad code that has a positive outcome.
But as a counter proposal, I will submit that code can get so bad, that the product no longer produces a desirable result, no longer produces it in a manner that people want, or doesn't produce it at all.
Code that produces no desirable result or no result at all. Is as bad as it's going to get. It's a worthless pile of ASCII characters.
Everyone likes to pretend that LLMs are only capable of writing mudballs, but it's trivially debunked by using LLMs yourself to refactor code, pay back debt, and fan out agents to look for debt to repay.
We're already at the point with sota models where I'm not even sure you can get the sort of mudballs OP is talking about; the LLM's inherent "taste" forbids it, and it can just end-to-end refactor as requirements change.
The hypermudballs were distinctly a human creation due to how expensive it is to generalize and refactor brittle, incremental production code.
I'm willing to keep an open mind but the only example you've shared of such a success story and debunk was a LLM clearing the technical debt of a codebase that was fully LLM-written from the very first commit?
If LLMs are so good at fixing tech debts, why did it write so much to begin with? And why do you assume that now there's no more debt?
It's like trying to argue with someone saying they actually are going to build the whole plane out of the black box. At some point we should just cut our losses.
It's an incremental project covering a complex domain. At any time, the code has to stay balanced in a spot that addresses correctness and performance across things like pty parsing, a terminal grid, stateful pty behavior, rendering, scrollback history, font glyphs, and like all software there is a fractal of idiosyncrasies that sprout up. And that's just the terminal side.
Unless you can see the future, like most software, so much of the project is trying things and seeing what happens at the periphery, like how to represent "row damage" and apply it in a way that works with macOS Core Graphics and 1000 other things.
e.g. Can you tell me off the top of your head what the trade-offs are of representing the terminal contents/scrollback as a pre-wrapped grid vs. a list of logical lines that are soft-wrapped on the fly? Did you enumerate them all and correctly weigh them such that you can pick the ideal solution ahead of time? No. You just kinda make educated decisions and find out in practice what the exact weight of the trade-offs are.
> why do you assume that now there's no more debt?
I don't. Like all software, you try to ratchet into better and better positions, and you try to come up with metrics that can tell you "You are here" with regard to rather fuzzy goals like "it should be correct and performant and nice to use".
But I can look at individual findings and their solution to go "yes, this improves the code." And on principle I think that, e.g. correct/DRY/simple by-construction is superior to by-convention, and I assume it helps future agents reason about the system and make future edits, and it gives them singular places to make changes that impact multiple components just like it would a human. But how much did it actually matter?
While I agree and I have done more than one rewrite / cleanup of messy codebases quite successfully with LLMs in the past few months, I can assure you plenty of people are still using the latest models to accrue technical debt faster than I thought was ever possible. The model "taste", assuming it has one, does not survive bad instructions
Like real life, you want to accrue debt strategically.
When you're experimenting, vetting ideas, mapping out the requirements, exploring solutions, then technical debt keeps churn cheap.
e.g. You don't want to overly commit to static types that keep "impossible state unrepresentable" too early on since you aren't even sure yet what impossible state looks like in the system.
So we regard debt repayment as some sort of transformation we'll do in the future that never comes because it's hard and expensive, but not to LLMs.
Because we have evidence otherwise, and even the worst code can be refactored by LLMs.
That a high velocity project might accumulate technical debt isn't interesting to me if LLMs can also pay it back or if you can decide to work at a different pace where you polish the architecture as you go instead of accumulating debt.
I'd make the opposite claim to you: people really don't want my claims to be true, probably because it robs us of our value and expertise as software engineers. But it's getting a bit late in the game to still be dancing around that pill to swallow.
the models have gotten very good at doing what you ask. if you ask for changes that will accrue tech debt, you'll get it. if you ask for changes that pay down that debt, you'll get it.
I recently got a thank you and a refund for towels I had returned from Amazon.
The towels were damp from bring used, I had never initiated a refund return, nor had I in fact returned them.
I reported this to amazon and had to find a human since the system had no "refund for nonexistent returns" option. The human didn't get it and pinged his team lead. After 20 minutes they told me to just keep my towels and refund.
Yet, our organization was hundreds of people, and the system had grown so large and complex that it had become impossible to learn how it all worked.
Which is what makes Bezos' service mandate [0] a classic exposition of Conway's Law [1].
All teams will henceforth expose their data and functionality through service interfaces.
Teams must communicate with each other through these interfaces.
The worst I have experienced:
A 30,000-line .cs file.
A giant monolithic program tangled together by Singleton abuse, 6-deep nested if/else blocks, and the list goes on.
There are too many to count. And this is a sight I see every single day.
This is the exact landscape I encounter at many companies when I go in for maintenance work.
Encapsulation completely shattered as a direct reflection of the subcontracting power hierarchy, among other things...
Code that completely disregards collaboration, utilizing outdated algorithms in the name of "optimization" and making it utterly unmaintainable for anyone else.
We learn about ADTs, Result-first approaches, Composition Roots, and code hygiene, yet at some point, code that simply "works" becomes what ships to production.
Communities advocate for building "good software," but the reality in the field is that most of it is "code written just to clock out for the day." I hear "it works, so don't touch it" in dozens of different variations.
The quality of an open-source project contrasts sharply with that of delivered enterprise software. People settle for the "if it works, it's fine" mindset because in open source, the code itself represents personal reputation, whereas in contract deliveries, meeting the deadline is ultimately the core objective...
I was mainly .NET dev in the first 8 years of my career and it's a pretty common thing among .NET shops/corporates. Most .NET devs I worked with seemed to be on average older guys who are stuck in .NET Framework 3.x days. I joined a place that used .NET 5 and one guy was still writing hash code and equal methods for every single data class he made(by hand too). When I asked him why, he said that he didn't know he doesn't have to do that anymore. Those people are mostly alright/mediocre devs who just refuse to keep up with the times, clock in, clock out, don't think too much, just write some more code to make it work. If someone a decade ago started some convoluted architectural pattern, they'll just continue doing it, no thought given as to why or what for, whether things can be done better.
While I have different, but equally annoying issues with Node/Js world I work in now, I don't miss those .NET days.
Older developers in the Node ecosystem still sometimes write callback hells, but it's true that it's less prevalent than in the .NET world.
The problem is that I am the one who has to maintain that code. The reality is that code ultimately follows the prevailing trends and paradigms of its era. Most developers tend to stop growing around their 5th to 7th year (this is purely my observation). They settle into the complacency of having shipped a single complete program, an experience they usually hit around their 4th year. As a result, when you do actual maintenance work, you end up facing things like C++98 code or .NET codebases where the dependency graph is a complete disaster.
Compounding the issue is that these types of companies have very little technical exchange with the outside world. And that specific way of writing code ends up acting as a sort of gatekeeping mechanism.
Sometimes, I feel like programming ultimately boils down to how well you can read and adapt to these shifts in paradigms. The coding trends of today will likely become the outdated legacy of tomorrow, and someone might eventually tell me, "You write old code." It is always a difficult problem.
We found a 4000 Loc single-file single-function c program running a functional safety domain. With nesting so deep that you need to change the font-size or get a wider monitor. We're not allowed to touch or replace it because it's pre-certified.
Deeply nested if/else blocks are logically the same if you split them up into classes or functions that perform the same logic. They are harder to understand, harder to re-use, harder to work with in an editor... but should not be harder to test (you need to test all the possible use cases either way).
I've had a very similar experience. Ironically, simple frameworks like WinForms, or tools that promise easy and rapid development, often turn into the absolute worst-case scenarios for maintenance.
This tendency is especially strong in code-behind frameworks. Because the domain logic directly relies on the state of UI controls, the business logic becomes entirely entangled with the view itself. And what is the result? Every single architectural layer ends up stuffed inside a specific event handler.
I could understand if this came from a small outsourcing shop, but what baffled me was that this was the codebase of the top company in the world in that specific domain.
And to top it off, you sometimes have to integrate with an SDK where the methods are literally named function1, function2, and so on. It is always a thrilling experience.
The funny thing is that these days they would make it a lot worse with LLMs, perhaps post lots of wins on LinkedIn too. It’s not impossible to steer that ship but it’s never gonna be one shotted in “do refactor, make no mistake” way
i've heard the same thing about pain which may be an interesting argument against the resurrection or at least that Jesus died for my sins and not because of them
One of my first jobs, was as a maintenance engineer, on a 100KLoC+ codebase of 1979s-era FORTRAN IV.
No comments.
No subroutines (what we now call “functions”).
No variable name longer than 4 characters.
Fun. The most effective debug tool, was a Ouija board. It made me an expert WAGger.
It was the main reason that I am so anal about code Quality, these days. I never want to subject anyone else to that.
BTW: with today’s LLMs, there’s really no excuse for badly-documented, or badly-formatted code. You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
> LLMs, there’s really no excuse
I'd phrase it differently; we're now able to accumulate technical debt faster than ever, without even building the institutional knowable needed to keep it sane. While the models writes novels about what it's doing that no human or LLM will find any use for.
But at the same time; if the LLM makes coding 5-10x faster, there's plenty of left-over time we can now spend doing things properly. Document, test, plan, refactor, lint, use CI tooling. There is no excuse now that LLMs reduce the pain threshold for all of them.
From what I've heard, there is no time for any of those things because management also expects features 5-10x faster.
> we're now able to accumulate technical debt faster than ever
LLM's just enable you to speed run your way into a legacy code base.
> without even building the institutional knowable needed to keep it sane
Does feature XXX move the needle? Did you gain more business or retain existing business because a feature exits? Has AI tooling helped your product team move the needle? No? Why not?
How easy is it to remove the feature that NO ONE uses is a question no one is asking. How many people got promoted for "removing the most garbage" from the system?
We, as an industry, might need to have a candid conversation about what we're doing and how we do it.
Remove a feature no one uses:
How do you properly measure this? Do you setup a database that monitors when every feature is used? Is the feature only used once a year for some reason? Is the feature only used on some rare data that is uncommon but still can occur?
And that really applies to something fully in your control. In systems controlled by customers its far harder.
> How do you properly measure this?
The same way you measure utilization and cost at a customer level.
I can name at lest three companies where their biggest clients are also the bulk of their costs - they lift revenue but drag the margins. I can name another couple who spent years marking their products entirely wrong because they simply had no clue how their product was really being used.
> Is the feature only used on some rare data that is uncommon but still can occur?
This is where the competence of your product team comes into play. We're building features faster with AI but none of it is moving the needle. That has little to nothing to do with code quality, and everything to do with product teams.
And yet technical debt is still much, much worse than it used to be thanks to LLMs.
As a technological innovation theyre a bit like amphetamines for code. Sure, maybe some people are using the "tankers chocolate" their employer handed them responsibly...I guess.
There are certainly a lot of people online who have gone all in who swear blind theyre not abusing them.
I've seen "clean" codebases that conform to "best practices" which are even less decipherable than what you describe.
100KLoC sounds like paradise compared to the latest codebase I touched. Having the signal to noise ratio fluctuate wildly at every member & file is highly distracting. When the information is dense and consistent, you can drop into a flow state more easily.
Four character variable names might sound awful but they can have an advantage. It's a form of compression once you are adapted to it. It forces you to keep things simple. When we can write an entire novel for a variable name, we may be tempted to inflate the scope of a solution.
No comments is universally a feature. If I want justification for a section of code, I am going to check git blame, PRs, linked issues, email, project management system, etc. The only code comments I value less than those written by humans are those written by LLMs. It is beyond pointless to shit up a codebase with this stuff. You could just ask the LLM to give you a live interpretation of the current state of the code instead of risking something falling out of sync.
Eh. I've found a "happy medium." I generally restrict comments to headerdocs at the function and property level. I use descriptive names, and keep the functions relatively small and task-oriented, so they do a fairly decent job of documenting themselves.
Every now and then, I'll insert a "why" comment, if I feel it's necessary.
I wrote about my technique (and reasons), here: https://littlegreenviper.com/leaving-a-legacy/
Greetings fellow traveler, I did the same. FORTRAN IV, I was tasked with finding a way to introduce subroutines. But it was a tangled mess of gotos with loops inside loops and overlapping loops and spectacular jumps in and out. It was an obscure undocumented algorithm for calculating the thermodynamic properties of turbine stages or some such.
Good times.
Running code through a deterministic formatter definitely helps a lot, but I've generally found LLMs' comments to add no-to-negative value. They just say what code does, typically very verbosely that just inflates files more while also just stating the most obvious aspects of the code and not adding any meaning behind it because they obviously don't know that. Not to the mention all the cases where they make mistakes and then you have code you don't intuitively understand and also have comments telling you it does something it doesn't.
Or worse, the comments are a tutorial through the changes for your review, referring to something that now doesn't exist because of the changes you're making. Less than informative, literally backwards facing.
But that's what stackoverflow looks like.
The worst thing about Ouija board debugging is the latency and bitrate. Honestly, I can tolerate a puzzle project. It’s almost satisfying. But sitting there and waiting for the response is agonizing.
Perhaps not the most reasonable thing ever to rely on an LLM for formatting given that there exist good old zero-token-using formatters for pretty much any language under the sky.
> BTW: with today’s LLMs, there’s really no excuse for badly-documented, or badly-formatted code. You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
And how do you know that the documentation is correct? Because if it's not, it's worse than being absent. The verification work sounds close to as hard as writing it in the first place.
This is what I never grasp when people suggest LLMs for anything precise (outside of cases where the LLM output is in a machine-verifiable language).
This has been a nice thing about using doctests in elixir. doc examples are compiled and run so they can’t drift. The prose around the examples can still drift of course
Only problem is the LLM comments are the exact same quality as the code. In other words, LLM comments also have to be thoroughly reviewed.
> You can write a thousand lines of commercial-grade spaghetti, and tell the LLM to format and document it.
Yes, you can tell the LLM to format and document it. You can also tell an effigy of Richard Nixon, or write it on a piece of paper and burn it. Of course you can do such things, but the important question is what that gains you.
Yesterday I vibe slop coded something, being very lazy about it, it being a throwaway experiment. Gemini wrote this for me:
Note the function name "loadFolder", and the short distance between the definition and where it gets called... So after a bunch of other changes, one change it made completely broke everything. I didn't check any of the code, but just described the symptoms etc.first fix attempt:
second fix attempt: third: and finally:> Most likely, your original code either passed childNode to a function like selectFolder(node) or sent a specific fetch() request. Here is the updated renderTree function [..]
I can only imagine what is going on out there right now, but I fully assume most of it is not very good, lots of it horrifying crap that technically, kinda works.The problem with using LLMs to write Javascript is that they’re trained on Javascript written by humans. Which is not the training corpus I’d choose to teach a neural net to write code.
Well then you'd have somewhat of a chicken-and-egg problem. But humans came first so at least there's that.
I laughed. Point noted. And yet, what do you have that's better?
We could train it on a carefully-curated best-practices corpus, but that's likely to be considerably smaller. It might not have the breadth of coverage that one would wish for a training corpus.
I’m saying that Javascript is particularly problematic here. The comparisons to “function” in the examples above are something you’d be unlikely to see in most other languages.
Using Javascript is like prompting the LLM “You are a marginally competent developer…” Not that there are no good Javascript developers, but the average quality of JS code tends to be lower, encouraged by the language’s many misfeatures.
I’ve seen an LLM fix an import error in Python by wrapping it in an exception and trying a different import if it fails. Again, this is a function of the language’s weaknesses. In languages with good module systems, an LLM couldn’t do this because it’s typically not an available feature.
> Technical debt has no bankruptcy, no clean reset
There definitely is a technical debt bankruptcy option.
You can stop using a particular piece of code or technology, e.g. by replacing it (with new code, or a third-party/vendor solution), or re-architecting a system or business process so that the code's function is no longer needed.
This is exactly what the technical debt metaphor means. Short-lived systems can accumulate a lot of technical debt without as much concern, because you're planning to declare bankruptcy (deprecate and decommission) the code soon anyways; long-lived systems must plan to pay off their technical debt on the usual installment plan.
Yes, a rewrite or replacement is the software analogue of the debt metaphor.
Section 3 of the article elaborates more on the technical debt statement in the TL;DR, but the gist of the argument there is that for a lot of systems, the rewrite is not really an option. Pretending it is an option is a partial contributor to the attitudes that make the debt spiral (or so I argue).
Yes, sometimes a rewrite is possible but the success stories are uncommon and are far outnumbered by failures.
Insightful article. The author makes some great points about the dynamics of technical debt in organizations.
I think a sinking ship is a reasonably good metaphor though. A sunken ship is a horrible, terrible outcome.
In the metaphor, a codebase is sunken when it’s unviable to continue using it. The organization stops working on it and no longer runs it. At this point, the code is completely worthless to the business, like a ship at the bottom of the ocean. True, you could imagine the code getting worse theoretically. But in reality, it will just sit there and rot.[0] The business might attempt a rewrite or just discontinue the product.
It’s possible, but not a given, that a business sinks with one of its codebases. This can happen with a ship too, if a business relies heavily on it.
[0] https://en.wikipedia.org/wiki/Software_rot
2 things
1. LLM can fix this in my experience. an LLM, especially the more modern ones, have way more short term memory than most humans (or at least way more than me). They can dig through this kind of code and figure out all the edge cases, write test, suggest various paths to make things better and then execute on those paths. On request they will happily setup dev systems, staging systems, whatever it takes to make the transition safe. At least that's my experience. They can dig much further than I ever would.
2. Short of and maybe separate to the LLM fix, this pattern of technical debt I think is nearly inevitable, at least with humans. In a perfect world, every human and every reviewer knows exactly what architecture to write and what tests to convey all of the rules and assumptions because no matter what, people are going to leave. I've never seen that code base though. So the rules and assumptions are at best half written down, maybe in some comments or docs, comments or docs that the next person to edit that part of the code base may or may not see. And so it goes.
I work on a code base that runs on Windows, Mac, Linux, Android, iOS. Those OSes change over time, their requirements change, their APIs change, the world changes and new APIs are needed for new things people do, and our original choices for cross platform solutions, no longer fit perfectly. We need to keep moving and shipping and we can't just stop the world and re-architect. Further, like the OP, not everything is written down and communicating across hundreds of programmers is hard. So yes, not every decision is perfect. It never will be. Fortunately we work to fix these things and pay down our technical debt but it can take 5 to 10 years before we can finally remove some old path while we wait on usage numbers for old OSes to go down far enough that we can remove those paths and switch everyone over to the new. And then the cycle repeats.
You are the very first person I encountered who claim LLM can fix technical debt. Usually I seem comments and articles saying LLM can only produce it. And I can agree with those articles and comments
Trivially debunked with the prompt "find technical debt to fix".
https://github.com/danneu/danterm/blob/749942ffa1198f520c8b7...
Here's the result of a prompt that specifically looks for correctness and simplifications to make "by construction" before I added "Design bar: ensure correctness by construction rather than by convention" to AGENTS.md:
https://github.com/danneu/danterm/blob/749942ffa1198f520c8b7...
Linking to an 8800 line claude written wall of text reinforces the parent's point.
How? Because it found too many opportunities to improve the code beyond what we'd want to review as a human?
Each file is a list of findings that includes the justification/verification of each finding in the same file.
This is just what work looks like, especially tech debt repayment. It's analysis, reasoning, and justification. Those produce words.
Sounds good, have fun with that.
I did. Just about all of the findings were re-verified to be good and then implemented by a rigorous pipeline of clankers, and the project was improved.
If it weren't for me insisting that I manually sign off on the solution of every finding, then it would have been fully automated too.
There is a lot of noise out there from anti-AI types who never seriously tried to clean up messes with AI. If you just ask for new features and never look at the code, coding agents will make a mess. If you often ask questions about how to clean up messy code, coding agents are great help for that, too. But you have to ask.
Agents (and humans) don't know what cleanups are safe or a good idea. They can get stuck in loops when different goals (conciseness, performance, etc.) clash. Their "improvements" can break things unintentionally. Targeted improvements can be good. Giant lists of "fix everything" are bad and will break things.
Tech debt comes with insufficient tests, so you won't know what you've broken until too late in many cases.
Yeah that's why step 0 is to tell the LLM to write a test suite for the thing you want to rewrite. It'll do that, but it does require adult supervision. Once you have that test suite, you can send the LLM into an inner dev loop. Like human programmers, LLMs are great when they have a fast inner loop.
The fact that LLMs exist doesn't mean you can just stop thinking. It mean the things you have to think about will be different. You have to treat them as savants with absolutely no ambition.
There’s a trap here though: the LLMs if you let them will spend all their time maintaining that test suite, they can become quite rigid in the way they design them. So it’ll write some code, break a test, which breaks a contract so then it updates the test and the contract but updating the test also broke another contract… soon enough you’re spending all your tokens fixing contracts instead of writing code.
In a sense the LLM having a fast inner loop is its blessing and curse. A blessing because it gets feedback quickly, but a curse because it becomes naval gazing and cannot see the forest for the trees.
At least this is my experience with Sol maby other models behave differently.
Software engineering also applies to the test suite. So its management requires supervision.
Sounds like one of the teams I left before project completion 13 years ago.
asking to clean up is not enough, it is needed to understand the code and fix the things the agents are getting wrong.
It's true that you need to ask good questions to get good results. If you try to understand the code yourself, you will ask better questions.
Yes like any tool, LLMs can be used poorly. But they do enable large-scale rewrites or poor-quality code because they are capable of understanding it and enumerating its behaviors in a matter of minutes or hours. It's the kind of understanding that would take a human months or years to develop.
Add to that their ability to basically set up what is essentially a perfect testing environment when asked, and you've got a feedback loop that lets you just step back while the agent cranks out a new implementation in a memory-safe language with a full test suite and bug-for-bug compatibility. I'm not kidding or exaggerating. This stuff is possible now, people just need to look past their anxieties about being replaced.
They can help with it if you point them in the right direction.
I’ll be the second then. I’ve found it’s very good at creating debt but equally good at resolving it. The thing is the default mode is creating debt so you have to intentionally pay it down.
Refactors that used to take me a month now take a week, it’s very handy. You can instruct them to move functions around, change interfaces, add or remove abstractions, remove redundant authorities, untangle spaghetti, rename identifiers across a codebase, and it will return very good results.
> an LLM, especially the more modern ones, have way more short term memory than most humans
But it's not unlimited. So they can write code that they "understand" within the limits of their working context. When it grows beyond that, they can't cope with it in its entirety. And a human, at that point, has no prayer.
For 1, effectiveness still depends on the size and complexity of the codebase.
From my experience, as complexity and size grow, each prompt takes longer, does less, and is prone to more mistakes and disruptions to other parts of the codebase.
The code base I work on is > 30 million lines (excluding comments and blank lines) and most (but not all) of the people on the team are finding it pretty effective. That doesn't mean it wouldn't be more effective on a smaller code base.
If you have ever worked with low-cost offshore contractors, you know this it true…
So true.
I worked one place where I had a team of 4 offshore devs replace my two good people. Three months of some of the worst quality work I’ve ever seen. And I was responsible for it on paper.
Decided fuck it, promoted one of them to manager and gave him a project to finish in 3 months, leading the other 3.
In that three months I redid that entire project, did their current project and a week of another one. All in a 25 hour week calendar.
Then I quit.
A lot of people are going to get a painful lesson in this if companies keep using LLMs to do all their programming.
Maintaining AI code is the new maintaining offshore code. I don't envy software engineers who are far from retirement.
The worst experience I had was with a telecom company. They had an internal web app that allowed them to upload a gigantic CSV file with millions of records from new phone activations. The web app took 59 minutes to process it!
The culprit: a single 30K .js file with one function, more and more recursive functions, and deeply nested if/else statements, up to 10 levels.
We spent one month undoing that Gordian knot and got the same process down to less than a minute. With AI, we probably would have done it in less than a day, and with a better programming language, probably even faster.
If it only took 59 minutes to process millions of records, it was nowhere near as bad as it could have been. At around one millisecond per record, it sounds like you probably don’t even have any accidental quadratic time complexity on the number of records! What are you complaining about?
I know a phone company where just the cronjob that sends bills took over 3 days to run. I guess it's not a problem until it takes more than a month.
As I read this article, as a non-coder (for a living) I noticed that what is being assumed is that the code produces a desired result. To that I will agree. You can really produce some bad code that has a positive outcome.
But as a counter proposal, I will submit that code can get so bad, that the product no longer produces a desirable result, no longer produces it in a manner that people want, or doesn't produce it at all.
Code that produces no desirable result or no result at all. Is as bad as it's going to get. It's a worthless pile of ASCII characters.
Oh and by the way, I will also submit that there is a form of bankruptcy for technical debt.
The company folds, or the software is no longer maintained or in use.
When was the last time you used word Perfect? Or MS-DOS?
This is a good example of what LLMs save us from.
Everyone likes to pretend that LLMs are only capable of writing mudballs, but it's trivially debunked by using LLMs yourself to refactor code, pay back debt, and fan out agents to look for debt to repay.
We're already at the point with sota models where I'm not even sure you can get the sort of mudballs OP is talking about; the LLM's inherent "taste" forbids it, and it can just end-to-end refactor as requirements change.
The hypermudballs were distinctly a human creation due to how expensive it is to generalize and refactor brittle, incremental production code.
I'm willing to keep an open mind but the only example you've shared of such a success story and debunk was a LLM clearing the technical debt of a codebase that was fully LLM-written from the very first commit?
If LLMs are so good at fixing tech debts, why did it write so much to begin with? And why do you assume that now there's no more debt?
It's like trying to argue with someone saying they actually are going to build the whole plane out of the black box. At some point we should just cut our losses.
Good questions.
> why did it write so much to begin with?
It's an incremental project covering a complex domain. At any time, the code has to stay balanced in a spot that addresses correctness and performance across things like pty parsing, a terminal grid, stateful pty behavior, rendering, scrollback history, font glyphs, and like all software there is a fractal of idiosyncrasies that sprout up. And that's just the terminal side.
Unless you can see the future, like most software, so much of the project is trying things and seeing what happens at the periphery, like how to represent "row damage" and apply it in a way that works with macOS Core Graphics and 1000 other things.
e.g. Can you tell me off the top of your head what the trade-offs are of representing the terminal contents/scrollback as a pre-wrapped grid vs. a list of logical lines that are soft-wrapped on the fly? Did you enumerate them all and correctly weigh them such that you can pick the ideal solution ahead of time? No. You just kinda make educated decisions and find out in practice what the exact weight of the trade-offs are.
You should see the amount of iterations a human needs on these kinds of projects. https://github.com/ghostty-org/ghostty has almost 18,000 commits. So does https://github.com/gnachman/iterm2. So does https://github.com/kovidgoyal/kitty.
> why do you assume that now there's no more debt?
I don't. Like all software, you try to ratchet into better and better positions, and you try to come up with metrics that can tell you "You are here" with regard to rather fuzzy goals like "it should be correct and performant and nice to use".
But I can look at individual findings and their solution to go "yes, this improves the code." And on principle I think that, e.g. correct/DRY/simple by-construction is superior to by-convention, and I assume it helps future agents reason about the system and make future edits, and it gives them singular places to make changes that impact multiple components just like it would a human. But how much did it actually matter?
While I agree and I have done more than one rewrite / cleanup of messy codebases quite successfully with LLMs in the past few months, I can assure you plenty of people are still using the latest models to accrue technical debt faster than I thought was ever possible. The model "taste", assuming it has one, does not survive bad instructions
Like real life, you want to accrue debt strategically.
When you're experimenting, vetting ideas, mapping out the requirements, exploring solutions, then technical debt keeps churn cheap.
e.g. You don't want to overly commit to static types that keep "impossible state unrepresentable" too early on since you aren't even sure yet what impossible state looks like in the system.
So we regard debt repayment as some sort of transformation we'll do in the future that never comes because it's hard and expensive, but not to LLMs.
People refuse to accept that there's more bad code/behavior/people that AI will empower and amplify than good.
Because we have evidence otherwise, and even the worst code can be refactored by LLMs.
That a high velocity project might accumulate technical debt isn't interesting to me if LLMs can also pay it back or if you can decide to work at a different pace where you polish the architecture as you go instead of accumulating debt.
I'd make the opposite claim to you: people really don't want my claims to be true, probably because it robs us of our value and expertise as software engineers. But it's getting a bit late in the game to still be dancing around that pill to swallow.
the models have gotten very good at doing what you ask. if you ask for changes that will accrue tech debt, you'll get it. if you ask for changes that pay down that debt, you'll get it.
I recently got a thank you and a refund for towels I had returned from Amazon.
The towels were damp from bring used, I had never initiated a refund return, nor had I in fact returned them.
I reported this to amazon and had to find a human since the system had no "refund for nonexistent returns" option. The human didn't get it and pinged his team lead. After 20 minutes they told me to just keep my towels and refund.
“Yeah, you’re right, but let’s do that refactor in a separate PR” and it never happens
Well, the "limit" is disassembled/obfuscated code, isn't it?
Removing complexity is 2 to +infinity times harder than adding it in the first place.
1. https://en.wikipedia.org/wiki/Conway%27s_law
The worst I have experienced: A 30,000-line .cs file. A giant monolithic program tangled together by Singleton abuse, 6-deep nested if/else blocks, and the list goes on. There are too many to count. And this is a sight I see every single day. This is the exact landscape I encounter at many companies when I go in for maintenance work. Encapsulation completely shattered as a direct reflection of the subcontracting power hierarchy, among other things...
Code that completely disregards collaboration, utilizing outdated algorithms in the name of "optimization" and making it utterly unmaintainable for anyone else.
We learn about ADTs, Result-first approaches, Composition Roots, and code hygiene, yet at some point, code that simply "works" becomes what ships to production.
Communities advocate for building "good software," but the reality in the field is that most of it is "code written just to clock out for the day." I hear "it works, so don't touch it" in dozens of different variations.
The quality of an open-source project contrasts sharply with that of delivered enterprise software. People settle for the "if it works, it's fine" mindset because in open source, the code itself represents personal reputation, whereas in contract deliveries, meeting the deadline is ultimately the core objective...
It feels like the world is always polarized.
I was mainly .NET dev in the first 8 years of my career and it's a pretty common thing among .NET shops/corporates. Most .NET devs I worked with seemed to be on average older guys who are stuck in .NET Framework 3.x days. I joined a place that used .NET 5 and one guy was still writing hash code and equal methods for every single data class he made(by hand too). When I asked him why, he said that he didn't know he doesn't have to do that anymore. Those people are mostly alright/mediocre devs who just refuse to keep up with the times, clock in, clock out, don't think too much, just write some more code to make it work. If someone a decade ago started some convoluted architectural pattern, they'll just continue doing it, no thought given as to why or what for, whether things can be done better. While I have different, but equally annoying issues with Node/Js world I work in now, I don't miss those .NET days.
Older developers in the Node ecosystem still sometimes write callback hells, but it's true that it's less prevalent than in the .NET world.
The problem is that I am the one who has to maintain that code. The reality is that code ultimately follows the prevailing trends and paradigms of its era. Most developers tend to stop growing around their 5th to 7th year (this is purely my observation). They settle into the complacency of having shipped a single complete program, an experience they usually hit around their 4th year. As a result, when you do actual maintenance work, you end up facing things like C++98 code or .NET codebases where the dependency graph is a complete disaster.
Compounding the issue is that these types of companies have very little technical exchange with the outside world. And that specific way of writing code ends up acting as a sort of gatekeeping mechanism.
Sometimes, I feel like programming ultimately boils down to how well you can read and adapt to these shifts in paradigms. The coding trends of today will likely become the outdated legacy of tomorrow, and someone might eventually tell me, "You write old code." It is always a difficult problem.
I can believe that... We've had logging macros break because line numbers no longer fit in an u16
I would love a TV show like "Hell's kitchen" except it's code bases and development processes.
> 6-deep nested if/else blocks
We found a 4000 Loc single-file single-function c program running a functional safety domain. With nesting so deep that you need to change the font-size or get a wider monitor. We're not allowed to touch or replace it because it's pre-certified.
Deeply nested if/else blocks are logically the same if you split them up into classes or functions that perform the same logic. They are harder to understand, harder to re-use, harder to work with in an editor... but should not be harder to test (you need to test all the possible use cases either way).
Last place I worked they had an ASP.Net MVC controller with basically the entire product in it. The .cs file was nearly 2 megs.
I've had a very similar experience. Ironically, simple frameworks like WinForms, or tools that promise easy and rapid development, often turn into the absolute worst-case scenarios for maintenance.
This tendency is especially strong in code-behind frameworks. Because the domain logic directly relies on the state of UI controls, the business logic becomes entirely entangled with the view itself. And what is the result? Every single architectural layer ends up stuffed inside a specific event handler.
I could understand if this came from a small outsourcing shop, but what baffled me was that this was the codebase of the top company in the world in that specific domain.
And to top it off, you sometimes have to integrate with an SDK where the methods are literally named function1, function2, and so on. It is always a thrilling experience.
The funny thing is that these days they would make it a lot worse with LLMs, perhaps post lots of wins on LinkedIn too. It’s not impossible to steer that ship but it’s never gonna be one shotted in “do refactor, make no mistake” way
i've heard the same thing about pain which may be an interesting argument against the resurrection or at least that Jesus died for my sins and not because of them