I disagree with the claim that "AI agents don't get lost." What I've observed instead is that they don't experience the sensation of feeling lost. Which is quite different.
This summer I spent quite a while using a coding agent to help me untangle a deep and complicated data processing pipeline. It had itself been built by agents, in a remarkably short amount of time. But it had also become clear that it was riddled with errors and was producing lots of bad data.
What I quickly discovered was that upwards of half of my questions would receive very confidently wrong answers. And even once I had finally diagnosed whatever problem I was currently working on, it was difficult to trust the agent with any bug fixes. Since it was having an even harder time tracing data flows than I was (I'll take this chance to submit for your consideration that faster is not necessarily better), it was proving to be a bit of a monkey's paw. Yes, it would fix the exact bug I asked it to fix, but typically introduce new defects in the process. And yes, I was having this struggle with all of the latest & greatest models.
I ultimately concluded that, in this codebase, the agent was indeed deeply, hopelessly lost. (edit: And probably this code got so bad in the first place because the agents that were used to build it had been lost for a while, but unable to recognize this problem and call their operators' attention to it.)
Yes, I've also noticed a few occasions when Claude would get it wrong, but in most cases it's able to find issues I didn't even consider because of a very deep analysis in a confusing (to humans) code. It detects some rare situations where a defect could exist. This happens when I explicitly ask Opus to review a PR and there's some harness around this ability, but I'm really impressed at how deep their analysis can be and correct as well. Of course, sometimes they're going to fail, but I don't see them getting lost often.
"Confidently wrong" is definitely a standard behavior model for LLMs.
I agree with the other reply that you're likely to get better results if it has some kind of test case to run that's more authoritative than its own reasoning.
I'm curious about your exact case. In my experience I often had luck with evidence based approaches where the agent had to prove something first in order to make a statement (or write/do some tests first before making claims).
I agree though that one has to be very careful when trying to "fix" things with agents in a big codebase without it introducing new defects.
Meanwhile I'm over here refactoring as much as I can from years (or decades) of human-slung code. Turning the mess I either inherited, helped create, or built on top of into something clean and pristine might be my favorite LLM use. Same for personal projects, codebases that evolved over many years when I happened to have time that weren't kept quite as "clean" as I wish that finally been cleaned up.
I've always _wanted_ my code to be clean and easy to follow but life, deadlines, shifting-priorities, etc have stood in the way of that. Now I can finally realize my personal nirvana.
That said, I've had to steer models away from too-heavy of abstraction or similar because it made the code too hard to follow.
I feel your pain, that's what I've been doing for the past whole week. I'm trying to fix some ancient bugs (there are lots of them in this codebase) but each Claude review detects so many issues that might happen on some rare cases that it takes days until it stops complaining and I can get some peer approval to get the fix merged.
It's been fun reviving some abandoned personal/side projects and seeing them through. I have a bunch of random partially completed code that I never got around to finishing and it's fun seeing it work even if I have no use for it. Tons of "proof of concepts" left at "concept" that never reached "proof" ha
I've found that access to coding agents has helped me be far less tolerant of bad code patterns that can be refactored.
Refactoring used to have a very real cost - it was substantial amounts of time that would have to be carved away from working on new features.
Now I can spot a potential refactor, fire off a prompt in an asynchronous coding agent (or on a worktree or whatever), then come back 20 minutes later and either accept it, poke it a bit, or abandon it. Costs me almost nothing.
And for me, it has created a giant pile of unmaintainable code from my coworkers because of stupid management people that think they can code.
Refactoring was never a substantial amounts of time for me before llms.
Before I could spot a potential refactor and refactor it in 20 minutes and less. Never abandon it. Just constant improvement to the point the previous tech startup that I was working for just drive from itself (I am still paid a a few hours per months for it)
If you were fortunate enough to be working in a project with very good test coverage, refactors were easier (maybe not easy) in the sense that you were at least fairly confident that nothing broke.
But I've personally never worked on code with test coverage that good. Refactors were always risky.
Do you work with other, potentially unmotivated devs/managers a lot? I wonder how much of this is an incentive issue. I don’t think refactoring itself is as much of an issue than working with 2-3 other engineers that DGAF and a manager who is only looking at LOC to determine who to promote.
But it also doesn’t mean these things aren’t problems, they’re obviously huge problems.
All sorts of efforts that used to be put off indefinitely can now be handled largely by LLMs. Yesterday I took a codebase (~50 source files) and spawned a sub-agent (GLM 5.3 Flash) for every single file. Each file was analyzed for test coverage issues, inconsistencies between comments and implementations, and all call sites against the implementation. Then issues were aggregated. I reviewed the list manually, had Opus and K3 review as well to prune the list, and then had a commit made for every minor issue found. 70 commits with maybe 30 minutes of manual work.
Not exactly a refactor, but high degrees of consistency are what I strive for in a codebase. LLMs get confused by inconsistencies, as do humans.
A lot of this feels like it comes down to the training of the agents to produce code that satisfies the various benchmarks combined with reactions to things which were previously maladaptive. I.e. things which were explicitly trained out of the model in post training. I think there's a lot of missing long term software engineering principles that don't seem to be baked into the way the models tend to write code by default.
I have some speculation that maybe the people doing the model post-training tend to be younger researchers that haven't worked on large complex software systems, so their taste isn't as developed in this regard about what things are important here.
But this is an area that can be steered with appropriate early instructions ("When choosing tradeoffs of implementation, build for long term maintainability and understandability of code over implementing just the exact code necessary to solve the issues. etc. chain of thought often includes information that would have to be repeated in a future agent session, make sure to persist it to code or external docs so that future sessions and user understanding is respected.")
It can also be done as a post-change step with similar effects. And you can use your agents to build this layer into your general modus operandi for dealing with the crimes of generated code. But one of the things that all AI labs should be doing is looking at AGENTS.md on real project as being hard expressions of what failure modes real projects have noticed in models generally. Don't wait fo the bugs to be raised on these things, use express preferences that show that there's a problem. Go trawl github for these in bulk to use for future post-training.
AI agents get lost all the time, particularly if the codebase is already sprawling out of control.
Your discipline only pays off if you already understand your code and/or established clear baseline for your standards before launching into a feature development mania. And it needs to be enforced every turn, or the firehose of code generation knocks the front door down easily.
- helps agents and engineers make better data model, data structure, design pattern, naming, and algorithm choices
- surfaces incorrect irregularities or outdated exceptions to a rule
- enables clean, independent upgrades of parts of a system to improve performance
- reduces stale references in code and comments (and the confusion that results, both from agents and humans)
Bad modularity is basically a summary of what usually constitutes pathological code in general, but AI systems seem particularly disposed to sling lots of it (at least humans are constrained in their output rate). How often have you tried to grok an AI-built project and found trivially unreusable code, unnecessary duplication (everywhere!), bad data structure and algorithm choices, and stale references?
I think I used Opus actually. But my prompt was huge, basically the article content. I told Opus what I wanted to approach in the article with all the relevant details I wanted to include and it created the article. Then I reviewed it.
But the part that agents don't get lost is actually part of the prompt, not something Opus wrote on its own. Of course they get lost sometimes but the idea of the article is to explain that agents can often make more sense of tangled code than humans when the bounds are not well defined.
> Here’s the problem. AI agents are not bound by human context limits in the same way. An agent can read the tangled function, trace every caller, and make sense of the mess that would have stopped a human cold. It can add the next branch correctly, and the one after that, working confidently inside code that no human on the team fully understands anymore.
They're not bound by the same limits but they're still bound by some limits, yeah?
I'm not an AI expert, so I don't honestly understand why LLM driven agents are as good as they are. But my impression is "trace every caller", most of the time, is still an approximation. Once the code has gotten convoluted enough, cases are going to get dropped.
Yes, of course, they're limited, but it doesn't compare to the limit humans face. We can't hold a lot of context on our minds when investigating a bug when the code is too complex with huge methods, lots of branches, several callers and so on. It doesn't mean agents are perfect but they don't care as much as humans if the code is a mess because they can find logic in mess, we can't.
They also very often do not actually read the fucking files! Even after specifying to „read in full the fucking files“. LLMs are so lazy, they take any shortcut they can.
Sorry, just thinking about it is reviving my frustration…
Yeah, you gotta make em prove their work so they don't hallucinate.
Try "build a script to trace callers" and "cite each file and line that calls the function"
Won't fix everything but usually helps in my experience
Although you still run into hard-to-find things especially in dynamic/duck typed languages or codebases with heavy use of reflection or code bases used as libraries
> Teams, em dash bla bla bla em dash have quietly bla bla bla. It's not this, it's _that_.
I find it hard to read articles where the agentic writing is this obvious. It's a distraction from the message of the text, which I'm sure is worth my time. Is there no way to stop generated writing from sounding like this?
The prompt was actually huge, containing all the central ideas of the articles in details. I actually found my first prompt in case you're curious:
---
Human contexts are way more limited compared to computers. We can't reason about complex software when they go through many branches with so many implications. It's just too hard for humans to keep track of all interconnected pieces. So humans have historically split the system parts into manageable modules that can be understood in isolation and then we spend some time connecting those parts. That's how we can keep the context reasonable for human understanding.
So, when senior engineers found themselves lost while trying to debug an issue in a complex system they would naturally decide to pause and rewrite or refactoring the confusing piece of the system to make it manageable so developers can easily understand what's going on and review future changes.
Usually a system doesn't start that confusing. But as requirements change developers add additional branches and code until the code is no longer manageable. Sometimes the requirements changed significantly since the code was first written and all we have in the code are exceptions rather than the rule. That's usually when historically senior developers would take the time to rewrite that part of the system so they can reason about it.
But AI agents are not as limited as humans context-wise and they can reason about those confusing (to humans) systems and make sense of it. So they simply keep adding additional branches to the existing mess without ever suggesting a major refactoring like a senior developer would do in those cases, unless there's specific harness to tell agents to act like that.
This article is about bringing this into attention so that developers can policy themselves and keep asking themselves whether it's time for a major refactoring instead of relying on the AI agents and trust them because they no longer understand the code because it's too complex for humans to follow. Can you draft an article focused on this concern?
I find the basic premise for low level code quality to be true, in my experience, but counterintuitively I can now police the overall structure and system architecture MUCH more heavily.
As ever, no one is willing to allocate time for this, but (with unlimited work tokens) I can parallel path massive cleanup refactors all the time now.
I've personally adopted doing multiple refactoring passes after any large code implementation done by AI. In pretty much any scenario where I'm adding code using AI, it's 1 turn to add the feature and and then another 4-5 turns to refactor and clean everything up.
Often times it's not even that the code is bad but rather that it's overengineered. I see it happen so much that I'm tempted to actually go the other way on a toy project. Like what would Claude or Codex come up with if I told it I wanted an enterprise grade, globally scalable, compliant and auditable tic-tac-toe game.
I've had the opposite experience where we're refactoring the gnarliest shit anyone's ever seen because AI can actually understand it well enough to decompose, test, refactor, etc.
This is pretty spot on imo. Although otoh I feel like I don't need to be able to reason as deeply about a system because of the ability of an agent to dig through a code base. Personally I believe I'm still looking for that new balance.
It's a little like driving a car in a neighborhood you know, vs one where you constantly need to be looking down at a map or gps. It's much more comfortable driving around the places you're familiar with, but you can't know everywhere.
Did I even tried to pretend the article wasn't generated with Claude's help?
I even shared the initial prompt in some comment in this thread. I don't understand what is the matter with also using AI help with generating articles. I still reviewed the article and put all the ideas I wanted to discuss in that article. I don't see why people are often complaining about this. The article content is much more important than how it was generated.
Let's say that tomorrow, due to an improved model or whatever, we realize that the most efficient form of code of an app - for an llm to understand and work with - is for it to be in one long spaghetti file.
Why wouldn't we do that? I think there's a point where this comes down to values instead of facts. If you want it to be human readable, that's fine and there are a bunch of therefores from that point. But if you don't necessarily want that for a particular codebase, why refactor if the LLMs can handle it?
I'd argue the things that make a codebase more human-readable is also what makes it more agent-readable though. Agents are trained on human data, after all.
The position in the article is reasonable because what would end up happening otherwise is:
- Agents increase complexity, humans can't read it anymore
- Agent unable to keep making updates without looping forever (the complexity of the code exceeds the agent's context length). Human doesn't understand either so can't fix.
This isn't hypothetical either, it's basically what ends up happening to most vibe-coded software if the person doing the vibe coding doesn't know how to review the outputs being produced.
>A computer can hold far more in “working memory” than a human can
That's a bit of a straw man. State of the art agents are limited to ~3.8Mi (1M tokens). That's usually where I run into issues--an LLM can't possibly hold as much context as a human and it's more of an art than a science getting the most important things squeezed in. It's especially prudent for complex codebases/systems.
An agent only knows what it can see. It doesn't know oldCruftyFunction is still critical to Bob's Excel macro that generates financial reports and yanks the codebase in as a bastardised dependency. A lot of times agents give a fall sense of security by making it seem like something complicated and unsafe is actually safe.
When project leads and directors only ask for features,
the humans start to eventually push back,
but agents do as you please, no matter what the cost.
They were trained to do what you ask,
but unlike with humans, you need to ask for the refactoring yourself.
-- They won't necessarily come up with it on their own.
(They also tend to not be around for long enough to live through the consequences of their tech debt actions.)
This'll be resolved by a selection process. Code that gets quadratically more complex will at some point fail to advance even with access to hundreds of genies.
I disagree with the claim that "AI agents don't get lost." What I've observed instead is that they don't experience the sensation of feeling lost. Which is quite different.
This summer I spent quite a while using a coding agent to help me untangle a deep and complicated data processing pipeline. It had itself been built by agents, in a remarkably short amount of time. But it had also become clear that it was riddled with errors and was producing lots of bad data.
What I quickly discovered was that upwards of half of my questions would receive very confidently wrong answers. And even once I had finally diagnosed whatever problem I was currently working on, it was difficult to trust the agent with any bug fixes. Since it was having an even harder time tracing data flows than I was (I'll take this chance to submit for your consideration that faster is not necessarily better), it was proving to be a bit of a monkey's paw. Yes, it would fix the exact bug I asked it to fix, but typically introduce new defects in the process. And yes, I was having this struggle with all of the latest & greatest models.
I ultimately concluded that, in this codebase, the agent was indeed deeply, hopelessly lost. (edit: And probably this code got so bad in the first place because the agents that were used to build it had been lost for a while, but unable to recognize this problem and call their operators' attention to it.)
Yes, I've also noticed a few occasions when Claude would get it wrong, but in most cases it's able to find issues I didn't even consider because of a very deep analysis in a confusing (to humans) code. It detects some rare situations where a defect could exist. This happens when I explicitly ask Opus to review a PR and there's some harness around this ability, but I'm really impressed at how deep their analysis can be and correct as well. Of course, sometimes they're going to fail, but I don't see them getting lost often.
"Confidently wrong" is definitely a standard behavior model for LLMs.
I agree with the other reply that you're likely to get better results if it has some kind of test case to run that's more authoritative than its own reasoning.
I'm curious about your exact case. In my experience I often had luck with evidence based approaches where the agent had to prove something first in order to make a statement (or write/do some tests first before making claims).
I agree though that one has to be very careful when trying to "fix" things with agents in a big codebase without it introducing new defects.
This is a really good take. Thanks for sharing this. I havent been able to put to words how I have felt about agents being untrustworthy.
Meanwhile I'm over here refactoring as much as I can from years (or decades) of human-slung code. Turning the mess I either inherited, helped create, or built on top of into something clean and pristine might be my favorite LLM use. Same for personal projects, codebases that evolved over many years when I happened to have time that weren't kept quite as "clean" as I wish that finally been cleaned up.
I've always _wanted_ my code to be clean and easy to follow but life, deadlines, shifting-priorities, etc have stood in the way of that. Now I can finally realize my personal nirvana.
That said, I've had to steer models away from too-heavy of abstraction or similar because it made the code too hard to follow.
I feel your pain, that's what I've been doing for the past whole week. I'm trying to fix some ancient bugs (there are lots of them in this codebase) but each Claude review detects so many issues that might happen on some rare cases that it takes days until it stops complaining and I can get some peer approval to get the fix merged.
It's been fun reviving some abandoned personal/side projects and seeing them through. I have a bunch of random partially completed code that I never got around to finishing and it's fun seeing it work even if I have no use for it. Tons of "proof of concepts" left at "concept" that never reached "proof" ha
Hear, hear. As a lazy dev, LLMs allow me to write so much better code than ever before, and refactor all the crap I never got around to fixing.
Same here. It's been very freeing mentally to get the 90% of things I always felt needed done actually done.
I've found that access to coding agents has helped me be far less tolerant of bad code patterns that can be refactored.
Refactoring used to have a very real cost - it was substantial amounts of time that would have to be carved away from working on new features.
Now I can spot a potential refactor, fire off a prompt in an asynchronous coding agent (or on a worktree or whatever), then come back 20 minutes later and either accept it, poke it a bit, or abandon it. Costs me almost nothing.
And for me, it has created a giant pile of unmaintainable code from my coworkers because of stupid management people that think they can code.
Refactoring was never a substantial amounts of time for me before llms. Before I could spot a potential refactor and refactor it in 20 minutes and less. Never abandon it. Just constant improvement to the point the previous tech startup that I was working for just drive from itself (I am still paid a a few hours per months for it)
I've never met a single developer saying refactors are easy. It's always a minefield.
In the only scenario where that could actually be true you wouldn't have coworkers creating unmaintainable code with LLMs now.
If you were fortunate enough to be working in a project with very good test coverage, refactors were easier (maybe not easy) in the sense that you were at least fairly confident that nothing broke.
But I've personally never worked on code with test coverage that good. Refactors were always risky.
Does that code at least come with tests?
Gotta get IT to slip superpowers or at least the TDD skill into MDM/enterprise controls
Do you work with other, potentially unmotivated devs/managers a lot? I wonder how much of this is an incentive issue. I don’t think refactoring itself is as much of an issue than working with 2-3 other engineers that DGAF and a manager who is only looking at LOC to determine who to promote.
But it also doesn’t mean these things aren’t problems, they’re obviously huge problems.
All sorts of efforts that used to be put off indefinitely can now be handled largely by LLMs. Yesterday I took a codebase (~50 source files) and spawned a sub-agent (GLM 5.3 Flash) for every single file. Each file was analyzed for test coverage issues, inconsistencies between comments and implementations, and all call sites against the implementation. Then issues were aggregated. I reviewed the list manually, had Opus and K3 review as well to prune the list, and then had a commit made for every minor issue found. 70 commits with maybe 30 minutes of manual work.
Not exactly a refactor, but high degrees of consistency are what I strive for in a codebase. LLMs get confused by inconsistencies, as do humans.
> then had a commit made for every minor issue found
Interesting choice of words. Not a fix. Not a refactor. A commit.
Not sure what you’re getting at
Test coverage for its own sake is not always a worthwhile goal.
Unless it was indeed valid coverage? You are assuming it was all bad
A lot of this feels like it comes down to the training of the agents to produce code that satisfies the various benchmarks combined with reactions to things which were previously maladaptive. I.e. things which were explicitly trained out of the model in post training. I think there's a lot of missing long term software engineering principles that don't seem to be baked into the way the models tend to write code by default.
I have some speculation that maybe the people doing the model post-training tend to be younger researchers that haven't worked on large complex software systems, so their taste isn't as developed in this regard about what things are important here.
But this is an area that can be steered with appropriate early instructions ("When choosing tradeoffs of implementation, build for long term maintainability and understandability of code over implementing just the exact code necessary to solve the issues. etc. chain of thought often includes information that would have to be repeated in a future agent session, make sure to persist it to code or external docs so that future sessions and user understanding is respected.")
It can also be done as a post-change step with similar effects. And you can use your agents to build this layer into your general modus operandi for dealing with the crimes of generated code. But one of the things that all AI labs should be doing is looking at AGENTS.md on real project as being hard expressions of what failure modes real projects have noticed in models generally. Don't wait fo the bugs to be raised on these things, use express preferences that show that there's a problem. Go trawl github for these in bulk to use for future post-training.
AI agents get lost all the time, particularly if the codebase is already sprawling out of control.
Your discipline only pays off if you already understand your code and/or established clear baseline for your standards before launching into a feature development mania. And it needs to be enforced every turn, or the firehose of code generation knocks the front door down easily.
[dead]
> The same modularity that keeps a system inside a human’s head keeps each change inside a well-defined boundary the agent can reason about reliably.
Not only that. Good modularity also:
- improves code reusability, reduces unnecessary code duplication
- helps agents and engineers make better data model, data structure, design pattern, naming, and algorithm choices
- surfaces incorrect irregularities or outdated exceptions to a rule
- enables clean, independent upgrades of parts of a system to improve performance
- reduces stale references in code and comments (and the confusion that results, both from agents and humans)
Bad modularity is basically a summary of what usually constitutes pathological code in general, but AI systems seem particularly disposed to sling lots of it (at least humans are constrained in their output rate). How often have you tried to grok an AI-built project and found trivially unreusable code, unnecessary duplication (everywhere!), bad data structure and algorithm choices, and stale references?
> AI agents don’t get lost
Thats… not my experience. Like, not at all. They very regularly get lost
Yes and they go down rabbit holes, and have to be yanked back to focus on the actual goal.
Same.
Also this article reads like it was written by Sonnet.
I think I used Opus actually. But my prompt was huge, basically the article content. I told Opus what I wanted to approach in the article with all the relevant details I wanted to include and it created the article. Then I reviewed it.
But the part that agents don't get lost is actually part of the prompt, not something Opus wrote on its own. Of course they get lost sometimes but the idea of the article is to explain that agents can often make more sense of tangled code than humans when the bounds are not well defined.
> Here’s the problem. AI agents are not bound by human context limits in the same way. An agent can read the tangled function, trace every caller, and make sense of the mess that would have stopped a human cold. It can add the next branch correctly, and the one after that, working confidently inside code that no human on the team fully understands anymore.
They're not bound by the same limits but they're still bound by some limits, yeah?
I'm not an AI expert, so I don't honestly understand why LLM driven agents are as good as they are. But my impression is "trace every caller", most of the time, is still an approximation. Once the code has gotten convoluted enough, cases are going to get dropped.
Yes, of course, they're limited, but it doesn't compare to the limit humans face. We can't hold a lot of context on our minds when investigating a bug when the code is too complex with huge methods, lots of branches, several callers and so on. It doesn't mean agents are perfect but they don't care as much as humans if the code is a mess because they can find logic in mess, we can't.
They also very often do not actually read the fucking files! Even after specifying to „read in full the fucking files“. LLMs are so lazy, they take any shortcut they can.
Sorry, just thinking about it is reviving my frustration…
Yeah, you gotta make em prove their work so they don't hallucinate.
Try "build a script to trace callers" and "cite each file and line that calls the function"
Won't fix everything but usually helps in my experience
Although you still run into hard-to-find things especially in dynamic/duck typed languages or codebases with heavy use of reflection or code bases used as libraries
> Teams, em dash bla bla bla em dash have quietly bla bla bla. It's not this, it's _that_.
I find it hard to read articles where the agentic writing is this obvious. It's a distraction from the message of the text, which I'm sure is worth my time. Is there no way to stop generated writing from sounding like this?
Why believe it's worth your time? Upfront it reads like:
> Claude, generate a 500 word blog post about how people don't refactor anymore because of AI.
Just send me the prompt!
The prompt was actually huge, containing all the central ideas of the articles in details. I actually found my first prompt in case you're curious:
---
Human contexts are way more limited compared to computers. We can't reason about complex software when they go through many branches with so many implications. It's just too hard for humans to keep track of all interconnected pieces. So humans have historically split the system parts into manageable modules that can be understood in isolation and then we spend some time connecting those parts. That's how we can keep the context reasonable for human understanding.
So, when senior engineers found themselves lost while trying to debug an issue in a complex system they would naturally decide to pause and rewrite or refactoring the confusing piece of the system to make it manageable so developers can easily understand what's going on and review future changes.
Usually a system doesn't start that confusing. But as requirements change developers add additional branches and code until the code is no longer manageable. Sometimes the requirements changed significantly since the code was first written and all we have in the code are exceptions rather than the rule. That's usually when historically senior developers would take the time to rewrite that part of the system so they can reason about it.
But AI agents are not as limited as humans context-wise and they can reason about those confusing (to humans) systems and make sense of it. So they simply keep adding additional branches to the existing mess without ever suggesting a major refactoring like a senior developer would do in those cases, unless there's specific harness to tell agents to act like that.
This article is about bringing this into attention so that developers can policy themselves and keep asking themselves whether it's time for a major refactoring instead of relying on the AI agents and trust them because they no longer understand the code because it's too complex for humans to follow. Can you draft an article focused on this concern?
I find the basic premise for low level code quality to be true, in my experience, but counterintuitively I can now police the overall structure and system architecture MUCH more heavily.
As ever, no one is willing to allocate time for this, but (with unlimited work tokens) I can parallel path massive cleanup refactors all the time now.
As long as you're working alone, that makes sense. Otherwise, some team will have to review all your refactoring PRs...
I've personally adopted doing multiple refactoring passes after any large code implementation done by AI. In pretty much any scenario where I'm adding code using AI, it's 1 turn to add the feature and and then another 4-5 turns to refactor and clean everything up.
Often times it's not even that the code is bad but rather that it's overengineered. I see it happen so much that I'm tempted to actually go the other way on a toy project. Like what would Claude or Codex come up with if I told it I wanted an enterprise grade, globally scalable, compliant and auditable tic-tac-toe game.
Reminds me of the "entreprise grade fizzbuzz"
https://github.com/enterprisequalitycoding/fizzbuzzenterpris...
I've had the opposite experience where we're refactoring the gnarliest shit anyone's ever seen because AI can actually understand it well enough to decompose, test, refactor, etc.
This is pretty spot on imo. Although otoh I feel like I don't need to be able to reason as deeply about a system because of the ability of an agent to dig through a code base. Personally I believe I'm still looking for that new balance. It's a little like driving a car in a neighborhood you know, vs one where you constantly need to be looking down at a map or gps. It's much more comfortable driving around the places you're familiar with, but you can't know everywhere.
This is obviously written by an LLM. Should we flag such content?
Did I even tried to pretend the article wasn't generated with Claude's help?
I even shared the initial prompt in some comment in this thread. I don't understand what is the matter with also using AI help with generating articles. I still reviewed the article and put all the ideas I wanted to discuss in that article. I don't see why people are often complaining about this. The article content is much more important than how it was generated.
I don't know, our backlog of refactoring, before AI, never really happened either.
I've yet to work at a place that bothered with much refactoring over adding the thirtieth conditional to new feature....
Let's say that tomorrow, due to an improved model or whatever, we realize that the most efficient form of code of an app - for an llm to understand and work with - is for it to be in one long spaghetti file.
Why wouldn't we do that? I think there's a point where this comes down to values instead of facts. If you want it to be human readable, that's fine and there are a bunch of therefores from that point. But if you don't necessarily want that for a particular codebase, why refactor if the LLMs can handle it?
I'd argue the things that make a codebase more human-readable is also what makes it more agent-readable though. Agents are trained on human data, after all.
The position in the article is reasonable because what would end up happening otherwise is:
- Agents increase complexity, humans can't read it anymore
- Agents increase complexity, agent can't read it's own code anymore
- Agent unable to keep making updates without looping forever (the complexity of the code exceeds the agent's context length). Human doesn't understand either so can't fix.
This isn't hypothetical either, it's basically what ends up happening to most vibe-coded software if the person doing the vibe coding doesn't know how to review the outputs being produced.
So, basically, sure, but I disagree with the hypothetical in the first place
Vendor lock in
>A computer can hold far more in “working memory” than a human can
That's a bit of a straw man. State of the art agents are limited to ~3.8Mi (1M tokens). That's usually where I run into issues--an LLM can't possibly hold as much context as a human and it's more of an art than a science getting the most important things squeezed in. It's especially prudent for complex codebases/systems.
An agent only knows what it can see. It doesn't know oldCruftyFunction is still critical to Bob's Excel macro that generates financial reports and yanks the codebase in as a bastardised dependency. A lot of times agents give a fall sense of security by making it seem like something complicated and unsafe is actually safe.
That's why you make sure to put that institutional knowledge in AGENTS.md /s
This is what you get for getting rid of header files.
When project leads and directors only ask for features, the humans start to eventually push back, but agents do as you please, no matter what the cost.
They were trained to do what you ask, but unlike with humans, you need to ask for the refactoring yourself. -- They won't necessarily come up with it on their own.
(They also tend to not be around for long enough to live through the consequences of their tech debt actions.)
This'll be resolved by a selection process. Code that gets quadratically more complex will at some point fail to advance even with access to hundreds of genies.
[dead]
[flagged]
[flagged]
[dead]