I wish more people had a balanced view on this. Turning over all coding with no oversight, no research and planning, no validation, of course that's terrible code. I send FAR more tokens on the planning, research, and validation than on actual coding. If fact if I had to guess it's like 10-20% of the cost and time.
My teams process has 5x our output and reduced our quality issues. But we plan each ticket with Claude and have it research it up front, producing an acceptance criteria list that we validate with human eyes. Then when we work on that ticket, first step is nearly always a heavy research and validation step, then planning the work, then reviewing the research and planning by both agents and humans. Then the coding with is mostly Claude. Then we have a detailed QA/verification step, some of that automated but always requiring a human to run the code locally, test and validate the changes, and requires automated integration tests along with 90% test coverage on all new code. It blows my mind that people would just throw code into production without testing and validation! I'd never do that if I hand wrote all the code, why would I do that if an agent did?
I know the hype promised Agents doing all the work but honestly it mostly just replaced the boring parts for me. The design, planning, and validation is still all human driven. It helps I've been writing software for a living for over 25 years, so I know pretty quickly when something isn't correct. Also I read all the code it produces, it's a rare instance that the PR is so large that I can't and I often make Claude redo those. Though I'll admit Opus 5 seems to produce way too verbose comments and that makes it harder to read all of it.
Ticket based (Jira) it's a rough estimate but we are getting 5x as many tickets completed since adopting our current workflow/set of skills. Obviously tickets aren't a standard unit of measurement but we attempt to keep them the same size as pre-Agent work. I think our ticket size is the optimal size for a human to read and understand what that ticket is supposed to change and we do our best to keep to one change == one ticket. I've also found that if we go with larger tickets or less detailed tickets the work that Claude produces is far worse.
This has been my experience too. The initial euphoria of "wow it did that thing quickly", then I'm stick with code that's subtly broken and I don't understand.
Getting very close to treating AI tools as "read only" at this point.
>This last year has been exhausting. The PR departments of the frontier labs have done an excellent job convincing us that AI developments are occurring at an astounding, world-changing rate. But if you zoom out, it becomes clear that almost every “breakthrough” since last summer has concerned the narrow domains of computer code and math, which are defined by highly structured languages and come accompanied by massive amounts of specialized training data.
Theyve also been trying to gaslight everyone (especially investors) into thinking that software engineering is in some sense "solved" because otherwise the bubble goes pop and their share prices collapse.
I wish more people had a balanced view on this. Turning over all coding with no oversight, no research and planning, no validation, of course that's terrible code. I send FAR more tokens on the planning, research, and validation than on actual coding. If fact if I had to guess it's like 10-20% of the cost and time.
My teams process has 5x our output and reduced our quality issues. But we plan each ticket with Claude and have it research it up front, producing an acceptance criteria list that we validate with human eyes. Then when we work on that ticket, first step is nearly always a heavy research and validation step, then planning the work, then reviewing the research and planning by both agents and humans. Then the coding with is mostly Claude. Then we have a detailed QA/verification step, some of that automated but always requiring a human to run the code locally, test and validate the changes, and requires automated integration tests along with 90% test coverage on all new code. It blows my mind that people would just throw code into production without testing and validation! I'd never do that if I hand wrote all the code, why would I do that if an agent did?
I know the hype promised Agents doing all the work but honestly it mostly just replaced the boring parts for me. The design, planning, and validation is still all human driven. It helps I've been writing software for a living for over 25 years, so I know pretty quickly when something isn't correct. Also I read all the code it produces, it's a rare instance that the PR is so large that I can't and I often make Claude redo those. Though I'll admit Opus 5 seems to produce way too verbose comments and that makes it harder to read all of it.
My teams process has 5x our output
What is your output? I mean in terms of units, not asking for numbers.
Ticket based (Jira) it's a rough estimate but we are getting 5x as many tickets completed since adopting our current workflow/set of skills. Obviously tickets aren't a standard unit of measurement but we attempt to keep them the same size as pre-Agent work. I think our ticket size is the optimal size for a human to read and understand what that ticket is supposed to change and we do our best to keep to one change == one ticket. I've also found that if we go with larger tickets or less detailed tickets the work that Claude produces is far worse.
This has been my experience too. The initial euphoria of "wow it did that thing quickly", then I'm stick with code that's subtly broken and I don't understand.
Getting very close to treating AI tools as "read only" at this point.
Reads like LinkedIn slop.
>This last year has been exhausting. The PR departments of the frontier labs have done an excellent job convincing us that AI developments are occurring at an astounding, world-changing rate. But if you zoom out, it becomes clear that almost every “breakthrough” since last summer has concerned the narrow domains of computer code and math, which are defined by highly structured languages and come accompanied by massive amounts of specialized training data.
Theyve also been trying to gaslight everyone (especially investors) into thinking that software engineering is in some sense "solved" because otherwise the bubble goes pop and their share prices collapse.