(Use AGENTS.md or whatever agent docs you use, to tell your agent to review after it’s done working but before it stops.)
Congratulations! After you added hundreds more items to the agent’s review checklist, that work you did to build the checklist has automated the first 90% of the work. The second 90% is still.. reviewing the code. So no, you can’t escape code review.
It’s an asymptote. And it’s part of your job as a professional.
If someone asks you for a code review and you turn your LLM loose to do it - you’re doing nothing. Just tell the requester you can’t be bothered and have them ask their LLM to do the review directly.
Relaying one LLMs output to another is very poor use of a human’s time.
I don't think I was able to express my point well.
I'm specifically not asking you to run your LLM lose on it. I'm asking you to add the specific knowledge that you have that the author/LLM doesn't to the review. I tried to jot down such examples in the post.
Your point was expressed well and makes sense. I tend to have the LLM explain the PR to me and I'll do a back and forth with it on the various decisions in the PR and their ramifications. That seems like a better way than reviewing every line, and agrees with your point in the post.
I'm 100% opposed to AI generating my code, but I could see myself using it as an advanced linter. I suspect this will be considered best practices after the AI Slopocalypse
I'll believe that once AIs stop outputting garbage code that spends 15 lines defensively checking for situations that can never occur or re-implementing their own URL-parsing logic instead of using a library.
Maybe Fable is better at this? Maybe there's a set of prompts or skills that will reduce these tendencies?
You can radically reduce your review burden but you can’t eliminate it - review is the part where you make your code good.
But whatever you see in review, you must add to a review checklist for the agent. You do this so you never have to check for that thing again.
Example: https://github.com/cadamsdotcom/CodeLeash/blob/main/.claude/...
(Use AGENTS.md or whatever agent docs you use, to tell your agent to review after it’s done working but before it stops.)
Congratulations! After you added hundreds more items to the agent’s review checklist, that work you did to build the checklist has automated the first 90% of the work. The second 90% is still.. reviewing the code. So no, you can’t escape code review.
It’s an asymptote. And it’s part of your job as a professional.
If someone asks you for a code review and you turn your LLM loose to do it - you’re doing nothing. Just tell the requester you can’t be bothered and have them ask their LLM to do the review directly.
Relaying one LLMs output to another is very poor use of a human’s time.
I don't think I was able to express my point well.
I'm specifically not asking you to run your LLM lose on it. I'm asking you to add the specific knowledge that you have that the author/LLM doesn't to the review. I tried to jot down such examples in the post.
Your point was expressed well and makes sense. I tend to have the LLM explain the PR to me and I'll do a back and forth with it on the various decisions in the PR and their ramifications. That seems like a better way than reviewing every line, and agrees with your point in the post.
I can't tell if the title is a joke or not, the article seems to give some very good reasons why you shouldn't use an LLM to review pull requests
> I don't think you should waste time reviewing every single line of code in here and just use AI to review it!
> What you bring is the knowledge that the author nor the LLM doesn't know.
How can you possibly know what relevant context to provide the LLM unless you read the 10k loc? Now you've wasted double the time.
I'm 100% opposed to AI generating my code, but I could see myself using it as an advanced linter. I suspect this will be considered best practices after the AI Slopocalypse
> your line by line reviews have no place here.
I'll believe that once AIs stop outputting garbage code that spends 15 lines defensively checking for situations that can never occur or re-implementing their own URL-parsing logic instead of using a library.
Maybe Fable is better at this? Maybe there's a set of prompts or skills that will reduce these tendencies?