We’ve been building our own Review Pipeline, including automated review guides, risk detection and much more.
There’s SO much to be done around Code Review, I’m very surprised GitHub isn’t doing more.
They have the reach already, they have years of real review data. And yet, the PR itself is not using any of the newly available methods for more efficient review.
Agents are hammering PRs out left and right, and yet here I am still staring at the raw diff of 35 PRs and end up being called the bottleneck.
I don't get all the people who complain about reviewing PRs. To me being a critic is much less stressful than being the author. Perhaps so much so that it becomes borderline enjoyable.
35 PRs on my client's codebase would be no big deal because I took a lot of time to understand the fundamentals of the business around it. I can recognize within seconds if a PR is an easy merge or a 5 minute phone call.
If you are struggling with PR reviews, this is a sign that something is wrong with the coal mine, not with your birds.
> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call
I could, before LLM-authored PRs. They broke all my heuristics for what to check. LLMs can write code that looks incredibly professional and well thought out, and nowadays they almost never make the usual human mistakes (like off-by-one errors). But at the same time, they can make incredibly silly mistakes of the kind people would never make (like inventing brand-new parameters for some remote API call).
It feels like everything I’ve learned over the years about reviewing code no longer applies.
One thing that's interesting is code review is still very static in my experience across gitub, gitlab, and bitbucket. Moving from a text editor to an IDE with LSP made a big difference for coding for me with click-through links and autocomplete of valid functions and variables. Raw text diff for review feels very old in comparison, especially as PRs generated by LLM agents get bigger and bigger compared to human PRs of the past.
I watched someone vibe code with a top of the line model, something they did quite regularly, and it was the most insane mess of code I have ever seen.
> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call.
I'm glad you can.
Even before agents, I'm pulled to work on at least a couple different projects a year, the successful ones with teams of 10s of people working on them. If I stay longer than a couple of months, I eventually become the expert on some of the things I did, so I get pulled years later to review something I haven't touched in ages, which has changed requirements, and grown a bunch of dependents.
That was the reality of working on a team with hundreds of people and dozens of related products.
Now agents are helping people increase complexity and velocity N-fold.
You can call it a problem in the coal mine, but there's no going back. GitHub simply has no choice but to handle this new scale.
Use CI results to control quality, no nitpicking style means easier review
Review tests one by one, to make sure I understand what the submitter is trying to test and if they actually did so
Check files off one-by-one, compartmentalizing understanding as I go
Complain loudly when PRs try to change too many subsystems or different functionality, especially if less related changes are not separated into standalone commits
I often have Claude make code walks specifically to present walls of code more sanely, add diagrams, bring in code context outside the diff, etc. This is one manifestation I made but I think there are many ways people are solving this (or roll your own tool!) https://github.com/jeffbaumes/code-walk
This is a great idea. Combine this with a tool that allows you to mark the lines of code you have reviewed and understand, and you're on your way to systematically working through the PR.
Yes, but code can be presented in different ways. This depends of course to a degree on the submitter (large changeset touching a lot of files or smaller change sets building upon each other?) but also could be presented differently (highlighting changes to data structures, identifying core parts of code and helper routines - lots of room for imagination)
Also mind: one doesn't only have to review changes code, but also review what hasn't been changed (are preconditions of other code still met?)
I've been trying to auto generate diagrams on PRs I'm reviewing. Flows changes via sequence diagrams, state changes and schema changes, post it to the PR with a skill. Then look at the relevant code based on the diagrams. Then \code-reviews. Moderately helpful. YMMV
Code review is a problem only because it is a backstop for everything else. We skip architecture review, we skip prototyping, we one-shot a result in isolation from our team and then expect a human at the end of the line to do our job for us??
Code review is great if the code isn't a surprise, the architecture was already agreed on, the AI reviewers are already happy, and you just need a final pass.
Code review was already terrible before AI if you were slopping down weeks of work on a poor teammate to sift through.
I tell the younger engineers at my company the same thing today I said 5 and 10 years ago. Make the review easy for another engineer to do it, and they'll do it. Make it hard on them, and you won't ship shit.
But the "make review easy" part is one where I'm failing (a bit) to keep up, I feel.
Also, what does "a final pass" look like to you? E.g. if a PR is 3000 lines... I'd guess one approach is enforce smaller PRs. I've opened a few, looked at the line count, and closed them, to be redone/broken into stacked PRs.
If I'm reviewing a PR that I'm familiar with the plan/work on, my "a final pass" is often just "read and suggest touch-ups of the docs". But that's really just the first step of a "full review" too, so it skips other stuff, and I know that isn't a proper review in the back of my head, but on to the next thing! :|
(Because of several factors including joining a much less mature org git-wise this year, I'm full of internal questions/dilemmas around PRs that I'm working on feasible solves for)
Code review complexity is always proportional to the associated work unit's complexity, not to some imaginary "keep PRs nice and small and nifty" complexity. I agree that consequences of bad SDLC ofter float up during code review, all the way up to formulating requirements and managing human resources.
I don’t believe that nobody is doing it well. May I ask who is doing it well? Either in house or with an AI reviewer vendor.
For me, what works is highly templated directory structure and rules that linters can quickly check and enforce. Every PR is a cookie cutter design filled in. There are only a handful of things you need and they’re all just CRUD at the end of the day. You can employ the same very few design patterns and primitives across a lot of your stack. Review is so fast, if you trust your CI to do its job and the developer to actually QA their own work.
What isn’t working is the sheer volume of PRs. Even with CODEOWNERS directing attention to only high impact changes, it is a constant distraction. I’m not sure how to handle SOC2 where we need a human reviewer on all code. Lastly, agents shit the bed when they touch spaghetti, esp when it’s sitting next to structured patterns. IME, the best way to get around this is to just rewrite features from scratch. But in this current world of instant completion and reward, it’s been challenging to convince people to commit to more ambitious lifts.
Finally, I want review more than ever. A team can’t keep up with the pace of change even if it was all perfect. I like the idea of more up front planning, but the code is ultimately what matters.
I’m waiting to see someone who has any sort of reasonable way, but I have yet to see one. Code review was always a difficult part of the job. However, it was self regulating because the people who would do very large code dumps frequently were very very few high throughput individuals and they were either legitimately good and knew what they were doing, so you trusted them, or were quickly told to not do that. The process had natural breaks on it, and particularly large PR would be expected to take few days or week(s) to review.
This all has changed. I’ve only seen gaslighting (in the form of “of course we expect you to read the code LLMs is generating”), lying along the same lines, or the realistic people who acknowledge they don’t read any of it. The former camp is mostly traditional engineers, and the later is mostly new or traditional non-engineering roles like recent product management coding converts. I think it would help everyone if we just normalize the fact that no one if reading code much anymore. It’s just not feasible. Write a bunch of creative reviewing bots or SKILLs and just leave it at that. It’s not a great answer, but like I said I haven’t seen any alternatives.
> if you trust your CI to do its job and the developer to actually QA their own work.
I trust the CI, but the CI is doing what the CI is written to do. I don’t know how you can achieve the second part in a large team/corporate setting or an open source setting.
> of course we expect you to read the code LLMs is generating
Yes! I think a lot of struggle comes from not having the same reliance on LLMs across the team. If developer A is not opening IDE and not reading the code to do work and developer B does not use LLMs, types code letter by letter and reads every text they encounter - it will not go well for the team. I also think many devs rely on LLMs more than they're willing to admit to the management and the rest of the team.
With Theseus, we've been trying to approach the problem from a couple of standpoints:
1. Code review is now less about mechanical validation (which often happens in linting, AGENTS.md, etc.) and more about education and understanding what's changing
2. Engineers need a tool which allows them to focus their attention on parts of changes, without having to review everything all at once
UI change that will have small effect.
IMHO PRs are going to die in their current form.
We need some sort of summary mechanism or invariant prerequisites to keep track of with tools. When contributions start violating them, we might need a PR to update the architectural invariant or roll them back. This is a decision when a person needs to have a look and act.
Working on autonomous development with agents and no human code review. The future is human oversight, without humans in the loop. A PR centric cycle is like being in a factory and standing next to a conveyor belt of PRs. Humans can't possibly review properly. They have to take a few steps back and start looking at the entire machine, evaluating the system as a whole.
> A PR centric cycle is like being in a factory and standing next to a conveyor belt of PRs. Humans can't possibly review properly.
Then something is either wrong with your conception of PRs or whatever process you have been submitted to. PR is asking for feedback from people that already understand the software. It's also a learning place for anyone that is not yet familiar with the part being modified. So you have the author, the reviewers, and the learners. Which is three separate sets of people.
The PR review itself shouldn't take a lot of time (10mn is already a lot), because it's either a small change which is easily understood[0] or it's a more complex one [1] that should have a design discussion thread attached. This discussion is orthogonal to the code itself. Both should have a description good enough for the reviewer to have an idea why the change is necessary before looking at the code.
So a PR is an exchange of ideas around a real artifact, the code. If no idea is being exchanged, then it's merely a gatekeeping activity.
pull requests are not going anywhere, if you let claude take the wheel, a company will fail (or never make it), the agents are not that good, your pullboard web copy is a great example
Sorry for the shameless plug but I would really love some feedback and discussion on some of the features in the code forge I am building.
1. Hunk checkpoints - If it's a moderately sized PR (which most of the agentic PRs nowadays are), I as a developer find myself like to think in terms of hunks whether I finished reviewing that part or not. GitHub allows you to mark only files as reviewed and nothing else.
2. I am trying to add semantic diffs to code reviews. That way a developer can actually understand the changes in the structure of the code instead of relying on reading plain text changes only. For ex: a function rename can simply compress 20 lines of changes into a single line.
3. Any agentic reviews and/or automated deterministic reviews can use both of the above to mark some simple things as reviewed in order to help the reviewer more.
4. This is in addition to existing review features from GitHub and the interdiff feature from other review tools.
A small mockup is available on the marketing website at https://juju.bi
I appreciate any comments on this approach. What do you guys think?
I have been thinking about a similar “semantic diff” approach to code reviews for the past several weeks. I’d love to hear more about how you’re approaching this - there’s a handful of distinct scenarios I see frequently in PRs that I think should be reviewable in a more direct form (renames, moving blocks of code between files without changing the text, whitespace changes, etc - basically any action a modern IDE can do deterministically).
I’ve played with some bookmarklets that modify the GitHub PR ui and attempt highlight these situations, but my setup is brittle (just trying to identify basic patterns in the text diff, ex “these are [oldname] replaced with [newname]”, which is not necessarily the same thing as an IDE rename-this-function action). I’ve been wondering if the answer here is in doing a diff on a compiled intermediate form of the program, visualized in some way - this would have to be language-specific though, I guess, and is well outside my expertise.
Separate from the diff aspect of reviews, I would also appreciate some more control in a PR tool over how to layout a review (what files/hunks to be shown, in what order). My team attempts to do this with carefully organized commits, so that reviewers step through them one by one, but the GitHub UI leaves a lot to be desired here (files appear in a static order top to bottom; comments and discussions can’t be stuck to one commit and show up in other commits; general bugs in the web view). I would appreciate being able to control how others see my PR if there’s room for that in your product.
Anyways, I joined your waitlist and am curious to see what you come up with - I think there are improvements to that can be made to code reviewing (that don’t involve AI - everyone is already putting AI review in their PRs), and am willing to try anything that takes a swing at that.
Individual files, for sure. I feel like plenty of times You have a changeset that involves an new/updated internal helper module of some kind, and also changes/additions to other places that use it, and its helpful to see them in a particular order, but file or folder hierarchies put things into a different order. I feel like a PR comment along the lines of "Look here [link-to-file] first" happens more than occasionally on my team.
As for changing the display of hunks within one file, I think that could be interesting for a couple reasons (but I totally get that could be obnoxious to both build and present a UI for). I can try to dig up an example later, but off the top of my head there's two situations I've seen infrequently this might help with? (I'm sorry if I'm vague - I cannot share any of the code I do for work, which is 99% of the PRs I deal with).
1. A specific file was refactored in such a way that code was moved, deleted, and added, and the interpreted hunks together are intermingled. I feel like I see diffs where it's hard to tell what's going on because there are delete/addition/delete/addition blocks alternating, but they form separate contexts/actions (and often have different indentation levels or other styling quirks) it's just the diff algorithm displaying them poorly that is causing the problem. This is normally the spot where I might switch between a side-by-side diff viewer and a unified viewer, but sometimes neither is good. (This could probably be solved with more careful individual commits, although sometimes this is unavoidable).
2. We have a few large modules (singluar classes in 2000+ln files) that often have a wide public interface, with multiple functions eventually calling the same internal helper (a trivial example would be a multiple separate search functions providing different kinds of windows/orderings/etc of a data source, sharing an private function that really does most of the main work), but this private function is not first in the file. It'd be helpful to order this to be seen first - "here's what I changed' followed by 'here's how it is used now'.
Maybe what I want could just be solved with a commit-by-commit ordering of things, but just showing multiple commits together on screen, with later commits coming further down the page (as opposed to showing the whole unified diff of a group of commits - Github does that, and that is pretty much never what I want other than 'what's changed since I last saw this').
I can attempt to find some specific examples later this week and try to explain these situations clearer if the above is confusing.
Hi, not all organisations are the same. I do not operate my team this way. I am sorry that this is your experience but please don’t pretend that it’s universal.
I wish they would instead focus on fixing the regressions with PRs first...
For example: There is a nasty bug that got introduced recently where you click "Merge when ready" but your chances of things actually going to the merge queue seem to be only at about 60-70%. When it happens the way you usually find out if things still aren't merged the next day.
Their support has acknowledged the issue to us over a month ago but no fix as of now.
A bug that's been driving me crazy lately is the file list in the sidebar doesn't reliably link you to the right file when you click on a filename. I find myself clicking it several times while the code on the right jumps around until it finally lands on the right file.
Anyone else experience this?
On topic, regarding these changes. I'm not sure what all changed, but I like having the status checks and merge button at the top of the page now.
Same. What’s even worse is that there is no visibility into it.
Became so bad for us that we switched to an external merge queue provider (trunk.io, not affiliated)
P90 of merge queue entry _started_ dropped from up to 40! minutes to 16 seconds. And it’s the same underlying mechanism, creating a branch and running CI on it.
Github PR viewer UX is just horrible. The UI is bloated, slow and very very difficult to use. (For example, commenting on a Copilot comment frequently takes 5-8s to the UI to submit and update.)
Why not fix the garbage UI first?
If we could just see unresolved comments and CI status in a page that take 100ms to load and 1 second to scroll to the bottom, everything would be fine.
The problem is that it the PR Conversation page takes 20-25 seconds to load a scroll through; and is full of outdated reports from Copilot/ Devin that don't go away of themselves, and don't have a good UI to be dismissed, once they're irrelevant.
I'm outside the US so poor UI <-> server network interaction may be magnified. But for something that could be simple and reasonable, it's just overloaded with garbage and an appalling experience.
Fix the basics first. I'd fire the entire team and every single Product Manager/ UX Lead involved. Anybody from Github listening?
Just the other day my teammate approved my PR and despite the repo having that merge when ready feature enabled, it kept not working and throwing some random error.
IIRC we unapproved it, approved without that enabled, and then manually merged it.
Publication of a completely unrelated article while another team is failing to fix that bug is a side effect of the company having more than one employee.
Companies with more than one person also have the issue that people don't communicate 100% of what's going on. This means prioritization is imperfect so they often do things other than what you'd like them to. Also you'll find different cultures which differently emphasize fixes and new work.
Hope you're not having too bad of a week whule your bug persists.
Their workflow scheduler is broken too. I noticed that workflows were getting scheduled hours late or not at all, so I had my own workflow orchestrator trigger them.
This is a major improvement and still not great. I made CodeApprove (https://codeapprove.com) 6+ years ago just because the PR page on GitHub is so bad and I felt the urge to make it better.
IMO the correct design is pretty simple:
- Everything on one page, information dense
- Focus on code and unresolved discussions, maybe CI checks
- Keyboard navigation
Graphite gets pretty close! But it's become cluttered over the years. CodeApprove is far from perfect too (it's been a while since I worked on the UX much) but I'd hope GitHub could do a little better.
I have very little experience with the tools, but from the landing page my first reaction was that it structurally reminded me a bit of Gerrit. Was it a source of inspiration for CodeApprove?
Idk, I just ask my agent what the status of the PR is in terms of failing checks, outstanding comments, etc.
But this seems fine
If anything I think the place to invest a better UX is on reviewing code, even “my” (agent’s) own code before I ask for a review. In a world where we’re moving away from IDEs, and using terminal apps or desktop apps to code, I often don’t even see the code until it’s on GitHub. But I still cling on to some semblance of needing to review, in particular what’s important.
And yes, me one year ago would be shocked at what I just said.
I don’t like that they reversed the the comment thread order but didn’t include a native way to include previews. Now instead of having to scroll to bottom to see the merge button. I have to scroll to the bottom to see the preview.
I tried it, but I don't understand the fanfare when it's basically the same old UI + a couple of minor layout changes. It definitely doesn't address the needs of the AI dev workflows.
Been using Graphite all year. I had trouble adapting my GitHub mind to it at first and I still don’t love everything about Graphite, but _wow_ is Graphite’s PR review page organized better than GitHub. GitHub’s new PR stack is superior (atomic merges are life-changing) and having one fewer tokens to manage CLI to install is fantastic, but I’m still mostly Graphite because the page is just organized better.
Specifically, graphite wins on:
- I land on the page and immediately see an overview of my whole PR stack: what’s in the stack, what’s passing, what’s failing, what’s draft, unresolved and total comments. I know exactly where to spend my time. GitHub PR stacks offer no such convenience.
- Description is below stack details, then code, and I never have to switch tabs or pages. This hierarchy makes sense: top of the page is broad whole stack details, and as we go down we get more specific. GitHub forces me to jump between pages to see all of this.
- Comments in a drawer are nice. Being able to quickly scan them helps me get a sense of what’s there, who’s commenting, I can jump between them quickly.
- Approval, request, labels, etc are also logically oriented at the top near other metadata. GitHub sprawls this around the page. It’s especially rough on mobile browser.
I just checked the new overview. It’s a step up. Curious to see if it helps the PR stack experience. I’d love to fully migrate off Graphite, the non-atomic merge is truly a nightmare across a large stack, but at the end of the day I haven’t been able to give up the UX gains of managing my work on Graphite compared to GitHub.
I tried it this morning, I guess it's for business rather than open source? The first PR I opened was slop that violated the project's policies. The new UI is missing both report and block.
There's not really any improvement to the core problem with PRs. You still have to read up and down the entire comments tab and then again the entire diff tab (hunting and clicking each "load more" and re-opening every "resolved" thread to find invisible replies) to reconstruct the timeline of what's happened in your head because nothing's in temporal order and there's still no interdiff.
NGL, I know GH gets a lot of shit about the stability, that's well deserved, but they've been really doing nice UI improvements recently. Hopefully this is a new trend that will also continue with the stability.
But at the same time, do they still hide/mask/minimize review comments when there are more than 10 in a row?
That is the worst with their UX currently, only challenged by the stupidity of the code diff page that hide the most modified files so you can easily miss them and it is impossible to do a ctrl+f search on the page.
These were not really my beef with the PR review page. My beef is that the first round of PR comments becomes unmoored from ground truth after any subsequent commits, making it difficult to see whether comments were addressed, and how.
I have. Between all the force pushes showing up in the feed, and various automations adding or removing labels, reviewers, to project boards, etc, the 'feed' tends to get pretty cluttered for me.
I've been using this new PR view for a few weeks now and I much prefer it.
> The merge box, distilled
Nobody, and I mean NOBODY, can legitimately like this Apple-style way or structuring sentences. The only people to remotely get away with it in my mind are Apple themselves (and only becuase they’re grandfathered in), and I get that in the late ‘00s / early ‘10s there was a trend of companies doing this to look more Apple-like, but in 2016 let alone 2026 it’s just camp and cringe. However, LLMs LOVE writing like this whenever they get a slight whiff that they’re being asked to do something in a marketing context. Are GitHub / MS using clearly AI-generated prose just to prove a point or something? Are the internal [dis-]incentives really this bad? Am I going insane? Is this just how all Californians talk?
Shovel manufacturer responds to complaints about customers throwing piles of manure in neighbors’ yards: “We’re trialing a free window cleaning service to help you get a clearer view of the manure”
> As people need to review more and more pull requests, time spent scrolling adds up
I can’t believe people would write this as the justification. You can press PgDn on your keyboard a few times or just End once to go to the bottom. If you are on a MacBook and your keyboard doesn’t have these keys, press Fn-Down instead of PgDn and Fn-Right instead of End. When done, use the opposite keys (PgUp/Home/Fn-Up/Fn-Left). I have never once found that scrolling takes too much time.
I know it's boring at this point, but seeing Claudish headers like "The merge box, distilled" and "Keep your place. Focus your attention" just screams SLOP at me.
Can you fix the inbox first please before reinventing more UI? The inbox is utterly broken and slow. It frequently says my PRs are waiting for review when they’ve already been approved.
We’ve been building our own Review Pipeline, including automated review guides, risk detection and much more.
There’s SO much to be done around Code Review, I’m very surprised GitHub isn’t doing more.
They have the reach already, they have years of real review data. And yet, the PR itself is not using any of the newly available methods for more efficient review.
Agents are hammering PRs out left and right, and yet here I am still staring at the raw diff of 35 PRs and end up being called the bottleneck.
They dropped the ball so badly with copilot. I would not be surprised they entirely drop the ball on review too.
Pull request review is by and large going away everywhere I talk to. Everywhere else it's a bottleneck they're trying to solve.
GitHub needs to figure out how to position itself as an orchestration layer between team members and their agents before slack eats their lunch.
I don't get all the people who complain about reviewing PRs. To me being a critic is much less stressful than being the author. Perhaps so much so that it becomes borderline enjoyable.
35 PRs on my client's codebase would be no big deal because I took a lot of time to understand the fundamentals of the business around it. I can recognize within seconds if a PR is an easy merge or a 5 minute phone call.
If you are struggling with PR reviews, this is a sign that something is wrong with the coal mine, not with your birds.
> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call
I could, before LLM-authored PRs. They broke all my heuristics for what to check. LLMs can write code that looks incredibly professional and well thought out, and nowadays they almost never make the usual human mistakes (like off-by-one errors). But at the same time, they can make incredibly silly mistakes of the kind people would never make (like inventing brand-new parameters for some remote API call).
It feels like everything I’ve learned over the years about reviewing code no longer applies.
One thing that's interesting is code review is still very static in my experience across gitub, gitlab, and bitbucket. Moving from a text editor to an IDE with LSP made a big difference for coding for me with click-through links and autocomplete of valid functions and variables. Raw text diff for review feels very old in comparison, especially as PRs generated by LLM agents get bigger and bigger compared to human PRs of the past.
I watched someone vibe code with a top of the line model, something they did quite regularly, and it was the most insane mess of code I have ever seen.
I used to be able to say the same but 10k loc diffs at my org shattered this ability pretty quickly.
auto reject, ask to split it up and minimize the diff.
> I can recognize within seconds if a PR is an easy merge or a 5 minute phone call.
I'm glad you can.
Even before agents, I'm pulled to work on at least a couple different projects a year, the successful ones with teams of 10s of people working on them. If I stay longer than a couple of months, I eventually become the expert on some of the things I did, so I get pulled years later to review something I haven't touched in ages, which has changed requirements, and grown a bunch of dependents.
That was the reality of working on a team with hundreds of people and dozens of related products.
Now agents are helping people increase complexity and velocity N-fold.
You can call it a problem in the coal mine, but there's no going back. GitHub simply has no choice but to handle this new scale.
Code review process is the worst part of LLM-assisted SDLC at the moment, probably because nobody knows how to do it well yet.
The human experience of reviewing PRs hasn't really moved beyond "look at these 2000 lines of text and think really hard".
At least for me. Am I missing something?
I at least:
Use CI results to control quality, no nitpicking style means easier review
Review tests one by one, to make sure I understand what the submitter is trying to test and if they actually did so
Check files off one-by-one, compartmentalizing understanding as I go
Complain loudly when PRs try to change too many subsystems or different functionality, especially if less related changes are not separated into standalone commits
I often have Claude make code walks specifically to present walls of code more sanely, add diagrams, bring in code context outside the diff, etc. This is one manifestation I made but I think there are many ways people are solving this (or roll your own tool!) https://github.com/jeffbaumes/code-walk
This is a great idea. Combine this with a tool that allows you to mark the lines of code you have reviewed and understand, and you're on your way to systematically working through the PR.
I mean if it's a code review, you gotta review the code.
Yes, but code can be presented in different ways. This depends of course to a degree on the submitter (large changeset touching a lot of files or smaller change sets building upon each other?) but also could be presented differently (highlighting changes to data structures, identifying core parts of code and helper routines - lots of room for imagination)
Also mind: one doesn't only have to review changes code, but also review what hasn't been changed (are preconditions of other code still met?)
I've been trying to auto generate diagrams on PRs I'm reviewing. Flows changes via sequence diagrams, state changes and schema changes, post it to the PR with a skill. Then look at the relevant code based on the diagrams. Then \code-reviews. Moderately helpful. YMMV
Code review is a problem only because it is a backstop for everything else. We skip architecture review, we skip prototyping, we one-shot a result in isolation from our team and then expect a human at the end of the line to do our job for us??
Code review is great if the code isn't a surprise, the architecture was already agreed on, the AI reviewers are already happy, and you just need a final pass.
Code review was already terrible before AI if you were slopping down weeks of work on a poor teammate to sift through.
I tell the younger engineers at my company the same thing today I said 5 and 10 years ago. Make the review easy for another engineer to do it, and they'll do it. Make it hard on them, and you won't ship shit.
I'm fully with you on this philosophy.
But the "make review easy" part is one where I'm failing (a bit) to keep up, I feel.
Also, what does "a final pass" look like to you? E.g. if a PR is 3000 lines... I'd guess one approach is enforce smaller PRs. I've opened a few, looked at the line count, and closed them, to be redone/broken into stacked PRs.
If I'm reviewing a PR that I'm familiar with the plan/work on, my "a final pass" is often just "read and suggest touch-ups of the docs". But that's really just the first step of a "full review" too, so it skips other stuff, and I know that isn't a proper review in the back of my head, but on to the next thing! :|
(Because of several factors including joining a much less mature org git-wise this year, I'm full of internal questions/dilemmas around PRs that I'm working on feasible solves for)
Code review complexity is always proportional to the associated work unit's complexity, not to some imaginary "keep PRs nice and small and nifty" complexity. I agree that consequences of bad SDLC ofter float up during code review, all the way up to formulating requirements and managing human resources.
I don’t believe that nobody is doing it well. May I ask who is doing it well? Either in house or with an AI reviewer vendor.
For me, what works is highly templated directory structure and rules that linters can quickly check and enforce. Every PR is a cookie cutter design filled in. There are only a handful of things you need and they’re all just CRUD at the end of the day. You can employ the same very few design patterns and primitives across a lot of your stack. Review is so fast, if you trust your CI to do its job and the developer to actually QA their own work.
What isn’t working is the sheer volume of PRs. Even with CODEOWNERS directing attention to only high impact changes, it is a constant distraction. I’m not sure how to handle SOC2 where we need a human reviewer on all code. Lastly, agents shit the bed when they touch spaghetti, esp when it’s sitting next to structured patterns. IME, the best way to get around this is to just rewrite features from scratch. But in this current world of instant completion and reward, it’s been challenging to convince people to commit to more ambitious lifts.
Finally, I want review more than ever. A team can’t keep up with the pace of change even if it was all perfect. I like the idea of more up front planning, but the code is ultimately what matters.
There are only a handful of things you need and they’re all just CRUD
Idk why everyone says this. Yeah crud is tedious and common. No it is not the majority of my time spent, implementing, fixing, or reviewing.
I’m waiting to see someone who has any sort of reasonable way, but I have yet to see one. Code review was always a difficult part of the job. However, it was self regulating because the people who would do very large code dumps frequently were very very few high throughput individuals and they were either legitimately good and knew what they were doing, so you trusted them, or were quickly told to not do that. The process had natural breaks on it, and particularly large PR would be expected to take few days or week(s) to review.
This all has changed. I’ve only seen gaslighting (in the form of “of course we expect you to read the code LLMs is generating”), lying along the same lines, or the realistic people who acknowledge they don’t read any of it. The former camp is mostly traditional engineers, and the later is mostly new or traditional non-engineering roles like recent product management coding converts. I think it would help everyone if we just normalize the fact that no one if reading code much anymore. It’s just not feasible. Write a bunch of creative reviewing bots or SKILLs and just leave it at that. It’s not a great answer, but like I said I haven’t seen any alternatives.
> if you trust your CI to do its job and the developer to actually QA their own work.
I trust the CI, but the CI is doing what the CI is written to do. I don’t know how you can achieve the second part in a large team/corporate setting or an open source setting.
> of course we expect you to read the code LLMs is generating
Yes! I think a lot of struggle comes from not having the same reliance on LLMs across the team. If developer A is not opening IDE and not reading the code to do work and developer B does not use LLMs, types code letter by letter and reads every text they encounter - it will not go well for the team. I also think many devs rely on LLMs more than they're willing to admit to the management and the rest of the team.
With Theseus, we've been trying to approach the problem from a couple of standpoints:
1. Code review is now less about mechanical validation (which often happens in linting, AGENTS.md, etc.) and more about education and understanding what's changing 2. Engineers need a tool which allows them to focus their attention on parts of changes, without having to review everything all at once
Would love to hear what you think!
[0] https://theseus.dev
Chill
We're trying to solve this with Theseus. Would love to hear your thoughts! https://theseus.dev
Wow
UI change that will have small effect. IMHO PRs are going to die in their current form.
We need some sort of summary mechanism or invariant prerequisites to keep track of with tools. When contributions start violating them, we might need a PR to update the architectural invariant or roll them back. This is a decision when a person needs to have a look and act.
Yeah man, the PR-centric lifecycle has to change.
Working on autonomous development with agents and no human code review. The future is human oversight, without humans in the loop. A PR centric cycle is like being in a factory and standing next to a conveyor belt of PRs. Humans can't possibly review properly. They have to take a few steps back and start looking at the entire machine, evaluating the system as a whole.
https://github.com/pullboard-dev/pullboard
> A PR centric cycle is like being in a factory and standing next to a conveyor belt of PRs. Humans can't possibly review properly.
Then something is either wrong with your conception of PRs or whatever process you have been submitted to. PR is asking for feedback from people that already understand the software. It's also a learning place for anyone that is not yet familiar with the part being modified. So you have the author, the reviewers, and the learners. Which is three separate sets of people.
The PR review itself shouldn't take a lot of time (10mn is already a lot), because it's either a small change which is easily understood[0] or it's a more complex one [1] that should have a design discussion thread attached. This discussion is orthogonal to the code itself. Both should have a description good enough for the reviewer to have an idea why the change is necessary before looking at the code.
So a PR is an exchange of ideas around a real artifact, the code. If no idea is being exchanged, then it's merely a gatekeeping activity.
[0] https://cgit.freebsd.org/src/commit/?id=73b1453e6d87f9af4c47...
[1] https://cgit.freebsd.org/src/commit/?id=49a3cbc47f8e721a787e...
Yeah bro, it's a broken process. Not a misunderstanding.
pull requests are not going anywhere, if you let claude take the wheel, a company will fail (or never make it), the agents are not that good, your pullboard web copy is a great example
https://claudeourlord.com
So why bother with PRs at all, might as well just skip the charade and push to master.
atleast the merge button is better reachable here, that's my primary takeaway from using it
I wish I was paid what im being paid for only reviewing PRs
It was better when we just called them 'review requests' instead of this silly 'pr' nonsense.
https://www.reviewboard.org/
I miss Gerrit.
Have an agent do the reviewing?
Sorry for the shameless plug but I would really love some feedback and discussion on some of the features in the code forge I am building.
1. Hunk checkpoints - If it's a moderately sized PR (which most of the agentic PRs nowadays are), I as a developer find myself like to think in terms of hunks whether I finished reviewing that part or not. GitHub allows you to mark only files as reviewed and nothing else.
2. I am trying to add semantic diffs to code reviews. That way a developer can actually understand the changes in the structure of the code instead of relying on reading plain text changes only. For ex: a function rename can simply compress 20 lines of changes into a single line.
3. Any agentic reviews and/or automated deterministic reviews can use both of the above to mark some simple things as reviewed in order to help the reviewer more.
4. This is in addition to existing review features from GitHub and the interdiff feature from other review tools.
A small mockup is available on the marketing website at https://juju.bi
I appreciate any comments on this approach. What do you guys think?
I have been thinking about a similar “semantic diff” approach to code reviews for the past several weeks. I’d love to hear more about how you’re approaching this - there’s a handful of distinct scenarios I see frequently in PRs that I think should be reviewable in a more direct form (renames, moving blocks of code between files without changing the text, whitespace changes, etc - basically any action a modern IDE can do deterministically).
I’ve played with some bookmarklets that modify the GitHub PR ui and attempt highlight these situations, but my setup is brittle (just trying to identify basic patterns in the text diff, ex “these are [oldname] replaced with [newname]”, which is not necessarily the same thing as an IDE rename-this-function action). I’ve been wondering if the answer here is in doing a diff on a compiled intermediate form of the program, visualized in some way - this would have to be language-specific though, I guess, and is well outside my expertise.
Separate from the diff aspect of reviews, I would also appreciate some more control in a PR tool over how to layout a review (what files/hunks to be shown, in what order). My team attempts to do this with carefully organized commits, so that reviewers step through them one by one, but the GitHub UI leaves a lot to be desired here (files appear in a static order top to bottom; comments and discussions can’t be stuck to one commit and show up in other commits; general bugs in the web view). I would appreciate being able to control how others see my PR if there’s room for that in your product.
Anyways, I joined your waitlist and am curious to see what you come up with - I think there are improvements to that can be made to code reviewing (that don’t involve AI - everyone is already putting AI review in their PRs), and am willing to try anything that takes a swing at that.
> Separate from the diff aspect of reviews, I would also appreciate some more control in a PR tool over how to layout a review
I am quite interested in knowing more about this. Would you mind give me a small example? Do you want to control individual hunks order or just files?
Individual files, for sure. I feel like plenty of times You have a changeset that involves an new/updated internal helper module of some kind, and also changes/additions to other places that use it, and its helpful to see them in a particular order, but file or folder hierarchies put things into a different order. I feel like a PR comment along the lines of "Look here [link-to-file] first" happens more than occasionally on my team.
As for changing the display of hunks within one file, I think that could be interesting for a couple reasons (but I totally get that could be obnoxious to both build and present a UI for). I can try to dig up an example later, but off the top of my head there's two situations I've seen infrequently this might help with? (I'm sorry if I'm vague - I cannot share any of the code I do for work, which is 99% of the PRs I deal with).
1. A specific file was refactored in such a way that code was moved, deleted, and added, and the interpreted hunks together are intermingled. I feel like I see diffs where it's hard to tell what's going on because there are delete/addition/delete/addition blocks alternating, but they form separate contexts/actions (and often have different indentation levels or other styling quirks) it's just the diff algorithm displaying them poorly that is causing the problem. This is normally the spot where I might switch between a side-by-side diff viewer and a unified viewer, but sometimes neither is good. (This could probably be solved with more careful individual commits, although sometimes this is unavoidable).
2. We have a few large modules (singluar classes in 2000+ln files) that often have a wide public interface, with multiple functions eventually calling the same internal helper (a trivial example would be a multiple separate search functions providing different kinds of windows/orderings/etc of a data source, sharing an private function that really does most of the main work), but this private function is not first in the file. It'd be helpful to order this to be seen first - "here's what I changed' followed by 'here's how it is used now'.
Maybe what I want could just be solved with a commit-by-commit ordering of things, but just showing multiple commits together on screen, with later commits coming further down the page (as opposed to showing the whole unified diff of a group of commits - Github does that, and that is pretty much never what I want other than 'what's changed since I last saw this').
I can attempt to find some specific examples later this week and try to explain these situations clearer if the above is confusing.
>1. Hunk checkpoints
How long are your files that you can't easily "finish" a file?
I find that for even files that are less than 500 lines, if the changes are complex enough, I wish I could track viewed changes line-by-line.
Y'all are still reviewing code?
(Semi serious here.)
Just press accept, who cares. If they're not giving you time to review, they can get what they demand.
Hi, not all organisations are the same. I do not operate my team this way. I am sorry that this is your experience but please don’t pretend that it’s universal.
Welcome to the minority.
I wish they would instead focus on fixing the regressions with PRs first... For example: There is a nasty bug that got introduced recently where you click "Merge when ready" but your chances of things actually going to the merge queue seem to be only at about 60-70%. When it happens the way you usually find out if things still aren't merged the next day.
Their support has acknowledged the issue to us over a month ago but no fix as of now.
A bug that's been driving me crazy lately is the file list in the sidebar doesn't reliably link you to the right file when you click on a filename. I find myself clicking it several times while the code on the right jumps around until it finally lands on the right file.
Anyone else experience this?
On topic, regarding these changes. I'm not sure what all changed, but I like having the status checks and merge button at the top of the page now.
Overall, this isn’t a super great time for software users that value working features over new features.
Not great for developers either: Stakeholders want whatever Claude said! No time for planning! Pump up those numbers!
Indeed. Two sides of the same fleece-vested coin.
I also frequently see this. I assume it's related to how files on the right are lazy loaded.
Yes! The file clicking one bothers me so much.
I've seen that too. Takes one or two clicks for it to actually scroll to the correct file.
Yes I noticed the same issue
Same. What’s even worse is that there is no visibility into it.
Became so bad for us that we switched to an external merge queue provider (trunk.io, not affiliated)
P90 of merge queue entry _started_ dropped from up to 40! minutes to 16 seconds. And it’s the same underlying mechanism, creating a branch and running CI on it.
Github PR viewer UX is just horrible. The UI is bloated, slow and very very difficult to use. (For example, commenting on a Copilot comment frequently takes 5-8s to the UI to submit and update.)
Why not fix the garbage UI first?
If we could just see unresolved comments and CI status in a page that take 100ms to load and 1 second to scroll to the bottom, everything would be fine.
The problem is that it the PR Conversation page takes 20-25 seconds to load a scroll through; and is full of outdated reports from Copilot/ Devin that don't go away of themselves, and don't have a good UI to be dismissed, once they're irrelevant.
I'm outside the US so poor UI <-> server network interaction may be magnified. But for something that could be simple and reasonable, it's just overloaded with garbage and an appalling experience.
Fix the basics first. I'd fire the entire team and every single Product Manager/ UX Lead involved. Anybody from Github listening?
Just the other day my teammate approved my PR and despite the repo having that merge when ready feature enabled, it kept not working and throwing some random error.
IIRC we unapproved it, approved without that enabled, and then manually merged it.
Publication of a completely unrelated article while another team is failing to fix that bug is a side effect of the company having more than one employee.
Companies with more than one person also have the issue that people don't communicate 100% of what's going on. This means prioritization is imperfect so they often do things other than what you'd like them to. Also you'll find different cultures which differently emphasize fixes and new work.
Hope you're not having too bad of a week whule your bug persists.
That's why I ask my agents to merge my PRs for me, it's much more reliable and take the cognitive load off
Their workflow scheduler is broken too. I noticed that workflows were getting scheduled hours late or not at all, so I had my own workflow orchestrator trigger them.
This is a major improvement and still not great. I made CodeApprove (https://codeapprove.com) 6+ years ago just because the PR page on GitHub is so bad and I felt the urge to make it better.
IMO the correct design is pretty simple: - Everything on one page, information dense - Focus on code and unresolved discussions, maybe CI checks - Keyboard navigation
Graphite gets pretty close! But it's become cluttered over the years. CodeApprove is far from perfect too (it's been a while since I worked on the UX much) but I'd hope GitHub could do a little better.
I have very little experience with the tools, but from the landing page my first reaction was that it structurally reminded me a bit of Gerrit. Was it a source of inspiration for CodeApprove?
hey Sam! Glad to see you are still at it.
Idk, I just ask my agent what the status of the PR is in terms of failing checks, outstanding comments, etc.
But this seems fine
If anything I think the place to invest a better UX is on reviewing code, even “my” (agent’s) own code before I ask for a review. In a world where we’re moving away from IDEs, and using terminal apps or desktop apps to code, I often don’t even see the code until it’s on GitHub. But I still cling on to some semblance of needing to review, in particular what’s important.
And yes, me one year ago would be shocked at what I just said.
Yep I do PR reviews only via the terminal now. gh or tea cli, and an agent swarm that does an initial review pass.
I don’t like that they reversed the the comment thread order but didn’t include a native way to include previews. Now instead of having to scroll to bottom to see the merge button. I have to scroll to the bottom to see the preview.
I've been using this for a few weeks but I don't quite like it. It's too many clicks to navigate to the PR discussion / comments for my liking.
I haven’t tried the new view, it I really like the graphite review page, everything is on one page.
The GitHub review page is very frustrating navigating around to different tabs.
That has not changed they just took some stuff that used to be on the bottom and put it on the top in this new design
I tried it, but I don't understand the fanfare when it's basically the same old UI + a couple of minor layout changes. It definitely doesn't address the needs of the AI dev workflows.
I wish that instead of trying to make the UI look pretty GH would instead try to make the platform solid again
I still remember when GH was decent.
Of course we are now seeing the effects of many years of being Microsofted.
The ironic thing is that AI could've probably improved things, if they'd actually bother to use it to their advantage.
Discussion: https://github.com/orgs/community/discussions/212186
Been using Graphite all year. I had trouble adapting my GitHub mind to it at first and I still don’t love everything about Graphite, but _wow_ is Graphite’s PR review page organized better than GitHub. GitHub’s new PR stack is superior (atomic merges are life-changing) and having one fewer tokens to manage CLI to install is fantastic, but I’m still mostly Graphite because the page is just organized better.
Specifically, graphite wins on:
- I land on the page and immediately see an overview of my whole PR stack: what’s in the stack, what’s passing, what’s failing, what’s draft, unresolved and total comments. I know exactly where to spend my time. GitHub PR stacks offer no such convenience.
- Description is below stack details, then code, and I never have to switch tabs or pages. This hierarchy makes sense: top of the page is broad whole stack details, and as we go down we get more specific. GitHub forces me to jump between pages to see all of this.
- Comments in a drawer are nice. Being able to quickly scan them helps me get a sense of what’s there, who’s commenting, I can jump between them quickly.
- Approval, request, labels, etc are also logically oriented at the top near other metadata. GitHub sprawls this around the page. It’s especially rough on mobile browser.
I just checked the new overview. It’s a step up. Curious to see if it helps the PR stack experience. I’d love to fully migrate off Graphite, the non-atomic merge is truly a nightmare across a large stack, but at the end of the day I haven’t been able to give up the UX gains of managing my work on Graphite compared to GitHub.
I tried it this morning, I guess it's for business rather than open source? The first PR I opened was slop that violated the project's policies. The new UI is missing both report and block.
There's not really any improvement to the core problem with PRs. You still have to read up and down the entire comments tab and then again the entire diff tab (hunting and clicking each "load more" and re-opening every "resolved" thread to find invisible replies) to reconstruct the timeline of what's happened in your head because nothing's in temporal order and there's still no interdiff.
Hoping to see this in GHE as soon as we can
I'm liking the changes. Hopefully the "lines changed" gets updated when you choose a specific commit in the file viewer now too
NGL, I know GH gets a lot of shit about the stability, that's well deserved, but they've been really doing nice UI improvements recently. Hopefully this is a new trend that will also continue with the stability.
But at the same time, do they still hide/mask/minimize review comments when there are more than 10 in a row?
That is the worst with their UX currently, only challenged by the stupidity of the code diff page that hide the most modified files so you can easily miss them and it is impossible to do a ctrl+f search on the page.
Amen. I lose comments and get confused as to why merging is blocked on a regular basis.
Scrap the entire PR experience, start again from basics. Fire the people responsible.
Right off the bat the ux is inconsistent in the examples. The grey bar is optional checks but required approvals
These were not really my beef with the PR review page. My beef is that the first round of PR comments becomes unmoored from ground truth after any subsequent commits, making it difficult to see whether comments were addressed, and how.
I’ve never once thought I was spending too much time scrolling.
I have. Between all the force pushes showing up in the feed, and various automations adding or removing labels, reviewers, to project boards, etc, the 'feed' tends to get pretty cluttered for me.
I've been using this new PR view for a few weeks now and I much prefer it.
Am I blind or is there no "try overview page" button on the header of this site? Or on any of my PRs
too little too late - we migrated to self hosted gitea.
> The merge box, distilled Nobody, and I mean NOBODY, can legitimately like this Apple-style way or structuring sentences. The only people to remotely get away with it in my mind are Apple themselves (and only becuase they’re grandfathered in), and I get that in the late ‘00s / early ‘10s there was a trend of companies doing this to look more Apple-like, but in 2016 let alone 2026 it’s just camp and cringe. However, LLMs LOVE writing like this whenever they get a slight whiff that they’re being asked to do something in a marketing context. Are GitHub / MS using clearly AI-generated prose just to prove a point or something? Are the internal [dis-]incentives really this bad? Am I going insane? Is this just how all Californians talk?
> The merge box, distilled
Ugh
Shovel manufacturer responds to complaints about customers throwing piles of manure in neighbors’ yards: “We’re trialing a free window cleaning service to help you get a clearer view of the manure”
> As people need to review more and more pull requests, time spent scrolling adds up
I can’t believe people would write this as the justification. You can press PgDn on your keyboard a few times or just End once to go to the bottom. If you are on a MacBook and your keyboard doesn’t have these keys, press Fn-Down instead of PgDn and Fn-Right instead of End. When done, use the opposite keys (PgUp/Home/Fn-Up/Fn-Left). I have never once found that scrolling takes too much time.
It seems the consensus isn't humans did not write it
95% if not more if claude thinks it's ok it's going in the sheer volume of code changes we are now doing are simply not doable by hand anymore.
It's basically the wild wet now :D bots reviewing bots.
Ideally only risky merges should get human attention, and some guard rails to enforce it but mistakes have been made.
So we getting UI updates just as humans are looking less and less at these screens via github....
Finally approve button is available on the main screen without having to open diff view which takes ages to load. Still two clicks though.
You didn’t have to open the diff before.
I know it's boring at this point, but seeing Claudish headers like "The merge box, distilled" and "Keep your place. Focus your attention" just screams SLOP at me.
I can't read it. It's not really text.
Just more slopstick on the pig. :/
> The merge box, distilled
What does that sub heading even mean!
Can you fix the inbox first please before reinventing more UI? The inbox is utterly broken and slow. It frequently says my PRs are waiting for review when they’ve already been approved.
Loading PRs on GH is dog slow for me. If they could improve the performance and availability, otherwise Idgaf
Especially on PRs with a lot of commentary. It's tiring to keep clicking "load more"
I give zero fucks. Just make Actions, 'ya know, some reasonable (3?) amount of 9s again.