> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository.
I don’t think this scales. We recently have been doing “spec driven development” and we are committing the specs and prompts to our repo, alongside the generated code. At the beginning it seems fine: you wanna change something, you update the spec and ask the machine to regenerate the code. Easy. Over time, though, you have hundreds if not thousands if spec files in MD. It’s all English prose. There is duplication and subtle inconsistencies. It’s difficult to search for sections of a spec. Do you create a new file for this new requirement or update an existing one? What level of detail is enough here? Should I hint the machine about using the “saga” pattern or just let it know that we are dealing with non atomic transactions distributed across services? Etc. When a colleague opens a PR updating a spec, it’s hard to suggest objective changes (at least with code, you can demonstrate the presence of bugs… not so much with English prose. Sometimes I feel like a lawyer)
All in all, it seems as if maintenance of english prose is way worse than maintenance of actual code in big enough systems. You not only need to review the spec but also review the generated code. It’s painful
Exactly. The spec should go in a git comment, NOT vcs. Or, you keep the document in history for the duration of the PR and remove it post review or pre merge
Disagree completely. The spec is vital so that future changes continue to conform to it. Specs absolutely need to live in the VCS, because they continue to be needed to keep the code conformant. They essentially are a form of code now. And code goes in the VCS.
At scale, specs can only be vital to the degree to which their conformance testing is automated. Good specs should use a formal, runnable verification language. Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.
But conformance testing is where agents really excel at if you set things up right:
* Black-box testing: the agent writing the tests cannot see implementation and the agent writing implementation cannot see tests, they only agree on a spec and an interface (the minimum needed to write tests).
* Evaluate test coverage using code coverage, but when gaps are found communicate those gaps in terms of the specification.
Good specs should be grounded (complete and not ambiguous), they don't need to be formal. You should be able to re-run your agents when the spec changes on diffs to the spec, and if a change happens out of bad, you should have agents that go in and propose fixes to the spec. Since agents are doing deterministic codegen like a compiler would, this is all pretty straightforward.
You also need to consider public and internal specifications (the public specification being for reuse of the component), and you might test your integrating component with a test-double (built from the public specification alone) rather than the real component itself.
> Good specs should be grounded (complete and not ambiguous), they don't need to be formal
IMHO, this is a mistake. I guess we play with it because there’s isn’t anything better nowadays. Writing and maintaining “specs” in plain english is painful.
Natural language is incredibly expressive and fairly easy to read and write. Pick your favorite formal specification language...and you can just express some properties in them, and they are mostly niche at that. If you increase the expressiveness of your formal language too much, it just becomes code, then you are back to square one. LLMs are also incredibly proficient in processing natural language; i.e. writing, maintaining, and using a spec written in plain english is actually fairly viable with a modern LLM.
> Writing and maintaining “specs” in plain english is painful.
The subject of this entire post is development with agents. Writing specs in English is how you do that. If you don't like it, then this is probably not the right article for you to be commenting on.
This is about ensuring that when AI's make future changes, they aren't just looking at the existing code and making assumptions about intent. They should always be pulling the specs to ensure that changes maintain compatibility with the specified intents.
> Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.
Not if the changes you're making are always to the specs, as opposed to the code. The whole point here is that you don't change the code, you change the specs, then approve the code that the LLM changes as a result. This way the spec should never diverge from the code.
AI absolutely changes the dynamic so that code doesn't converge from the spec. That's the whole point, and the whole point of committing the specs like code.
Automating testing is great too, of course, but that's not the full picture. It ensures formal compliance, but doesn't encapsulate anything about the spirit or purpose of why the design in a certain way. Good specs do. The purpose/motivation sections and engineering guidelines are some of the most important for an LLM. Which is what helps the LLM figure out how to then best modify the existing code when features need to be changed or added.
> The whole point here is that you don't change the code, you change the specs,
> but doesn't encapsulate anything about the spirit or purpose of why the design in a certain way. Good specs do.
Specs are meta solutions. They describe the general shape of the solutions by refusing to make any technical decision that would leads to incidental problems and thus only needs to focus on the essential ones. So they're always simplistic, because they ignore the cascading effect of implementation decisions.
Generating code with AI is rolling the dice every time said generation is done. Proper implementation happens because with making decisions and going down a path, backtracking if necessary when it's no longer working. Going with AI is breaking down that continuity because they restart from scratch everytime.
> Specs are meta solutions. They describe the general shape of the solutions by refusing to make any technical decision that would leads to incidental problems and thus only needs to focus on the essential ones.
Yes, this is a feature not a bug. Then, code review ensures that the actual code makes the requisite technical decisions in a desirable way. This is much faster.
> Generating code with AI is rolling the dice every time said generation is done. Proper implementation happens because with making decisions and going down a path, backtracking if necessary when it's no longer working. Going with AI is breaking down that continuity because they restart from scratch everytime.
False. You only do the initial code generation an initial time, and then update the spec to specify (at a high level, ideally) the constraints that are missing. But then you have the AI modify the existing code to meet the new spec, not start from scratch. AI does not "restart from scratch everytime", it takes an updated spec plus existing code and updates the code to match the updated spec.
It's still "making decisions and going down a path," which you call proper implementation. That's the whole point.
The reason people don't bother with specs is that they're extraordinarily verbose, ambiguous, and simplistic at the same time. Which is why we have invented formal notations, because they're short, have one definition, and enough to create the whole thing, if not to understand the why. The latter can still be done in prose.
What a spec is truly useful for is that it lets you manipulate the solution at the idea stage, without tainting it with real world concerns. But to do it requires a lot knowledge about those real world concerns, so that your assumptions are not straight lies.
This is good for iteration. But any implementation ossifies your specs, and while code are more malleable than most, there's a certain points when the specs should be discarded and the actual implementation being the source of truth for any further development. In any project, that is done with Change Request, which only outline the specific area of changes, assuming the rest as being immutable.
> AI does not "restart from scratch everytime", it takes an updated spec plus existing code and updates the code to match the updated spec.
Maybe you can share an example of that workflow and an actual argument that would make it sustainable.
I've written the formal version of a spec (a suite of tests) and if there's one thing that would make the above idea ridiculous is how much wiggle room there is to follow the specs and be at the same time unusable.
The problem I see is that git commit meaaages are not as flexible at storing context information as files. Or easier to retrieve.
We do have --grep to search them but it's definitely not as poweful as all the tools we have to handle files at the OS level.
As verbose as storing specs in files is, I also prefer it
I find the value of someone steering an AI session to be practically nil for nearly everything. Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.
So I'm not sure why people going "chop, chop" and "nah make it more red and bigger" is a useful signal of much of anything.
No, the future is a complex "gate" that checks, weighs and measures everything before it gets committed --- a local (or remote) "CI" but... more granular and far more specific and turned to the needs of the project you work on.
> Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.
I think the issue that many may not want to think about is, how do we promote/reward/fire developers in the AI assisted programming age. As part of code reviews, I think it will be second nature to ask "how did you arrive at the code".
I think it makes no sense to share the conversation but I do think developers will be required to provide lessons, notes and checkpoints in the conversation.
What I think will happen is you will be required to run the conversation through a company required prompt that will include things like, does the conversation contain "think harder" without providing any guidance for how to correct things.
Basically, did the developer treat the conversation like a literal slot machine.
Not really, I don't think I've ever care about the how. I want to mostly know the trigger of a change (a bug, possible feature, or just tidying) and what are the impacts if that changes is included in the codebase. That is often just a paragraph or two.
The workflow is mostly worthless, unless it can generalize and evolve into a process. And that is more worthy of a blog post or a presentation.
Hi! I work at Entire. We actually have another feature that we announced called Trails which indicates Confidence, Risk, Drift of code changes to give the maintainer more confidence that the code is high quality.
Each of those signals -- confidence, risk, drift are determined by a workflow file that the maintainer can create.
And trails continuously evaluate every time a change is made.
So that's exactly what we're doing..a gate that checks and weighs and measure everything before it gets committed.
The point of us capturing the agent sessions is so we can do more things with it..like adding "gates" that verify quality!
Who would trust the classifiers, though? I would not.
A team of professionals will trust a gateway they built themselves -- by an AI, most likely, as that is boring work -- that works for their organisation and whatever weird quirks and house styles they have. In a year or two when things are a bit more stabilised that'll be dozens of subagents each testing a little thing each in isolation plus a bevvy of software that looks for antipatterns in addition to running tests and linters and what have you. It'll be tested against the ground truth specification and not the boring crap in the middle.
At no point will anybody's session matter. Results count; nothing more.
classifiers are determined by the team that's working on the repo. Not by us! They're configuration files..like how in CI/CD you write a workflow file that needs to have passing tests. Same thing.
When your AI consumes documents that are out of date it introduce bugs and incorrect behavior into the system. Code is the only source of truth that actually matters.
The only artifacts outside of code that I don't explicitly end-date are user-story style requirements documents that I periodically revisit with the AI to make sure they're up to date.
One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.
> One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.
This is actually the reason for me wanting to create what I call "Brain checkpoints". After chatting with the agent and letting it do its thing for discovery, I needed an easy way to understand what it knows at any given point. What files it has read, what tool calls were made and so forth.
Checkpoints work extremely well in Pi since you can create a branch, and have the agent generate a checkpoint based on what has changed since the last checkpoint and this information will not pollute the main conversation but you benefit from the token caching.
> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository.
Pi.dev has a feature where you can export the session as a html file and look at it later. I foresee that potentially you could store this in the same Git repository and get the benefit of reviewing how a particular code change came about during a session with an agent.
I guess the next step would be having the coding agent save that session context automatically in a folder in the git repository rather than requiring a human to export it.
This startup also seems to be operating in a similar space to tangled.org - moving code repos into a decentralised hosting environment.
I will make another update by the end of this week that contains what I call "brain checkpoints" that will make it easier for developers to debug and understand AI reasoning.
The idea is after a task has been finished, you would commit lessons, notes, and "brain checkpoints" that are designed to live with the code.
entire cli maintainer here: the entire cli does exactly this, with support for agent native clis but also pi or opencode. We take the raw session logs, put it in the repo with a stable link to the commit, and then you can render it in the cli or see it in on entire.io.
I don't see this taking off. LLM's produce absolute novels worth of verbose text; and nobody wants to read someone else's conversation with it, and certainly nobody wants to read a multi-year old session log that's 20,000 words long. Also, this is what commit messages are supposed to solve! And the only reason anyone looks at logs are to see what's going wrong, and usually a bisect is easier for finding bad commits rather than doing a deep dive into the specifics.
In the interest of not just being a hater and suggesting an alternative though: if you're implementing a feature (whether yourself or with the agent) have the agent write a spec for the feature and commit THAT. Now you have a nicely organized thing that AI's or people can consume which captures all the intent of the session log with much less noise.
I tend to find that such specs as committed to git best take the form of commit messages. And then a summary, in the pull request. We also did this in the before-time.
For what its worth, agents are very capable of navigating these. I highly recommend asking an ai to go do the same thing your colleague did half a year ago for this new context, or whatever.
This model has some significant downsides for humans, but less so for automatons. It eliminates some major problems of software development like merge conflicts, dependency hell, etc.
> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository. [...] Humans can more easily understand and verify what was built and why, providing a provenance layer that leads to far faster review periods.
Did they understand the coming issues (reviews), but just came up with a completely wrong hypothesis (session logs)? I mentioned it in another comment some time ago, I think the biggest issue in VCS software arises from the combination of two separate ones...
- A (technical): Reviews are currently not part of the VCS
- B (cultural): The amount of merged source code currently grants for higher authority/ownership of a code base than the review/verification of that code
These two things in combination are also part of the reason why we deem software engineers replaceable in the current hype cycle. Adding session logs to the VCS seems like treating the symptoms instead of the disease.
That is more along the lines of put all the agent communication into a DB that has git’s immutable history. They still use git for the code, and beads (well, DoltDB for storage/versioning) for the tasks and a bit of non-associative memory.
On the topic, I don’t find too much of the prompting to be worth referring to later, but I do always vibe code a search the sessions local web server so I can find things later. The harnesses seem to be getting better at this, but I make a ton of directories for different branches so I can’t always remember if a session was in de512.main or de1024.main
I have had luck with telling an agent to search thru old sessions to write up how to docs or sketch out a python script to automate something having done it a few times.
I think a Q&A-based approach is could also be a suitable way to capture the reasoning behind a project. Instead of writing documentation afterward, an AI could interview the developer throughout the process and preserve the questions and answers as project context.
Superpowers' brainstorming skill is another classic example of this. This skill with SOTA models is incredibly helpful. The results become very thorough specs/plans, ADRs, etc.
I suspect we'll move away from pull requests, because in the LLM world they're the worst way of accepting a contribution.
Verbose slop is painful to review, and it's dangerous to accept unreviewed code from a stranger.
For a maintainer it's way easier to tell their own agent to reimplement the same idea. It's still slop, but done their way, under their supervision.
For popular projects agent-made pull requests become a DoS attack. I wouldn't be surprised if projects start refusing to accept unsolicited PRs and switch to "don't call us, we'll call you". You could have an agent scanning forks of your projects to find what bugs users are fixing and what features they're adding, and use it as a roadmap, without the pressure of accepting any particular commit as-is.
I'd also like to move away from a binary merged-not-merged divide. Projects may have a stable manually-reviewed core that should be protected from agents messing it up, while allowing the sloppy parts to churn however LLMs like it.
Has anyone tried teaching the agents to use jj or gitbutler-cli to do their ideas of lightweight branches? I keep feeling like that would be a huge win for the multi-branch development I'm doing much more of with agents, but I haven't yet run that experiment.
The basic idea is, you always ensure you inject the instructions for how to use jj or any other scm. LLMs are pattern matchers and they can fully map behaviour, but due to heavy training on git, they can forget.
With hooks, you can easy detect when 'git' is used and basically tell the agent "As stated earlier you must use jj". So it will try again using `jj` and you block the agent when ever it tries to use git.
It is a wasted turn but the reality is, it won't cost must since you can leverage caching and you only waste tokens on the invalid command and not the output generated from git.
> Anyone else finds the LLMs version control discipline lacking?
No? How do you define discipline? Claude code knows how to read and write commits. My team just granted it limited force push access a week ago. We’ve taught it to make more human-friendly commit messages.
I think a useful question here is “what is a good commit?”, specifically at the MR stage. I don’t care what commits look like on your local machine.
I tend to like to treat commits at that stage as complete, atomic thoughts.
“This commit is a refactor of this function name”
“This commit defines all the data classes we’ll use”
“This commit writes the new function and all its tests”
“This commit injects the new code into the old path with a feature flag”
“This commit fixes the tests that broke from that change”
Some of these would be better served as separate MRs entirely; but I imagine my idea stands.
we store the whole session but with tools to query the summarized choices. It's not intended for the user to sit there and read through the whole session. The session acts as metadata.
"Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository."
Wasn't there an article on HN that went by in the last few weeks about someone actually implementing this, and it just made things worse on every metric?
Pre-AI people periodically wanted version control to be tracking every keystroke and I, along with a lot of other people, feel like they never successfully articulated exactly what we're supposed to get out of that. Session history seems like the same thing. A prompt history, maybe. But I don't want every bit of an AI's musing any more than I want every last stray through that passes through a developer's head either, and for the same reason in both cases: A finite mind has room for only so much stuff.
It is not a viable strategy for a finite mind, be it human or AI, to just "stuff everything into it and expect improvement to result". The first thing that finite mind will need to do to get any value out of it is to extract it into some much, much smaller and less detail-rich summarized version. Which is pretty close to what we already have, except without needing to burn AI time on the process. We have spent decades honing techniques for reducing cognitive load. The AIs benefit from them too, and make them more important than ever, not obsoleted.
Sure, it isn't exactly what we have today. But I think this is a case of the exceptions looming too large in our mind, precisely because they are exceptions. Yes, I've had a handful of cases in my career where I've wondered what were they thinking, and not just as a criticism, but as an actual question. But it's rare, and in the end, not necessarily all that valuable compared to questions like "what is the code doing" and "what is passing through the code that gets this thing done to it". In the vast majority of cases, the code is already only and exactly what I need.
The fact we have this code artifact that is a gateway and a checkpoint through which "intention" does not pass is a positive good thing and is one of the subtle reasons that people miss that makes both large-scale human software development and the current trend of AI-based software development possible in the first place. The highly effective process of trimming down all of this stuff into an artifact that has an extremely-precisely specified function and doesn't need external support from humans or process or history is an amazingly wonderful thing. Larding down the codebase with vast, vast quantities of very fluffy data that is only quite rarely useful is a positive step backwards, not forwards.
> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository.
I don’t think this scales. We recently have been doing “spec driven development” and we are committing the specs and prompts to our repo, alongside the generated code. At the beginning it seems fine: you wanna change something, you update the spec and ask the machine to regenerate the code. Easy. Over time, though, you have hundreds if not thousands if spec files in MD. It’s all English prose. There is duplication and subtle inconsistencies. It’s difficult to search for sections of a spec. Do you create a new file for this new requirement or update an existing one? What level of detail is enough here? Should I hint the machine about using the “saga” pattern or just let it know that we are dealing with non atomic transactions distributed across services? Etc. When a colleague opens a PR updating a spec, it’s hard to suggest objective changes (at least with code, you can demonstrate the presence of bugs… not so much with English prose. Sometimes I feel like a lawyer)
All in all, it seems as if maintenance of english prose is way worse than maintenance of actual code in big enough systems. You not only need to review the spec but also review the generated code. It’s painful
I’ve had good luck with this approach:
https://github.com/williamcotton/algraf/tree/main/docs
There’s also some tests in place to make sure some things from the master spec are up to date, eg, error codes.
Exactly. The spec should go in a git comment, NOT vcs. Or, you keep the document in history for the duration of the PR and remove it post review or pre merge
Disagree completely. The spec is vital so that future changes continue to conform to it. Specs absolutely need to live in the VCS, because they continue to be needed to keep the code conformant. They essentially are a form of code now. And code goes in the VCS.
At scale, specs can only be vital to the degree to which their conformance testing is automated. Good specs should use a formal, runnable verification language. Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.
But conformance testing is where agents really excel at if you set things up right:
* Black-box testing: the agent writing the tests cannot see implementation and the agent writing implementation cannot see tests, they only agree on a spec and an interface (the minimum needed to write tests).
* Evaluate test coverage using code coverage, but when gaps are found communicate those gaps in terms of the specification.
Good specs should be grounded (complete and not ambiguous), they don't need to be formal. You should be able to re-run your agents when the spec changes on diffs to the spec, and if a change happens out of bad, you should have agents that go in and propose fixes to the spec. Since agents are doing deterministic codegen like a compiler would, this is all pretty straightforward.
You also need to consider public and internal specifications (the public specification being for reuse of the component), and you might test your integrating component with a test-double (built from the public specification alone) rather than the real component itself.
> Good specs should be grounded (complete and not ambiguous), they don't need to be formal
IMHO, this is a mistake. I guess we play with it because there’s isn’t anything better nowadays. Writing and maintaining “specs” in plain english is painful.
Natural language is incredibly expressive and fairly easy to read and write. Pick your favorite formal specification language...and you can just express some properties in them, and they are mostly niche at that. If you increase the expressiveness of your formal language too much, it just becomes code, then you are back to square one. LLMs are also incredibly proficient in processing natural language; i.e. writing, maintaining, and using a spec written in plain english is actually fairly viable with a modern LLM.
> Writing and maintaining “specs” in plain english is painful.
The subject of this entire post is development with agents. Writing specs in English is how you do that. If you don't like it, then this is probably not the right article for you to be commenting on.
Not true.
This is about ensuring that when AI's make future changes, they aren't just looking at the existing code and making assumptions about intent. They should always be pulling the specs to ensure that changes maintain compatibility with the specified intents.
> Otherwise you'll accumulate specs that are right when they ship, wrong in subtle ways 3 months in, and wrong in glaring ways 6 months in. AI doesn't change this dynamic, it amplifies it.
Not if the changes you're making are always to the specs, as opposed to the code. The whole point here is that you don't change the code, you change the specs, then approve the code that the LLM changes as a result. This way the spec should never diverge from the code.
AI absolutely changes the dynamic so that code doesn't converge from the spec. That's the whole point, and the whole point of committing the specs like code.
Automating testing is great too, of course, but that's not the full picture. It ensures formal compliance, but doesn't encapsulate anything about the spirit or purpose of why the design in a certain way. Good specs do. The purpose/motivation sections and engineering guidelines are some of the most important for an LLM. Which is what helps the LLM figure out how to then best modify the existing code when features need to be changed or added.
> The whole point here is that you don't change the code, you change the specs,
> but doesn't encapsulate anything about the spirit or purpose of why the design in a certain way. Good specs do.
Specs are meta solutions. They describe the general shape of the solutions by refusing to make any technical decision that would leads to incidental problems and thus only needs to focus on the essential ones. So they're always simplistic, because they ignore the cascading effect of implementation decisions.
Generating code with AI is rolling the dice every time said generation is done. Proper implementation happens because with making decisions and going down a path, backtracking if necessary when it's no longer working. Going with AI is breaking down that continuity because they restart from scratch everytime.
> Specs are meta solutions. They describe the general shape of the solutions by refusing to make any technical decision that would leads to incidental problems and thus only needs to focus on the essential ones.
Yes, this is a feature not a bug. Then, code review ensures that the actual code makes the requisite technical decisions in a desirable way. This is much faster.
> Generating code with AI is rolling the dice every time said generation is done. Proper implementation happens because with making decisions and going down a path, backtracking if necessary when it's no longer working. Going with AI is breaking down that continuity because they restart from scratch everytime.
False. You only do the initial code generation an initial time, and then update the spec to specify (at a high level, ideally) the constraints that are missing. But then you have the AI modify the existing code to meet the new spec, not start from scratch. AI does not "restart from scratch everytime", it takes an updated spec plus existing code and updates the code to match the updated spec.
It's still "making decisions and going down a path," which you call proper implementation. That's the whole point.
The reason people don't bother with specs is that they're extraordinarily verbose, ambiguous, and simplistic at the same time. Which is why we have invented formal notations, because they're short, have one definition, and enough to create the whole thing, if not to understand the why. The latter can still be done in prose.
What a spec is truly useful for is that it lets you manipulate the solution at the idea stage, without tainting it with real world concerns. But to do it requires a lot knowledge about those real world concerns, so that your assumptions are not straight lies.
This is good for iteration. But any implementation ossifies your specs, and while code are more malleable than most, there's a certain points when the specs should be discarded and the actual implementation being the source of truth for any further development. In any project, that is done with Change Request, which only outline the specific area of changes, assuming the rest as being immutable.
> AI does not "restart from scratch everytime", it takes an updated spec plus existing code and updates the code to match the updated spec.
Maybe you can share an example of that workflow and an actual argument that would make it sustainable.
I've written the formal version of a spec (a suite of tests) and if there's one thing that would make the above idea ridiculous is how much wiggle room there is to follow the specs and be at the same time unusable.
The problem I see is that git commit meaaages are not as flexible at storing context information as files. Or easier to retrieve. We do have --grep to search them but it's definitely not as poweful as all the tools we have to handle files at the OS level. As verbose as storing specs in files is, I also prefer it
If only we had a way of describing exactly and in great detail to the machine what to do! Some sort of language, maybe, idk… /s
I find the value of someone steering an AI session to be practically nil for nearly everything. Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.
So I'm not sure why people going "chop, chop" and "nah make it more red and bigger" is a useful signal of much of anything.
No, the future is a complex "gate" that checks, weighs and measures everything before it gets committed --- a local (or remote) "CI" but... more granular and far more specific and turned to the needs of the project you work on.
> Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.
I think the issue that many may not want to think about is, how do we promote/reward/fire developers in the AI assisted programming age. As part of code reviews, I think it will be second nature to ask "how did you arrive at the code".
Can’t wait to get in trouble for having curse words in my chat with AI get flagged by the company’s anti-swearing-in-code automation.
I think it makes no sense to share the conversation but I do think developers will be required to provide lessons, notes and checkpoints in the conversation.
What I think will happen is you will be required to run the conversation through a company required prompt that will include things like, does the conversation contain "think harder" without providing any guidance for how to correct things.
Basically, did the developer treat the conversation like a literal slot machine.
> I think it will be second nature to ask "how did you arrive at the code".
What about “Do you understand the code well enough to argue about it, its past, and possible evolution paths”?
I think that would be implied by "how did you arrive at the code". I think the conversation will revove around how did you "guide" things.
Not really, I don't think I've ever care about the how. I want to mostly know the trigger of a change (a bug, possible feature, or just tidying) and what are the impacts if that changes is included in the codebase. That is often just a paragraph or two.
The workflow is mostly worthless, unless it can generalize and evolve into a process. And that is more worthy of a blog post or a presentation.
Hi! I work at Entire. We actually have another feature that we announced called Trails which indicates Confidence, Risk, Drift of code changes to give the maintainer more confidence that the code is high quality.
Each of those signals -- confidence, risk, drift are determined by a workflow file that the maintainer can create.
And trails continuously evaluate every time a change is made.
So that's exactly what we're doing..a gate that checks and weighs and measure everything before it gets committed.
The point of us capturing the agent sessions is so we can do more things with it..like adding "gates" that verify quality!
Who would trust the classifiers, though? I would not.
A team of professionals will trust a gateway they built themselves -- by an AI, most likely, as that is boring work -- that works for their organisation and whatever weird quirks and house styles they have. In a year or two when things are a bit more stabilised that'll be dozens of subagents each testing a little thing each in isolation plus a bevvy of software that looks for antipatterns in addition to running tests and linters and what have you. It'll be tested against the ground truth specification and not the boring crap in the middle.
At no point will anybody's session matter. Results count; nothing more.
classifiers are determined by the team that's working on the repo. Not by us! They're configuration files..like how in CI/CD you write a workflow file that needs to have passing tests. Same thing.
[dead]
When your AI consumes documents that are out of date it introduce bugs and incorrect behavior into the system. Code is the only source of truth that actually matters.
The only artifacts outside of code that I don't explicitly end-date are user-story style requirements documents that I periodically revisit with the AI to make sure they're up to date.
One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.
> One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.
This is actually the reason for me wanting to create what I call "Brain checkpoints". After chatting with the agent and letting it do its thing for discovery, I needed an easy way to understand what it knows at any given point. What files it has read, what tool calls were made and so forth.
Checkpoints work extremely well in Pi since you can create a branch, and have the agent generate a checkpoint based on what has changed since the last checkpoint and this information will not pollute the main conversation but you benefit from the token caching.
> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository.
Pi.dev has a feature where you can export the session as a html file and look at it later. I foresee that potentially you could store this in the same Git repository and get the benefit of reviewing how a particular code change came about during a session with an agent.
I guess the next step would be having the coding agent save that session context automatically in a folder in the git repository rather than requiring a human to export it.
This startup also seems to be operating in a similar space to tangled.org - moving code repos into a decentralised hosting environment.
This is what I am doing with my pi-brains extension for pi
https://github.com/gitsense/pi-brains
I will make another update by the end of this week that contains what I call "brain checkpoints" that will make it easier for developers to debug and understand AI reasoning.
The idea is after a task has been finished, you would commit lessons, notes, and "brain checkpoints" that are designed to live with the code.
> moving code repos into a decentralised hosting environment
It's unclear to me what Entire means by decentralized. Based on their most recent blog post (https://entire.io/blog/an-entirely-new-git-hosting-network) it seems like they just mean globally distributed, but all controlled by them.
In contrast, Tangled at least offers something where you can own your own data!
entire cli maintainer here: the entire cli does exactly this, with support for agent native clis but also pi or opencode. We take the raw session logs, put it in the repo with a stable link to the commit, and then you can render it in the cli or see it in on entire.io.
I don't see this taking off. LLM's produce absolute novels worth of verbose text; and nobody wants to read someone else's conversation with it, and certainly nobody wants to read a multi-year old session log that's 20,000 words long. Also, this is what commit messages are supposed to solve! And the only reason anyone looks at logs are to see what's going wrong, and usually a bisect is easier for finding bad commits rather than doing a deep dive into the specifics.
In the interest of not just being a hater and suggesting an alternative though: if you're implementing a feature (whether yourself or with the agent) have the agent write a spec for the feature and commit THAT. Now you have a nicely organized thing that AI's or people can consume which captures all the intent of the session log with much less noise.
I tend to find that such specs as committed to git best take the form of commit messages. And then a summary, in the pull request. We also did this in the before-time.
For what its worth, agents are very capable of navigating these. I highly recommend asking an ai to go do the same thing your colleague did half a year ago for this new context, or whatever.
I am anticipating a move away from git-style version control entirely.
One possible shape is something like unison-lang's CAS AST model:
https://www.unison-lang.org/docs/the-big-idea/
This model has some significant downsides for humans, but less so for automatons. It eliminates some major problems of software development like merge conflicts, dependency hell, etc.
> Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository. [...] Humans can more easily understand and verify what was built and why, providing a provenance layer that leads to far faster review periods.
Did they understand the coming issues (reviews), but just came up with a completely wrong hypothesis (session logs)? I mentioned it in another comment some time ago, I think the biggest issue in VCS software arises from the combination of two separate ones...
These two things in combination are also part of the reason why we deem software engineers replaceable in the current hype cycle. Adding session logs to the VCS seems like treating the symptoms instead of the disease.> Version Control Will Evolve for the Agent Boom
Isn't this the idea behind Yegg's "Beads"?
That is more along the lines of put all the agent communication into a DB that has git’s immutable history. They still use git for the code, and beads (well, DoltDB for storage/versioning) for the tasks and a bit of non-associative memory.
On the topic, I don’t find too much of the prompting to be worth referring to later, but I do always vibe code a search the sessions local web server so I can find things later. The harnesses seem to be getting better at this, but I make a ton of directories for different branches so I can’t always remember if a session was in de512.main or de1024.main
I have had luck with telling an agent to search thru old sessions to write up how to docs or sketch out a python script to automate something having done it a few times.
[flagged]
I think a Q&A-based approach is could also be a suitable way to capture the reasoning behind a project. Instead of writing documentation afterward, an AI could interview the developer throughout the process and preserve the questions and answers as project context.
Matt Pocock’s “grill-me” skill is a nice example of this idea: https://github.com/mattpocock/skills/tree/main/skills/produc...
Superpowers' brainstorming skill is another classic example of this. This skill with SOTA models is incredibly helpful. The results become very thorough specs/plans, ADRs, etc.
https://github.com/obra/superpowers/blob/main/skills/brainst...
I suspect we'll move away from pull requests, because in the LLM world they're the worst way of accepting a contribution.
Verbose slop is painful to review, and it's dangerous to accept unreviewed code from a stranger.
For a maintainer it's way easier to tell their own agent to reimplement the same idea. It's still slop, but done their way, under their supervision.
For popular projects agent-made pull requests become a DoS attack. I wouldn't be surprised if projects start refusing to accept unsolicited PRs and switch to "don't call us, we'll call you". You could have an agent scanning forks of your projects to find what bugs users are fixing and what features they're adding, and use it as a roadmap, without the pressure of accepting any particular commit as-is.
I'd also like to move away from a binary merged-not-merged divide. Projects may have a stable manually-reviewed core that should be protected from agents messing it up, while allowing the sloppy parts to churn however LLMs like it.
Take a look at his keynote at WeAreDevelopers..he starts around the 17:20 mark
https://www.youtube.com/watch?v=eSoHzgrhk6Q
He does hint around removing away from pull requests!
> For a maintainer it's way easier to tell their own agent to reimplement the same idea.
This only makes sense if you assume the original PR was just vibe-coded with minimal human effort. Maybe one day but I don't think we are there yet.
And it costs at least twice the amount of tokens, so a full win for LLM providers.
The security implications of scanning and merging external prompts at scale are going to be interesting.
Provocation: pull requests were always the worst way to do it.
There’s reason I wrote high info density documentation and comments rather than committed every StackOverflow page I viewed during development pre-AI.
Followup announcement post:
An Entirely New Git Hosting Network
https://entire.io/blog/an-entirely-new-git-hosting-network
(https://news.ycombinator.com/item?id=48833019)
Has anyone tried teaching the agents to use jj or gitbutler-cli to do their ideas of lightweight branches? I keep feeling like that would be a huge win for the multi-branch development I'm doing much more of with agents, but I haven't yet run that experiment.
I was interested in answering this question so I built a benchmark comparing git, jj and gitbutler in agentic context
https://vcbench.dev/
Disclaimer - I am a co-founder of GitButler
For Git did agents use the newer git history commands or prefer older multi step methods
> Has anyone tried teaching the agents to use jj or gitbutler-cli to do their ideas of lightweight branches?
This is very trivial if your coding agents supports hooks like Pi. I have demo repo that goes over how rules work at https://github.com/gitsense/gsc-rules-demos
The basic idea is, you always ensure you inject the instructions for how to use jj or any other scm. LLMs are pattern matchers and they can fully map behaviour, but due to heavy training on git, they can forget.
With hooks, you can easy detect when 'git' is used and basically tell the agent "As stated earlier you must use jj". So it will try again using `jj` and you block the agent when ever it tries to use git.
It is a wasted turn but the reality is, it won't cost must since you can leverage caching and you only waste tokens on the invalid command and not the output generated from git.
Claude works with jj just fine.
Recentish and relatedish:
Ex-GitHub CEO launches a new developer platform for AI agents - https://news.ycombinator.com/item?id=46961345 - Feb 2026 (577 comments)
Anyone else finds the LLMs version control discipline lacking?
The engineering practices in general are lacking. Not tests nor assumptions validation ever, unless explicitly asked.
Have you tried creating skills for your agents to follow your patterns?
> Anyone else finds the LLMs version control discipline lacking?
No? How do you define discipline? Claude code knows how to read and write commits. My team just granted it limited force push access a week ago. We’ve taught it to make more human-friendly commit messages.
What’s missing?
I think a useful question here is “what is a good commit?”, specifically at the MR stage. I don’t care what commits look like on your local machine.
I tend to like to treat commits at that stage as complete, atomic thoughts.
Some of these would be better served as separate MRs entirely; but I imagine my idea stands.I primarily care about the pull request title and description since we squash-merge and those values end up populating the squashed commit.
What do you do to keep the cognitive load of reviewers at a minimum?
I still dont let my agents use git.
We need to store CHOICES with the commit or branch. Not the whole session. Conscious, summarized choices
we store the whole session but with tools to query the summarized choices. It's not intended for the user to sit there and read through the whole session. The session acts as metadata.
Do we really need a new system for this? What’s wrong with just writing an elaborate commit message?
"Our hypothesis is simple: session logs are now the most important artifact in software development, and should be stored alongside the code itself in the repository."
Wasn't there an article on HN that went by in the last few weeks about someone actually implementing this, and it just made things worse on every metric?
Pre-AI people periodically wanted version control to be tracking every keystroke and I, along with a lot of other people, feel like they never successfully articulated exactly what we're supposed to get out of that. Session history seems like the same thing. A prompt history, maybe. But I don't want every bit of an AI's musing any more than I want every last stray through that passes through a developer's head either, and for the same reason in both cases: A finite mind has room for only so much stuff.
It is not a viable strategy for a finite mind, be it human or AI, to just "stuff everything into it and expect improvement to result". The first thing that finite mind will need to do to get any value out of it is to extract it into some much, much smaller and less detail-rich summarized version. Which is pretty close to what we already have, except without needing to burn AI time on the process. We have spent decades honing techniques for reducing cognitive load. The AIs benefit from them too, and make them more important than ever, not obsoleted.
Sure, it isn't exactly what we have today. But I think this is a case of the exceptions looming too large in our mind, precisely because they are exceptions. Yes, I've had a handful of cases in my career where I've wondered what were they thinking, and not just as a criticism, but as an actual question. But it's rare, and in the end, not necessarily all that valuable compared to questions like "what is the code doing" and "what is passing through the code that gets this thing done to it". In the vast majority of cases, the code is already only and exactly what I need.
The fact we have this code artifact that is a gateway and a checkpoint through which "intention" does not pass is a positive good thing and is one of the subtle reasons that people miss that makes both large-scale human software development and the current trend of AI-based software development possible in the first place. The highly effective process of trimming down all of this stuff into an artifact that has an extremely-precisely specified function and doesn't need external support from humans or process or history is an amazingly wonderful thing. Larding down the codebase with vast, vast quantities of very fluffy data that is only quite rarely useful is a positive step backwards, not forwards.
> But as code becomes increasingly abundant through agents, the context behind "why" the code was written is becoming vital:
That’s the commit message. If something is not immediately clear from the title and the diff, you write it it in the commit description, like this:
https://cgit.freebsd.org/src/commit/?id=ed7e0ebfa20e7e798d2e...
If you can’t summarize your changes in a similar manner, that usually means you don’t understand the code and shouldn’t push it to other people.
Not having to use Stack Overflow or Git directly are by far the biggest improvements LLMs have brought to my life.
[flagged]
[flagged]
[flagged]