I have been using Claude to build a clone of Bash in Rust (https://github.com/jdstanhope/huck). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests.
That being said, I am still guiding it on a regular basis and only recently let it run open ended trying fixing small to medium size bugs. The next stage will be try let it run in a loop trying to fix the 100 issues it has identified as well as getting most of the bash test suite to pass.
I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.
I find most of the time it would be fine on its own, but every now and again you avoid a really bad degradation through guidance. Some kinds of big change definitely problematic.
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that if I, human overseer, don't course correct it many times a day, it goes off into Neverland.
Yet I see comments and articles like above every week, where someone says "Yeah, I just left my agent run for a few nights, and then it made this awesome tool."
There is a huge gap here, but I can't pinpoint it yet. Either I (and everyone I talk to) is Doing It Wrong, or your code is actually flawed at its core and bugs will take exponentially longer to fix and never converge, or... I just need to let my agent run for 3 months??
I've been doing a lot of experimenting with this with a c++ compiler project[0]. I definitely experienced a lot of challenges, but I do believe it is possible to end-up in a consistent space by putting in some very careful guardrails. I covered some of the things i learned in the follow on blog posts. My more recent runs have been producing better code after continuing to improve harnesses, tests, and prompts. New blog post coming soon on v2 and v3 reruns. I'll be kicking off some v4 runs soon. Here is a teaser showing the later runs: https://storage.googleapis.com/ralph-run-viewer-zippy-960/in...
Please check out the code and let me know what you think. I have driven a few large refactorings that I don't think Claude would have done on its own. The two big ones were related to handling tokenization and parsing (Claude's original code had circular dependencies and would scan forward in the lever instead of using the parser to determine structure) and the second was for using subshells for command substitution instead of inline and having dual FD and buffer handling.
The only way I've found to make it work on long term AI-code-only projects is to have it constantly refactor, with the axes of refactoring usually provided by me.
It’s not human readability I care about. The problem with duplicate code is that the agent will fix one instance, but leave the others buggy. Hilarity ensues.
If that's true (I very much doubt it), then have the LLMs generate something like WASM binary and run that everywhere instead. No need for human readable languages. Or if LLMs are not that good at binary format, then some textual representation optimized for LLMs instead of humans that compiles to WASM, the JVM or Rust binary.
No need for multiple programming languages anymore. But again, doubt that happens.
> Code is no longer meant to be read by humans but by machines.
That isn't true. Everyone knows it and you know it. So why would you make a statement like this? Does it make you feel like a "thought leader"? This isn't X or LinkedIn. Bulshitting here just makes you look like a fool.
The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.
My experience exactly. I think we’re doing great, but about 3 weeks in I realized it’s an unworkable mess that keeps breaking, and I spend the next two weeks cleaning it up or rebuilding it
As few people have mentioned you have to guide the AI to clean up stuff, it doesn't seem to choose that on its own. If prompted it can come up with a lot of things to clean up.
1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here.
2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context needed to handle it well. If you let an agent just build a big pile of code, you’ll have issues.
Once you have the above, you’re more likely to be able to just let the agent run.
This is why you hear about things like SQLite being ported to Rust (by multiple groups!) It has a comprehensive unit test suite and a good existing architecture. Such porting projects are benefitting from already having done the upfront work that I mentioned, plus they have the original source code as a reference implementation.
Because of that, most of what I do involves a lot of interaction with the agent. That’s because I’m typically working on complex existing systems, that don’t necessarily have the above properties, and agents aren’t at the point where they can handle that autonomously.
This is exactly the case for "porting" Bash to Rust. There is a huge amount of existing tests that can be leverage and when in doubt you can run the same code in bash and Huck and compare the result and then fix the issue.
"Reproduce an existing piece of software" doesn't seem like a problem that generalizes well to "develop a new piece of software," for a fair number of reasons.
I agree...if there is a manual that documents well specified software including edge/corner cases i would kinda consider it a waste of tokens unless the aim is to build a better version of the software
You say that, but I asked Codex to build a Asteroids (but FPS) as a web app, and it wrote it in under 4K LOC, and a Tempest implementation in 6.5K LOC. Those seem like really reasonable line counts to me.
I also built a caching proxy server for apt packages, that's a bigger project, and that is 30.5K lines of golang, which doesn't seem bad compared to apt-cacher-ng which is 19.3K of C++ (my project does significantly more).
I'm going to take it that you were being sarcastic.
However, in the case of the apt cacher, the DDoS of Ubuntu infrastructure showed that the existing apt cachers left a *LOT* of room for improvement, not to mention that apt-cacher-ng has regularly (over the last 13 years of use) gotten wedged and needed intervention.
This cacher understands Deb package repo structure, and when it finds new meta-data it grabs and checks the meta-data, downloads new version of any package you regularly download (configurable), and then makes a snapshot of that, which it then promotes for cache users to see.
I can shut down my Internet connection and then install a fresh Ubuntu from ISO, then "apt update; apt dist-upgrade" successfully via the cache.
If you are saying that AI isn't solving novel problems, you are waaaaaay off.
But you're also missing the point: This branch of the conversation is about AI tools writing wildly inflated code. Which I proposed a few data-points refuting.
Yeah came here to say the same thing. The Hard Problem of software engineering isn't making a complex monstrosity with millions of lines of code and endless feature creep, but not doing that. This was as true 20 years as it is today.
I'm experimenting with large-ish software projects using Claude Fable. The results are surprising to me.
- Project portfolio management software. Good results so far. This is unguided experimentation: Fable researches feature ideas and plans them, then Opus 5 implements them, without me in the loop. The features include things such as an auth server, full-text search, audit logging, resource leveling, HA/DR, IaC, and more.
- FHIR & SNOMED medical software experimentation in demo-only environments. This has mixed results so far. This is guided experimentation: the FHIR specifications and SNOMED specifications are thorough and I'm in the loop. Fable plans OK, but Opus 5 is thrashing a lot, going down many rabbit holes.
We (Team of 3) built a retrieval engine from scratch in .NET for investigative search, with AI doing most of the backend implementation while humans handled the design and architecture and front end. It was developed using a task based, incremental approach. Includes more than 3,000 tests and extensive documentation, and is optimized for asynchronous ingestion using NATS and a disk backed WAL.
It includes a custom WAL, structure aware semantic chunking, ingestion and indexing pipelines, hybrid (rrf), text and vector retrieval, ACL aware filtering at query level, query classifier, citation tracking, and a CrateDB backend that performs fused retrieval over a single table.
total LOC: 180 K.
Status: evaluation by State agencies for investigative search on legacy documents.
tool used - Codex, Luna, and earlier codex 5.3, and gpt 4 mini.
i've been working on several rather large projects these past few months, and i'm trying to write as little code as possible.
I don't think i wrote more than 10 lines of code in the largest project i'm working on. Lines of code: Java: 900_635, typescript: 725_418, C++: 180_445, Dart: 96_181.
It's been obvious from the start that no model, as good as it is, can do large(-ish) amounts of work by its own without supervision, control, criticism, etc ... If left unsupervised, models usually do half the work, leaving stubs and todos everywhere.
Quality comes from applying software engineering principles as much as possible, just like you would do with teams of junior devs: planning sessions and implementation sessions with adversarial critiques, specifying as much as possible upfront, planning unit/smoke/integration tests, etc ...
Many systems rely on swarm of agents to build software but i've found it very difficult to get good results without lots of overhead/token waste because of inter agent communications mostly.
So instead i built what is mostly a workflow engine to structure / organize processes into workflows with different agents assigned different roles. I've setup a basic landing page here https://kodfactory.com if anyone wants to follow along.
Seems like most of developers and humans vibecoding new projects with LLMs. But I can see they have problems in the codebase with AI generated code, or security breachs etc. Whats the limit of it? In other words, I'm curious about what the future holds.
For really hard problems (maths) it's far far worse than Fable 5, not even as good as Sol 4.6. Current benchmarks are inadequate as they fail to capture how much worse Opus 5 is than Fable 5.
I'm a bit suspicious of this as there are open source excel clones already.
It's impressive AI can put together it's own version, but I can't help but feel it's probably just relying on existing excel clones as it's a well trodden space.
Novel applications are what impress me. Not a clone.
Agreed. Novel would be very impressive. AI tech is struggling with that.
It is important not to wait for today's AI tech to get to AGI. It is better to exploit the useful things it can do today while waiting for it to get better tomorrow.
Cloning existing software might be useful today.
It allows the model to
1) learn from its mistakes
2) mix known solutions (embed product A inside product B)
3) mine value (turn product A into a set of reusable libraries)
What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost.
The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining performant, scalable, and understandable to humans. I am unaware of what is the currently longest-lived vibe coded project but it cannot be more than a few years old.
So to my mind, we don't have very much real data yet.
I'm trying to build a competitor to Parasolid and... it's a struggle getting towards the end of the first 90% of the work. It's getting back to the point where I'm buying textbooks and reading papers because moderate domain knowledge is not enough to drive LLMs to completion here.
The space is too big and the code too extensive for a 1M context window and the capabilities of the current SOTA models.
I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions. I wonder what kind of structure AI-only code would have.
> I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions.
In the linked paper the researchers state "AI solutions must match the original program’s output exactly on end-to-end tests, including held-out tests. " And a bit further on "The AI agent is given execute-only access to the original program and a set of visible test cases, but does not have access to the original source code. "
At a guess, their agents are not going off into the weeds (like mine do occasionally) is because they have both access to a reference runtime implementation to use as a definitive end to end test to guide development. And also a relatively large token budget to run enough characterisation tests to allow them to home in on solutions.
It's decent at design and implementation, but strangely terrible at language consistency - and expectedly architecture.
Steering it on language consistency is easy. Steering it on architecture has required me to learn a lot more about compilers than I wanted to do for this project (since that's a lot more than I wanted to take on for something that was supposed to be a "fun" "side" project).
I'm not sure about architecture. When I first implemented the slot system in my language, I designed it around that implementation, and in the end, the architecture suffered. There were quite a few cases where consistency was simply unavoidable.
It's not easy to create a completely new architecture with something that doesn't exist in existing languages. Consistency doesn't come together as easily as you'd think.
In that sense, I really admire the people who first introduced something new.
When I started implementing it, I focused on one killer feature and just went for it—with the feeling that if this succeeds, my language will succeed. But once I started building around it, it turned out to be very different from other compiler architectures.
With CRUD or other well-established domains, it's easier to maintain consistency. But achieving internal consistency in a program seems to be a different kind of challenge.
Is it like self driving cars, where 100% non-human code would be arguably safer?
But then it always comes down to having some human who is accountable for verifying the behavior. I think ultimately we just need a better set of languages and frameworks to minimally encode the types of applications we want to build today, so that human comprehension and verification of the source code becomes easier.
I've noticed that when only AI writes the code, there are cases where it ends up being unusable by humans. For small code snippets or function-level units, AI is often much better than me. But when I look at the whole picture, AI tends to perform poorly.
I have been using Claude to build a clone of Bash in Rust (https://github.com/jdstanhope/huck). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests.
That being said, I am still guiding it on a regular basis and only recently let it run open ended trying fixing small to medium size bugs. The next stage will be try let it run in a loop trying to fix the 100 issues it has identified as well as getting most of the bash test suite to pass.
> it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests.
you ever worked with human engineers before? :D
I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.
I find most of the time it would be fine on its own, but every now and again you avoid a really bad degradation through guidance. Some kinds of big change definitely problematic.
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that if I, human overseer, don't course correct it many times a day, it goes off into Neverland.
Yet I see comments and articles like above every week, where someone says "Yeah, I just left my agent run for a few nights, and then it made this awesome tool."
There is a huge gap here, but I can't pinpoint it yet. Either I (and everyone I talk to) is Doing It Wrong, or your code is actually flawed at its core and bugs will take exponentially longer to fix and never converge, or... I just need to let my agent run for 3 months??
I've been doing a lot of experimenting with this with a c++ compiler project[0]. I definitely experienced a lot of challenges, but I do believe it is possible to end-up in a consistent space by putting in some very careful guardrails. I covered some of the things i learned in the follow on blog posts. My more recent runs have been producing better code after continuing to improve harnesses, tests, and prompts. New blog post coming soon on v2 and v3 reruns. I'll be kicking off some v4 runs soon. Here is a teaser showing the later runs: https://storage.googleapis.com/ralph-run-viewer-zippy-960/in...
[0]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...
Please check out the code and let me know what you think. I have driven a few large refactorings that I don't think Claude would have done on its own. The two big ones were related to handling tokenization and parsing (Claude's original code had circular dependencies and would scan forward in the lever instead of using the parser to determine structure) and the second was for using subshells for command substitution instead of inline and having dual FD and buffer handling.
The only way I've found to make it work on long term AI-code-only projects is to have it constantly refactor, with the axes of refactoring usually provided by me.
> tons of duplication that lead to whack-a-mole
The future is duplication. Code is no longer meant to be read by humans but by machines.
It’s not human readability I care about. The problem with duplicate code is that the agent will fix one instance, but leave the others buggy. Hilarity ensues.
How near/far is that future ?
Cuz what if a human has to intervene as the LLM can’t get it right?
Assuming a human then has till be able to comprehend it?
If that's true (I very much doubt it), then have the LLMs generate something like WASM binary and run that everywhere instead. No need for human readable languages. Or if LLMs are not that good at binary format, then some textual representation optimized for LLMs instead of humans that compiles to WASM, the JVM or Rust binary.
No need for multiple programming languages anymore. But again, doubt that happens.
> Code is no longer meant to be read by humans but by machines.
That isn't true. Everyone knows it and you know it. So why would you make a statement like this? Does it make you feel like a "thought leader"? This isn't X or LinkedIn. Bulshitting here just makes you look like a fool.
The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.
My experience exactly. I think we’re doing great, but about 3 weeks in I realized it’s an unworkable mess that keeps breaking, and I spend the next two weeks cleaning it up or rebuilding it
As few people have mentioned you have to guide the AI to clean up stuff, it doesn't seem to choose that on its own. If prompted it can come up with a lot of things to clean up.
Yeah, it always wants to add another layer or append another fix. It almost never goes back and fixes the original thing in place.
IME you can get good results if:
1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here.
2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context needed to handle it well. If you let an agent just build a big pile of code, you’ll have issues.
Once you have the above, you’re more likely to be able to just let the agent run.
This is why you hear about things like SQLite being ported to Rust (by multiple groups!) It has a comprehensive unit test suite and a good existing architecture. Such porting projects are benefitting from already having done the upfront work that I mentioned, plus they have the original source code as a reference implementation.
Because of that, most of what I do involves a lot of interaction with the agent. That’s because I’m typically working on complex existing systems, that don’t necessarily have the above properties, and agents aren’t at the point where they can handle that autonomously.
This is exactly the case for "porting" Bash to Rust. There is a huge amount of existing tests that can be leverage and when in doubt you can run the same code in bash and Huck and compare the result and then fix the issue.
[dead]
"Reproduce an existing piece of software" doesn't seem like a problem that generalizes well to "develop a new piece of software," for a fair number of reasons.
I agree...if there is a manual that documents well specified software including edge/corner cases i would kinda consider it a waste of tokens unless the aim is to build a better version of the software
I believe they can make ANY software project the largest it can be!
This solution should look for more problems.
You say that, but I asked Codex to build a Asteroids (but FPS) as a web app, and it wrote it in under 4K LOC, and a Tempest implementation in 6.5K LOC. Those seem like really reasonable line counts to me.
I also built a caching proxy server for apt packages, that's a bigger project, and that is 30.5K lines of golang, which doesn't seem bad compared to apt-cacher-ng which is 19.3K of C++ (my project does significantly more).
Those are amazing problems that definitely needed solving.
You should really be proud of those LOC numbers!
Good job!
I'm going to take it that you were being sarcastic.
However, in the case of the apt cacher, the DDoS of Ubuntu infrastructure showed that the existing apt cachers left a *LOT* of room for improvement, not to mention that apt-cacher-ng has regularly (over the last 13 years of use) gotten wedged and needed intervention.
This cacher understands Deb package repo structure, and when it finds new meta-data it grabs and checks the meta-data, downloads new version of any package you regularly download (configurable), and then makes a snapshot of that, which it then promotes for cache users to see.
I can shut down my Internet connection and then install a fresh Ubuntu from ISO, then "apt update; apt dist-upgrade" successfully via the cache.
If you are saying that AI isn't solving novel problems, you are waaaaaay off.
But you're also missing the point: This branch of the conversation is about AI tools writing wildly inflated code. Which I proposed a few data-points refuting.
The benchmark does not reward producing more code.
Yeah came here to say the same thing. The Hard Problem of software engineering isn't making a complex monstrosity with millions of lines of code and endless feature creep, but not doing that. This was as true 20 years as it is today.
I apologize for such a big program — I didn't have enough tokens to write a small one.
- my coworkers
I'm experimenting with large-ish software projects using Claude Fable. The results are surprising to me.
- Project portfolio management software. Good results so far. This is unguided experimentation: Fable researches feature ideas and plans them, then Opus 5 implements them, without me in the loop. The features include things such as an auth server, full-text search, audit logging, resource leveling, HA/DR, IaC, and more.
- FHIR & SNOMED medical software experimentation in demo-only environments. This has mixed results so far. This is guided experimentation: the FHIR specifications and SNOMED specifications are thorough and I'm in the loop. Fable plans OK, but Opus 5 is thrashing a lot, going down many rabbit holes.
We (Team of 3) built a retrieval engine from scratch in .NET for investigative search, with AI doing most of the backend implementation while humans handled the design and architecture and front end. It was developed using a task based, incremental approach. Includes more than 3,000 tests and extensive documentation, and is optimized for asynchronous ingestion using NATS and a disk backed WAL.
It includes a custom WAL, structure aware semantic chunking, ingestion and indexing pipelines, hybrid (rrf), text and vector retrieval, ACL aware filtering at query level, query classifier, citation tracking, and a CrateDB backend that performs fused retrieval over a single table. total LOC: 180 K. Status: evaluation by State agencies for investigative search on legacy documents. tool used - Codex, Luna, and earlier codex 5.3, and gpt 4 mini.
i've been working on several rather large projects these past few months, and i'm trying to write as little code as possible.
I don't think i wrote more than 10 lines of code in the largest project i'm working on. Lines of code: Java: 900_635, typescript: 725_418, C++: 180_445, Dart: 96_181.
It's been obvious from the start that no model, as good as it is, can do large(-ish) amounts of work by its own without supervision, control, criticism, etc ... If left unsupervised, models usually do half the work, leaving stubs and todos everywhere.
Quality comes from applying software engineering principles as much as possible, just like you would do with teams of junior devs: planning sessions and implementation sessions with adversarial critiques, specifying as much as possible upfront, planning unit/smoke/integration tests, etc ...
Many systems rely on swarm of agents to build software but i've found it very difficult to get good results without lots of overhead/token waste because of inter agent communications mostly.
So instead i built what is mostly a workflow engine to structure / organize processes into workflows with different agents assigned different roles. I've setup a basic landing page here https://kodfactory.com if anyone wants to follow along.
Seems like most of developers and humans vibecoding new projects with LLMs. But I can see they have problems in the codebase with AI generated code, or security breachs etc. Whats the limit of it? In other words, I'm curious about what the future holds.
This actually proves the hardest part of software engineering is defining requirements.
> MirrorCode only covers programs with plain-text output. MirrorCode does not test AI abilities in software that is inherently visual
It’s an interesting eval but this is a distinct limitation — most applications that consumers use directly are not purely text-based.
I am curious how Claude Opus 5 fares - similar, better, or (my guess) worse than Fable 5.
For really hard problems (maths) it's far far worse than Fable 5, not even as good as Sol 4.6. Current benchmarks are inadequate as they fail to capture how much worse Opus 5 is than Fable 5.
I agree, Opus 5 is terrible. Worse than Opus 4.8.
Running a Loop on the well defined goal seems promising: https://www.youtube.com/shorts/EtxdB7gpFqY
I'm a bit suspicious of this as there are open source excel clones already.
It's impressive AI can put together it's own version, but I can't help but feel it's probably just relying on existing excel clones as it's a well trodden space.
Novel applications are what impress me. Not a clone.
Agreed. Novel would be very impressive. AI tech is struggling with that.
It is important not to wait for today's AI tech to get to AGI. It is better to exploit the useful things it can do today while waiting for it to get better tomorrow.
Cloning existing software might be useful today. It allows the model to
1) learn from its mistakes
2) mix known solutions (embed product A inside product B)
3) mine value (turn product A into a set of reusable libraries)
etc.
> without access to the original source code
All models in the leaderboard probably have had access to the original source code in their training data.
Which is discussed in the article, tbf
my biggest so far was https://apps.apple.com/at/app/aifcc-ai-first-computer/id6782... a whole linux distro in a mac app. claude worked for about 2 weeks.
Couldn't find what exact tests they are running. The GitHub repo is very obscure to be read by my human brain.
What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost.
The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining performant, scalable, and understandable to humans. I am unaware of what is the currently longest-lived vibe coded project but it cannot be more than a few years old.
So to my mind, we don't have very much real data yet.
isn't this a harness engineering problem at its core (and the human equivalent being a product org problem)?
I'm trying to build a competitor to Parasolid and... it's a struggle getting towards the end of the first 90% of the work. It's getting back to the point where I'm buying textbooks and reading papers because moderate domain knowledge is not enough to drive LLMs to completion here.
The space is too big and the code too extensive for a 1M context window and the capabilities of the current SOTA models.
Why just the commercial ones !??
I'm curious about that too.
I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions. I wonder what kind of structure AI-only code would have.
> I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions.
In the linked paper the researchers state "AI solutions must match the original program’s output exactly on end-to-end tests, including held-out tests. " And a bit further on "The AI agent is given execute-only access to the original program and a set of visible test cases, but does not have access to the original source code. "
At a guess, their agents are not going off into the weeds (like mine do occasionally) is because they have both access to a reference runtime implementation to use as a definitive end to end test to guide development. And also a relatively large token budget to run enough characterisation tests to allow them to home in on solutions.
Also building a language with AI.
It's decent at design and implementation, but strangely terrible at language consistency - and expectedly architecture.
Steering it on language consistency is easy. Steering it on architecture has required me to learn a lot more about compilers than I wanted to do for this project (since that's a lot more than I wanted to take on for something that was supposed to be a "fun" "side" project).
I have a similar problem.
I'm not sure about architecture. When I first implemented the slot system in my language, I designed it around that implementation, and in the end, the architecture suffered. There were quite a few cases where consistency was simply unavoidable.
It's not easy to create a completely new architecture with something that doesn't exist in existing languages. Consistency doesn't come together as easily as you'd think.
In that sense, I really admire the people who first introduced something new.
When I started implementing it, I focused on one killer feature and just went for it—with the feeling that if this succeeds, my language will succeed. But once I started building around it, it turned out to be very different from other compiler architectures.
With CRUD or other well-established domains, it's easier to maintain consistency. But achieving internal consistency in a program seems to be a different kind of challenge.
Is it like self driving cars, where 100% non-human code would be arguably safer?
But then it always comes down to having some human who is accountable for verifying the behavior. I think ultimately we just need a better set of languages and frameworks to minimally encode the types of applications we want to build today, so that human comprehension and verification of the source code becomes easier.
How would 100% non-human code possibly be safer?
well if driving is supposedly safer when automated, why not code
Is it actually safer? If you can't understand the code you can't verify it's safety, and AI code often has bugs in it. You can't trust implicitly.
The qualities of computation that make autopilots safer than human drivers aren’t the same qualities that produce better designed software products.
It’s supposedly safer than the average driver, not safer than professional drivers.
Category error. Self-driving cars are still meticulously coded by humans.
I've noticed that when only AI writes the code, there are cases where it ends up being unusable by humans. For small code snippets or function-level units, AI is often much better than me. But when I look at the whole picture, AI tends to perform poorly.