>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them;
This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around the corner' and will do it for us? And people have been making this point for years now, and it's not like my job got any easier. I just got more AI.
And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered?
Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm.
Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems.
Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless.
Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.
It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".
Can you imagine scenarios that would make it less silly? I will give an example:
- The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
The Hallucinations are becoming less, significantly by now.
It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy.
We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less?
I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc.
Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
Hallucinations were low hanging fruit in some ways. As someone working on a large-ish complex-ish distributed system that has to be maintained and support customers, it's still very high value to have Claude in the mix, but the core problem of needing to monitor, advise, course correct, and make sure you don't end up with more code and complexity than you need is, at least in my experience, still roughly the same. The sharp edges are being filed off very rapidly, but the core experience of "make and maintain a large system" isn't advancing nearly as fast, IMO.
> The Hallucinations are becoming less, significantly by now.
Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection.
> We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology.
> . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations.
> Chinese models becoming very good as open models
I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway?
> There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way.
> Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds:
1. how long it takes to add new features?
2. which ones act more according to spec once added?
3. when adding features, which ones have more bugs?
4. in the face of an error, will the agent delete my whole AWS infra (count the money losses if possible also)?
5. will I understand (or need to understand, but I bet yes) this code at some point in the future?
You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)...
> shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.
You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.
> We do not have even enough compute power for current technology.
Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.
It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.
> Now add the human factor of requirements and back and forth
Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.
I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.
People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.
So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.
I think you are missing a lot of details here, honestly.
> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.
The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?
> It doesn't need to be perfect
I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.
In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.
If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.
This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ...
Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate.
To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel.
To collectives, it's a tool to work on consensus and conventions, to share and gather.
It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity.
> why then get bothered
Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ...
The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm.
We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically.
How many associations will we be able to make if AIs will do everything without human intervention?
Yes I double checked the quote is not in the article. HN is probably the best place on the internet for people actually reading the article, but this being the top comment here suggests that the majority of voters still do not read the article
Same reason some think preserving the environment is pointless because the believers will ascend to heaven, either way. It’s a religion. It’s dogmatic nihilism.
> I, as the human, still have to do the thinking as Claude still 'can't jump'
I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down. I agree LLMs are not good at abduction but very few humans are either and very few jobs/tasks require it. I can't talk for researchers jobs though. But perhaps fewer researchers would be desired by these labs (not none).
A fully automated utopia isn't just going to happen. Even with frontier models, the integrations, the evals, the UX, need a lot of work and someone needs to do it. After I've automated this thing I'll move on to the next task, this is what it means to be a software engineer.
It still makes a massive difference for me if they only need a handfull people now.
Generating a good looking UI for example, is so much easier now with LLM.
For a joke I asked ChatGPT yesterday to make a short promoimage for a 'joke' idea i had, it was above avg. I have for sure seen worse Marketing Images than what ChatGPT generated.
It looked similiar to plenty of other Marketing Images but its not that anyone cares.
although, if i'm out of tokens and have to wait a full day, i won't bother doing some things manually because the day i'll spend doing something won't take more than 1 hour the next day when tokens are available again.
That seems like a somewhat orthogonal point? Like, if I'm a carpenter and my batteries all run out / I can't actually power my power tools then the best course of action is to go home and recharge all the batteries instead of trying to hand-cut 100 pieces of lumber today. After all, the power tools can do it a lot faster (and with less effort) than I can.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
I read that more so as, I'm a carpenter and my batteries have all ran flat, so I'll put them on charge and do something else today. I'll cut up the lumber tomorrow when the batteries have charged.
AI has made 'jumps' in demanding fields like leading mathematical research and has made advancements in AI research itself. Is now a good time to start a maths career? Is there a field of research (yours?) which is inherently (more) AI proof?
Btw, I think the discussion of Einstein's career in the paper you link is historically wrong in many respects, particularly the argument about 'weak signal'. Einstein was in fact working on some of the most mainstream and widely discussed problems in physics of the day, he is admired for the creativity of his solutions to those problems, and much of his work built incrementally on ideas and breakthroughs that came (long) before (as all research does).
Article suggests that a central motivation of Einstein's work was resolving action-at-a-distance in Newtonian mechanics - yet Maxwell introduced the same Lagrangian field theories for electromagnetism we use today 50 years earlier to solve the same problem for Farraday's laws of electromagnetism. Similar wave equations existed even earlier. Heaviside in 1893 extended this technique to gravity (matching 'weak field' GR) 20 years earlier. So this is perhaps the one aspect of gravity that had actually already been solved before Einstein. Authors might be conflating his work on action-at-a-distance in QM.
Einstein's GR extended the linear 'weak field' understanding of gravity to include the non-linear self-referential case where masses themselves create gravity. This was mathematically incredibly difficult but was necessary precisely because SR's mass energy equivalence created so many strong signals that were unresolved. For example: if finite energy is mass, then mass changes as objects accelerate past a large mass like a start, and hence their propagation in space could not be explained by linear EM style field equations. Many such considerations were causing very 'strong signals' in SR, and there were analogous problems in QM atomic models being developed at the same time.
SR was also a solution to a problem that was actively being worked by many of the leading physicists of the day. SR actually does match Newtonian mechanics for a single observer - it resolves contradictions in the case of separate observers, by allowing them to assign different values to the speeds, masses, etc of objects such that each object appears to follow Newtonian mechanics for each observer. Again, this was necessary because of a lot of contradictions related to the behavior of light that had been well-known for ~20 years at the time.
Personally, I don't consider this kind of reasoning to be beyond the capabilities of future LLMs (even current LLMs if the task was broken into technical rather than philosophical problems). Personally, I doubt that such problems could stand open for 20+ years waiting for a creative genius to solve them in the modern world.
Also people tend to forget that LLMs still just work on compressed data... Where are the MAJOR breakthroughs? Where is all the "crazy" AI output going? Software seemed to degrade in quality a lot in the recent years. All "improvements" LLMs go through are simply improvements on how to burn more tokens out of my pockets given that Claude now want an actual browser extension to "visually" confirm small changes every time I use it for UI.
They are still just data parrots.
From what I see most benefits are for people that work with LLMs, but usually smaller percentages never 50% or more because of the LLMs (OK, unless you were doing basic, repetitive stuff, but then that's not to write about).
Which kind of answers the original question "why bother working?" with "because now, I can do a bit more than before".
I also see bad quality (in code, documents, presentations). It comes from people that had no clue how to do something before and now they imagine that just asking Claude is solving well the problem. And is annoying (and hard) to explain to it them, and then they get frustrated.
Idk, maybe some people get crazy productivity out of LLMs. To me, going deep into the AI bubble, reading about terms I've never seen before just feels like some crypto bro bubble with people being too deep into the sauce to notice that these things are not the wonder machines they believe so hard in...
It also seems kinda tone deaf. If someone basically told me I was wasting my time and asked what I would do in the future, I would not bother giving them a particularly thoughtful answer because trying to spend effort justifying my life choices to them would be the actual waste of time.
By trade I'm a UX Researcher/Designer who designs in code (HTML/CSS) and have done so since 2009. Recently I vibe coded an entire python app with a database and each time I didnt know what to do I would just feed screenshots to Gemini or Codex for guidance (i think i could share my screen with Codex and it can guide me via a voice conversation). I know I could follow up and build a companion iPhone and Android app using these tools.
Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career. Where do you see the opportunity where I just see a bleak one where anyone can do this stuff by typing or talking to AI? Myself, after 17 years in the field I am begrudingly back in school for a new medical career. As well, anytime an IT recruiter reaches out I am getting responses back only after under-cutting the hourly rate I use to demand and what others probably are still trying to get. And with it feels even bleaker as it becomes a race to the bottom!
In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
> In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles from professional software developers because they are trying to get quality output, but if you don't have a bar for quality, just about everyone can create their own software.
A poster a few months ago had a Show HN about his 7 year old kid, barely able to read, who was happily vibing up games.
I can attest to the other side as well, that I have seen professional software developers outputting code of lower quality than AI. And I would say that during my career (18 or so years) I have met a small number of quality software developers or engineers.
Although that might be because I was not in Silicon Valley where most of the smart/hotshot engineers converge.
The industry has vast (and increasing) oversupply of “programmers” versus diminishing demand. Add to this, the adoption of AI.
> Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career.
I think until the market better achieves some equilibrium, there is no way general software programming (sorry “engineering”) should be considered as a career. That said, there will always be opportunities in particular markets or specialties.
I also work in UX and SWE, and heavily use GenAI in my work. I don’t have a positive outlook for people who limit their career to one of those fields, but I do have a positive outlook for generalist, multi-disciplinary careers. When you have the experience and skill to steer product development from end-to-end, you can produce high-quality products super-quickly. The experience and skills are the differentiator — if you lack those you can still use GenAI to move fast but probably in the wrong direction.
> > Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.
> I imagine someone being asked this question, and how they should respond. I think like so - ‘Fuck off Richard’.
> This is partly because I imagine this question being asked in a kind of snarky, gotcha kind of way, with some sort of nerdy superiority. Like ‘ha your behaviour is inconsistent with your implied preferences, you idiot, do you even von Neumann–Morgenstern?’
> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes just thinking really hard about problems I ran into.
Now... I still do those things, but I'm doing many of them faster - and I'm often doing them while my coding agents are churning away on code.
There's also this weird effect where the harder a problem is the more I can get done in parallel with it, because an agent might need to spend 20 minutes on it without my involvement.
You read my mind. I suspect it is a transient spike while I blow through the backlog and I'll run out of things that AI can vastly accelerate in, say, 6 months. But right now? I'm spending 80-90% of my time blasting through implementing features, finding bugs, fixing old bugs, writing and improving my tools... Code was never the majority of my job. But right now the bar to doing things is so low and the sheer joy of blasting through some previously tedious, low to medium hanging fruit is thrilling.
>> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
> Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
Not really, as once it is time to write code, the problem has been defined/understood (to the degree possible with knowledge acquired at the time), and encoding it is largely an exercise in typing along with verifying assumptions via test suites.
Does GenAI quicken some portions of the above workflow? Sure, in the same way IDEs with contextual code snippet suggestions can make encoding faster.
I used it to write SQL and make dashboards. Back in the day, I would spend a lot of time doing that, then I changed roles. I dipped my toe in it recently and used AI exclusively. I would send a prompt, see the output, decide if that is what I wanted or not. I kept my brain in "what-if mode" and I let the LLM handle the technical specs.
I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future.
I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
I think this is THE issue of our time as programmers to be honest: do you review every line of code an agent writes?
An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
I've found that engineers on a large team do read every line, mainly due to the fact that the skill levels run the gamut from intern to lead, and only 1 or 2 people out of 12 might have knowledge of the application being modified.
It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Claude produces is extremely verbose in some places and anemic in others, and I can't tell at a glance what's right and what looks right with that much ground to cover.
Goodness, how is that being tolerated? I guess it can’t be stopped without a lot of political capital; but 16k lines of code is HUGE, and I cannot imagine that it’s actually 16k lines of value - I’ve written whole new subsystems of a product in fewer lines. Are these all written in an exceptionally verbose language like Go or Java? Are they VERY well documented? Are they doing things they shouldn’t be doing???
> I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.
> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?
But if you observe that the agent day after day do handle user input safely; and also routinely run an agent that scans for security vulnerabilities and observe it finding cases where input is not handled safely in existing code, you may conclude that the chance of an issue is at the same level at, or probably lower than, if a human wrote it and a human reviewed it.
("Escaping" user input is not good practice though, use parameters, assuming you are talking about SQL.)
> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???
I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this.
For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.
I think it also depends on what you're building. Some solo project or basic html thing? Sure no need to review every line. It's a bit different when you're working on foundational libraries that a business relies on, anything touching a production database, etc.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
We don’t because everyone is accountable for his or her own mistakes. So everyone is incentivized for their recklessness to not be the root cause of some bug.
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you ever asked your users? What about bug reports? Is the amount and rate decreasing?
Yeah. I spend most of my day driving agents to write code, verifying the results, orchestrating work streams, and so on. The rest of the time, a Fable agent is organizing work in Linear/Jira and making sure coworkers are getting their stuff done in a way that won’t conflict.
With GenAI, we can now produce something without caring about it - or while caring about it very little.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
Yep. ADHD very strong in this one, so LLM code generation takes pretty much all joy out of coding. It’s like watching a computers play chess. Yeah, no thanks.
I feel like all you need to know about how seriously to take this is that they cite that ancient early-2025 METR study, and describe it in the text as "recently one even found..."
Same thought - 80% through reading it occurred to me to check the citations. A few items from 2025 and most well before that.
So much has changed since late 2025 one can’t really draw any conclusions from this.
In fact, I’m guessing things will continue to move so fast that by the time one were to execute a survey of developers, many of the responses and findings are no longer relevant.
Your point really goes both ways, we really don't know anything about how LLM usage is affecting anything. No one knows, it's the wild wild west, which is whatever. But I think no one can really draw conclusions from what's happening in tech right now.
Reminds me of COVID and how everyone was fighting over early trends during that time.
The whole point of the 2025 one is that they found the self-reporting to be significantly inflated, which is why self-reported surveys like this one are hard to trust.
Yes, but their newer write-up discusses that (and shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate).
They also have an update -- linked from the original study! -- explaining that it's out of date and no longer reliable, and explaining why they had to cancel a follow-up study because it was understating productivity gains (but also was showing wins for the people who carried over from their previous study): https://metr.org/blog/2026-02-24-uplift-update/
The authors of this paper decided to ignore all of METR's follow-up data and discussion, and to report only the ancient number from early 2025 (a time when Windsurf was state of the art). And then, rather than apologizing for it, and caveating it as a number not to be taken seriously, they described it as a study done "recently."
That's either shockingly dishonest or incredibly out-of-touch.
> shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate
Or the hype machine is even stronger, making people inflate those numbers even more.
If I wanted to be uncharitable, the question of "how much slower you'd be without AI" compares against people's AI-less productivity from 2026, which could have become much less than 2025 AI-less productivity, if hypothetically AI use makes people helpless and deskilled.
> f developers spend only about 15 percent of their time typing in the editor
I think this is missing an important detail. Lots of time was spent on non-coding stuff, because coding used to be more committal and hence expensive. With how quickly one can code up a quick prototype or even production-ready code these days, the code becomes the communication tool as well.
I hope that some of the executives out there will read the list. I know they won't spend the time to read the full blog, but at least the headers should be enough
I don't understand Myth 1 (Developers Spend Most of Their Time Writing Code).
They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day.
The problem with this argument is that once you have code, some (not all) of the precursors to code go away.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
How often do you talk about low code details in the meetings? It is mostly about requirements and goals (IMHO) and "architecture"/design, and now I wouldn't even bother my peers with questions about the code as I can let AI tell me how things work exactly as written in the code, not some thing that someone remembers or thinks how it works (in large codebases, most developers only know in detail the things they are working on currently or from recent history)
It helps me. Case in point, I recently had a API refactoring project that was 80% busywork and only a small amount of code. It's crap work, but it has to be done and it's part of my role. I told AI to generate the initial design doc. It took it upon itself to reference the local headers on my system and found some APIs that should be included that I hadn't even considered would also need to be touched for consistency. It saved me hours typing up the doc, requirements, test plan, etc and then saved me embarrassment by preventing me from making a stupid oversight on a tedious and boring task.
From the patterns I have seen people would tend to use the time to build demo's using AI instead of design and then have a back and forth on the demo itself. One has to consider the fact that demo's look more convincing/complete and give an impression that the design decision has been made. In some ways, how agile got rid of the need for explicit & detailed documentation, the AI created demo's will cut the requirements of high level designs too.
The concept of MVP has been there for a while, it felt deceptive too and incurred lot of tech debt. However one would feel the deception only if they understand the missing details. For a lot of roles making decisions it benefits them to ignore the details unless it really bites them
The article mentions that coding is only a fraction of dev time and thus accelerating that part of the job can only create incremental gains, and oh PS, lines of code and similar metrics are a bad way if measuring dev productivity, anyway, and we've known that for decades.
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
Isn't this falling into the "lines of code" trap TFA mentions?
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
I'm planning an article at the moment in defense of lines of code. Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
> Lines of code are akshually a good metric now, because that's the only metric LLMs can optimize and I really, really, really love LLMs, they're the bee's knees.
That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code. That sync up with the QA engineer you did to hand it off to them? Don't need that meeting if you're not writing the code. That half hour you spent installing vim extensions? Don't need 'em if you don't open vim anymore.
There are engineers whose jobs go well beyond coding, of course. Staff engineers and principal engineers have had their jobs radically change because of AI, but not because it's writing all their code.
But there are also a lot of engineers -- your standard mid-level engineer, or even senior engineers at a lot of orgs with title inflation -- whose job is almost entirely about delivering code, and who spend all day either writing code or engaging in scaffolding around code-writing activities. Let's not pretend that automating away that code writing is a 15% boost.
In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
> In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
It’s because you are engaging in first-order thinking. You imagine the immediate result which is that software engineers are replaced.
If you engaged in second-order thinking you would realize that if software engineers can be replaced, _any_ work can eventually be automated. So why stress about it, we’ll figure out a replacement for capitalism. It seems like the world is starting the slow swing to the left so eventually the problem will be solved.
Of course might be those engineers you mention are engaged in no thinking.
Sure, but why doesn't said business partner just give a requirement to the AI to do the business partner's own job? Since it is hypothetically superhuman at fulfilling requirements at such a point...
PMs seem pretty easy to replace. In fact, given any single role to fixate on, it's probably replaceable
Oh, some of us see a massive disruption coming, just maybe not in the way you think.
Talented, experienced devs leaving the field out of disgust would be pretty disruptive.
Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands would definitely disrupt businesses.
Skill atrophy, and an entire generation of "developers" that have never actually written code will be disruptive when something breaks and no one knows how to fix it (this isn't a hypothetical, I'm seeing this phenomenon in a lot of large enterprise groups).
The total elimination of novel solutions, new ideas and new approaches to development and the resulting industry wide stagnation won't be disruptive, but will certainly be a drag.
> Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands
Replace "AI slop" with "legacy code", and you've got basically any large enterprise today.
The timescales are different though: it takes years if not decades to accumulate classic legacy cruft, while LLMs can spew up huge amounts of incomprehensible spaghetti in seconds.
>> How are you going to prompt the LLM or validate its output if you don't understand the requirements?
> Your PM can write the ticket, and your QA can test the output.
If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)?
And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write the verification prompt(s)?
And if your PM can write those prompts, why not cut out the PM and have your CSRs write the feature request/verification prompts?
And if your CSRs can write the feature request/verification prompts, why not cut them out and have the organization's customers write the feature request along with acceptance prompts?
And if customers can write those prompts, why would they pay you?
Also AI is now drafting design docs, generating PLC work products, entering it all in Jira, characterizing and root causing bugs... It's speeding up the 86% of my job that isn't coding. The article is a bit myopic and frankly contradicts itself.
If Claude told you to work on a task that you don't want to work on, or make a design choice that you think is wrong, would you do it? If not, then it can't really replace things like design or meetings. (Note that this is subtly but importantly different than the "vibecoding" model, where you just don't bother to supervise Claude's decisions.)
I don't know about others, but at work, the reason I only spend like 14% of my day coding is because I'm lazy, not because I'm actually doing other stuff.
Like many others in the comments, I feel there are a lot of assumptions in this piece. Before, coding is only 14% therefore, small slice. I think that's a very superficial assumption. That was because coding was expensive and we needed to be sure we didn't code the wrong thing. If code is as cheap as it is now, we will optimize differently, we will structure around it. Instead of so many meetings we will code 5 different versions of the same thing and choose, etc.
Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.
Lots os engineering projects took as long as you mentioned because they were horribly specified. Then the engineer either waits for an answer (which they generally do not get) or takes a decision (which might not be ideal without domain knowledge). Then you release and the customer suddenly starts explaining what they want.
You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are the bottleneck and the problem itself is complex it will not be 10x improvement.
Of course, maybe we talk about different type of project, there are also the type of projects like "a web-shop that sells stuff" that someone considered "expensive to code".
Is it? The temptation to start without a thorough design is now much stronger because the implementation osnperceived to be cheap and easy to replace. But if you start building the wrong thing fast, you still get the right thing later than when you had checked properly at the start.
Basically yes, there will be more coding in that phase, more prototyping, the PRD phases will be shorter too, there will be more pressure to deliver quickly and the PRDs will be under more pressure to move more quickly. This is what I'm already seeing to be honest.
You forgot to add "coordination" to that pipeline. That is easily far and away the biggest source of delays.
That includes talking to vendors, meetings with every layer of stakeholder when just one person digs their heels, etc.
That is truly the final frontier for "AI", and one that it will likely never cross. That would be when even the execs and upper management feel threatened by "AI". But, since they also delegate so much, you often see someone at the bottom of the totem pole in those meetings. This is why nobody is getting replaced by "AI". We really need to move this discussion away from the scifi stupidity already. There is no singularity or godlike AGI about to take over the world.
I hate to use awful terms like "synergy" and "teamwork", but they do have a lot more substance and truth to them than any perceived threat from "AI".
I think the paper would have been stronger if it acknowledged how quickly the underlying evidence is becoming outdated. AI-assisted development in 2026 isn't just better models. The way many devs including myself work has changed and matured quite a bit as compared to last year
We switched from looking at an UI (claude webui) and waiting for code generation to using claude exclusivlie on the cli and claude doing a lot more stuff in the background with smaller prompts.
For me it changes in a way that i would like to have a 24/7 workspace vm setup outside of my work laptop for keeping it running if it wants and looking at it remotely if i want.
The workspace thing would also allow it to have more permissions like downloading, configuring and using headless chrome instead of highjacking my chrome session.
I think the tooling around the models themself has improved _a lot_ - they are really good at giving the models the correct context, even in big code bases
- In biz development, a dev usually spends 30-40% time on coding, and more time on requirement discussion, integration testing (especially when the tests involves mobilephone or car)
- coding time can be reduced to 30%, which means reduce 20%-30% time of the full pipeline
- meanwhile, every phase and role is using LLM now, for example, product manager can produce longer requirement doc easily (we can use LLM to read it anyway:) Meeting sometimes is more than before, because more document output leads to more reading and discussion.
- I hope to find new ways to express biz requirements, in a more efficient and automatic manner.
- Shorten the requirement-dev-test-deploy loop is very important. OUTPUT is not OUTCOME. It is equal when we can see the final result, instead of intermediate metric.
- Agentic infra is extremely useful, or every one will find a way to access the database, report and ops system, in some weird fragile method.
Writing code is indeed the bottleneck for same resource constrained companies.
Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.
Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.
I’m very suspicious of this objection, because when Claude first landed the same people now saying “code is not the bottleneck” were saying “the generated code doesn’t work.” Smacks of moving goalposts.
The only solid objection to “AI is going replace developers” is “AI is an accelerant.” It helps developers move faster. I haven’t seen anywhere it has fully replaced developers.
Whether this leads to a large number of job losses depends on whether you think we can increase software output by the same factor as the acceleration and still be profitable. I think we can, latent software demand is extremely high. I also think we’re nearing the limit of capability with current models.
Situation could change if more advanced models emerge, but some of the more foreseeable advances probably have compute requirements beyond today’s hardware.
With all the myths and hyperbolae circulating regarding AI, I'd love to know what it's like at large software companies adjusting to this brave new world.
It's easy for a small team to adjust workflows and roles, but I just imagine the office politics must be a waking nightmare in big organisations right now.
As an individual contributor I do not have insights into office politics. While we have >10k developers, it is not a software company.
Top leadership and/or investors believe in the benefits of AI.
Thus, skeptics stay silent. Who wants to loudly contradict their bosses' boss?
In practice, I do not see a big shift yet in workflows that would require a lot of politics. We just have the agent implement the code, and then it still goes through the usual code review and QA processes.
Only now with more effective models and harnesses do many developers realize how good these tools are at investigating bugs, etc.
Before the price-decreased GPT 5.6 Luna we barely had access to enough cheap AI to last for a month of work.
In my company GitHub CoPilot was rolled out on a global scale last year.
LLMs from Anthropic and co we get through a central tool which buys api tokens from Azure and AWS.
Its surprisingly fast, probably thefastest i have seen technology getting rolled out.
Im more worried about the small/middle sized companies which are software companies but don't get that they are. You know the companies 100% depending on IT but not having the right or good or enough people who often in interviews struggle with basics like code review.
I’ve had people tell me, with a completely straight face, that they expected 10-100x productivity improvements. This is at the executive and VC level. The mania is extreme.
I for sure do plenty of things with AI a lot faster.
Instead of searching some linux issue, i will prompt claude to generate a small analyser script for checking wha tlinux i have, i will tell it what hardware i have and it fixed my issue in like 5 minutes? That would have been a lot longer before.
11-18% of time spent in coding is still very high number I think. For a large org with lots of process and risk aversion, this number could be as low as 5%. Even for 14%, the 10x improvement could mean 86+(14/10) => 87.4/100 => 12.6% overall time saved.
I mostly agree with the part where it is stated that AI is a tool which received massive investments without knowing how to maximize its utility. I think that we will pay for it in the near future.
> studies at Microsoft and elsewhere showing it’s closer to 14 percent
This is a depressing stat. The real productivity gains come from leaving soul sucking big tech companies where nothing gets done with any sort of urgency.
In my open source work I believe this is the same. I don't have numbers, but I'm sure the vast majority of my time isn't spent writing code. Of course, it depends on how you define "writing code".
It is not urgency. Large production systems mean you are doing mostly unsexy operational planning. If I had a dollar each time I hear the word "data migration" I reckon I could do well.
A lot of this rings true, but I think it's still too narrow. Sure, coding does not equal productivity, that is well debunked already. But I would argue that productivity is a product of engineering delivery + product decision making. Now where is the line between product and engineering? It varies by company, team and individual, but I don't think productivity can be measured for those functions independently, and in fact I see gains from AI on both the coding AND the product management side.
Basically as a senior tech lead in a large company engineering org, I don't have the bandwidth to individually validate every assertion from engineers on other teams OR from every product manager that comes with a half-baked ask. In the past I would be limited by the influence I could get through human relationships to strong SMEs with good judgment, and those folks always thin out as a company grows and calcifies. The number of creative and innovative thinkers dwindles, and the number of people protecting their turf and doing the minimum not to get fired increases. As a result many good ideas can get blocked by random gatekeeprs with poor imagination, poor expertise or both. However with AI I can follow up on gut instincts and fact check a lot more things, and ask incisive questions that can cut through a lot of organizational bullshit.
That's where I think most of the AI gains are today. Of course once AI plateaus and normalizes I think it will be baked into the org structures of tomorrow. But for now it offers real competitive advantage to those with the expertise to ask the right questions.
This reads like a critique of 2023 tooling published in 2026. Their Amdahl-style arithmetic (speed up a 14% slice, cap your gains at 14%) holds only if "AI" means autocomplete. Current frontier models do far more than that: research, code comprehension, review, test authoring, debugging, exploratory prototyping, ideation. That's most of the rest of the working day or "86%".
The only point that still holds is that organizational policies and procedures that automate AI use and lower the barrier to entry are more efficient than leaving it up to each individual. Every other point they make is either stale or was never true to begin with.
It's kind of weird how we blame agents for hallucinations as if humans don't fall for that as well, while agents can run the build-fail-fix-repeat in much faster cycles than coders.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
I must be a crap developer, because I probably spend twice as much time bugfixing and testing than "coding". (Both of which actually involve coding stuff, so I really don't like that distinction they make)
This is stuff AI can be really good at, so brushing that part under the table distorts the picture.
Having said that, I do agree with most of the myths they present.
LLMs are also quite good in writing unit tests and understanding bugs a lot faster than I do, now.
Just a few month back i looked at some yaml stuff for like 20 minutes, played around with it, looked at formatting etc. then i asked the LLM, it immediadly told me what was wrong. I was just blind to that particular wrong char.
even an AI assist that makes coding twice as fast would, in theory, improve developers’ overall productivity by less than 15 percent. The other 85 percent of their time remains untouched
I stopped reading after this. AI has massively impacted most aspects of my non-coding work including the mentioned planning, understanding legacy code bases, setting up environments, etc etc.
Either this article is written by people with skill issues or - given the platform - its a biased and protectionist take that will fall quickly under the march of reality.
All very sensible points which I think all senior programmers who have used AI would largely to agree with.
For those more junior - keep in mind that a lot of the maximalist rhetoric are from people either selling models, or the cottage industry of people selling you courses or tools to help you use the models. Try and keep in mind software is not a mature industry, it's an immature one, and it's prone to hype and fads.
The 14% coding time figure is one of those stats that sounds surprising until you actually track your own time. When I started building a coding agent with persistent state, I realized how some days are spent with minimal actual typing, most of it is design, reading code, debugging, problem solving, and context-switching.
But I'd push back on one thing the article implies that AI is automatically a productivity win. It's not. Some days I've shipped two months of work in a few days with AI. Other days, like today, I've burned a whole day and gotten almost nothing done because the proper research was not done by me or multiple agents.
The bottleneck for AI can be the human understanding of how to optimally use the tool. While the bottleneck for the human can be not maximizing multiple agents, or the input the user enters, then the retention of the output. If the user's input is lost, the output falters. If the user doesn't understand what the AI output is, there is going to be a problem eventually.
The article touches on adoption barriers (Myth 7), but it doesn't really get into the ego piece. There's still a wave of experienced devs who either refuse to adopt AI, or use it quietly and don't share what they're doing. That slows the whole team's learning curve. At this point, I think it's pretty much understood that you should be using AI as a dev — not to replace your skills, but to accelerate them. That means still learning new languages, still writing code, still troubleshooting. The tools change, but the craft doesn't.
I think the article is right that the real leverage is organizational, not individual. The teams that succeed with AI aren't the ones giving everyone a license — they're the ones rethinking how they review, test, and maintain code.
What I'm still uncertain about is how to measure whether AI is actually making systems better, not just faster. Lines of code is clearly a bad metric, but I haven't seen a good alternative yet. What metrics are people actually using that feel meaningful?
I'm getting tired of these articles telling me what AI will or won't do to my career when every day I see something different first hand. I'm about to stop arguing with people. If you think it's all BS then fine. Good luck.
I don’t trust point estimates like 14%. It’s like calculating an average salary and saying it’s $120k. Completely meaningless. What does the actual distribution look like that this was pulled from? No standard deviation. Is it even symmetric? What’s the 10th and 90th percentiles? Just giving a statistic on its own tells me nothing.
>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them;
This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around the corner' and will do it for us? And people have been making this point for years now, and it's not like my job got any easier. I just got more AI.
https://www.poetryfoundation.org/poems/51294/waiting-for-the...
And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
[1] https://www.tomzahavy.com/files/llms-cant-jump.pdf
> This is such a weird point to make
I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered?
Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm.
Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems.
Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless.
Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.
It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".
Can you imagine scenarios that would make it less silly? I will give an example:
- The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
Relevant xkcd: https://xkcd.com/989/
Why spend money and time making the new flagship model when a future flagship model can make you the flagship model?
Then why not stop researching and doing the definitve model that will solve every problem? Why some people are not doing it?
Bc they are aware of the marketing and limitations. If they did believe it, then they would switch area of research.
No we don't know your 'points'
The Hallucinations are becoming less, significantly by now.
It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy.
We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less?
I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc.
Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
Hallucinations were low hanging fruit in some ways. As someone working on a large-ish complex-ish distributed system that has to be maintained and support customers, it's still very high value to have Claude in the mix, but the core problem of needing to monitor, advise, course correct, and make sure you don't end up with more code and complexity than you need is, at least in my experience, still roughly the same. The sharp edges are being filed off very rapidly, but the core experience of "make and maintain a large system" isn't advancing nearly as fast, IMO.
> The Hallucinations are becoming less, significantly by now.
Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection.
> We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology.
> . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations.
> Chinese models becoming very good as open models
I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway?
> There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way.
> Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds:
You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)...Not easy at all.
> shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.
You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.
> We do not have even enough compute power for current technology.
Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.
It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.
> Now add the human factor of requirements and back and forth
Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.
I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.
People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.
> I would say yes and it will scale
So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.
I think you are missing a lot of details here, honestly.
> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.
The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?
> It doesn't need to be perfect
I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.
In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.
If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.
The AI researchers are not the ones making the marketing, much less believing in it.
I agree. But this is not what you see on the headlines and what money-incentivized stakeholders are saying.
> inherent limitations of the paradigm
This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ...
Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate.
To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel.
To collectives, it's a tool to work on consensus and conventions, to share and gather.
It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity.
> why then get bothered
Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ...
The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm.
We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically.
How many associations will we be able to make if AIs will do everything without human intervention?
I think you meant to post this in response to https://news.ycombinator.com/item?id=49174900?
Yes I double checked the quote is not in the article. HN is probably the best place on the internet for people actually reading the article, but this being the top comment here suggests that the majority of voters still do not read the article
Same reason some think preserving the environment is pointless because the believers will ascend to heaven, either way. It’s a religion. It’s dogmatic nihilism.
> I, as the human, still have to do the thinking as Claude still 'can't jump'
I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down. I agree LLMs are not good at abduction but very few humans are either and very few jobs/tasks require it. I can't talk for researchers jobs though. But perhaps fewer researchers would be desired by these labs (not none).
> I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down
Don't worry, I'm sure you'll hit your goal of zero thinking soon!
Trending towards the Homo Amens Mechanicus: the mechanical thoughtless human. What a goal
Only bc of this I will keep balancing what I do with my brain with what machines can do. dangerous outcome.
The IQ willl drop if we just become mechanical acritical people the same way muscles get worse if you do not exercise.
> I agree LLMs are not good at abduction but very few humans are either
I, too, am glad that few humans seem good at abduction.
And I hope LLMs don't get better at it.
"Abduction" being the way the article used it as in "explanatory reasoning in justifying hypotheses". https://plato.stanford.edu/entries/abduction/
A fully automated utopia isn't just going to happen. Even with frontier models, the integrations, the evals, the UX, need a lot of work and someone needs to do it. After I've automated this thing I'll move on to the next task, this is what it means to be a software engineer.
An actual utopia would require never-before-seen democratic mandate from people who are currently on the brink of hot civil war
It still makes a massive difference for me if they only need a handfull people now.
Generating a good looking UI for example, is so much easier now with LLM.
For a joke I asked ChatGPT yesterday to make a short promoimage for a 'joke' idea i had, it was above avg. I have for sure seen worse Marketing Images than what ChatGPT generated.
It looked similiar to plenty of other Marketing Images but its not that anyone cares.
although, if i'm out of tokens and have to wait a full day, i won't bother doing some things manually because the day i'll spend doing something won't take more than 1 hour the next day when tokens are available again.
That seems like a somewhat orthogonal point? Like, if I'm a carpenter and my batteries all run out / I can't actually power my power tools then the best course of action is to go home and recharge all the batteries instead of trying to hand-cut 100 pieces of lumber today. After all, the power tools can do it a lot faster (and with less effort) than I can.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
I read that more so as, I'm a carpenter and my batteries have all ran flat, so I'll put them on charge and do something else today. I'll cut up the lumber tomorrow when the batteries have charged.
According to the article that part being automated isn’t more than 18% of your day anyway
Robot powered factories still have humans there, the gist is that they are a tiny fraction of what a classical factory would require 50 years ago.
AI has made 'jumps' in demanding fields like leading mathematical research and has made advancements in AI research itself. Is now a good time to start a maths career? Is there a field of research (yours?) which is inherently (more) AI proof?
Btw, I think the discussion of Einstein's career in the paper you link is historically wrong in many respects, particularly the argument about 'weak signal'. Einstein was in fact working on some of the most mainstream and widely discussed problems in physics of the day, he is admired for the creativity of his solutions to those problems, and much of his work built incrementally on ideas and breakthroughs that came (long) before (as all research does).
Article suggests that a central motivation of Einstein's work was resolving action-at-a-distance in Newtonian mechanics - yet Maxwell introduced the same Lagrangian field theories for electromagnetism we use today 50 years earlier to solve the same problem for Farraday's laws of electromagnetism. Similar wave equations existed even earlier. Heaviside in 1893 extended this technique to gravity (matching 'weak field' GR) 20 years earlier. So this is perhaps the one aspect of gravity that had actually already been solved before Einstein. Authors might be conflating his work on action-at-a-distance in QM.
Einstein's GR extended the linear 'weak field' understanding of gravity to include the non-linear self-referential case where masses themselves create gravity. This was mathematically incredibly difficult but was necessary precisely because SR's mass energy equivalence created so many strong signals that were unresolved. For example: if finite energy is mass, then mass changes as objects accelerate past a large mass like a start, and hence their propagation in space could not be explained by linear EM style field equations. Many such considerations were causing very 'strong signals' in SR, and there were analogous problems in QM atomic models being developed at the same time.
SR was also a solution to a problem that was actively being worked by many of the leading physicists of the day. SR actually does match Newtonian mechanics for a single observer - it resolves contradictions in the case of separate observers, by allowing them to assign different values to the speeds, masses, etc of objects such that each object appears to follow Newtonian mechanics for each observer. Again, this was necessary because of a lot of contradictions related to the behavior of light that had been well-known for ~20 years at the time.
Personally, I don't consider this kind of reasoning to be beyond the capabilities of future LLMs (even current LLMs if the task was broken into technical rather than philosophical problems). Personally, I doubt that such problems could stand open for 20+ years waiting for a creative genius to solve them in the modern world.
And don't get me started on the philosophy.
Sorry where in the article does it make this point? I cannot find the text "On my visits to the Bay Area" anywhere in the original article.
Also people tend to forget that LLMs still just work on compressed data... Where are the MAJOR breakthroughs? Where is all the "crazy" AI output going? Software seemed to degrade in quality a lot in the recent years. All "improvements" LLMs go through are simply improvements on how to burn more tokens out of my pockets given that Claude now want an actual browser extension to "visually" confirm small changes every time I use it for UI. They are still just data parrots.
From what I see most benefits are for people that work with LLMs, but usually smaller percentages never 50% or more because of the LLMs (OK, unless you were doing basic, repetitive stuff, but then that's not to write about).
Which kind of answers the original question "why bother working?" with "because now, I can do a bit more than before".
I also see bad quality (in code, documents, presentations). It comes from people that had no clue how to do something before and now they imagine that just asking Claude is solving well the problem. And is annoying (and hard) to explain to it them, and then they get frustrated.
LLM don't work on 'compressed data'. LLM compress data into their latent space which allows them to become general.
They learn the concept of things and how to do them because this is better compression than learning concepts one by one.
Which means, if an LLM 'learns' the concept of a poem, it can put everything into the formad of a poem instead of learning a billion poems.
There’s a small group of established architects talking about harness engineering, but I’m not sure anyone is actually listening to them.
And those same architects are quietly extracting real productivity from GenAI.
And even this write up skips that info by waving, “Some people…”
Mind sharing some names or something else we can learn about?
Idk, maybe some people get crazy productivity out of LLMs. To me, going deep into the AI bubble, reading about terms I've never seen before just feels like some crypto bro bubble with people being too deep into the sauce to notice that these things are not the wonder machines they believe so hard in...
The only thing I noticed in some of the software I use is more design changes, but they are often not better than before.
It also seems kinda tone deaf. If someone basically told me I was wasting my time and asked what I would do in the future, I would not bother giving them a particularly thoughtful answer because trying to spend effort justifying my life choices to them would be the actual waste of time.
What kind of answers were they expecting to get?
By trade I'm a UX Researcher/Designer who designs in code (HTML/CSS) and have done so since 2009. Recently I vibe coded an entire python app with a database and each time I didnt know what to do I would just feed screenshots to Gemini or Codex for guidance (i think i could share my screen with Codex and it can guide me via a voice conversation). I know I could follow up and build a companion iPhone and Android app using these tools.
Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career. Where do you see the opportunity where I just see a bleak one where anyone can do this stuff by typing or talking to AI? Myself, after 17 years in the field I am begrudingly back in school for a new medical career. As well, anytime an IT recruiter reaches out I am getting responses back only after under-cutting the hourly rate I use to demand and what others probably are still trying to get. And with it feels even bleaker as it becomes a race to the bottom!
In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
> In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles from professional software developers because they are trying to get quality output, but if you don't have a bar for quality, just about everyone can create their own software.
A poster a few months ago had a Show HN about his 7 year old kid, barely able to read, who was happily vibing up games.
I can attest to the other side as well, that I have seen professional software developers outputting code of lower quality than AI. And I would say that during my career (18 or so years) I have met a small number of quality software developers or engineers. Although that might be because I was not in Silicon Valley where most of the smart/hotshot engineers converge.
The industry has vast (and increasing) oversupply of “programmers” versus diminishing demand. Add to this, the adoption of AI.
> Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career.
I think until the market better achieves some equilibrium, there is no way general software programming (sorry “engineering”) should be considered as a career. That said, there will always be opportunities in particular markets or specialties.
I also work in UX and SWE, and heavily use GenAI in my work. I don’t have a positive outlook for people who limit their career to one of those fields, but I do have a positive outlook for generalist, multi-disciplinary careers. When you have the experience and skill to steer product development from end-to-end, you can produce high-quality products super-quickly. The experience and skills are the differentiator — if you lack those you can still use GenAI to move fast but probably in the wrong direction.
It reminds me of Richard Hamming’s notorious question. I like the summary at https://bestjelly.substack.com/p/hamming-questions (which starts out with a quote from another site):
> > Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.
> I imagine someone being asked this question, and how they should respond. I think like so - ‘Fuck off Richard’.
> This is partly because I imagine this question being asked in a kind of snarky, gotcha kind of way, with some sort of nerdy superiority. Like ‘ha your behaviour is inconsistent with your implied preferences, you idiot, do you even von Neumann–Morgenstern?’
Also, if you believe your well-paying job is eventually going to be automated you would be prudent to bank the money while you prepare for the future.
> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes just thinking really hard about problems I ran into.
Now... I still do those things, but I'm doing many of them faster - and I'm often doing them while my coding agents are churning away on code.
There's also this weird effect where the harder a problem is the more I can get done in parallel with it, because an agent might need to spend 20 minutes on it without my involvement.
You read my mind. I suspect it is a transient spike while I blow through the backlog and I'll run out of things that AI can vastly accelerate in, say, 6 months. But right now? I'm spending 80-90% of my time blasting through implementing features, finding bugs, fixing old bugs, writing and improving my tools... Code was never the majority of my job. But right now the bar to doing things is so low and the sheer joy of blasting through some previously tedious, low to medium hanging fruit is thrilling.
>> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
> Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
Not really, as once it is time to write code, the problem has been defined/understood (to the degree possible with knowledge acquired at the time), and encoding it is largely an exercise in typing along with verifying assumptions via test suites.
Does GenAI quicken some portions of the above workflow? Sure, in the same way IDEs with contextual code snippet suggestions can make encoding faster.
I used it to write SQL and make dashboards. Back in the day, I would spend a lot of time doing that, then I changed roles. I dipped my toe in it recently and used AI exclusively. I would send a prompt, see the output, decide if that is what I wanted or not. I kept my brain in "what-if mode" and I let the LLM handle the technical specs.
Does the code get reviewed? How do you deal with increased amount of code that may need to be looked at?
I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future.
I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
I think this is THE issue of our time as programmers to be honest: do you review every line of code an agent writes?
An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
I've found that engineers on a large team do read every line, mainly due to the fact that the skill levels run the gamut from intern to lead, and only 1 or 2 people out of 12 might have knowledge of the application being modified.
It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Claude produces is extremely verbose in some places and anemic in others, and I can't tell at a glance what's right and what looks right with that much ground to cover.
Goodness, how is that being tolerated? I guess it can’t be stopped without a lot of political capital; but 16k lines of code is HUGE, and I cannot imagine that it’s actually 16k lines of value - I’ve written whole new subsystems of a product in fewer lines. Are these all written in an exceptionally verbose language like Go or Java? Are they VERY well documented? Are they doing things they shouldn’t be doing???
> I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.
> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?
But if you observe that the agent day after day do handle user input safely; and also routinely run an agent that scans for security vulnerabilities and observe it finding cases where input is not handled safely in existing code, you may conclude that the chance of an issue is at the same level at, or probably lower than, if a human wrote it and a human reviewed it.
("Escaping" user input is not good practice though, use parameters, assuming you are talking about SQL.)
> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???
I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this. For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.
Once you've seen the AI not make mistakes like that a few dozen times you start to trust it not to mess that up in the future.
We're pretty far past this if you're using anything close to the sota models.
But you could be defensive with a security checklist in agents.md and have adversarial review, if you wanted.
I think it also depends on what you're building. Some solo project or basic html thing? Sure no need to review every line. It's a bit different when you're working on foundational libraries that a business relies on, anything touching a production database, etc.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
We don’t because everyone is accountable for his or her own mistakes. So everyone is incentivized for their recklessness to not be the root cause of some bug.
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you ever asked your users? What about bug reports? Is the amount and rate decreasing?
If you're going to be reading the code you might as well write it by hand instead of using AI.
Yeah. I spend most of my day driving agents to write code, verifying the results, orchestrating work streams, and so on. The rest of the time, a Fable agent is organizing work in Linear/Jira and making sure coworkers are getting their stuff done in a way that won’t conflict.
I've found that using LLMs for significant amounts of code generation completely drain the result from any dopamine I would get doing it myself.
Have others noticed this as well? This is going so far as to me losing interest in side projects because I have "lost touch" with the code base.
With GenAI, we can now produce something without caring about it - or while caring about it very little.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
Yep. ADHD very strong in this one, so LLM code generation takes pretty much all joy out of coding. It’s like watching a computers play chess. Yeah, no thanks.
I feel like all you need to know about how seriously to take this is that they cite that ancient early-2025 METR study, and describe it in the text as "recently one even found..."
Same thought - 80% through reading it occurred to me to check the citations. A few items from 2025 and most well before that.
So much has changed since late 2025 one can’t really draw any conclusions from this.
In fact, I’m guessing things will continue to move so fast that by the time one were to execute a survey of developers, many of the responses and findings are no longer relevant.
Your point really goes both ways, we really don't know anything about how LLM usage is affecting anything. No one knows, it's the wild wild west, which is whatever. But I think no one can really draw conclusions from what's happening in tech right now.
Reminds me of COVID and how everyone was fighting over early trends during that time.
Exactly. I saw them using things from 2025... AI sorta sucked then and didn't really "take off" until that Opus drop in December or whatever it was.
I felt the same and why didn’t the authors look over METR’s recent material?
https://metr.org/blog/2026-05-11-ai-usage-survey/
The whole point of the 2025 one is that they found the self-reporting to be significantly inflated, which is why self-reported surveys like this one are hard to trust.
Yes, but their newer write-up discusses that (and shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate).
They also have an update -- linked from the original study! -- explaining that it's out of date and no longer reliable, and explaining why they had to cancel a follow-up study because it was understating productivity gains (but also was showing wins for the people who carried over from their previous study): https://metr.org/blog/2026-02-24-uplift-update/
The authors of this paper decided to ignore all of METR's follow-up data and discussion, and to report only the ancient number from early 2025 (a time when Windsurf was state of the art). And then, rather than apologizing for it, and caveating it as a number not to be taken seriously, they described it as a study done "recently."
That's either shockingly dishonest or incredibly out-of-touch.
> shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate
Or the hype machine is even stronger, making people inflate those numbers even more.
If I wanted to be uncharitable, the question of "how much slower you'd be without AI" compares against people's AI-less productivity from 2026, which could have become much less than 2025 AI-less productivity, if hypothetically AI use makes people helpless and deskilled.
> f developers spend only about 15 percent of their time typing in the editor
I think this is missing an important detail. Lots of time was spent on non-coding stuff, because coding used to be more committal and hence expensive. With how quickly one can code up a quick prototype or even production-ready code these days, the code becomes the communication tool as well.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
that's such a weird metric, why exclude bug fixing and testing? depending on the phase of the project I might spend 100% of my coding time bug fixing
I hope that some of the executives out there will read the list. I know they won't spend the time to read the full blog, but at least the headers should be enough
I don't understand Myth 1 (Developers Spend Most of Their Time Writing Code).
They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day.
The problem with this argument is that once you have code, some (not all) of the precursors to code go away.
Okay.
Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load.
My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.
> Or reducing meeting load.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
How often do you talk about low code details in the meetings? It is mostly about requirements and goals (IMHO) and "architecture"/design, and now I wouldn't even bother my peers with questions about the code as I can let AI tell me how things work exactly as written in the code, not some thing that someone remembers or thinks how it works (in large codebases, most developers only know in detail the things they are working on currently or from recent history)
> It is mostly about requirements and goals and architecture/design
People vibe code because they have no clue about any of that. Not because they're slow typers.
It helps me. Case in point, I recently had a API refactoring project that was 80% busywork and only a small amount of code. It's crap work, but it has to be done and it's part of my role. I told AI to generate the initial design doc. It took it upon itself to reference the local headers on my system and found some APIs that should be included that I hadn't even considered would also need to be touched for consistency. It saved me hours typing up the doc, requirements, test plan, etc and then saved me embarrassment by preventing me from making a stupid oversight on a tedious and boring task.
AI is helpful in such projects. Less so in other projects where design is heavy
From the patterns I have seen people would tend to use the time to build demo's using AI instead of design and then have a back and forth on the demo itself. One has to consider the fact that demo's look more convincing/complete and give an impression that the design decision has been made. In some ways, how agile got rid of the need for explicit & detailed documentation, the AI created demo's will cut the requirements of high level designs too.
> demo's look more convincing/complete ...
In the case of an LLM generated demo, usually deceptively so.
> and give an impression that the design decision has been made.
In the case of a vibed design, this is the opposite of useful for the team.
The concept of MVP has been there for a while, it felt deceptive too and incurred lot of tech debt. However one would feel the deception only if they understand the missing details. For a lot of roles making decisions it benefits them to ignore the details unless it really bites them
What kind of shape of evidence would you find convincing?
Self-reported or observational data capturing time spent for categories of task ala every other similar study in this space?
This isn't exactly novel territory, here, Simon. Let's not pretend I'm asking for something strange, unprecedented, or unreasonable.
How's this? https://simonwillison.net/2026/Jul/13/datasette-code-frequen...
I have similar charts across my three main open source projects:
https://github.com/simonw/datasette/graphs/code-frequency
https://github.com/simonw/llm/graphs/code-frequency
https://github.com/simonw/sqlite-utils/graphs/code-frequency
The article mentions that coding is only a fraction of dev time and thus accelerating that part of the job can only create incremental gains, and oh PS, lines of code and similar metrics are a bad way if measuring dev productivity, anyway, and we've known that for decades.
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
I ask for evidence.
In response you give me... code output metrics?
I don't have anything else to hand that I can think of. I don't keep a time tracking diary.
I don't think volume of code changed was the metric the other commenter was asking for.
Isn't this falling into the "lines of code" trap TFA mentions?
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
I'm planning an article at the moment in defense of lines of code. Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
> Lines of code are akshually a good metric now, because that's the only metric LLMs can optimize and I really, really, really love LLMs, they're the bee's knees.
Really now?
No.
People working fewer hours :)
Damn, looks like we have lower productivity than cavemen!
Sir, this is Capitalism.
It certainly helps as a research assistant for design work. But it can’t do it for you.
It does because you can now just steamroll features out the door and make everyone look bad that’s sitting around in meetings all day
Are you aware literally everyone else also has access to AI stuff? You’re not special for using an agent. You’re part of the crowd
Yeah, this seriously drives me nuts.
That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code. That sync up with the QA engineer you did to hand it off to them? Don't need that meeting if you're not writing the code. That half hour you spent installing vim extensions? Don't need 'em if you don't open vim anymore.
There are engineers whose jobs go well beyond coding, of course. Staff engineers and principal engineers have had their jobs radically change because of AI, but not because it's writing all their code.
But there are also a lot of engineers -- your standard mid-level engineer, or even senior engineers at a lot of orgs with title inflation -- whose job is almost entirely about delivering code, and who spend all day either writing code or engaging in scaffolding around code-writing activities. Let's not pretend that automating away that code writing is a 15% boost.
> That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code.
How are you going to prompt the LLM or validate its output if you don't understand the requirements?
In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
> In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
It’s because you are engaging in first-order thinking. You imagine the immediate result which is that software engineers are replaced.
If you engaged in second-order thinking you would realize that if software engineers can be replaced, _any_ work can eventually be automated. So why stress about it, we’ll figure out a replacement for capitalism. It seems like the world is starting the slow swing to the left so eventually the problem will be solved.
Of course might be those engineers you mention are engaged in no thinking.
Sure, but why doesn't said business partner just give a requirement to the AI to do the business partner's own job? Since it is hypothetically superhuman at fulfilling requirements at such a point...
PMs seem pretty easy to replace. In fact, given any single role to fixate on, it's probably replaceable
Oh, some of us see a massive disruption coming, just maybe not in the way you think.
Talented, experienced devs leaving the field out of disgust would be pretty disruptive.
Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands would definitely disrupt businesses.
Skill atrophy, and an entire generation of "developers" that have never actually written code will be disruptive when something breaks and no one knows how to fix it (this isn't a hypothetical, I'm seeing this phenomenon in a lot of large enterprise groups).
The total elimination of novel solutions, new ideas and new approaches to development and the resulting industry wide stagnation won't be disruptive, but will certainly be a drag.
> Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands
Replace "AI slop" with "legacy code", and you've got basically any large enterprise today.
The timescales are different though: it takes years if not decades to accumulate classic legacy cruft, while LLMs can spew up huge amounts of incomprehensible spaghetti in seconds.
You will have artisans or software SWAT teams on one side and code factories on the other. For most problems code factories will be good enough.
Your PM can write the ticket, and your QA can test the output.
I think we would, at some point realize that the roles of PM and QA are best done by an LLM, while coding is left to humans.
So an 180 from the current coding-automation frenzy..
>> How are you going to prompt the LLM or validate its output if you don't understand the requirements?
> Your PM can write the ticket, and your QA can test the output.
If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)?
And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write the verification prompt(s)?
And if your PM can write those prompts, why not cut out the PM and have your CSRs write the feature request/verification prompts?
And if your CSRs can write the feature request/verification prompts, why not cut them out and have the organization's customers write the feature request along with acceptance prompts?
And if customers can write those prompts, why would they pay you?
And finally: Why bother making new software at all when AI can just do the thing you wanted done, directly?
Because customers have better things to do.
Do they though? I assume they’re also getting replaced by LLM in this hypothetical.
Also AI is now drafting design docs, generating PLC work products, entering it all in Jira, characterizing and root causing bugs... It's speeding up the 86% of my job that isn't coding. The article is a bit myopic and frankly contradicts itself.
If Claude told you to work on a task that you don't want to work on, or make a design choice that you think is wrong, would you do it? If not, then it can't really replace things like design or meetings. (Note that this is subtly but importantly different than the "vibecoding" model, where you just don't bother to supervise Claude's decisions.)
I don't know about others, but at work, the reason I only spend like 14% of my day coding is because I'm lazy, not because I'm actually doing other stuff.
Not lazy it is a taxing task like doing an exam.
Like many others in the comments, I feel there are a lot of assumptions in this piece. Before, coding is only 14% therefore, small slice. I think that's a very superficial assumption. That was because coding was expensive and we needed to be sure we didn't code the wrong thing. If code is as cheap as it is now, we will optimize differently, we will structure around it. Instead of so many meetings we will code 5 different versions of the same thing and choose, etc.
> That was because coding was expensive and we needed to be sure we didn't code the wrong thing.
Coding has never been expensive as it is nothing more than a reification of a solution to a problem as it is understood at that time.
It is the underlying understanding of the problem which has always been expensive and remains so.
Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.
Lots os engineering projects took as long as you mentioned because they were horribly specified. Then the engineer either waits for an answer (which they generally do not get) or takes a decision (which might not be ideal without domain knowledge). Then you release and the customer suddenly starts explaining what they want.
You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are the bottleneck and the problem itself is complex it will not be 10x improvement.
Of course, maybe we talk about different type of project, there are also the type of projects like "a web-shop that sells stuff" that someone considered "expensive to code".
Is it? The temptation to start without a thorough design is now much stronger because the implementation osnperceived to be cheap and easy to replace. But if you start building the wrong thing fast, you still get the right thing later than when you had checked properly at the start.
So you're suggesting that coding will take more of the PRD phase?
Basically yes, there will be more coding in that phase, more prototyping, the PRD phases will be shorter too, there will be more pressure to deliver quickly and the PRDs will be under more pressure to move more quickly. This is what I'm already seeing to be honest.
|--------|-------|------|------|-------|------|
|Contract|Product|Design|Coding|Testing|Deploy|
Writing Code Isn't the Bottleneck, until writing code is the bottleneck, until it's not again.
Getting a usable PRD is often the bottleneck.
The real bottleneck is Omega Star getting their shit together. And I ain't joking.
Or SPS Commerce. Evidence is becoming unconvincing that they're even in business anymore.
You forgot to add "coordination" to that pipeline. That is easily far and away the biggest source of delays.
That includes talking to vendors, meetings with every layer of stakeholder when just one person digs their heels, etc.
That is truly the final frontier for "AI", and one that it will likely never cross. That would be when even the execs and upper management feel threatened by "AI". But, since they also delegate so much, you often see someone at the bottom of the totem pole in those meetings. This is why nobody is getting replaced by "AI". We really need to move this discussion away from the scifi stupidity already. There is no singularity or godlike AGI about to take over the world.
I hate to use awful terms like "synergy" and "teamwork", but they do have a lot more substance and truth to them than any perceived threat from "AI".
I think the paper would have been stronger if it acknowledged how quickly the underlying evidence is becoming outdated. AI-assisted development in 2026 isn't just better models. The way many devs including myself work has changed and matured quite a bit as compared to last year
how so?
We switched from looking at an UI (claude webui) and waiting for code generation to using claude exclusivlie on the cli and claude doing a lot more stuff in the background with smaller prompts.
For me it changes in a way that i would like to have a 24/7 workspace vm setup outside of my work laptop for keeping it running if it wants and looking at it remotely if i want.
The workspace thing would also allow it to have more permissions like downloading, configuring and using headless chrome instead of highjacking my chrome session.
I think the tooling around the models themself has improved _a lot_ - they are really good at giving the models the correct context, even in big code bases
Many devs now work in YOLO mode letting LLMs automate their tasks.
Some experience from my work:
- In biz development, a dev usually spends 30-40% time on coding, and more time on requirement discussion, integration testing (especially when the tests involves mobilephone or car)
- coding time can be reduced to 30%, which means reduce 20%-30% time of the full pipeline
- meanwhile, every phase and role is using LLM now, for example, product manager can produce longer requirement doc easily (we can use LLM to read it anyway:) Meeting sometimes is more than before, because more document output leads to more reading and discussion.
- I hope to find new ways to express biz requirements, in a more efficient and automatic manner.
- Shorten the requirement-dev-test-deploy loop is very important. OUTPUT is not OUTCOME. It is equal when we can see the final result, instead of intermediate metric.
- Agentic infra is extremely useful, or every one will find a way to access the database, report and ops system, in some weird fragile method.
> Myth 2: Writing Code Is the Bottleneck
Writing code is indeed the bottleneck for same resource constrained companies.
Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.
Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.
I’m very suspicious of this objection, because when Claude first landed the same people now saying “code is not the bottleneck” were saying “the generated code doesn’t work.” Smacks of moving goalposts.
The only solid objection to “AI is going replace developers” is “AI is an accelerant.” It helps developers move faster. I haven’t seen anywhere it has fully replaced developers.
Whether this leads to a large number of job losses depends on whether you think we can increase software output by the same factor as the acceleration and still be profitable. I think we can, latent software demand is extremely high. I also think we’re nearing the limit of capability with current models.
Situation could change if more advanced models emerge, but some of the more foreseeable advances probably have compute requirements beyond today’s hardware.
With all the myths and hyperbolae circulating regarding AI, I'd love to know what it's like at large software companies adjusting to this brave new world.
It's easy for a small team to adjust workflows and roles, but I just imagine the office politics must be a waking nightmare in big organisations right now.
As an individual contributor I do not have insights into office politics. While we have >10k developers, it is not a software company.
Top leadership and/or investors believe in the benefits of AI.
Thus, skeptics stay silent. Who wants to loudly contradict their bosses' boss?
In practice, I do not see a big shift yet in workflows that would require a lot of politics. We just have the agent implement the code, and then it still goes through the usual code review and QA processes.
Only now with more effective models and harnesses do many developers realize how good these tools are at investigating bugs, etc.
Before the price-decreased GPT 5.6 Luna we barely had access to enough cheap AI to last for a month of work.
In my company GitHub CoPilot was rolled out on a global scale last year.
LLMs from Anthropic and co we get through a central tool which buys api tokens from Azure and AWS.
Its surprisingly fast, probably thefastest i have seen technology getting rolled out.
Im more worried about the small/middle sized companies which are software companies but don't get that they are. You know the companies 100% depending on IT but not having the right or good or enough people who often in interviews struggle with basics like code review.
This is actually true at my company. They expect employees to be 10× more productive now that we have AI.
I’ve had people tell me, with a completely straight face, that they expected 10-100x productivity improvements. This is at the executive and VC level. The mania is extreme.
I for sure do plenty of things with AI a lot faster.
Instead of searching some linux issue, i will prompt claude to generate a small analyser script for checking wha tlinux i have, i will tell it what hardware i have and it fixed my issue in like 5 minutes? That would have been a lot longer before.
> … with a completely straight face, that they expected 10-100x productivity improvements.
A great alternative future to look forward to - 1 employee expected to do the work of 10-100. Those others fired to save money.
So then they should get 10-100x more revenue, now that AI does all the marketing and selling.
It seems that this could have been expanded or contracted to any Fibonacci number of myths.
11-18% of time spent in coding is still very high number I think. For a large org with lots of process and risk aversion, this number could be as low as 5%. Even for 14%, the 10x improvement could mean 86+(14/10) => 87.4/100 => 12.6% overall time saved.
And time in coding is like time on highways for taxi drivers. A fairly useless metric.
> A June 2025 study of Microsoft developers
A year ago feels like forever
Not so much due to length of time, but because of the Opus shockwave than fell within that period.
I mostly agree with the part where it is stated that AI is a tool which received massive investments without knowing how to maximize its utility. I think that we will pay for it in the near future.
> studies at Microsoft and elsewhere showing it’s closer to 14 percent
This is a depressing stat. The real productivity gains come from leaving soul sucking big tech companies where nothing gets done with any sort of urgency.
In my open source work I believe this is the same. I don't have numbers, but I'm sure the vast majority of my time isn't spent writing code. Of course, it depends on how you define "writing code".
It is not urgency. Large production systems mean you are doing mostly unsexy operational planning. If I had a dollar each time I hear the word "data migration" I reckon I could do well.
A lot of this rings true, but I think it's still too narrow. Sure, coding does not equal productivity, that is well debunked already. But I would argue that productivity is a product of engineering delivery + product decision making. Now where is the line between product and engineering? It varies by company, team and individual, but I don't think productivity can be measured for those functions independently, and in fact I see gains from AI on both the coding AND the product management side.
Basically as a senior tech lead in a large company engineering org, I don't have the bandwidth to individually validate every assertion from engineers on other teams OR from every product manager that comes with a half-baked ask. In the past I would be limited by the influence I could get through human relationships to strong SMEs with good judgment, and those folks always thin out as a company grows and calcifies. The number of creative and innovative thinkers dwindles, and the number of people protecting their turf and doing the minimum not to get fired increases. As a result many good ideas can get blocked by random gatekeeprs with poor imagination, poor expertise or both. However with AI I can follow up on gut instincts and fact check a lot more things, and ask incisive questions that can cut through a lot of organizational bullshit.
That's where I think most of the AI gains are today. Of course once AI plateaus and normalizes I think it will be baked into the org structures of tomorrow. But for now it offers real competitive advantage to those with the expertise to ask the right questions.
This reads like a critique of 2023 tooling published in 2026. Their Amdahl-style arithmetic (speed up a 14% slice, cap your gains at 14%) holds only if "AI" means autocomplete. Current frontier models do far more than that: research, code comprehension, review, test authoring, debugging, exploratory prototyping, ideation. That's most of the rest of the working day or "86%".
The only point that still holds is that organizational policies and procedures that automate AI use and lower the barrier to entry are more efficient than leaving it up to each individual. Every other point they make is either stale or was never true to begin with.
one still has to think. Also about this ai automating stuff and humans playing around. I believe there is a time for this and time for that
It's kind of weird how we blame agents for hallucinations as if humans don't fall for that as well, while agents can run the build-fail-fix-repeat in much faster cycles than coders.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
I must be a crap developer, because I probably spend twice as much time bugfixing and testing than "coding". (Both of which actually involve coding stuff, so I really don't like that distinction they make)
This is stuff AI can be really good at, so brushing that part under the table distorts the picture.
Having said that, I do agree with most of the myths they present.
Then yeah you might not be a good developer.
LLMs are also quite good in writing unit tests and understanding bugs a lot faster than I do, now.
Just a few month back i looked at some yaml stuff for like 20 minutes, played around with it, looked at formatting etc. then i asked the LLM, it immediadly told me what was wrong. I was just blind to that particular wrong char.
Logic bugs? Yeah it can find them too.
even an AI assist that makes coding twice as fast would, in theory, improve developers’ overall productivity by less than 15 percent. The other 85 percent of their time remains untouched
I stopped reading after this. AI has massively impacted most aspects of my non-coding work including the mentioned planning, understanding legacy code bases, setting up environments, etc etc.
Either this article is written by people with skill issues or - given the platform - its a biased and protectionist take that will fall quickly under the march of reality.
> Despite this decade-old research, many organizations still rely on lines of code as a measure of developer productivity.
No, they don't! It's easy to dispel myths when the myths are built on straw men. Dumb article.
All very sensible points which I think all senior programmers who have used AI would largely to agree with.
For those more junior - keep in mind that a lot of the maximalist rhetoric are from people either selling models, or the cottage industry of people selling you courses or tools to help you use the models. Try and keep in mind software is not a mature industry, it's an immature one, and it's prone to hype and fads.
ChatGPT, summarize
The 14% coding time figure is one of those stats that sounds surprising until you actually track your own time. When I started building a coding agent with persistent state, I realized how some days are spent with minimal actual typing, most of it is design, reading code, debugging, problem solving, and context-switching.
But I'd push back on one thing the article implies that AI is automatically a productivity win. It's not. Some days I've shipped two months of work in a few days with AI. Other days, like today, I've burned a whole day and gotten almost nothing done because the proper research was not done by me or multiple agents.
The bottleneck for AI can be the human understanding of how to optimally use the tool. While the bottleneck for the human can be not maximizing multiple agents, or the input the user enters, then the retention of the output. If the user's input is lost, the output falters. If the user doesn't understand what the AI output is, there is going to be a problem eventually.
The article touches on adoption barriers (Myth 7), but it doesn't really get into the ego piece. There's still a wave of experienced devs who either refuse to adopt AI, or use it quietly and don't share what they're doing. That slows the whole team's learning curve. At this point, I think it's pretty much understood that you should be using AI as a dev — not to replace your skills, but to accelerate them. That means still learning new languages, still writing code, still troubleshooting. The tools change, but the craft doesn't.
I think the article is right that the real leverage is organizational, not individual. The teams that succeed with AI aren't the ones giving everyone a license — they're the ones rethinking how they review, test, and maintain code.
What I'm still uncertain about is how to measure whether AI is actually making systems better, not just faster. Lines of code is clearly a bad metric, but I haven't seen a good alternative yet. What metrics are people actually using that feel meaningful?
I'm getting tired of these articles telling me what AI will or won't do to my career when every day I see something different first hand. I'm about to stop arguing with people. If you think it's all BS then fine. Good luck.
I don’t trust point estimates like 14%. It’s like calculating an average salary and saying it’s $120k. Completely meaningless. What does the actual distribution look like that this was pulled from? No standard deviation. Is it even symmetric? What’s the 10th and 90th percentiles? Just giving a statistic on its own tells me nothing.