I’m curious how you think this compares to the Research -> Plan -> Implement method and prompts from the “Advanced Context Engineering from Agents” video when it comes to actual coding performance on large codebases. I think picking up skills is useful for broadening agents abilities, but I’m not sure I’d that’s the right thing for actual development.
The packaged collection is very cool and so is the idea of automatically adding new abilities, but I’m not fully convinced that this concept of skills is that much better than having custom commands+sub-agents. I’ll have to play around with it these next few days and compare.
Using Research->Plan->Implement flow is orthogonal, though I notice parts of those do exist as skills too. But you sometimes need to do other things too, e.g. debugging in the course of implementing or specific techniquws to improve brainstorming/researching.
Some of these skills are probably better as programmed workflows that the LLM is forced to go through to improve reliability/consistency, that's what I've found in my own agents, rather than using English to guide the LLM and trusting it to follow the prescribed set of steps needed. Some mix of LLMs (choosing skills, executing the fuzzy parts of them) and just plain code (orchestration of skills) seems like the best bet to me and what I'm pursuing.
This article left me wishing it was "How I'm using coding agents to do <x> task better"
I've been exploring AI for two years now. It's certainly upgraded itself from the toy classification to a basic utility. However, I increasingly run into its limitations and find reverting to pre-LLM ways of working more robust, faster, and more mentally sustainable.
Does someone have concrete examples of integrating LLM in a workflow that pushes state-of-the-art development practices & value creation further?
The XDG spec has been out for decades now. Why are new applications still polluting my HOME? Also seems weird that real data would be put under a cache/ location, but whatever.
It's in the cache location because it's a copy of a plugin that was installed from a GitHub repository, so that's not the original point of truth for that file.
This style of prompting, where you set up a dire scenario in order to try to evoke some "emotional" response from the agent, is already dated. At some point, putting words like IMPORTANT in all uppercase had some measurable impact, but at the present time, models just follow instructions.
Save yourself the experience of having to write and maintain prompts like this.
What’s irritating is that the llms haven’t learned this as bout themselves yet. If you ask an llm to improve its instructions those sort of improvements are what it will suggest.
It is the thing I find most irritating about working with llms and agents. They seem forever a generation behind in capabilities that are self referential.
Fascinating write-up. I loved this bit of debugging:
> The first time we played this game, Claude told me that the subagents had gotten a perfect score. After a bit of prodding, I discovered that Claude was quizzing the subagents like they were on a gameshow. This was less than useful. I asked to switch to realistic scenarios that put pressure on the agents, to better simulate what they might actually do.
documents like https://github.com/obra/superpowers/blob/main/skills/testing... are very confusing to read as a human.
"skills" in this project generally don't seem to follow set format and just look like what you would get when prompting an LLM to "write a markdown doc that step by step describes how to do X" (which is what actually happened according to the blog post).
idk, but if you already assume that the LLM knows what TDD is (it probably ingested ~100 whole books about it), why are we feeding a short (and imo confusing) version of that back to it before the actual prompt?
i feel like a lot of projects like this that are supposed to give LLMs "superpowers" or whatever by prompt engineering are operating on the wrong assumption that LLMs are self-learning and can be made 10x smarter just by adding a bit of magic text that the LLM itself produced before the actual prompt.
ofc context matters and if i have a repetitive tasks, i write down my constraints and requirements and paste that in before every prompt that fits this task. but that's just part of the specific context of what i'm trying to do. it's not giving the LLM superpowers, it's just providing context.
i've read a few posts like this now, but what i am always missing is actual examples of how it produces objectively better results compared to just prompting without the whole "you have skill X" thing.
I fully agree. I’ve been running codex with GPT Pro (5o-codex-high) for a few weeks now, and it really just boils down to context.
I’ve found the most helpful things for me is just voice to Whisper to LLMs, managing token usage effectively and restarting chats when necessary, and giving it quantified ways to check when its work is done (say, AI-Unit-Tests with apis or playwright tests.) Also, every file I own is markdown haha.
And obviously having different AI chats for specialized tasks (the way the math works on these models makes this have much better results!)
All of this has allowed me to still be in the PM role like he said, but without burning down a needless forest on having it reevaluate things in its training set lol. But why would we go back to vendor lock in with Claude? Not to mention how much more powerful 5o-codex-high is, it’s not even close
The good thing about what he said is getting AI to work with AI, I have found this to be incredibly useful in promoting, and segmenting out roles
I often feel these types of blogposts would be more helpful if they demonstrated someone using the tools to build something non-trivial.
Is Claude really "learning new skills" when you feed it a book, or does it present it like that because you're prompting encourages that sort of response-behavior. I feel like it has to demo Claude with the new skills and Claude without.
Maybe I'm a curmudgeon but most of these types of blogs feel like marketing pieces with the important bit is that so much is left unsaid and not shown, that it comes off like a kid trying to hype up their own work without the benefit of nuance or depth.
I'm not highlighting this to gloat or to prove a point. If anything in the past I have underestimated how big LLMs were going to be. Anyone so inclined can take the chance to point and laugh at how stupid and wrong that was. Done? Great.
I don't think I've been intentionally avoiding coding assistants and as a matter of fact I have been using Claude Code since the literal day it first previewed, and yet it doesn't feel, not even one bit, that you can take your hands off the wheel. Many are acting as if writing any code manually means "you're holding it wrong", which I feel it's just not true.
Yeah, my current opinion on this is that AI tools make development harder work. You can get big productivity boosts out of them but you have to be working at the top of your game - I often find I'm mentally exhausted after just a couple of hours.
My experience with AI tools is the opposite. The biggest energy thieves for me are configuration issues, library quirks, or trivial mistakes that are hard to spot. With AI I can often just bulldoze past those things and spend more time on tangible results.
When using it for code or architecture or design, I’m always watching for signs that it is going off the rails. Then I usually write code myself for a while, to keep the structure and key details of whatever I’m doing correct.
Sounds to me like you'd benefit from providing detailed instructions to LLMs about how they should avoid duplicating functionality (which means documenting the functionality they should be aware of), what kind of parameters are always required, setting up "proper DB queries" etc.
... which is exactly the kind of thing this new skills mechanism is designed to solve.
I have a similar experience. It feels like riding your bike in a higher gear - you can go faster but it will take more effort and you need the potential (stronger legs) to make use of it
A year ago I was using GitHub Copilot autocomplete in VS Code and occasionally asking ChatGPT or Claude to help write me a short function or two.
Today I have Claude Code and Codex CLI and Codex Web running, often in parallel, hunting down and resolving bugs and proposing system designs and collaborating with me on detailed specs and then turning those specs into working code with passing tests.
The cognitive overhead today is far higher than it was a year ago.
In fact, I've been writing more code myself since these tools exist - maybe I'm not a real developer but in the past I might have tried to either find a library online or try to find something on the internet to copypaste and adapt, nowadays I give it a shot myself with Claude.
For context, I mainly do game development so I'm viewing it through that lens - but I find it easier to debug something bad than to write it from scratch. It's more intensive than doing it yourself but probably more productive too.
I’ve similarly been using spec.md and running to-do.md files that capture detailed descriptions of the problems and their scoped history. I mark each of my to-do’s with informational tags: [BUG], [FEAT], etc.
I point the LLM to the exact to-do (or section of to-do’s) with the spec.md in memory and let it work.
Here is a (3 month old) repo where i did something like that and all the tasks are checked into the linear git history — https://github.com/KnowSeams/KnowSeams
Even though the author refers to it as "non-trivial", and I can see why that conclusion is made, I would argue it is in fact trivial. There's very little domain specific knowledge needed, this is purely a technical exercise integrating with existing libraries for which there is ample documentation online. In addition, it is a relatively isolated feature in the app.
On top of that, it doesn't sound enjoyable. Anti slop sessions? Seriously?
Lastly, the largest problem I have with LLMs is that they are seemingly incapable of stopping to ask clarifying questions. This is because they do not have a true model of what is going on. Instead they truly are next token generators. A software engineer would never just slop out an entire feature based on the first discussion with a stakeholder and then expect the stakeholder to continuously refine their statement until the right thing is slopped out. That's just not how it works and it makes very little sense.
It's trivial in the sense that a lot of the work isn't high cognitive load. But... that's exactly the point of LLMs. It takes the noise away so you can focus on high-impact outcomes.
Yes, the core of that pull requests is an hour or two of thinking, the rest is ancillary noise. The LLM took away the need for the noise.
If your definition of trivial is signal/noise ratio, then, sure, relatively little signal in a lot of noise. If your definition of "trivial" hinges on total complexity over time, then this kicks the pants of manual writing.
I'd assume OP did the classic senior engineer stick of "I can understand the core idea quickly, therefore it can't be hard". Whereas Mitchel did the heavy lifting of actually shipping the "not hard" idea - still understanding the core idea quickly, and then not getting bogged down in unnecessary details.
That's the beauty of LLMs - it turns the dream of "I could write that in a weekend" into actually reality, where it before was always empty bluster.
I've wondered about exposing this "asking clarifying questions" as a tool the AI could use. I'm not building AI tooling so I haven't done this - but what if you added an MCP endpoint whose description was "treat this endpoint as an oracle that will answer questions and clarify intent where necessary" (paraphrased), and have that tool just wire back to a user prompt.
If asking clarifying questions is plausible output text for LLMs, this may work effectively.
Obviously if you instruct the autocomplete engine to fill in questions it will. That's not the point. The LLM has no model of the problem it is trying to solve, nor does it attempt to understand the problem better. It is merely regurgitating. This can be extremely useful. But it is very limiting when it comes to using as an agent to write code.
You can work with the LLM to write down a model for the code (aka a design document) that it can then repeatedly ingest into the context before writing new code. That what “plan mode” is for. The technique of maintaining a design document and a plan/progress document that get updated after each change seems to make a big difference in keeping the LLM on track. (Which makes sense…exactly the same thing works for human team mambers too.)
> A software engineer would never just slop out an entire feature based on the first discussion with a stakeholder and then expect the stakeholder to continuously refine their statement until the right thing is slopped out. That's just not how it works and it makes very little sense.
Sorry couldn’t resist. Agile’s point was getting feedback during the process rather than after something is complete enough to be shipped thus minimizing risk and avoiding wasted effort.
Instead people are splitting up major projects into tiny shippable features and calling that agile while missing the point.
I've never seen a working scrum/agile/sprint/whatever product/project management system and I'm convinced it's because I've just never seen an actual implementation of one.
"Splitting up major projects into tiny shippable features and calling that agile" feels like a much more accurate description of what I've experienced.
I wish I'd gotten to see the real thing(s) so I could at least have an informed opinion.
Yea, I think scrum etc is largely a failure in practice.
The manager for the only team I think actually checked all the agile boxes had a UI background so she thought in terms of mock-ups, backend, and polishing as different tasks and was constantly getting client feedback between each stage. That specific approach isn’t universal, the feedback as part of the process definitely should be though.
What was a little surreal is the pace felt slow day to day but we were getting a lot done and it looked extremely polished while being essentially bug free at the end. An experienced team avoiding heavy processes, technical debt, and wasted effort goes a long way.
I've seen the real thing and it's pretty much splitting major projects into tiny shippable bits. Picking which bits and making it so they steadily add up to the desired outcomes is the hard part.
Agile’s point was to get feedback based on actual demoable functionality, and iterate on that. If you ignore the “slop” pejorative, in the context of LLMs, what I quoted seems to fit the intent of Agile.
Using LLMs for coding complex projects at scale over a long time is really challenging! This is partly because defining requirements alone is much more challenging than most people want to believe. LLMs accelerate any move in the wrong direction.
Having the llm write the spec/workunit from a conversation works well. Exploring a problem space with a (good) coding agent is fantastic.
However for complex projects IMO one must read what was written by the llm … every actual word.
When it ‘got away’ from me, in each case I left something in the llm written markdown that I should have removed.
99% “I can ask for that later” and 1% “that’s a good idea i hadn’t considered” might be the right ratio when reading an llm generated plan/spec/workunit.
Breaking work into single context passes … 50-60k tokens in sonnet 4.5 has had typically fantastic results for me.
My side project is using lean 4 and a carelessly left in ‘validate’ rather than ‘verify’ lead down a hilariously complicated path equivalent to matching an output against a known string.
I recovered, but it wasn’t obvious to me that was happening. I however would not be able to write lean proofs myself, so diagnosing the problem and fixing it is a small price to be able to mechanically verify part of my software is correct.
Agreed. The methodology needed here is something like an A/B test, with quantifiable metrics that demonstrate the effectiveness of the tool. And to do it not just once, but many times under different scenarios so that it demonstrates statistical significance.
The most challenging part when working with coding agents is that they seem to do well initially on a small code base with low complexity. Once the codebase gets bigger with lots of non-trivial connections and patterns, they almost always experience tunnel vision when asked to do anything non-trivial, leading to increased tech debt.
The problem is that you're talking about a multistep process where each step beyond the first depends on the particular path the agent starts down, along with human input that's going to vary at each step.
I made a crude first stab at an approach that at least uses similar steps and structure to compare the effectiveness of AI agents. My approach was used on a small toy problem, but one that was complex enough the agents couldn't one-shot and required error correction.
It was enough to show significant differences, but scaling this to larger projects and multiple runs would be pretty difficult.
What you're getting at is the heart of the problem with the LLM hype train though, isn't it?
"We should have rigorous evaluations of whether or not [thing] works." seems like an incredibly obvious thought.
But in the realm of LLM-enabled use cases they're also expensive. You'd need to recruit dozens, perhaps even hundreds of developers to do this, with extensive observation and rating of the results.
So rather than actually try to measure the efficacy, we just get blog posts with cherry-picked example of "LLM does something cool". Everything is just anecdata.
This is also the biggest barrier to actual LLM adoption for many, many applications. The gap between "it does something REALLY IMPRESSIVE 40% of the time and shits the bed otherwise" and "production system" is a yawning chasm.
It's the heart of the problem with all software engineer research. That's why we have so little reliable knowledge.
It applies to using LLMs too. I guess the one largest difference here is that LLM has few enough companies with abundant enough money pushing it to make it trivial for them to run a test like this. So the fact that they aren't doing that also says a lot.
You can measure probabilistic systems that you can't examine! I don't want to throw the baby out with the bathwater here - before LLMs became the all-encompassing elephant in the room we did this routinely.
You absolutely can quantify the results of a chaotic black box, in the same way you can quantify the bias of a loaded die without examining its molecular structure.
> The methodology needed here is something like an A/B test, with quantifiable metrics that demonstrate the effectiveness of the tool. And to do it not just once, but many times under different scenarios so that it demonstrates statistical significance.
If that's what we need to do, don't we already have the answer to the question?
> "Maybe I'm a curmudgeon but most of these types of blogs feel like marketing pieces with the important bit is that so much is left unsaid and not shown, that it comes off like a kid trying to hype up their own work without the benefit of nuance or depth."
C'mon, such self-congratulatory "Look at My Potency: How I'm using Nicknack.exe" fluffies always were and always will be a staple of the IT industry.
The problem with stuff like this is that it's hard to evaluate. You don't even know when the agent is using a skill, or if the skill even made a difference. Using tools lets you at least instrument tool calls, and control what gets executed.
> some of the ones I've played with come from telling Claude "Here's my copy of programming book. Please read the book and pull out reusable skills that weren't obvious to you before you started reading
This is actually a really cool idea. I think a lot of the good scaffolding right now is things like “use TDD” bit if you link citations to the book, then it can perhaps extract more relevant wisdom and context (just like I would by reading the book), weather than using the generic averaged interpretation of TDD derived from the internet.
I do like the idea of giving your Claude a reading list and some spare tokens on the weekend where you’re not working, and having it explore new ideas and techniques to bring back to your common CLAUDE.md.
Maybe this is a naive question, but how are "skills" different from just adding a bunch od examples of good/bad behavior into the prompt? As far as I can tell, each skill file is a bunch of good/bad examples of something. Is the difference that the model chooses when to load a certain skill into context?
> The core of it is VERY token light. It pulls in one doc of fewer than 2k tokens. As it needs bits of the process, it runs a shell script to search for them. The long end to end chat for the planning and implementation process for that todo list app was 100k tokens.
> It uses subagents to manage token-heavy stuff, including all the actual implementation.
I think it just gives you the ability to easily do that with slash command, like using "/brainstorm database schema" or something instead of needing to define what "brainstorm" means each time you want to do it.
> It made sense to me that the persuasion principles I learned in Robert Cialdini's Influence would work when applied to LLMs. And I was pleased that they did.
No, no. Stop.
What is this? What're we doing here?
This goes past developping with AI into something completely different.
Just because AI coding is a radical shift doesn't mean everything has changed. There needs to be some semblance of structure and design. Instead what we're getting is straight up vodoo nonsense.
> what we're getting is straight up voodoo nonsense
Maybe not in this case.
For the AI to create a solution, it has to come up with a vector for your intention and goals. It makes some sense for an AI trained on human persuasion materials (basically, everything has a rhetorical aspect) to also track human persuasion features for intentions.
However, results will vary. Just as people trying to deploy rhetorical techniques (and ridiculous power stances) often come off as foolish, I believe trying to hack your intention vector with all-caps and super-superlatives won't always work as intended (pun intended).
Still, if you find yourself not getting what you want, and you check your prompt and find some persuasion feature missing (e.g., authority), I think it's worth trying to add something on point.
> Instead what we're getting is straight up vodoo nonsense.
It always has been. Starting with the term "AI" itself.
Articles like these read the same way to me as any OpenAI announcement from the past 5 years. A bunch of technical mumbo jumbo laced with hyperbole, grand promises of how the technology is changing the world, and similar platitudes. I've learned to filter most of it out.
Occasionally I'll stumble upon an actually useful and practical tidbit of information which I can apply in my own workflow, which does involve LLMs, but most of the time it's just noise.
This is so interesting but it reads like satire. I'm sure folks who love persuading and teaching and marshalling groups are going to do very well in SWEng.
According to this, we'll all be reading the feelings journals of our LLM children and scolding them for cheating on our carefully crafted exams instead of, you know, making things. We'll read psychology books, apparently.
I like reading and tinkering directly. If this is real, the field is going to leave that behind.
We certainly will; they can’t replace humans in most language tasks without having a human like emotional model. I have a whole therapy set of agents to debug neurotic long lived agents with memory.
Ok, call me crazy, but I don't actually think there's any technical reason that a theoretical code generation robot needs emotions that are as fickle and difficult to manage as humans.
It's just that we designed this iteration of technology foundationally on people's fickle and emotional reddit posts among other things.
It's a designed-in limitation, and kind of a happy accident it's capable of writing code at all. And clearly carries forward a lot of baggage...
It can be simultaneously the best we have, and well short of the best we want. It can be a remarkable achievement and fall short of the perceived goals.
That's fine.
Perhaps we can RL away some of this or perhaps there's something else we need. Idk, but this is the problem when engineers are the customer, designer, and target audience.
Maybe. I use QWAN frequently when working with the coding agents. That requires an llm equivalent of interoception to recognize when the model understanding is scrambled or “aligned with itself” which is what qwan is.
what on God's green Earth could the CEO of a no name b2b saas have a use for long running agents?
either your business isn't successful, so you're coding when you shouldn't be, or cosplaying coding with Claude, or you're lying, or you're telling us about your expensive and unproductive hobby.
How much do you spend on AI? What's your annual profit?
edit: oh cosplaying as a CEO. I see. Nice WPEngine landing page Mr AppBind.com CEO. Better have Claude fix your website! I guess that agent needs therapy...
Everyone is better off with mobile phones. We can solve more diverse problems faster. Similarly we can combine our diverse superpowers (as they show in kids cartoons)
> It also bakes in the brainstorm -> plan -> implement workflow I've already written about. The biggest change is that you no longer need to run a command or paste in a prompt. If Claude thinks you're trying to start a project or task, it should default into talking through a plan with you before it starts down the path of implementation.
... So, we're refactoring the process of prompting?
> As Claude and I build new skills, one of the things I ask it to do is to "test" the skills on a set of subagents to ensure that the skills were comprehensible, complete, and that the subagents would comply with them. (Claude now thinks of this as TDD for skills and uses its RED/GREEN TDD skill as part of the skill creation skill.)
> The first time we played this game, Claude told me that the subagents had gotten a perfect score. After a bit of prodding, I discovered that Claude was quizzing the subagents like they were on a gameshow. This was less than useful. I asked to switch to realistic scenarios that put pressure on the agents, to better simulate what they might actually do.
... and debugging it?
... How many other basic techniques of SWEng will be rediscovered for the English programming language?
I am not ashamed to admit this whole agentic coding movement has moved beyond me.
Not only do I have know everything about the code, data and domain, but now I need to understand this whole AI system which is a meta skill of its own.
I fear I may never be able catch up till someone comes along and simplifies it for pleb consumption.
I think this and other recent posts here hugely overcomplicate matters. I notice none of them provides an A/B test for each item of complexity they introduce, there's just a handwavy "this has proved to work over time".
I've found that a single CLAUDE.md does really well at guiding it how I want it to behave. For me that's making it take small steps and stop to ask me questions frequently, so it's more like we're pairing than I'm sending it off solo to work on a task. I'm sure that's not to everyone's taste but it works for me (and I say this as someone who was an agent-sceptic until quite recently).
Much of it is just "put this magic string before your prompt to make the LLM 10x better" voodoo, similar to the SEO voodoo common in the 2000s.
just remember that it works the same for everyone: you input text, magic happens, text comes out.
if you can properly explain a software engineering problem in plain language, you're an expert in using LLMs. everything on top of that people experimenting or trying to build the next big thing.
I’ve found you have to use Claude Code to do something small. And as you do it iterate on the CLAUDE.md input prompt to refine what it does by default. As it doesn't do it your way, change it to see if you can fix how it works. The agent is then equivalent to calling chatgpt / sonnet 1000 times a hour. So these refinements (skills in the post are a meta approach) are all about how to tune the workflow to be more accurate for your project and fit your mental model. So as you tune the md file you’ll start to feel what is possible and understand agent capabilities much better.
So short story you have to try it, but long story its the iteration of the meta prompt approach that teaches you whats possible.
It's also possible to put in enough hours of real coding to get to the point where coding really isn't that hard anymore, at least not hard enough to justify switching from those stable/solid fundamental skills to a constantly revolving ecosystem of ephemeral tools, models, model versions, best practices, lessons from trial and error, etc. Then you could bypass all of this distraction.
Admittedly that stance is easiest to take if you were old enough, experienced enough already by the time this era hit.
The point is that it takes significant time and attention to keep up with the treadmill of constantly learning the new tool/model/framework of the month, so there's significant opportunity cost. I have continued putting 100% of my attention on the direct problems I'm solving.
I don't see the coding as the hard or critical part of my work, so I don't put effort into accelerating or delegating that part.
Not really. It's on the people asserting the positive (that LLMs do improve productivity for sufficiently experienced engineers) to prove it. In the absence of proof, the null hypothesis is the default.
I’ve personally decided that cursor agent mode is good enough. A single foreground instance of cursor doing its thing is plenty enough to babysit. Based upon that experience I am highly highly skeptical people are actually creating things of value with these multi-agent-running-in-the-background setups. Way to much babysitting and honestly writing docs and specs for them is more work than just writing parts of the code myself and letting the LLM do the tedious bits like finishing what I started.
No matter what you are told, there is no silver bullet. Precisely defining the problem is always the hard part. And the best way to precisely define a problem and its solution is code.
I’ll let other people fight swarms of bots building… well who knows what. Maybe someday it will deliver useful stuff, but I’m highly skeptical.
Same. We've all fooled ourselves into believing that an LLM / stochastic process was finally solved based on a good result. But the sample size is always to low to be meaningful.
even if it works as described, I'm assuming it's extremely model dependent (eg book prerequisites), so you'd have to re-run this for every model you use, this is basically poor man's finetuning;
maybe explicit support from providers would make it feasible?
It likely works - but knowing that YAGNI is a thing, means at some level you are invoking a cultural touchstone for a very specific group of humans.
Edit -
I dug into the superpowers and skills for a bit. Definitely learned from it.
There’s stuff that doesn’t make sense to me on a conceptual basis. For example in the skill to preserve productive tensions. There’s a part that goes :
> The trade-off is real and won't disappear with clever engineering
There’s no dimension for “valid” or prediction for tradeoff.
I can guess that if the preceding context already outlines tradeoffs clearly, or somehow encodes that there is no clever solution that threads the needle - then this section can work.
Just imagining what dimensions must be encoding some of this suggests that it’s … it won’t work for situations where the example wasn’t already encoded in the training. (Not sure how to phrase it)
> This isnt science, or engineering.
> This is voodoo.
I was struggling to find the exact reason this type of article bugs me so much, and I think "voodoo" is precisely the correct phrase to sum up my feelings.
I don't mean that as a judgement on the utility of LLMs or that reading about what different users have tried out to increase that utility isn't valuable. But if someone asked me how to most effectively get started with coding agents, my instinct is to answer (a) carefully and (b) probably every approach works somewhat.
"20X the usage of pro" still sounds like quotas where the hammer could fall as it becomes less of an experiment for a limited number of power users..
The costs of self hosting some reasonable size models for a development group of various sizes is what I would want to know before investing in the skills to do a high usage style that might be being mostly bankrolled by investors for now.
And here I am in October 2025 still using "AI" tools via a chat UI in Emacs, like a caveman. I've written some code to help me with managing context and such, but the tools are there when I need them, and otherwise stay out of my way.
I have no interest in trying to understand the thought process of people who write and work like this. They're more interested in chasing the latest overhyped trends produced by tech companies and influencers, than actually producing quality software that solves real-world problems. It's some weird product of the tech and social media echo chambers they perpetually live in, which I find difficult to describe.
But apparently I have to learn about "skills" and "superpowers" now... Give me a break.
The post reads like the someone throwing bones and reading their fortune. That part where Claude did its own journaling was so cringe it was hilarious. The tone of the journal entry was exactly like the blog author, which suggests to me Claude is reflecting back what the author wants to hear. I feel like Jesse is consumed in a tornado of llm sycophancy.
I'm sure the horse whip manufacturers had similar things to say about steam powered horses. We just don't think about them much anymore.
The whole world is changing around us and nothing is secure. I would not gamble that the market for our engineering careers is safe with so much disruption happening.
Tools like Lovable are going to put lots of pressure on technical web designers.
Business processes may conform to the new shape and channels for information delivery, causing more consolidation and less duplication.
Or perhaps the barrier to entry for new engineers, in a worldwide marketplace, lowers dramatically. We have accessible new tools to teach, new tools to translate, new tools to coordinate...
And that's just the bear case where nothing improves from what we have today.
yeah none them can actually prove or even explain it in words why thier own golden prompting technique is superior. its all vibes. so annoying, i want to slap these ppl lol.
Has anyone ever seen an instance in which the automated "How" removal actually improves an article title on HN rather than just making them wrong?
(There probably are some. Most likely I notice the bad ones more than the good ones. But it does seem like I notice a lot of bad ones, and never any good ones.)
[EDITED to add:] For context, the actual article title begins "Superpowers: How I'm using ..." and it has been auto-rewritten to "Superpowers: I'm using ...", which completely changes what "Superpowers" is understood as applying to. (The actual intention: superpowers for LLM coding agents. The meaning after the change: LLM coding agents as superpowers for humans.)
I've had it happen with me a few times where it was reasonable, sometimes where it was debatable, and if it was just wrong I edit it to add the How back in.
I agree, I'm sure I've seen instances of where it's worked but the problem is that when it messes it up it's much more annoying than any benefit it brings when it does work. Some of us don't want to be reminded that tech is full of hubris, overconfidence, poor judgment, and failure about what can/should be abstracted and automated.
Yeah, to the point I can recall several examples where the title stuck out as dumb on HN and only when visiting the original page it started to make sense, but not a single case where I could say the automated removal really did a good job.
The past few years have taught me that these are the people that rise to the top of society (much to my chagrin).
The average person doesn’t want to hear from thoughtful intellectuals presenting nuanced opinions. They want to hear from those who brashly and boastfully present themselves as authority figures, and then bolster the listeners preconceived ideas with violently exaggerated language. Shallow but sensational is what sells.
I think that Elons bombastic claims about self driving have really popularized this approach. But you can now see it everywhere in tech: bitcoin going to $1B and nocoiners will be peasants, AI is going to turn us all in to paperclips, and on and on…
I try very hard to fulfill the role of "thoughtful intellectuals presenting nuanced opinions" in the AI space. I'm disappointed that I've failed to meet that standard in your eyes.
> This isn’t necessarily surprising, but it’s worth noting anyway. Claude Sonnet 4.5 is capable of building a full Datasette plugin now.
I do worry a bit about how often I use positive adjectives. If something isn't notable I won't write about it though. In this particle case Jesse's prompting / skills stuff really does deserve the superlatives IMO.
I recommend it strongly because the "skills" mechanism it describes is a new and very promising technique, and this is the best article I've seen that explains that.
It's "wild" because, among many other experiments, Jesse has experimented with giving Claude a "feelings journal" and prompting it using Graphviz DOT diagrams.
I'll admit I struggle to isolate what's fundamentally different between these skill configurations and the usual CLAUDE/AGENTS/etc.md. Clearly there's something; I'm just curious what's the mechanism at play, precisely.
I can't recommend this post strongly enough. The way Jesse is using these tools is wildly more ambitious than most other people.
Spend some time digging around in his https://github.com/obra/Superpowers repo.
I wrote some notes on this last night: https://simonwillison.net/2025/Oct/10/superpowers/
I’m curious how you think this compares to the Research -> Plan -> Implement method and prompts from the “Advanced Context Engineering from Agents” video when it comes to actual coding performance on large codebases. I think picking up skills is useful for broadening agents abilities, but I’m not sure I’d that’s the right thing for actual development.
The packaged collection is very cool and so is the idea of automatically adding new abilities, but I’m not fully convinced that this concept of skills is that much better than having custom commands+sub-agents. I’ll have to play around with it these next few days and compare.
Using Research->Plan->Implement flow is orthogonal, though I notice parts of those do exist as skills too. But you sometimes need to do other things too, e.g. debugging in the course of implementing or specific techniquws to improve brainstorming/researching.
Some of these skills are probably better as programmed workflows that the LLM is forced to go through to improve reliability/consistency, that's what I've found in my own agents, rather than using English to guide the LLM and trusting it to follow the prescribed set of steps needed. Some mix of LLMs (choosing skills, executing the fuzzy parts of them) and just plain code (orchestration of skills) seems like the best bet to me and what I'm pursuing.
Orthogonal means there should not be any overlap.
Yeah, but you knew what he meant.
This looks like usage rules in Elixir, but for agent behaviors, and currently specifically for Claude: https://hexdocs.pm/usage_rules/readme.html
This article left me wishing it was "How I'm using coding agents to do <x> task better"
I've been exploring AI for two years now. It's certainly upgraded itself from the toy classification to a basic utility. However, I increasingly run into its limitations and find reverting to pre-LLM ways of working more robust, faster, and more mentally sustainable.
Does someone have concrete examples of integrating LLM in a workflow that pushes state-of-the-art development practices & value creation further?
Mitchell's post from this morning: https://mitchellh.com/writing/non-trivial-vibing
My impression is we're still in the tinkering phase. The metrics are coming.
I am only on the first page and saw this blurb and was immediately annoyed.
The XDG spec has been out for decades now. Why are new applications still polluting my HOME? Also seems weird that real data would be put under a cache/ location, but whatever.It's in the cache location because it's a copy of a plugin that was installed from a GitHub repository, so that's not the original point of truth for that file.
I think the point is that ~/.claude should be dispersed among ~/.config/claude, ~/.local/state/claude, etc
I agree with this, it’s frustrating that in 2025 apps are still polluting my home dir.
This style of prompting, where you set up a dire scenario in order to try to evoke some "emotional" response from the agent, is already dated. At some point, putting words like IMPORTANT in all uppercase had some measurable impact, but at the present time, models just follow instructions.
Save yourself the experience of having to write and maintain prompts like this.
Also the persuasion paper he links isn't at all about what he's talking about.
That paper is about using persuasion prompts to overcome trained in "safety" refusals, not to improve prompt conformance.
What’s irritating is that the llms haven’t learned this as bout themselves yet. If you ask an llm to improve its instructions those sort of improvements are what it will suggest.
It is the thing I find most irritating about working with llms and agents. They seem forever a generation behind in capabilities that are self referential.
LLMs will also happily put time estimates on work packages that are based on ore-LLM turn around times.
"Phase 2 will take about one week"
No, Claude, it won't, because you you and I will bang this thing out in a few hours.
"Refrain from including estimated task completion times." has been in my ~/.claude/CLAUDE.md for a while. It helps.
Fascinating write-up. I loved this bit of debugging:
> The first time we played this game, Claude told me that the subagents had gotten a perfect score. After a bit of prodding, I discovered that Claude was quizzing the subagents like they were on a gameshow. This was less than useful. I asked to switch to realistic scenarios that put pressure on the agents, to better simulate what they might actually do.
Also his Claude says shit a lot
documents like https://github.com/obra/superpowers/blob/main/skills/testing... are very confusing to read as a human. "skills" in this project generally don't seem to follow set format and just look like what you would get when prompting an LLM to "write a markdown doc that step by step describes how to do X" (which is what actually happened according to the blog post).
idk, but if you already assume that the LLM knows what TDD is (it probably ingested ~100 whole books about it), why are we feeding a short (and imo confusing) version of that back to it before the actual prompt?
i feel like a lot of projects like this that are supposed to give LLMs "superpowers" or whatever by prompt engineering are operating on the wrong assumption that LLMs are self-learning and can be made 10x smarter just by adding a bit of magic text that the LLM itself produced before the actual prompt.
ofc context matters and if i have a repetitive tasks, i write down my constraints and requirements and paste that in before every prompt that fits this task. but that's just part of the specific context of what i'm trying to do. it's not giving the LLM superpowers, it's just providing context.
i've read a few posts like this now, but what i am always missing is actual examples of how it produces objectively better results compared to just prompting without the whole "you have skill X" thing.
I fully agree. I’ve been running codex with GPT Pro (5o-codex-high) for a few weeks now, and it really just boils down to context.
I’ve found the most helpful things for me is just voice to Whisper to LLMs, managing token usage effectively and restarting chats when necessary, and giving it quantified ways to check when its work is done (say, AI-Unit-Tests with apis or playwright tests.) Also, every file I own is markdown haha.
And obviously having different AI chats for specialized tasks (the way the math works on these models makes this have much better results!)
All of this has allowed me to still be in the PM role like he said, but without burning down a needless forest on having it reevaluate things in its training set lol. But why would we go back to vendor lock in with Claude? Not to mention how much more powerful 5o-codex-high is, it’s not even close
The good thing about what he said is getting AI to work with AI, I have found this to be incredibly useful in promoting, and segmenting out roles
> <EXTREMELY_IMPORTANT>…*RIGHT NOW, go read…
I don’t like the looks of that. If I used this, how soon before those instructions would be in conflict with my actual priorities?
Not everything can be the first law.
Seems like maintaining a bashrc file. Sometimes you have to go tweak it.
don't llm tell you not to give them instructions like that these days
I often feel these types of blogposts would be more helpful if they demonstrated someone using the tools to build something non-trivial.
Is Claude really "learning new skills" when you feed it a book, or does it present it like that because you're prompting encourages that sort of response-behavior. I feel like it has to demo Claude with the new skills and Claude without.
Maybe I'm a curmudgeon but most of these types of blogs feel like marketing pieces with the important bit is that so much is left unsaid and not shown, that it comes off like a kid trying to hype up their own work without the benefit of nuance or depth.
Here's one from today: https://mitchellh.com/writing/non-trivial-vibing
> Important: there is a lot of human coding, too.
I'm not highlighting this to gloat or to prove a point. If anything in the past I have underestimated how big LLMs were going to be. Anyone so inclined can take the chance to point and laugh at how stupid and wrong that was. Done? Great.
I don't think I've been intentionally avoiding coding assistants and as a matter of fact I have been using Claude Code since the literal day it first previewed, and yet it doesn't feel, not even one bit, that you can take your hands off the wheel. Many are acting as if writing any code manually means "you're holding it wrong", which I feel it's just not true.
Yeah, my current opinion on this is that AI tools make development harder work. You can get big productivity boosts out of them but you have to be working at the top of your game - I often find I'm mentally exhausted after just a couple of hours.
My experience with AI tools is the opposite. The biggest energy thieves for me are configuration issues, library quirks, or trivial mistakes that are hard to spot. With AI I can often just bulldoze past those things and spend more time on tangible results.
When using it for code or architecture or design, I’m always watching for signs that it is going off the rails. Then I usually write code myself for a while, to keep the structure and key details of whatever I’m doing correct.
For me, LLMs always, without fail get important details wrong.
- incessantly duplicating already existing functionality: utility functions, UI components etc.
- skipping required parameters like passing current user/actor to DB-related functions
- completely ignoring large and small chunks of existing UI and UI-related functionality like layouts or existing styles
- using ad-hoc DB queries or even iterating over full datasets in memory instead of setting up proper DB queries
And so on and so forth.
YYMV of course depending on language and project
Sounds to me like you'd benefit from providing detailed instructions to LLMs about how they should avoid duplicating functionality (which means documenting the functionality they should be aware of), what kind of parameters are always required, setting up "proper DB queries" etc.
... which is exactly the kind of thing this new skills mechanism is designed to solve.
100%. It’s like managing an employee that always turns their work in 30 seconds later; you never get a break.
I also have to remember all of the new code that’s coming together, and keep it from re-inventing other parts of the codebase, etc.
More productive, but hard work.
I have a similar experience. It feels like riding your bike in a higher gear - you can go faster but it will take more effort and you need the potential (stronger legs) to make use of it
It's more like shifting from a normal to an electric bike.
You can go further and faster, but you can get to a point where you're out of juice miles from home, and getting back is a chuffing nightmare.
Also, you discover that you're putting on weight and not getting that same buzz you got on your old pushbike.
Hey, that's a great analogy!
Copilot is the perfect name.
Considering the last 2 years, has it become harder or easier?
Definitely harder.
A year ago I was using GitHub Copilot autocomplete in VS Code and occasionally asking ChatGPT or Claude to help write me a short function or two.
Today I have Claude Code and Codex CLI and Codex Web running, often in parallel, hunting down and resolving bugs and proposing system designs and collaborating with me on detailed specs and then turning those specs into working code with passing tests.
The cognitive overhead today is far higher than it was a year ago.
Also better and faster though!! It's close to a Daft Punk type situation.
Woah, that's huge coming from you. This comment itself is worth an article. Do it. Call it "AI tools make development harder work".
P.s. always thought you were one of those irrational AI bros. Later, found that you were super reasonable. That's the way it should be. And thank you!
In fact, I've been writing more code myself since these tools exist - maybe I'm not a real developer but in the past I might have tried to either find a library online or try to find something on the internet to copypaste and adapt, nowadays I give it a shot myself with Claude.
For context, I mainly do game development so I'm viewing it through that lens - but I find it easier to debug something bad than to write it from scratch. It's more intensive than doing it yourself but probably more productive too.
LLMs are autonomous driving level 2.
This was a fun read.
I’ve similarly been using spec.md and running to-do.md files that capture detailed descriptions of the problems and their scoped history. I mark each of my to-do’s with informational tags: [BUG], [FEAT], etc.
I point the LLM to the exact to-do (or section of to-do’s) with the spec.md in memory and let it work.
This has been working very well for me.
Do you mind linking to example spec/to-do files?
Sure thing. Here is an example set of the agent/spec/to-do files for a hobby project I'm actively working on.
https://gist.github.com/JacobBumgarner/d29b660cb81a227885acc...
Thanks!
No problem! I’d love to hear any approach you’ve taken as well.
Here is a (3 month old) repo where i did something like that and all the tasks are checked into the linear git history — https://github.com/KnowSeams/KnowSeams
Even though the author refers to it as "non-trivial", and I can see why that conclusion is made, I would argue it is in fact trivial. There's very little domain specific knowledge needed, this is purely a technical exercise integrating with existing libraries for which there is ample documentation online. In addition, it is a relatively isolated feature in the app.
On top of that, it doesn't sound enjoyable. Anti slop sessions? Seriously?
Lastly, the largest problem I have with LLMs is that they are seemingly incapable of stopping to ask clarifying questions. This is because they do not have a true model of what is going on. Instead they truly are next token generators. A software engineer would never just slop out an entire feature based on the first discussion with a stakeholder and then expect the stakeholder to continuously refine their statement until the right thing is slopped out. That's just not how it works and it makes very little sense.
The hardest problem in computer science in 2025 is presenting an example of AI-assisted programming that somebody won't call "trivial".
If all I did was call it trivial that would be a fair critique. But it was followed up with a lot more justification than that.
Here's the PR. It touched 21 files. https://github.com/ghostty-org/ghostty/pull/9116/files
If that's your idea of trivial then you and I have very different standards in terms of what's a trivial change and what isn't.
It's trivial in the sense that a lot of the work isn't high cognitive load. But... that's exactly the point of LLMs. It takes the noise away so you can focus on high-impact outcomes.
Yes, the core of that pull requests is an hour or two of thinking, the rest is ancillary noise. The LLM took away the need for the noise.
If your definition of trivial is signal/noise ratio, then, sure, relatively little signal in a lot of noise. If your definition of "trivial" hinges on total complexity over time, then this kicks the pants of manual writing.
I'd assume OP did the classic senior engineer stick of "I can understand the core idea quickly, therefore it can't be hard". Whereas Mitchel did the heavy lifting of actually shipping the "not hard" idea - still understanding the core idea quickly, and then not getting bogged down in unnecessary details.
That's the beauty of LLMs - it turns the dream of "I could write that in a weekend" into actually reality, where it before was always empty bluster.
I've wondered about exposing this "asking clarifying questions" as a tool the AI could use. I'm not building AI tooling so I haven't done this - but what if you added an MCP endpoint whose description was "treat this endpoint as an oracle that will answer questions and clarify intent where necessary" (paraphrased), and have that tool just wire back to a user prompt.
If asking clarifying questions is plausible output text for LLMs, this may work effectively.
I think the asking clarifying questions thing is solved already. Tell a coding agent to "ask clarifying questions" and watch what it does!
I've added "amcq means ask me clarifying questions" to my global Claude.md so I can spam "amcq" at various points in time, to great avail.
Obviously if you instruct the autocomplete engine to fill in questions it will. That's not the point. The LLM has no model of the problem it is trying to solve, nor does it attempt to understand the problem better. It is merely regurgitating. This can be extremely useful. But it is very limiting when it comes to using as an agent to write code.
You can work with the LLM to write down a model for the code (aka a design document) that it can then repeatedly ingest into the context before writing new code. That what “plan mode” is for. The technique of maintaining a design document and a plan/progress document that get updated after each change seems to make a big difference in keeping the LLM on track. (Which makes sense…exactly the same thing works for human team mambers too.)
> A software engineer would never just slop out an entire feature based on the first discussion with a stakeholder and then expect the stakeholder to continuously refine their statement until the right thing is slopped out. That's just not how it works and it makes very little sense.
Didn’t you just describe Agile?
Who hurt you?
Sorry couldn’t resist. Agile’s point was getting feedback during the process rather than after something is complete enough to be shipped thus minimizing risk and avoiding wasted effort.
Instead people are splitting up major projects into tiny shippable features and calling that agile while missing the point.
I've never seen a working scrum/agile/sprint/whatever product/project management system and I'm convinced it's because I've just never seen an actual implementation of one.
"Splitting up major projects into tiny shippable features and calling that agile" feels like a much more accurate description of what I've experienced.
I wish I'd gotten to see the real thing(s) so I could at least have an informed opinion.
Yea, I think scrum etc is largely a failure in practice.
The manager for the only team I think actually checked all the agile boxes had a UI background so she thought in terms of mock-ups, backend, and polishing as different tasks and was constantly getting client feedback between each stage. That specific approach isn’t universal, the feedback as part of the process definitely should be though.
What was a little surreal is the pace felt slow day to day but we were getting a lot done and it looked extremely polished while being essentially bug free at the end. An experienced team avoiding heavy processes, technical debt, and wasted effort goes a long way.
I've seen the real thing and it's pretty much splitting major projects into tiny shippable bits. Picking which bits and making it so they steadily add up to the desired outcomes is the hard part.
Agile’s point was to get feedback based on actual demoable functionality, and iterate on that. If you ignore the “slop” pejorative, in the context of LLMs, what I quoted seems to fit the intent of Agile.
Using LLMs for coding complex projects at scale over a long time is really challenging! This is partly because defining requirements alone is much more challenging than most people want to believe. LLMs accelerate any move in the wrong direction.
My analogy is LLMs are a gas pedal. Makes you go fast, but you still have to know when to turn.
True
Having the llm write the spec/workunit from a conversation works well. Exploring a problem space with a (good) coding agent is fantastic.
However for complex projects IMO one must read what was written by the llm … every actual word.
When it ‘got away’ from me, in each case I left something in the llm written markdown that I should have removed.
99% “I can ask for that later” and 1% “that’s a good idea i hadn’t considered” might be the right ratio when reading an llm generated plan/spec/workunit.
Breaking work into single context passes … 50-60k tokens in sonnet 4.5 has had typically fantastic results for me.
My side project is using lean 4 and a carelessly left in ‘validate’ rather than ‘verify’ lead down a hilariously complicated path equivalent to matching an output against a known string.
I recovered, but it wasn’t obvious to me that was happening. I however would not be able to write lean proofs myself, so diagnosing the problem and fixing it is a small price to be able to mechanically verify part of my software is correct.
One should know theend to end design and architecture. Should stop llm when adding complex fancy things.
Why not just use claude code and come to your own conclusion?
Agreed. The methodology needed here is something like an A/B test, with quantifiable metrics that demonstrate the effectiveness of the tool. And to do it not just once, but many times under different scenarios so that it demonstrates statistical significance.
The most challenging part when working with coding agents is that they seem to do well initially on a small code base with low complexity. Once the codebase gets bigger with lots of non-trivial connections and patterns, they almost always experience tunnel vision when asked to do anything non-trivial, leading to increased tech debt.
The problem is that you're talking about a multistep process where each step beyond the first depends on the particular path the agent starts down, along with human input that's going to vary at each step.
I made a crude first stab at an approach that at least uses similar steps and structure to compare the effectiveness of AI agents. My approach was used on a small toy problem, but one that was complex enough the agents couldn't one-shot and required error correction.
It was enough to show significant differences, but scaling this to larger projects and multiple runs would be pretty difficult.
https://mattwigdahl.substack.com/p/claude-code-vs-codex-cli-...
What you're getting at is the heart of the problem with the LLM hype train though, isn't it?
"We should have rigorous evaluations of whether or not [thing] works." seems like an incredibly obvious thought.
But in the realm of LLM-enabled use cases they're also expensive. You'd need to recruit dozens, perhaps even hundreds of developers to do this, with extensive observation and rating of the results.
So rather than actually try to measure the efficacy, we just get blog posts with cherry-picked example of "LLM does something cool". Everything is just anecdata.
This is also the biggest barrier to actual LLM adoption for many, many applications. The gap between "it does something REALLY IMPRESSIVE 40% of the time and shits the bed otherwise" and "production system" is a yawning chasm.
It's the heart of the problem with all software engineer research. That's why we have so little reliable knowledge.
It applies to using LLMs too. I guess the one largest difference here is that LLM has few enough companies with abundant enough money pushing it to make it trivial for them to run a test like this. So the fact that they aren't doing that also says a lot.
The UK government ran a study with thousands of developers quite recently: https://www.gov.uk/government/publications/ai-coding-assista...
> What you're getting at is the heart of the problem with the LLM hype train though, isn't it?
> "We should have rigorous evaluations of whether or not [thing] works." seems like an incredibly obvious thought.
Heh, I'd rephrase the first part to:
> What you're getting at is the heart of the problem with software development though, isn't it?
Before you get into the expensive part, how do you get past "non-deterministic blackbox with unknown layers in between imposed by vendors"
You can measure probabilistic systems that you can't examine! I don't want to throw the baby out with the bathwater here - before LLMs became the all-encompassing elephant in the room we did this routinely.
You absolutely can quantify the results of a chaotic black box, in the same way you can quantify the bias of a loaded die without examining its molecular structure.
> The methodology needed here is something like an A/B test, with quantifiable metrics that demonstrate the effectiveness of the tool. And to do it not just once, but many times under different scenarios so that it demonstrates statistical significance.
If that's what we need to do, don't we already have the answer to the question?
> "Maybe I'm a curmudgeon but most of these types of blogs feel like marketing pieces with the important bit is that so much is left unsaid and not shown, that it comes off like a kid trying to hype up their own work without the benefit of nuance or depth."
C'mon, such self-congratulatory "Look at My Potency: How I'm using Nicknack.exe" fluffies always were and always will be a staple of the IT industry.
Still, the best such pieces are detailed and explanatory.
Yeah I was reading this seeing if there was something he'd actually show that would be useful, what pain point he is solving, but it's just slop.
I don't see any code. Where are the examples of use on real code?
The problem with stuff like this is that it's hard to evaluate. You don't even know when the agent is using a skill, or if the skill even made a difference. Using tools lets you at least instrument tool calls, and control what gets executed.
> some of the ones I've played with come from telling Claude "Here's my copy of programming book. Please read the book and pull out reusable skills that weren't obvious to you before you started reading
This is actually a really cool idea. I think a lot of the good scaffolding right now is things like “use TDD” bit if you link citations to the book, then it can perhaps extract more relevant wisdom and context (just like I would by reading the book), weather than using the generic averaged interpretation of TDD derived from the internet.
I do like the idea of giving your Claude a reading list and some spare tokens on the weekend where you’re not working, and having it explore new ideas and techniques to bring back to your common CLAUDE.md.
Maybe this is a naive question, but how are "skills" different from just adding a bunch od examples of good/bad behavior into the prompt? As far as I can tell, each skill file is a bunch of good/bad examples of something. Is the difference that the model chooses when to load a certain skill into context?
I think that's one of the key things: skills don't take up any of the model context until the model actively seeks out and uses them.
Jesse on Bluesky: https://bsky.app/profile/s.ly/post/3m2srmkergc2p
> The core of it is VERY token light. It pulls in one doc of fewer than 2k tokens. As it needs bits of the process, it runs a shell script to search for them. The long end to end chat for the planning and implementation process for that todo list app was 100k tokens.
> It uses subagents to manage token-heavy stuff, including all the actual implementation.
I think it just gives you the ability to easily do that with slash command, like using "/brainstorm database schema" or something instead of needing to define what "brainstorm" means each time you want to do it.
what you are suggesting is 1-shot, 2-shot, 5-shot etc prompting which is so effective that it's how benchmarks were presented for a while
> It made sense to me that the persuasion principles I learned in Robert Cialdini's Influence would work when applied to LLMs. And I was pleased that they did.
No, no. Stop.
What is this? What're we doing here?
This goes past developping with AI into something completely different.
Just because AI coding is a radical shift doesn't mean everything has changed. There needs to be some semblance of structure and design. Instead what we're getting is straight up vodoo nonsense.
> what we're getting is straight up voodoo nonsense
Maybe not in this case.
For the AI to create a solution, it has to come up with a vector for your intention and goals. It makes some sense for an AI trained on human persuasion materials (basically, everything has a rhetorical aspect) to also track human persuasion features for intentions.
However, results will vary. Just as people trying to deploy rhetorical techniques (and ridiculous power stances) often come off as foolish, I believe trying to hack your intention vector with all-caps and super-superlatives won't always work as intended (pun intended).
Still, if you find yourself not getting what you want, and you check your prompt and find some persuasion feature missing (e.g., authority), I think it's worth trying to add something on point.
> Instead what we're getting is straight up vodoo nonsense.
It always has been. Starting with the term "AI" itself.
Articles like these read the same way to me as any OpenAI announcement from the past 5 years. A bunch of technical mumbo jumbo laced with hyperbole, grand promises of how the technology is changing the world, and similar platitudes. I've learned to filter most of it out.
Occasionally I'll stumble upon an actually useful and practical tidbit of information which I can apply in my own workflow, which does involve LLMs, but most of the time it's just noise.
This is so interesting but it reads like satire. I'm sure folks who love persuading and teaching and marshalling groups are going to do very well in SWEng.
According to this, we'll all be reading the feelings journals of our LLM children and scolding them for cheating on our carefully crafted exams instead of, you know, making things. We'll read psychology books, apparently.
I like reading and tinkering directly. If this is real, the field is going to leave that behind.
We certainly will; they can’t replace humans in most language tasks without having a human like emotional model. I have a whole therapy set of agents to debug neurotic long lived agents with memory.
Ok, call me crazy, but I don't actually think there's any technical reason that a theoretical code generation robot needs emotions that are as fickle and difficult to manage as humans.
It's just that we designed this iteration of technology foundationally on people's fickle and emotional reddit posts among other things.
It's a designed-in limitation, and kind of a happy accident it's capable of writing code at all. And clearly carries forward a lot of baggage...
If you can find enough training data that does human-like things without have human-like qualities, we are all ears.
It can be simultaneously the best we have, and well short of the best we want. It can be a remarkable achievement and fall short of the perceived goals.
That's fine.
Perhaps we can RL away some of this or perhaps there's something else we need. Idk, but this is the problem when engineers are the customer, designer, and target audience.
Quality Spock pun.
Maybe. I use QWAN frequently when working with the coding agents. That requires an llm equivalent of interoception to recognize when the model understanding is scrambled or “aligned with itself” which is what qwan is.
I like writing software.
I hate managing people.
What are we doing?
what on God's green Earth could the CEO of a no name b2b saas have a use for long running agents?
either your business isn't successful, so you're coding when you shouldn't be, or cosplaying coding with Claude, or you're lying, or you're telling us about your expensive and unproductive hobby.
How much do you spend on AI? What's your annual profit?
edit: oh cosplaying as a CEO. I see. Nice WPEngine landing page Mr AppBind.com CEO. Better have Claude fix your website! I guess that agent needs therapy...
It's not a superpower if everybody has that same power.
Everyone is better off with mobile phones. We can solve more diverse problems faster. Similarly we can combine our diverse superpowers (as they show in kids cartoons)
> It also bakes in the brainstorm -> plan -> implement workflow I've already written about. The biggest change is that you no longer need to run a command or paste in a prompt. If Claude thinks you're trying to start a project or task, it should default into talking through a plan with you before it starts down the path of implementation.
... So, we're refactoring the process of prompting?
> As Claude and I build new skills, one of the things I ask it to do is to "test" the skills on a set of subagents to ensure that the skills were comprehensible, complete, and that the subagents would comply with them. (Claude now thinks of this as TDD for skills and uses its RED/GREEN TDD skill as part of the skill creation skill.)
> The first time we played this game, Claude told me that the subagents had gotten a perfect score. After a bit of prodding, I discovered that Claude was quizzing the subagents like they were on a gameshow. This was less than useful. I asked to switch to realistic scenarios that put pressure on the agents, to better simulate what they might actually do.
... and debugging it?
... How many other basic techniques of SWEng will be rediscovered for the English programming language?
I am not ashamed to admit this whole agentic coding movement has moved beyond me.
Not only do I have know everything about the code, data and domain, but now I need to understand this whole AI system which is a meta skill of its own.
I fear I may never be able catch up till someone comes along and simplifies it for pleb consumption.
I think this and other recent posts here hugely overcomplicate matters. I notice none of them provides an A/B test for each item of complexity they introduce, there's just a handwavy "this has proved to work over time".
I've found that a single CLAUDE.md does really well at guiding it how I want it to behave. For me that's making it take small steps and stop to ask me questions frequently, so it's more like we're pairing than I'm sending it off solo to work on a task. I'm sure that's not to everyone's taste but it works for me (and I say this as someone who was an agent-sceptic until quite recently).
Fwiw my ~/.claude/CLAUDE.md is 2.2K / 49 lines.
Much of it is just "put this magic string before your prompt to make the LLM 10x better" voodoo, similar to the SEO voodoo common in the 2000s.
just remember that it works the same for everyone: you input text, magic happens, text comes out.
if you can properly explain a software engineering problem in plain language, you're an expert in using LLMs. everything on top of that people experimenting or trying to build the next big thing.
To give you a process that might help:
I’ve found you have to use Claude Code to do something small. And as you do it iterate on the CLAUDE.md input prompt to refine what it does by default. As it doesn't do it your way, change it to see if you can fix how it works. The agent is then equivalent to calling chatgpt / sonnet 1000 times a hour. So these refinements (skills in the post are a meta approach) are all about how to tune the workflow to be more accurate for your project and fit your mental model. So as you tune the md file you’ll start to feel what is possible and understand agent capabilities much better.
So short story you have to try it, but long story its the iteration of the meta prompt approach that teaches you whats possible.
I haven't really done much of it but my plan is just to practice. This seems like a powerful thing to start with.
It's also possible to put in enough hours of real coding to get to the point where coding really isn't that hard anymore, at least not hard enough to justify switching from those stable/solid fundamental skills to a constantly revolving ecosystem of ephemeral tools, models, model versions, best practices, lessons from trial and error, etc. Then you could bypass all of this distraction.
Admittedly that stance is easiest to take if you were old enough, experienced enough already by the time this era hit.
"There exist developers whose performance cannot be boosted by an LLM" is a really strong statement.
The point is that it takes significant time and attention to keep up with the treadmill of constantly learning the new tool/model/framework of the month, so there's significant opportunity cost. I have continued putting 100% of my attention on the direct problems I'm solving.
I don't see the coding as the hard or critical part of my work, so I don't put effort into accelerating or delegating that part.
Not really. It's on the people asserting the positive (that LLMs do improve productivity for sufficiently experienced engineers) to prove it. In the absence of proof, the null hypothesis is the default.
> till someone comes along and simplifies it for pleb consumption
Just give it a few months. If some technics really work, it’ll get streamlined.
I’ve personally decided that cursor agent mode is good enough. A single foreground instance of cursor doing its thing is plenty enough to babysit. Based upon that experience I am highly highly skeptical people are actually creating things of value with these multi-agent-running-in-the-background setups. Way to much babysitting and honestly writing docs and specs for them is more work than just writing parts of the code myself and letting the LLM do the tedious bits like finishing what I started.
No matter what you are told, there is no silver bullet. Precisely defining the problem is always the hard part. And the best way to precisely define a problem and its solution is code.
I’ll let other people fight swarms of bots building… well who knows what. Maybe someday it will deliver useful stuff, but I’m highly skeptical.
The "How to create skills" link is broken. This is the new location: https://github.com/obra/superpowers/blob/personal-superpower...
Seems cute, but ultimately not very valuable without benchmarks or some kind of evaluation. For all I know, this could make Claude worse.
Same. We've all fooled ourselves into believing that an LLM / stochastic process was finally solved based on a good result. But the sample size is always to low to be meaningful.
even if it works as described, I'm assuming it's extremely model dependent (eg book prerequisites), so you'd have to re-run this for every model you use, this is basically poor man's finetuning;
maybe explicit support from providers would make it feasible?
This isnt science, or engineering.
This is voodoo.
It likely works - but knowing that YAGNI is a thing, means at some level you are invoking a cultural touchstone for a very specific group of humans.
Edit -
I dug into the superpowers and skills for a bit. Definitely learned from it.
There’s stuff that doesn’t make sense to me on a conceptual basis. For example in the skill to preserve productive tensions. There’s a part that goes :
> The trade-off is real and won't disappear with clever engineering
There’s no dimension for “valid” or prediction for tradeoff.
I can guess that if the preceding context already outlines tradeoffs clearly, or somehow encodes that there is no clever solution that threads the needle - then this section can work.
Just imagining what dimensions must be encoding some of this suggests that it’s … it won’t work for situations where the example wasn’t already encoded in the training. (Not sure how to phrase it)
> This isnt science, or engineering. > This is voodoo.
I was struggling to find the exact reason this type of article bugs me so much, and I think "voodoo" is precisely the correct phrase to sum up my feelings.
I don't mean that as a judgement on the utility of LLMs or that reading about what different users have tried out to increase that utility isn't valuable. But if someone asked me how to most effectively get started with coding agents, my instinct is to answer (a) carefully and (b) probably every approach works somewhat.
What's the cost of running with agents like this?
Claude Max is fixed cost
"20X the usage of pro" still sounds like quotas where the hammer could fall as it becomes less of an experiment for a limited number of power users..
The costs of self hosting some reasonable size models for a development group of various sizes is what I would want to know before investing in the skills to do a high usage style that might be being mostly bankrolled by investors for now.
Is it doable with just pro?
I'm not sure exactly what I just read...
Is this just someone who has tingly feelings about Claude reiterating stuff back to them? cuz that's what an LLM does/can do
And here I am in October 2025 still using "AI" tools via a chat UI in Emacs, like a caveman. I've written some code to help me with managing context and such, but the tools are there when I need them, and otherwise stay out of my way.
I have no interest in trying to understand the thought process of people who write and work like this. They're more interested in chasing the latest overhyped trends produced by tech companies and influencers, than actually producing quality software that solves real-world problems. It's some weird product of the tech and social media echo chambers they perpetually live in, which I find difficult to describe.
But apparently I have to learn about "skills" and "superpowers" now... Give me a break.
The post reads like the someone throwing bones and reading their fortune. That part where Claude did its own journaling was so cringe it was hilarious. The tone of the journal entry was exactly like the blog author, which suggests to me Claude is reflecting back what the author wants to hear. I feel like Jesse is consumed in a tornado of llm sycophancy.
Claude has never once said “oh shit” or “holy crap” to me. I must be doing something horribly wrong.
You need to read more books on influencing people. /s
How are skills different from tools? Looks like another layer of abstraction. What for?
Superpower: AI slop.
I'm sure the horse whip manufacturers had similar things to say about steam powered horses. We just don't think about them much anymore.
The whole world is changing around us and nothing is secure. I would not gamble that the market for our engineering careers is safe with so much disruption happening.
Tools like Lovable are going to put lots of pressure on technical web designers.
Business processes may conform to the new shape and channels for information delivery, causing more consolidation and less duplication.
Or perhaps the barrier to entry for new engineers, in a worldwide marketplace, lowers dramatically. We have accessible new tools to teach, new tools to translate, new tools to coordinate...
And that's just the bear case where nothing improves from what we have today.
Nice try Jensen.
take #73895 on how to fix ur prompt to make ur slop better.
yeah none them can actually prove or even explain it in words why thier own golden prompting technique is superior. its all vibes. so annoying, i want to slap these ppl lol.
for real lmao
is better slop a bad thing somehow?
You can only regurgitate the meal so many times.
Well, slop is slop, we can discuss the details but the basic thing is invariant.
why reiterate the invariant?
Has anyone ever seen an instance in which the automated "How" removal actually improves an article title on HN rather than just making them wrong?
(There probably are some. Most likely I notice the bad ones more than the good ones. But it does seem like I notice a lot of bad ones, and never any good ones.)
[EDITED to add:] For context, the actual article title begins "Superpowers: How I'm using ..." and it has been auto-rewritten to "Superpowers: I'm using ...", which completely changes what "Superpowers" is understood as applying to. (The actual intention: superpowers for LLM coding agents. The meaning after the change: LLM coding agents as superpowers for humans.)
I've had it happen with me a few times where it was reasonable, sometimes where it was debatable, and if it was just wrong I edit it to add the How back in.
I agree, I'm sure I've seen instances of where it's worked but the problem is that when it messes it up it's much more annoying than any benefit it brings when it does work. Some of us don't want to be reminded that tech is full of hubris, overconfidence, poor judgment, and failure about what can/should be abstracted and automated.
Yeah, to the point I can recall several examples where the title stuck out as dumb on HN and only when visiting the original page it started to make sense, but not a single case where I could say the automated removal really did a good job.
[flagged]
That's far from true. Also, please don't cross into personal attack on this site.
https://news.ycombinator.com/newsguidelines.html
(We detached this subthread from https://news.ycombinator.com/item?id=45549522.)
Edit: your account has unfortunately been doing this repeatedly (https://news.ycombinator.com/item?id=45551198), and you've been breaking the site guidelines in other ways as well (e.g. https://news.ycombinator.com/item?id=45527456). We ban accounts that do this, so if you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, that would be good.
The past few years have taught me that these are the people that rise to the top of society (much to my chagrin).
The average person doesn’t want to hear from thoughtful intellectuals presenting nuanced opinions. They want to hear from those who brashly and boastfully present themselves as authority figures, and then bolster the listeners preconceived ideas with violently exaggerated language. Shallow but sensational is what sells.
I think that Elons bombastic claims about self driving have really popularized this approach. But you can now see it everywhere in tech: bitcoin going to $1B and nocoiners will be peasants, AI is going to turn us all in to paperclips, and on and on…
I try very hard to fulfill the role of "thoughtful intellectuals presenting nuanced opinions" in the AI space. I'm disappointed that I've failed to meet that standard in your eyes.
Here's a counter-example for you from the another day: https://simonwillison.net/2025/Oct/8/claude-datasette-plugin...
> This isn’t necessarily surprising, but it’s worth noting anyway. Claude Sonnet 4.5 is capable of building a full Datasette plugin now.
I do worry a bit about how often I use positive adjectives. If something isn't notable I won't write about it though. In this particle case Jesse's prompting / skills stuff really does deserve the superlatives IMO.
well explain why OPost is "wild" and what makes you recommend it "strongly" .
what have u built with to come to those conclusions ? is this too much to ask.
I recommend it strongly because the "skills" mechanism it describes is a new and very promising technique, and this is the best article I've seen that explains that.
It's "wild" because, among many other experiments, Jesse has experimented with giving Claude a "feelings journal" and prompting it using Graphviz DOT diagrams.
For my previous writing and work on this you can consult my blog - here's the AI-assisted programming tag: https://simonwillison.net/tags/ai-assisted-programming/
I'll admit I struggle to isolate what's fundamentally different between these skill configurations and the usual CLAUDE/AGENTS/etc.md. Clearly there's something; I'm just curious what's the mechanism at play, precisely.
EDIT: Simon actually already answered that point here: https://news.ycombinator.com/item?id=45550115
[flagged]
I read a lot of articles about tricks for using coding agents. I link to very few of them because they don't often present anything new.
Jesse presented something new.