I uploaded this to Fermat's Library because I think it's the most relevant five page paper in CS right now, even though it has nothing to do with software.
Bainbridge's argument about process control operators maps almost perfectly to AI-assisted software engineering:
The operator is asked to monitor a system that was automated because it does the job better than them. That's every engineer reviewing AI generated PRs, you're expected to verify decisions you couldn't have produced at that speed yourself.
Manual and cognitive skills decay without practice.
Her plant operators couldn't smoothly control processes they'd stopped doing by hand.
We're building the same dynamic with junior engineers who never debug without an agent.
The designer automates away the "unreliable human" and leaves them the residual tasks, which are always the hardest, most context-dependent edge cases, handed to someone progressively deskilled by the system now failing.
Her sharpest observation: the most reliable automated systems need the greatest investment in human training, because the operators get the least practice.
We should be thinking hard about what this means for how we train engineers, structure code review, and allocate work between humans and AI tools.
I uploaded this to Fermat's Library because I think it's the most relevant five page paper in CS right now, even though it has nothing to do with software.
Bainbridge's argument about process control operators maps almost perfectly to AI-assisted software engineering:
The operator is asked to monitor a system that was automated because it does the job better than them. That's every engineer reviewing AI generated PRs, you're expected to verify decisions you couldn't have produced at that speed yourself.
Manual and cognitive skills decay without practice.
Her plant operators couldn't smoothly control processes they'd stopped doing by hand.
We're building the same dynamic with junior engineers who never debug without an agent.
The designer automates away the "unreliable human" and leaves them the residual tasks, which are always the hardest, most context-dependent edge cases, handed to someone progressively deskilled by the system now failing.
Her sharpest observation: the most reliable automated systems need the greatest investment in human training, because the operators get the least practice. We should be thinking hard about what this means for how we train engineers, structure code review, and allocate work between humans and AI tools.
[dead]