I'd seperate "the agent never receives the secret" from the "agent can use the secret".
What I find particularly useful is secrets never leave the local machine of the user and are never saved. The safest thing of course is agent can never see and or use the secret but then running tests and experiments could be hard.
Personally I've built a service that runs the harness inside a microVM where I inject the secret value via a separate service so the harness only ever sees an arbitrary value. My goal is to separate the network request from the harness logic so we can keep it secure.
In general I find that anything you put in the harness is likely to be owned, compromised or at risk. I just don't think the security primitives are widely there for these tools no matter what we're told. Agents will find a way to do something if they're given access in the harness and to the LLM.
In most harness it's already built in not to print the secrets, you can use also services like AWS secretes or 1password that allow using temporary access
I'd seperate "the agent never receives the secret" from the "agent can use the secret".
What I find particularly useful is secrets never leave the local machine of the user and are never saved. The safest thing of course is agent can never see and or use the secret but then running tests and experiments could be hard.
Personally I've built a service that runs the harness inside a microVM where I inject the secret value via a separate service so the harness only ever sees an arbitrary value. My goal is to separate the network request from the harness logic so we can keep it secure.
In general I find that anything you put in the harness is likely to be owned, compromised or at risk. I just don't think the security primitives are widely there for these tools no matter what we're told. Agents will find a way to do something if they're given access in the harness and to the LLM.
In most harness it's already built in not to print the secrets, you can use also services like AWS secretes or 1password that allow using temporary access
[flagged]
[flagged]