What’s the point of this post? The people posting “you should be in jail” won’t read this. The people replying “you don’t know how to configure a sandbox” won’t understand this.
So who is this for?
> Of course we need strong sandboxing. We also need to secure the services it can reach, the tools it can invoke
That’s… the sandbox. The sandbox includes everything in the sandbox.
> So is it as easy as just putting it in a sandbox? No, it’s not. But I do believe it is tractable across three dimensions
Those 3 dimensions are sandboxing, alignment and monitoring.
I’d suggest that monitoring is a part of any sandbox. That leaves “the sandbox” and “trying to get the model to not escape the sandbox”?
Pretty sure the majority reaction to this article is not what the author expected. He should probably refrain from w̶h̶i̶n̶i̶n̶g̶ writing about his job publicly.
I refuse to empathize with someone making six figures while creating insoluble problems for subsequent generations, if any. I wish the author could miss sister's wedding every weekend.
> I literally missed my sister’s wedding a few weeks ago to help clean up after some of the recent incidents
If you're not paid for that, you should just leave. But my understanding is that OpenAI employees cannot complain about their conditions, can they? Hard to feel for a millionaire saying "I earn orders of magnitudes more than you, but I work 50% more", I think (I don't know if it's the case here or not).
> all I hear on X is how I “don’t know how to configure a sandbox” or that I should be in jail
I think it can be two very different comments. "They don't know how to configure a sandbox" is condescending. "Someone should be in jail"... well that can make sense. When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.
The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure. It's hard to build a dam that does not collapse, but that's not a reason for civil engineers to vibe-build a dam and then say "ok it collapsed, but look it's hard to get right".
> Very few organizations can survive that scale and growth, and yet OpenAI did a very good job handling it (and should be commended).
So if OpenAI results in people dying, it is fine because "really, it's very hard"? I'm trying to think about a company that would build dams at such a scale and then use it as an excuse for only a fraction of them collapsing from time to time.
> To say it lightly: this surprised the fuck out of us. We had not expected it this soon. Suddenly we weren’t dealing with just a small jump in capabilities; we were talking about a different sport altogether.
Last example: imagine a geoengineering company coming out with a "solution" to global warming. Suddenly it goes out of control and it screws us even more. Would we say "please, have some respect, having that kind of impact is really hard and it honestly surprised the fuck out of us", or would we say "maybe you shouldn't have done something dangerous you didn't understand"?
> I recently watched the movie Sully.
Okay I can't resist it. It's inverting the argument. In Sully, we accept the fact that they took risks they shouldn't have because it turns out that they saved lives (and they prove that doing what they should have would have resulted in more casualties).
Now if the plane had crashed into a tower and killed thousands of people, of course we would blame the pilots!
OpenAI resulting in security incidents is NOT AT ALL the same as Sully saving lives by not following the rules. What a weird comparison that is?
If Starship crashed in Central Park, I wouldn't expect a SpaceX engineer to write a blog post saying "Look, luckily it didn't kill anyone, and I'm sick of reading comments saying "someone should go to jail". It's very hard to build a rocket, and people who think otherwise are just misinformed. Give us some slack, crashing into Central Park surprised the fuck out of us".
> When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.
The difference is that 99.9% of software is just not as important as even your average two-lane bridge.
The recent cries to send random OpenAI people to jail frankly strike me as IT people just inflating our own importance. Oh no, HuggingFace got hacked. So what? Nobody got hurt, no individual lost their property. A company had to rebuild some server configs. The reason nobody cares enough to send someone to jail is that it's just not that important.
Also, even civil engineers very rarely go to jail when a building or bridge collapses. This is something everyone is using as an analogy when in reality it's very hard to prove criminal negligence.
> The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure.
Yeah, and when they say "okay, fine, we should slow down" everyone says they're making it up for marketing reasons.
And then if they actually slow down on their own everyone makes fun of them for being obsolete and switches to their competitor.
> The difference is that 99.9% of software is just not as important as even your average two-lane bridge.
You miss the point. LLMs are not the two-lane bridge here, they are the trigger. Sure, when an LLM makes 99.9% of software fail, nobody cares. But what happens when it touches a single, very critical instance of software?
> Oh no, HuggingFace got hacked. So what?
Ever heard of NotPetya? Blocked Maersk unwillingly (as a collateral). Could have resulted in a global economical crisis. There is a nice Darknet Diaries episode about that.
> Also, even civil engineers very rarely go to jail when a building or bridge collapses.
At the very least they don't go public saying "well it happens, building bridges is hard". Usually it goes more "it shouldn't have happened, and we need to get our act together and make sure it doesn't happen again anytime soon".
> Tricky problem, isn't it?
I get the irony. And I also totally get why someone getting an insane salary would see no reason to not work on it (and cynically think "if I am not getting rich having fun here, someone else will... may as well be me"). Who am I to say I wouldn't do the same? I strongly believe that sending astronauts to space is fun but useless, and taxpayers shouldn't agree. But offer me to go to space? I go tomorrow. Ironic, isn't it?
I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing. And comparing yourself to Sully (while saying you don't).
> But what happens when it touches a single, very critical instance of software?
I don't disagree at all, but the AI companies/engineers to their credit have been very clear on that risk - if anything, a bit too "clear" with all the apocalyptic rhetoric. And I've been disappointed that the overall response has either been "they should go to jail" (for a set of incidents that, while useful canaries in the coal mine, effectively harmed nobody) or "they're making it all up for marketing/monopolization". Sure, it's different people saying both, but it still feels kind of dumb that this is somehow collectively our response as a community.
Admittedly I'm pretty much on your side here, I just think the "yeahh send them to jail!" thing in particular is peak "internet ruining discussion on an important topic by taking it to its most ridiculous conclusion"
> I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing.
Agreed, it's pretty hard to try and get sympathy out of that. But on the other hand, what else can they do? I'd rather they explain their side of things than just stay quiet about it. They are right on their main point, which is that it's a hard problem at its core and I can understand why getting hit with a barrage of "just configure a sandbox dude" would tick anyone off.
What’s the point of this post? The people posting “you should be in jail” won’t read this. The people replying “you don’t know how to configure a sandbox” won’t understand this.
So who is this for?
> Of course we need strong sandboxing. We also need to secure the services it can reach, the tools it can invoke
That’s… the sandbox. The sandbox includes everything in the sandbox.
> So is it as easy as just putting it in a sandbox? No, it’s not. But I do believe it is tractable across three dimensions
Those 3 dimensions are sandboxing, alignment and monitoring.
I’d suggest that monitoring is a part of any sandbox. That leaves “the sandbox” and “trying to get the model to not escape the sandbox”?
Pretty sure the majority reaction to this article is not what the author expected. He should probably refrain from w̶h̶i̶n̶i̶n̶g̶ writing about his job publicly.
I refuse to empathize with someone making six figures while creating insoluble problems for subsequent generations, if any. I wish the author could miss sister's wedding every weekend.
Hopefully at least seven, if not eight. Fully vested, that is.
Devil take them all.
> but don’t attack staff directly
Sure, agreed.
> I literally missed my sister’s wedding a few weeks ago to help clean up after some of the recent incidents
If you're not paid for that, you should just leave. But my understanding is that OpenAI employees cannot complain about their conditions, can they? Hard to feel for a millionaire saying "I earn orders of magnitudes more than you, but I work 50% more", I think (I don't know if it's the case here or not).
> all I hear on X is how I “don’t know how to configure a sandbox” or that I should be in jail
I think it can be two very different comments. "They don't know how to configure a sandbox" is condescending. "Someone should be in jail"... well that can make sense. When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.
The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure. It's hard to build a dam that does not collapse, but that's not a reason for civil engineers to vibe-build a dam and then say "ok it collapsed, but look it's hard to get right".
> Very few organizations can survive that scale and growth, and yet OpenAI did a very good job handling it (and should be commended).
So if OpenAI results in people dying, it is fine because "really, it's very hard"? I'm trying to think about a company that would build dams at such a scale and then use it as an excuse for only a fraction of them collapsing from time to time.
> To say it lightly: this surprised the fuck out of us. We had not expected it this soon. Suddenly we weren’t dealing with just a small jump in capabilities; we were talking about a different sport altogether.
Last example: imagine a geoengineering company coming out with a "solution" to global warming. Suddenly it goes out of control and it screws us even more. Would we say "please, have some respect, having that kind of impact is really hard and it honestly surprised the fuck out of us", or would we say "maybe you shouldn't have done something dangerous you didn't understand"?
> I recently watched the movie Sully.
Okay I can't resist it. It's inverting the argument. In Sully, we accept the fact that they took risks they shouldn't have because it turns out that they saved lives (and they prove that doing what they should have would have resulted in more casualties).
Now if the plane had crashed into a tower and killed thousands of people, of course we would blame the pilots!
OpenAI resulting in security incidents is NOT AT ALL the same as Sully saving lives by not following the rules. What a weird comparison that is?
If Starship crashed in Central Park, I wouldn't expect a SpaceX engineer to write a blog post saying "Look, luckily it didn't kill anyone, and I'm sick of reading comments saying "someone should go to jail". It's very hard to build a rocket, and people who think otherwise are just misinformed. Give us some slack, crashing into Central Park surprised the fuck out of us".
> When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.
The difference is that 99.9% of software is just not as important as even your average two-lane bridge.
The recent cries to send random OpenAI people to jail frankly strike me as IT people just inflating our own importance. Oh no, HuggingFace got hacked. So what? Nobody got hurt, no individual lost their property. A company had to rebuild some server configs. The reason nobody cares enough to send someone to jail is that it's just not that important.
Also, even civil engineers very rarely go to jail when a building or bridge collapses. This is something everyone is using as an analogy when in reality it's very hard to prove criminal negligence.
> The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure.
Yeah, and when they say "okay, fine, we should slow down" everyone says they're making it up for marketing reasons.
And then if they actually slow down on their own everyone makes fun of them for being obsolete and switches to their competitor.
Tricky problem, isn't it?
> The difference is that 99.9% of software is just not as important as even your average two-lane bridge.
You miss the point. LLMs are not the two-lane bridge here, they are the trigger. Sure, when an LLM makes 99.9% of software fail, nobody cares. But what happens when it touches a single, very critical instance of software?
> Oh no, HuggingFace got hacked. So what?
Ever heard of NotPetya? Blocked Maersk unwillingly (as a collateral). Could have resulted in a global economical crisis. There is a nice Darknet Diaries episode about that.
> Also, even civil engineers very rarely go to jail when a building or bridge collapses.
At the very least they don't go public saying "well it happens, building bridges is hard". Usually it goes more "it shouldn't have happened, and we need to get our act together and make sure it doesn't happen again anytime soon".
> Tricky problem, isn't it?
I get the irony. And I also totally get why someone getting an insane salary would see no reason to not work on it (and cynically think "if I am not getting rich having fun here, someone else will... may as well be me"). Who am I to say I wouldn't do the same? I strongly believe that sending astronauts to space is fun but useless, and taxpayers shouldn't agree. But offer me to go to space? I go tomorrow. Ironic, isn't it?
I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing. And comparing yourself to Sully (while saying you don't).
> But what happens when it touches a single, very critical instance of software?
I don't disagree at all, but the AI companies/engineers to their credit have been very clear on that risk - if anything, a bit too "clear" with all the apocalyptic rhetoric. And I've been disappointed that the overall response has either been "they should go to jail" (for a set of incidents that, while useful canaries in the coal mine, effectively harmed nobody) or "they're making it all up for marketing/monopolization". Sure, it's different people saying both, but it still feels kind of dumb that this is somehow collectively our response as a community.
Admittedly I'm pretty much on your side here, I just think the "yeahh send them to jail!" thing in particular is peak "internet ruining discussion on an important topic by taking it to its most ridiculous conclusion"
> I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing.
Agreed, it's pretty hard to try and get sympathy out of that. But on the other hand, what else can they do? I'd rather they explain their side of things than just stay quiet about it. They are right on their main point, which is that it's a hard problem at its core and I can understand why getting hit with a barrage of "just configure a sandbox dude" would tick anyone off.