OpenAI's release explicitly says No. But then also caveats that with "we cannot rule out that de-identified data derived from their usage of our products" impacted things.
What's most striking to me, and what may or may not be true, is the "we cannot rule out" bit.
"We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models . However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced)."
https://openai.com/index/navier-stokes-solution/
If one of the researchers forgot to check "don't train on this" a single time, the pretrain is going to know what they were working on. The question of whether they remembered to check "don't train on this" every single time is super concrete, and embarrassing to _both_ sides if the answer is "no, he didn't read the eula."
These models will absolutely remember a brilliant insight that appeared a single time in the pretrain corpus, because if they couldn't-- they would get a slightly worse loss. I don't know what this wishy-washing "well maybe we trained on it but we didn't read it" is supposed to mean.
Just an FYI, despite the wording on this, my "don't train on this" checkbox never changes (I don't need to re-toggle it, or change it on every device, etc...). This post reads as if its something that needs to be done repeatedly. Also if you turn on the advanced security option, ChatGPT locks to no training and cannot even be toggled on if you wanted it to.
The fact that this link is a Reddit thread should probably be taken as evidence that we don't have a clear picture of what happened yet, and speculation is rampant.
It's possible that OpenAI was mining prominent researcher's chats for inspiration to tackle these problems, but it's also entirely possible the leak came from his collaborator's end as he works at Anthropic and I'm sure there's plenty of corporate espionage going on between those two. That would also explain why they didn't want to comment on the source of the prompt.
Everyone I think assumes training in the general “it affects the distribution” sense. But if it fishes precise novel insights out as alleged then it makes these products dramatically less valuable. At least the non zdr ones
This is a problem with every technology product. Before Facebook and Gmail had strong institutional controls, any employee could just open anybody's email and private messages on a whim. Now things are a bit more strict for random employees, but if the company as a whole wants to do something, you bet they can.
Can you opt out of sharing data for “analytical purposes”? I think you can’t. And what does “analytical purposes” even mean?
I think people straw man the whole data argument on “opt out of training purposes” and ignore the mandatory analytic purposes. Which probably includes figuring out trends and important research problem data to steal from users and even business ideas.
OpenAI's release explicitly says No. But then also caveats that with "we cannot rule out that de-identified data derived from their usage of our products" impacted things.
What's most striking to me, and what may or may not be true, is the "we cannot rule out" bit.
"We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models . However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced)." https://openai.com/index/navier-stokes-solution/
If one of the researchers forgot to check "don't train on this" a single time, the pretrain is going to know what they were working on. The question of whether they remembered to check "don't train on this" every single time is super concrete, and embarrassing to _both_ sides if the answer is "no, he didn't read the eula."
These models will absolutely remember a brilliant insight that appeared a single time in the pretrain corpus, because if they couldn't-- they would get a slightly worse loss. I don't know what this wishy-washing "well maybe we trained on it but we didn't read it" is supposed to mean.
(Example of Fable knowing the content of a deeply unimportant LessWrong post I wrote: https://claude.ai/share/a907b46c-bf7b-4fca-9c71-8582cf8507cc a working Navier Stokes solution would be way more salient)
Just an FYI, despite the wording on this, my "don't train on this" checkbox never changes (I don't need to re-toggle it, or change it on every device, etc...). This post reads as if its something that needs to be done repeatedly. Also if you turn on the advanced security option, ChatGPT locks to no training and cannot even be toggled on if you wanted it to.
The fact that this link is a Reddit thread should probably be taken as evidence that we don't have a clear picture of what happened yet, and speculation is rampant.
It's possible that OpenAI was mining prominent researcher's chats for inspiration to tackle these problems, but it's also entirely possible the leak came from his collaborator's end as he works at Anthropic and I'm sure there's plenty of corporate espionage going on between those two. That would also explain why they didn't want to comment on the source of the prompt.
If true it could have quite an impact.
Everyone I think assumes training in the general “it affects the distribution” sense. But if it fishes precise novel insights out as alleged then it makes these products dramatically less valuable. At least the non zdr ones
This is a problem with every technology product. Before Facebook and Gmail had strong institutional controls, any employee could just open anybody's email and private messages on a whim. Now things are a bit more strict for random employees, but if the company as a whole wants to do something, you bet they can.
Can you opt out of sharing data for “analytical purposes”? I think you can’t. And what does “analytical purposes” even mean?
I think people straw man the whole data argument on “opt out of training purposes” and ignore the mandatory analytic purposes. Which probably includes figuring out trends and important research problem data to steal from users and even business ideas.
They are currently being sued for trade secret theft so it seems likely.
100% their data was stolen. You can't enter any private data into AI and expect it to remain private.
I'm surprised solutions weren't trained off their Office365 or Google Docs..
Yes.
A rare exception to Betteridge's law of headlines
[dupe] Discussion: https://news.ycombinator.com/item?id=49605915
And currently: https://news.ycombinator.com/item?id=49613262