- Mr. Liu not only downloaded a confidential Apple circuit schematic but also used it in his work at OpenAI;
- Far from his unauthorized access to Apple’s third-party cloud storage being unknown to him, Mr. Liu and others at OpenAI were well-aware of that access;
- Mr. Liu, upon learning of Apple’s internal investigation of him, sent instructions for destroying evidence to an OpenAI colleague who confirmed she would comply; and
- Mr. Liu used a tool in his work at OpenAI that has the same name as an internal Apple engineering application used for Apple development work.
Sounds like the LLM is equally good a participating in a criminal conspiracy and keeping notes on it. Not the team mate we all want, but one we deserve.
Good lord. Smart people doing the dumbest things with their work equipment.
I have a former coworker who was the VP of Finance at my company. He suddenly got fired out of the blue. The company was in the midst of trying to find a buyer (this was announced) so this was extra surprising. Turns out, he knew a lot of bad news about the company, and they caught him looking for a job. They fired him because he was too much of a liability.
How did they catch him? He was putting resumes and cover letters on OneDrive. "Yeah, in retrospect, that's one of the dumber things I've ever done," he said to me later.
I am constantly dumb founded at the people who conduct deeply personal business on a company laptop. Some individuals do not even own their own computer. Has every job hunt they have pursued has been done on their previous employers equipment?
I have more personally owned computers than is reasonable, but I can understand people who get issued a company laptop and use it for everything.
It’s a couple hundred bucks a year to keep a personal laptop and if you’re mostly accessing cloud services, I can easily understand the temptation to use a company laptop for it all.
But when I said the exact same thing [0], people were talking about how it's their "culture" to use corporate laptops for personal usage, and I even got downvoted and flagged if you read the thread (turn showdead on in your HN settings). It's baffling indeed, but I guess people will learn the hard way, that companies can see everything you store and even type on there.
> Another turned his computer that “wouldn’t behave” into IT, full of porn.
Every tech support worker has seen this countless times. So many people are completely blind to the idea that turning your device over to support means support can actually see what’s on your device…
To paraphrase the sage Jimmy McMillan: the number of people who interview on their work laptops is too damn high. I'm always flabbergasted when they admit this and not once has somebody who did so passed. IME there's a strong correlation between good judgment and technical prowess in this regard but I suppose stories like this prove me wrong.
I can't find the stat, but if memory serves me correctly, NZ over-indexes in CSAM-related crime. It's unfortunate, but it's not always good news down on our little islands.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.”
This is somewhat of a high impact argument to test. I wonder if the case will eventually get to working this point out.
There's already some precedent when dealing with humans reverse engineering systems. We use "cleanroom" approaches where the ones doing the reverse engineering are segregated from those doing the later design and development work. Instead of producing a design document based on the thing being studied, the reverse engineers produce a specification from which a new design, untainted by knowledge of the original design, is produced.
People using ML to try and reverse-engineer and create a "clean" version of things will likely need to use a similar approach. You can't ask one LLM to take in a circuit design as input and produce a specification and design within the same context. The resulting design will be at least partially informed by knowledge of the original design. The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. At least then you know your LLM was not aware of the original design.
Of course, LLM agents "cheat", so you'll also want to be careful to ensure a clean environment if you're using an agent that does not provide access to the original design material.
> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design.
I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered.
This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).
> This seems as acceptable as asking a human to do the same.
I wouldn't be surprised if self-bias in the model weights and imperceptible grammar/word/punctuation choices could cause it to duplicate more information than a human-to-human transfer Probably not enough to matter, but...
To put it another way, imagine the task was one human viewing a painting, and then describing it over the phone for another human to paint, so that the final product wasn't really a "copy". Assume everyone has the same eyeballs and art-skills.
I'd expect a massive improvement if the humans on each end were freshly-made clones from a teleporter accident, sharing identical brain-structures and 99.9999% of their memories.
Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.
You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.
If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design.
If a windows DLL, distributed without a license that says anything regarding, is it a "clean room implementation" if there is some nonzero chance the source code was leaked into the weights at pre-train time? I guess there should be some sort of method for subpoenaing frontier labs to ask "can you grep for this code in the training set for this model", but that might not be practical or feasible.
> If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design.
You can't ensure it properly segregates its knowledge so it's a legal risk. If you believe your LLM can generate a design from a spec without knowledge of the original, why would you take that unnecessary risk? A lot of the behavior in this area is meant to avoid the appearance of impropriety, because the appearance of impropriety forces you to defend and demonstrate there was none.
If you isolate the two behaviors (reverse engineer design to spec; convert spec to new design) then you have a legal defense. You can claim that any coincidentally too similar design elements are a consequence of standard design patterns or something, not a consequence of inherent knowledge of the original design.
This is why we separate the people into distinct roles, there's no reason not to do the same (or expect the same) with machines. If you don't want the legal cover, of course, by all means take the risk and enjoy a trillion dollar company taking you to court.
"However, we also find that Llama 3.1 70B memorizes some books, like Harry Potter and the Sorcerer’s Stone and 1984, almost entirely. In fact, Harry Potter is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim. "
> With our specific experiments, we find that the largest LLMs don’t memorize most books–either in whole or in part.
This technique has only ever been made to work with a vanishingly small number of extremely popular works, probably because they are so overrepresented in the training data set.
The risk with IP, however, is a lot more grave. You may not even need to memorize the details of the IP verbatim, just the broad idea may be enough. It may lurk encoded in the weights forever, just waiting to be activated by the right prompt to start a chain of thought that unlocks further details. Heck, it may even appear as if the model suggested the idea itself.
Not sure why people are latching on to this. I'm 95% sure the AI agent was not "trained" on this intellectual property but rather the IP was just analyzed and it's usage and details were recorded to the AI agents memory (typically markdown/JSON summary of previous sessions).
The defense IMO is just trying to muddy the waters between "irreversible AI training that updates weights" and "AI learning by just storing secrets in text files".
Even in the 5% chance this AI model actually was fine-tuned or fully-trained (i.e the weights were updated), the employee did this while employed at Apple and used Apple IP as the training data. Even if the output is under fair-use (because it's transformative), (1) any IP created while employed during work duties almost always belongs to your employer and (2) the act of accessing Apple's IP would be (trade secret or regular) theft if done outside normal work duties.
> Additionally, Apple learned about Liu’s use of the schematic because he used it on a Mac mini which later synced via iCloud to the MacBook he took from Apple. Apple now also wants access to that Mac mini.
I'm very curious about the privacy implications of this. I know that anything I do and store on my company's laptop can be tracked, but I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information.
The files syncing to the company laptop's disk is a layer of nuance that makes this situation tricky to evaluate.
encourages, but doesn’t require. My work phone is an iPhone 14 that I only use for Slack notifications & to check the next day’s schedule when I don’t want to bother opening their laptop.
When Ashley Gjovik was fired, she complained the pressure to use work devices for personal stuff was considerable. I stopped carrying back then & haven’t felt any pressure.
A “carry” or “live on” device doesn’t have to be used for personal email and messages. It should just be used for more real-world non-test workloads. The lessons of only testing iPhone 4 in stealth cases hasn’t been forgottten.
I do agree that Apple likes to have it both ways, though.
I can't open pretty much anything, including shoghoth face and linked in, but thankfully this imageboard and YouTube are spared. I still wonder how the powers that be didn't bother to ban npm on dev machines
> I'm very curious about the privacy implications of this. I know that anything I do and store on my company's laptop can be tracked, but I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information.
If it's work-related, work computer only. If it's anything personal, personal computer only. This is especially true for situations where companies are using screen capturing tools that capture the screen every 'x' seconds[0].
The seeming convenience of using your work computer for everything isn't worth the risk but some people don't consider the legal implications, as you've noted in your anecdote.
Once your personal message to Bob or Jane enters an official record because it's included in the eDiscovery (even if accidental) result(s), it's over.
Better to not have a surface risk like that, than assume the process will keep your personal data safe.
I was under the impression that Apple requires its employees to login with personal Apple ID accounts for cloud storage.
> “You go through these steps of getting all your software set up, and all your devices provisioned to access internal Apple networks, and things like that. And it explicitly says you cannot use your corporate Apple account to set this up.”
How's that supposed to work? Apple requires employees to own Apple products or have a personal account in the first place? That doesn't sound believable.
That seems crazy to me. I've always had my employers either provide or offer to provide me with anything I'll use for work. Asking someone to use a non-corporate-managed account for work seems especially insane to me. I've never heard of that.
This may have been an optional iCloud 2TB promotion for employees, but still relevant to the lawsuit.
I'm also under the impression that Apple requires its employees to dogfood their products and services. Hoping someone with personal experience and a lapsed NDA can chime in.
Techies tend to associate access control with legal control. Nothing could be farther from the truth, especially when it comes to legal discovery and subpoenas.
And it gets worse. The 4th amendment protects against unreasonable searches from the government without a warrant, not all searches. If you have evidence on your personal devices that a judge believes could be relevant to the outcome of a criminal or civil trial, be prepared to give it up or face a potential evidence tampering / obstruction charge. Doesn't matter if you never signed into your personal Gmail on your work computer or not. In a trial where work records are important, your non-work devices and accounts can be subpoenaed if there is a good reason to believe you have work materials on your Gmail or personal laptop. Like if in the first review of work e-mails, they find you've e-mailed one work file from your work to your personal Gmail one time or even just have been signed onto your personal Gmail while at work.
> I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information
I don't think this is true, this would still be unauthorized access on your employer's end and would be considered illegal. They're not allowed to pose as you to access your services iirc.
You should obviously still sign out (or never sign in in the first place) of course!
These things all vary across legal jurisdictions and with what you sign away in your contracts. If your contract doesn't say anything, then yes, your boss opening up your laptop while you're on lunch and reading through your logged-in but password-protected personal e-mail is not just a civil violation but gets into CFAA/wiretapping territory. California has some particularly strong default protections.
But you can 'freely' sign away these rights in your contract. Or if your contract binds you to follow internal policy and it's in the internal policy, that's usually enough notice and consent for the courts. When you're given a work device from a BigCo with a legal department that knows what they're doing, it usually comes with a EULA-style contract you don't read that authorizes everything.
California law does now say you can't be forced to give your employer your personal e-mail or social login, or other way of scanning your personal e-mail or socials. You can't sign this right away (just like you can't sign away your right to a minimum wage or workplace safety), but if you freely sign in while on a company device, network, and time, and your contract or policy is worded so broadly that anything you do on that device/network/time is the company's...
Yeah, from what I remember they have a funky iCloud file share thing set up for the whole company, and it requires them to either sign in with their personal iCloud account, or carry two iPhones (one for personal, one for work). Most employees understandably opt for the first option.
Reminds me of the story of an ex-Coca-Cola employee who offered to sell the secret recipe to Pepsi. Pepsi immediately let Coca-Cola know and it was handled. Not a good look for OpenAI. They come off as desperate and unprofessional.
Funny to imagine that food scientists at these companies must have cracked these recipes decades ago, and the secrecy might now only serve as a loyalty test for the uninitiated in the food and beverage guild.
That's because pepsi already had coca-cola's secret recipe. I'm sure if they didn't have it already they would have been more than happy to at least have some knowledge before reporting it, but not like they didn't have the talent, money or technology to reverse engineer it at least a decade ago at that point.
No, it's because Pepsi isn't trying to be Coke, they're trying to be better than Coke. So having the recipe is kind of moot.
Even if they were to take the recipe and publish it for all to see, you would still need all of the other machinery that is Coca-Cola to make Coca-Cola.
And then you have issues of quality. Coca-Cola has certain standards, it's not a guarantee that everyone else will have those same standards. Like, people buy the expensive brand of milk despite all milk being the same. It's one ingredient. With quality standards given by the government. There is really no room for interpretation.
Is this an AI response? coca-cola was replicated by one guy and confirmed in blind taste test by several other people, no magical machinery needed to make it once you know the ingredients.
What about quality? That has nothing to do with it.
Of course they're trying to be the better coke, that's why they tried making coke with the same color with more sugar and made the flavors stronger.
Coca-Cola is the only company that can legally import and process raw coca leaves into a flavoring agent. It’s literally impossible to duplicate their exact recipe.
If you live in the US sure. I live in a Latin American country where it’s legal to buy coca leaves (cocaine is illegal obviously, but the leaves aren’t and are commonly sold in health food stores of all places for tea).
I don't know if you have seen it, but the recipe is 1:1 of what cocacola is, scientifically proven. (Altho it tastes closer to original cola since replicating coca 1:1 is impossible)
Not really OpenAI but today's society. I say this started with the "just do it and ask for forgiveness later" attitude that started maybe 20 or 30 years ago.
At one time most people had respect, even people at the top, now many people just do what they want. Plus most of the time they get away with it.
is what the entire LLM industry is based on. They swallowed up all of society’s copyrighted texts without really asking for permission from anyone. This is just par for the course for them it seems unfortunately.
> Peer pressure and community sanctions of bad actors will result in the changes we need.
It's been demonstrated over centuries that this is unreliable. We end up needing a third party (commonly governments) to step in and establish rules and referee behavior. We wouldn't have an EPA, FCC, SEC, or many other organizations and laws if private actors could successfully police themselves.
Private actors are inevitably part of groups. Those groups need to hold the individual members accountable. That obviously doesn't work any more in SV among wealth-obsessed tech-bros. It still works in other parts of the economy though.
> Apple alleges that Liu ran a simulation in March using the circuit schematic file in LTspice, an electrical engineering tool. In messages from around that time, Liu said his AI “agent” learned to run LTspice and review the results.
"I didn't steal it, I fed it to an agent who then fed it back to me".
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.
I'm not arguing for either side, but here's a fun thought experiment I've been thinking about a lot lately. If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets?
What if I trained a team of agents to think and code like me at work, then exported those agentic skills and took them with me to another company? At what point would they still be considered the old company's property? Let's say I didn't take any code or proprietary information from
the old company, just the coding style and best practices my agents learned from the old company's live metrics. Is that considered stealing from the company? Is that any different from me learning those patterns and skills personally and then reapplying them elsewhere?
There doesn't have to be any meaningful analogy here. Laws don't have to treat humans and machines equally. Laws are for people and allow things that are less than ideal, but necessary for people to function.
Humans can't leave their old brains at their previous employer, can't delete their experience learned on proprietary data. It wouldn't make sense for the law to fight this.
But the law can control what can be done with information stored outside of your brain.
Your training of a team of agents is a product you developed while working for that company and would be owned by them. You could in theory train a new team of agents after leaving the company, but it could not be trained on anything that isn't in your brain. If you gave it anything, like the old company's live metrics, then you'd be stealing trade secrets. The key difference from you personally learning those skills is that an agent trained on company data turns personal knowledge into a portable, structured artifact, which crosses from "your skill" into something that looks a lot more like company property.
This is my personal conclusion as well. But think about where this leads. In the future, agents will do most of the coding and our job is to guide and train them. Most of our skills would be externalized somehow, living in the agents instead of in our heads. Which means once you leave a company, you may have effectively lost a lot of your hard earned skills and experiences.
I'm pretty sure Marx has some things to say on this. When your skills live in company-owned agents instead of in your head, you've been alienated from your own labor in a way Marx predicted. This is exactly why companies like Meta have been so aggressive to integrate AI tools into the day-to-day work for their employees. The more your skills live in their systems, the less you can take with you.
I don't understand the point of your question. The "problem" of an ex-employee consciously or subconsciously bringing his learned experiences at your company to a competitor is literally the reason why non-competes exist. In fact, this topic is quite old.
My real point is this: if you're not allowed to bring your experiences with you, then the company exists to extract your creative thinking and skills. You can argue that's a fair trade for money. I don't necessarily disagree. But I think most of us find joy in learning, in building our own skills, in being able to reapply them elsewhere. And before LLMs and agents, I couldn't lose those skills and experiences no matter where I worked. As we progress, though, we've hit a point where we have to ask ourselves: at what price are we willing to trade complete ownership of our thinking and experiences for money?
> If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets?
The fact that California allows this to happen (banning non-competes and rejecting "inevitable disclosure") is exactly why Silicon Valley started here and remains here. It's exactly why ex-OpenAI people could start Anthropic. It's why neither OpenAI nor Anthropic have a monopoly on AI today. It's great both for employees and for the general public.
> At what point would they still be considered the old company's property?
It depends on your contracts (aka NDAs). Sometimes employers do ban for that very same reason from joining a competitor(s) for x years from date of separation
It also depends on your law. In California, some companies still try to put these restrictions into contracts, but that contradicts state law so it has no legal effect. The intended effect is intimidation, however, which may work with some people.
> If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets?
Human brains are in many ways privileged agents in a legal and copyright system, because it exists to serve the needs of humans.
Obviously it is. Just like when you give a monkey a camera and it takes a selfie, the photo is obviously yours. But the legal system doesn't care about obviousness.
Has anyone seen Pandora? I want to have a look inside that box she's been carrying around...it's time we start suing the pants off this "generative" stuff...
Happened at a (large) company I worked for in the past. A devops guy stole the DB ahead of going to work with a rival, he was caught and the rival of course wanted nothing to do with it. I'm not sure of his fate but it couldn't have been much fun.
That might be the most embarrassing implication for OpenAI out of the whole affair: That OpenAI, supposedly one of the vanguards in AI, saw a need to appropriate intellectual property of Apple, supposedly a complete laggard in that field.
A mature organization does not want the liability and if contacted by previous employer with evidence of these kinds of allegations, immediately gives said employees the boot.
Happens occasionally at big corporations, even the bloodsucking vampire. Best to distance yourself from such employees - lest they will do to you what they have done to others.
The difference is, relative to OpenAi they have standards and morals.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.”
But they've accepted the business model of taking works without permission for training AI; they need that to be fair use.
I know it's unethical, but when I read this, I can't help but hope a pseudo-cleanroom/ai-laundered Linux GPU driver appears for MacBooks. What I wouldn't give to run Linux on my Apple hardware.
It is cute that anyone thinks intellectual property can be defended much longer. All intelligence will get folded into whichever models have the loosest restrictions on training on IP, and those models will become the smartest and most capable so everyone will use them.
But also like, do not steal IP you could easily have re-created again. Those involved in this scheme were doubly idiots because they stole the IP in brazen ways that were easily traced back to them. Do people really not understand that every corporate laptop and server these days tracks -everything- you do? That corpo paycheck comes at a price of no privacy of any kind.
California has protections on you owning whatever you can take in your brain and these days that is all anyone needs with LLMs being as powerful as they are.
One of the things I always told kids who came into these big trading firms is the owner(s) is/are a billionaire, and likely came up from one of the trading floors or something adjacent. They are not the “forgiving” types when it comes to shit like this, and they have infinitely more resources than you.
So maybe don’t try to steal code, or trade concepts.
Hasn’t stopped some people I know from trying…
(For clarification I am not ex-CitSec and do not know this poor SOB, but he serves as the perfect poster child for “Don’t do that”)
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning "may create irreversible and continually propagating uses of the trade secret."
Yes. Yes, please make this argument, Apple. Some fascinating other conclusions follow from this.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.
The claims:
- Mr. Liu not only downloaded a confidential Apple circuit schematic but also used it in his work at OpenAI;
- Far from his unauthorized access to Apple’s third-party cloud storage being unknown to him, Mr. Liu and others at OpenAI were well-aware of that access;
- Mr. Liu, upon learning of Apple’s internal investigation of him, sent instructions for destroying evidence to an OpenAI colleague who confirmed she would comply; and
- Mr. Liu used a tool in his work at OpenAI that has the same name as an internal Apple engineering application used for Apple development work.
Sounds like Apple might soon own some Jalapeño
I think they'll settle for some of the sweet, sweet DRAM supply that Altman has locked up.
Sounds like the LLM is equally good a participating in a criminal conspiracy and keeping notes on it. Not the team mate we all want, but one we deserve.
Good lord. Smart people doing the dumbest things with their work equipment.
I have a former coworker who was the VP of Finance at my company. He suddenly got fired out of the blue. The company was in the midst of trying to find a buyer (this was announced) so this was extra surprising. Turns out, he knew a lot of bad news about the company, and they caught him looking for a job. They fired him because he was too much of a liability.
How did they catch him? He was putting resumes and cover letters on OneDrive. "Yeah, in retrospect, that's one of the dumber things I've ever done," he said to me later.
I am constantly dumb founded at the people who conduct deeply personal business on a company laptop. Some individuals do not even own their own computer. Has every job hunt they have pursued has been done on their previous employers equipment?
I have more personally owned computers than is reasonable, but I can understand people who get issued a company laptop and use it for everything.
It’s a couple hundred bucks a year to keep a personal laptop and if you’re mostly accessing cloud services, I can easily understand the temptation to use a company laptop for it all.
Also, when you travel or even commute using public transport, you don't always want to carry two laptops around.
But when I said the exact same thing [0], people were talking about how it's their "culture" to use corporate laptops for personal usage, and I even got downvoted and flagged if you read the thread (turn showdead on in your HN settings). It's baffling indeed, but I guess people will learn the hard way, that companies can see everything you store and even type on there.
[0] https://news.ycombinator.com/item?id=47851948#47854312
>Has every job hunt they have pursued has been done on their previous employers equipment?
Yes. And the resume was updated and the applications filed while on the clock.
https://youtu.be/WcyNWbx8kAo?t=4
(the timestamp is important)
If you're salaried, every moment of time in your life is on the clock.
That depends on jurisdiction
A coworker of mine once answered the wrong cell phone and said the wrong company name. (He was working for a competitor as well as our company.)
Another turned his computer that “wouldn’t behave” into IT, full of porn.
> Another turned his computer that “wouldn’t behave” into IT, full of porn.
Every tech support worker has seen this countless times. So many people are completely blind to the idea that turning your device over to support means support can actually see what’s on your device…
To paraphrase the sage Jimmy McMillan: the number of people who interview on their work laptops is too damn high. I'm always flabbergasted when they admit this and not once has somebody who did so passed. IME there's a strong correlation between good judgment and technical prowess in this regard but I suppose stories like this prove me wrong.
being dodgy is hard work. it takes effort to maintain and the longer you maintain it the cumulatively harder it is to get away with.
We have a plague here in NZ of top-level Police officers using work-issued devices to access and store CSAM material.
Wow that got dark real quick, mind sharing any links/evidence? Never would have thought NZ has a CSAM problem.
https://www.rnz.co.nz/news/crime-and-justice/1101088/aucklan...
https://www.nzherald.co.nz/nz/second-auckland-police-officer...
https://www.rnz.co.nz/news/crime-and-justice/567348/revealed...
It's not all sunshine and rainbows in our little country
I mean, the headline says "second". Anything greater than 0 is bad but seems a little short of a "plague".
Anything greater than 0, in leadership roles of the organization that should be one the most highly trusted in the country, is a plague.
so you would call it a "plague" if it was just one guy? hopefully you can see how ridiculous that would work out in practice.
It’s horrible, it deserves severe legal punishment, but “plague” to me implies a broad pattern.
It's rarely sunshine, but there are a lot of rainbows!
https://www.nzherald.co.nz/nz/auckland-police-officer-matthe...
Here’s one. There are more. Don’t be so naïve as to think any single country is free from these kinds of issues.
I can't find the stat, but if memory serves me correctly, NZ over-indexes in CSAM-related crime. It's unfortunate, but it's not always good news down on our little islands.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.”
This is somewhat of a high impact argument to test. I wonder if the case will eventually get to working this point out.
There's already some precedent when dealing with humans reverse engineering systems. We use "cleanroom" approaches where the ones doing the reverse engineering are segregated from those doing the later design and development work. Instead of producing a design document based on the thing being studied, the reverse engineers produce a specification from which a new design, untainted by knowledge of the original design, is produced.
People using ML to try and reverse-engineer and create a "clean" version of things will likely need to use a similar approach. You can't ask one LLM to take in a circuit design as input and produce a specification and design within the same context. The resulting design will be at least partially informed by knowledge of the original design. The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. At least then you know your LLM was not aware of the original design.
Of course, LLM agents "cheat", so you'll also want to be careful to ensure a clean environment if you're using an agent that does not provide access to the original design material.
> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design.
I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered.
This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).
> This seems as acceptable as asking a human to do the same.
I wouldn't be surprised if self-bias in the model weights and imperceptible grammar/word/punctuation choices could cause it to duplicate more information than a human-to-human transfer Probably not enough to matter, but...
To put it another way, imagine the task was one human viewing a painting, and then describing it over the phone for another human to paint, so that the final product wasn't really a "copy". Assume everyone has the same eyeballs and art-skills.
I'd expect a massive improvement if the humans on each end were freshly-made clones from a teleporter accident, sharing identical brain-structures and 99.9999% of their memories.
Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.
You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.
question to clarify:
If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design.
If a windows DLL, distributed without a license that says anything regarding, is it a "clean room implementation" if there is some nonzero chance the source code was leaked into the weights at pre-train time? I guess there should be some sort of method for subpoenaing frontier labs to ask "can you grep for this code in the training set for this model", but that might not be practical or feasible.
> If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design.
You can't ensure it properly segregates its knowledge so it's a legal risk. If you believe your LLM can generate a design from a spec without knowledge of the original, why would you take that unnecessary risk? A lot of the behavior in this area is meant to avoid the appearance of impropriety, because the appearance of impropriety forces you to defend and demonstrate there was none.
If you isolate the two behaviors (reverse engineer design to spec; convert spec to new design) then you have a legal defense. You can claim that any coincidentally too similar design elements are a consequence of standard design patterns or something, not a consequence of inherent knowledge of the original design.
This is why we separate the people into distinct roles, there's no reason not to do the same (or expect the same) with machines. If you don't want the legal cover, of course, by all means take the risk and enjoy a trillion dollar company taking you to court.
A related argument could be whether the output of an LLM that has been trained on GPL code should also be considered to be under a GPL licence.
the fact that llms can spit out books they've been trained on near verbatim lends some credence to this.
"However, we also find that Llama 3.1 70B memorizes some books, like Harry Potter and the Sorcerer’s Stone and 1984, almost entirely. In fact, Harry Potter is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim. "
https://reglab.stanford.edu/publications/extracting-memorize...
A more important quote from that link:
> With our specific experiments, we find that the largest LLMs don’t memorize most books–either in whole or in part.
This technique has only ever been made to work with a vanishingly small number of extremely popular works, probably because they are so overrepresented in the training data set.
The risk with IP, however, is a lot more grave. You may not even need to memorize the details of the IP verbatim, just the broad idea may be enough. It may lurk encoded in the weights forever, just waiting to be activated by the right prompt to start a chain of thought that unlocks further details. Heck, it may even appear as if the model suggested the idea itself.
Not sure why people are latching on to this. I'm 95% sure the AI agent was not "trained" on this intellectual property but rather the IP was just analyzed and it's usage and details were recorded to the AI agents memory (typically markdown/JSON summary of previous sessions).
The defense IMO is just trying to muddy the waters between "irreversible AI training that updates weights" and "AI learning by just storing secrets in text files".
Even in the 5% chance this AI model actually was fine-tuned or fully-trained (i.e the weights were updated), the employee did this while employed at Apple and used Apple IP as the training data. Even if the output is under fair-use (because it's transformative), (1) any IP created while employed during work duties almost always belongs to your employer and (2) the act of accessing Apple's IP would be (trade secret or regular) theft if done outside normal work duties.
> Additionally, Apple learned about Liu’s use of the schematic because he used it on a Mac mini which later synced via iCloud to the MacBook he took from Apple. Apple now also wants access to that Mac mini.
I'm very curious about the privacy implications of this. I know that anything I do and store on my company's laptop can be tracked, but I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information.
The files syncing to the company laptop's disk is a layer of nuance that makes this situation tricky to evaluate.
> I forgot to sign out of my personal Gmail
Never, ever, ever sign into personal mail/messaging on work machines. Even the appearance of having done so just sets up for bad things.
This was kinda iffy 20 years ago, now its crazy to do. We all have phones now, there's no good reason to do this.
> Never, ever, ever sign into personal mail/messaging on work machines
apple explicitly encourages their employees to do exactly this - it's called "carry"ing your work device.
encourages, but doesn’t require. My work phone is an iPhone 14 that I only use for Slack notifications & to check the next day’s schedule when I don’t want to bother opening their laptop.
i don't see your point - my point is apple actively encourages this thing that is being painted as terrible character flaw.
When Ashley Gjovik was fired, she complained the pressure to use work devices for personal stuff was considerable. I stopped carrying back then & haven’t felt any pressure.
A “carry” or “live on” device doesn’t have to be used for personal email and messages. It should just be used for more real-world non-test workloads. The lessons of only testing iPhone 4 in stealth cases hasn’t been forgottten.
I do agree that Apple likes to have it both ways, though.
> I stopped carrying back then & haven’t felt any pressure.
if you're who i suspect you are, then you're being wholly disingenuous because you're not a dev. for a dev there is much more pressure.
you're not wrong, but its really annoying carrying 2 laptops and 2 phones and 2 charging bricks [0].
Especially when work picks out the 16.2" laptop for you.
[0] - Yes, I know there are 3rd party multi-port USB-C charging products, but the wattage isn't as high as having 2 different bricks.
Hell, some employers are even helpful enough to block gmail, yahoo etc domains on their network.
I can't open pretty much anything, including shoghoth face and linked in, but thankfully this imageboard and YouTube are spared. I still wonder how the powers that be didn't bother to ban npm on dev machines
> I'm very curious about the privacy implications of this. I know that anything I do and store on my company's laptop can be tracked, but I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information.
If it's work-related, work computer only. If it's anything personal, personal computer only. This is especially true for situations where companies are using screen capturing tools that capture the screen every 'x' seconds[0].
The seeming convenience of using your work computer for everything isn't worth the risk but some people don't consider the legal implications, as you've noted in your anecdote.
Once your personal message to Bob or Jane enters an official record because it's included in the eDiscovery (even if accidental) result(s), it's over.
Better to not have a surface risk like that, than assume the process will keep your personal data safe.
0 - https://desktime.com/features/time-tracking-with-screenshots
I was under the impression that Apple requires its employees to login with personal Apple ID accounts for cloud storage.
> “You go through these steps of getting all your software set up, and all your devices provisioned to access internal Apple networks, and things like that. And it explicitly says you cannot use your corporate Apple account to set this up.”
https://cybersecurityventures.com/does-apple-spy-on-its-empl...
How's that supposed to work? Apple requires employees to own Apple products or have a personal account in the first place? That doesn't sound believable.
Requiring them to make a (free) Apple account doesn’t sound unreasonable.
“Go to https://account.apple.com/ and make one that doesn’t use your Apple corporate email” seems like a fair and legal work instruction to me.
That seems crazy to me. I've always had my employers either provide or offer to provide me with anything I'll use for work. Asking someone to use a non-corporate-managed account for work seems especially insane to me. I've never heard of that.
This may have been an optional iCloud 2TB promotion for employees, but still relevant to the lawsuit.
I'm also under the impression that Apple requires its employees to dogfood their products and services. Hoping someone with personal experience and a lapsed NDA can chime in.
Techies tend to associate access control with legal control. Nothing could be farther from the truth, especially when it comes to legal discovery and subpoenas.
And it gets worse. The 4th amendment protects against unreasonable searches from the government without a warrant, not all searches. If you have evidence on your personal devices that a judge believes could be relevant to the outcome of a criminal or civil trial, be prepared to give it up or face a potential evidence tampering / obstruction charge. Doesn't matter if you never signed into your personal Gmail on your work computer or not. In a trial where work records are important, your non-work devices and accounts can be subpoenaed if there is a good reason to believe you have work materials on your Gmail or personal laptop. Like if in the first review of work e-mails, they find you've e-mailed one work file from your work to your personal Gmail one time or even just have been signed onto your personal Gmail while at work.
There is a difference between judge signed warrant and nosy CISO or HR person having feelings. A whole world of difference
> I hadn't considered that if I forgot to sign out of my personal Gmail on it that they could legally search that information
I don't think this is true, this would still be unauthorized access on your employer's end and would be considered illegal. They're not allowed to pose as you to access your services iirc.
You should obviously still sign out (or never sign in in the first place) of course!
These things all vary across legal jurisdictions and with what you sign away in your contracts. If your contract doesn't say anything, then yes, your boss opening up your laptop while you're on lunch and reading through your logged-in but password-protected personal e-mail is not just a civil violation but gets into CFAA/wiretapping territory. California has some particularly strong default protections.
But you can 'freely' sign away these rights in your contract. Or if your contract binds you to follow internal policy and it's in the internal policy, that's usually enough notice and consent for the courts. When you're given a work device from a BigCo with a legal department that knows what they're doing, it usually comes with a EULA-style contract you don't read that authorizes everything.
California law does now say you can't be forced to give your employer your personal e-mail or social login, or other way of scanning your personal e-mail or socials. You can't sign this right away (just like you can't sign away your right to a minimum wage or workplace safety), but if you freely sign in while on a company device, network, and time, and your contract or policy is worded so broadly that anything you do on that device/network/time is the company's...
I think I've heard about this before, apparently Apple encourages its employees to use the same iCloud account for personal and business use.
Yeah, from what I remember they have a funky iCloud file share thing set up for the whole company, and it requires them to either sign in with their personal iCloud account, or carry two iPhones (one for personal, one for work). Most employees understandably opt for the first option.
Reminds me of the story of an ex-Coca-Cola employee who offered to sell the secret recipe to Pepsi. Pepsi immediately let Coca-Cola know and it was handled. Not a good look for OpenAI. They come off as desperate and unprofessional.
Funny to imagine that food scientists at these companies must have cracked these recipes decades ago, and the secrecy might now only serve as a loyalty test for the uninitiated in the food and beverage guild.
https://www.theguardian.com/media/2006/jul/07/marketingandpr... - I had only vague memories of this, but here's an article from the time about it.
That's because pepsi already had coca-cola's secret recipe. I'm sure if they didn't have it already they would have been more than happy to at least have some knowledge before reporting it, but not like they didn't have the talent, money or technology to reverse engineer it at least a decade ago at that point.
No, it's because Pepsi isn't trying to be Coke, they're trying to be better than Coke. So having the recipe is kind of moot.
Even if they were to take the recipe and publish it for all to see, you would still need all of the other machinery that is Coca-Cola to make Coca-Cola.
And then you have issues of quality. Coca-Cola has certain standards, it's not a guarantee that everyone else will have those same standards. Like, people buy the expensive brand of milk despite all milk being the same. It's one ingredient. With quality standards given by the government. There is really no room for interpretation.
Is this an AI response? coca-cola was replicated by one guy and confirmed in blind taste test by several other people, no magical machinery needed to make it once you know the ingredients.
What about quality? That has nothing to do with it.
Of course they're trying to be the better coke, that's why they tried making coke with the same color with more sugar and made the flavors stronger.
Coca-Cola is the only company that can legally import and process raw coca leaves into a flavoring agent. It’s literally impossible to duplicate their exact recipe.
If you live in the US sure. I live in a Latin American country where it’s legal to buy coca leaves (cocaine is illegal obviously, but the leaves aren’t and are commonly sold in health food stores of all places for tea).
Stepan Company, actually. But you’re right that Coca-Cola is the exclusive buyer.
where did you find that? (super interesting)
The Acquired episode on coca cola
https://en.wikipedia.org/wiki/Stepan_Company#Coca_extraction
The “machinery” being the marketing, distribution, and general scale, not the physical vats used to mix it.
Even if you could replicate the exact flavor, you’d still need to do it at scale and get it into stores and convince people to buy it.
As to “just make it yourself”, everyone can make a sandwich, Jersey Mike’s and several other sandwich shops exist successfully.
I don't know if you have seen it, but the recipe is 1:1 of what cocacola is, scientifically proven. (Altho it tastes closer to original cola since replicating coca 1:1 is impossible)
Are you actually reading my responses?
I’m saying the formula is immaterial. It’s also about being able to produce at scale, market, and distribute.
Based on all evidence so far Sam Altman has negative scruples so I'm pretty sure this would be incentivised if anything.
Not really OpenAI but today's society. I say this started with the "just do it and ask for forgiveness later" attitude that started maybe 20 or 30 years ago.
At one time most people had respect, even people at the top, now many people just do what they want. Plus most of the time they get away with it.
One could make the argument that
"just do it and ask for forgiveness later"
is what the entire LLM industry is based on. They swallowed up all of society’s copyrighted texts without really asking for permission from anyone. This is just par for the course for them it seems unfortunately.
One could argue that it is what Trevithick did to make better steam engines, or what Cisco did with routers. Or countless others.
IP theft can really pay off.
Well the asking for forgiveness part is also missing in this case, entirely.
It's actually forbidden, by advice of legal counsel, because in court the other side would present it as evidence of knowing culpability.
In legal speak "please forgive me" is pronounced "so sue me!"
We can fix this by holding each other accountable. Peer pressure and community sanctions of bad actors will result in the changes we need.
> Peer pressure and community sanctions of bad actors will result in the changes we need.
It's been demonstrated over centuries that this is unreliable. We end up needing a third party (commonly governments) to step in and establish rules and referee behavior. We wouldn't have an EPA, FCC, SEC, or many other organizations and laws if private actors could successfully police themselves.
Private actors are inevitably part of groups. Those groups need to hold the individual members accountable. That obviously doesn't work any more in SV among wealth-obsessed tech-bros. It still works in other parts of the economy though.
Guessing you haven't take the HR training - thats not allowed anymore.
Evidently not in that industry.
> Apple alleges that Liu ran a simulation in March using the circuit schematic file in LTspice, an electrical engineering tool. In messages from around that time, Liu said his AI “agent” learned to run LTspice and review the results.
"I didn't steal it, I fed it to an agent who then fed it back to me".
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.
Ok I'm hooked
I'm not arguing for either side, but here's a fun thought experiment I've been thinking about a lot lately. If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets? What if I trained a team of agents to think and code like me at work, then exported those agentic skills and took them with me to another company? At what point would they still be considered the old company's property? Let's say I didn't take any code or proprietary information from the old company, just the coding style and best practices my agents learned from the old company's live metrics. Is that considered stealing from the company? Is that any different from me learning those patterns and skills personally and then reapplying them elsewhere?
There doesn't have to be any meaningful analogy here. Laws don't have to treat humans and machines equally. Laws are for people and allow things that are less than ideal, but necessary for people to function.
Humans can't leave their old brains at their previous employer, can't delete their experience learned on proprietary data. It wouldn't make sense for the law to fight this.
But the law can control what can be done with information stored outside of your brain.
Your training of a team of agents is a product you developed while working for that company and would be owned by them. You could in theory train a new team of agents after leaving the company, but it could not be trained on anything that isn't in your brain. If you gave it anything, like the old company's live metrics, then you'd be stealing trade secrets. The key difference from you personally learning those skills is that an agent trained on company data turns personal knowledge into a portable, structured artifact, which crosses from "your skill" into something that looks a lot more like company property.
This is my personal conclusion as well. But think about where this leads. In the future, agents will do most of the coding and our job is to guide and train them. Most of our skills would be externalized somehow, living in the agents instead of in our heads. Which means once you leave a company, you may have effectively lost a lot of your hard earned skills and experiences.
I'm pretty sure Marx has some things to say on this. When your skills live in company-owned agents instead of in your head, you've been alienated from your own labor in a way Marx predicted. This is exactly why companies like Meta have been so aggressive to integrate AI tools into the day-to-day work for their employees. The more your skills live in their systems, the less you can take with you.
I don't understand the point of your question. The "problem" of an ex-employee consciously or subconsciously bringing his learned experiences at your company to a competitor is literally the reason why non-competes exist. In fact, this topic is quite old.
My real point is this: if you're not allowed to bring your experiences with you, then the company exists to extract your creative thinking and skills. You can argue that's a fair trade for money. I don't necessarily disagree. But I think most of us find joy in learning, in building our own skills, in being able to reapply them elsewhere. And before LLMs and agents, I couldn't lose those skills and experiences no matter where I worked. As we progress, though, we've hit a point where we have to ask ourselves: at what price are we willing to trade complete ownership of our thinking and experiences for money?
It is also illegal in CA.
> If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets?
The fact that California allows this to happen (banning non-competes and rejecting "inevitable disclosure") is exactly why Silicon Valley started here and remains here. It's exactly why ex-OpenAI people could start Anthropic. It's why neither OpenAI nor Anthropic have a monopoly on AI today. It's great both for employees and for the general public.
> At what point would they still be considered the old company's property?
It depends on your contracts (aka NDAs). Sometimes employers do ban for that very same reason from joining a competitor(s) for x years from date of separation
It also depends on your law. In California, some companies still try to put these restrictions into contracts, but that contradicts state law so it has no legal effect. The intended effect is intimidation, however, which may work with some people.
The difference is you’re a human with the right to self.
> If I learned things at one company, took my experience and memory to another company, and applied what I had learned there, am I technically stealing that company's secrets?
Human brains are in many ways privileged agents in a legal and copyright system, because it exists to serve the needs of humans.
Artifacts produced by human brains are not.
You’ve just described how consulting works
Let's do intellectual property from books next!
lol oh no the plagiarism machine got plagiarized!
Apple is essentially arguing that once an LLM system is tainted with IP that the whole system is tainted...
<gets out popcorn>
is it not?
Obviously it is. Just like when you give a monkey a camera and it takes a selfie, the photo is obviously yours. But the legal system doesn't care about obviousness.
I wonder how they trained siri
I thought Siri was just Google Gemini
It's not nearly that simple. Apple used Google frontier models in training Siri, but Siri is not just a pass-through to Gemini.
definitely not by someone asking it about circuit schematics
Has anyone seen Pandora? I want to have a look inside that box she's been carrying around...it's time we start suing the pants off this "generative" stuff...
In IP law, the party with the most money wins.
Leopard ate my face.
Happened at a (large) company I worked for in the past. A devops guy stole the DB ahead of going to work with a rival, he was caught and the rival of course wanted nothing to do with it. I'm not sure of his fate but it couldn't have been much fun.
> the rival of course wanted nothing to do with it
The key difference here seems to be that OpenAI very much wants something to do with it.
That might be the most embarrassing implication for OpenAI out of the whole affair: That OpenAI, supposedly one of the vanguards in AI, saw a need to appropriate intellectual property of Apple, supposedly a complete laggard in that field.
Were they trying to get Apple's AI IP? I haven't followed that closely, but it seems like a lot of it has been around hardware design, not AI.
It's almost laughable how their behavior aligns with the "Scam Altman" snarky insult.
A mature organization does not want the liability and if contacted by previous employer with evidence of these kinds of allegations, immediately gives said employees the boot.
Happens occasionally at big corporations, even the bloodsucking vampire. Best to distance yourself from such employees - lest they will do to you what they have done to others.
The difference is, relative to OpenAi they have standards and morals.
In this case there is a big chance he was hired by OpenAI because of that - or even was asked to do it.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.”
But they've accepted the business model of taking works without permission for training AI; they need that to be fair use.
I know it's unethical, but when I read this, I can't help but hope a pseudo-cleanroom/ai-laundered Linux GPU driver appears for MacBooks. What I wouldn't give to run Linux on my Apple hardware.
that's not unethical at all.
It is cute that anyone thinks intellectual property can be defended much longer. All intelligence will get folded into whichever models have the loosest restrictions on training on IP, and those models will become the smartest and most capable so everyone will use them.
But also like, do not steal IP you could easily have re-created again. Those involved in this scheme were doubly idiots because they stole the IP in brazen ways that were easily traced back to them. Do people really not understand that every corporate laptop and server these days tracks -everything- you do? That corpo paycheck comes at a price of no privacy of any kind.
California has protections on you owning whatever you can take in your brain and these days that is all anyone needs with LLMs being as powerful as they are.
Citadel had an employee take code, and had to use divers to recover it from a canal:
https://www.businessinsider.com/yihao-ben-pu-citadel-2011-11
One of the things I always told kids who came into these big trading firms is the owner(s) is/are a billionaire, and likely came up from one of the trading floors or something adjacent. They are not the “forgiving” types when it comes to shit like this, and they have infinitely more resources than you.
So maybe don’t try to steal code, or trade concepts.
Hasn’t stopped some people I know from trying…
(For clarification I am not ex-CitSec and do not know this poor SOB, but he serves as the perfect poster child for “Don’t do that”)
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning "may create irreversible and continually propagating uses of the trade secret."
Yes. Yes, please make this argument, Apple. Some fascinating other conclusions follow from this.
What are you referring to?
That using AI to create derivative works of proprietary or copyrighted content would count as intellectual property theft.
I'd love that, but that generalizes the argument, and it does not naturally follow that this is true.
It doesn't strictly follow, but it very much naturally (or arguably) follows.
> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.
The warez want to to be free
I wonder how many times this has happened without anyone being caught.
Or being caught, but without sufficiently rigorous evidence to take to court.
I wonder how much rope Apple gave these people to hang themselves with.