In an old Simpsons episode Lisa gets to visit the Teachers room, where all the staff are making fun of the children. Groundskeeper Willie is pantomiming Milhouse “Oh I am Milhouse, I tell all my secrets to Willie since I have no friends!” and the teachers laugh. Later something embarrassing happens to Milhouse and he immediately runs away crying “I have to tell this to Willie!”.
Inequality has eroded trust in society in ~50 years, a lot of the old models (heh) of how we see the world aren't relevant. It's hard to exist when everything around us is adversarial.
I love how some people think the scale of a problem doesnt matter. Like yeah we has data brokerw 50 years ago, way less of them and the scope they could collect data about was significantly more limited. That matters a lot
"A difference in degree becomes a difference in kind" seems to escape too many people when it comes to negative social impact, especially those making money in the scaling business.
Most of YC and VC (and the internet) is built on the math of network effects, but you are correct, so many people here don't think scale differentiates things. It might not always be a distinction with a difference, but in some cases, it clearly is. That's why we have the whole darn concept of network effects lol.
It's not inequality. It's who we've chosen to reward. Adversarial people have eaten a lot of the world and that's because we let them.
People love inequality when it's a celebrity they adore living large. Someone who hasn't scammed them and has demonstrably improved their life and the lives of others in a tangible way. The deeper the con, like Trump and Elon, the more damage to trust.
Yes, the media has conditioned most of us to believe that anyone starting or investing in a tech company and at least a billion dollars in equity is "doing good" and should be emulated.
It’s just inequality full stop. Trust is a word by wannabe-Bernays polsci people who would just so dearly want the proles to trust their betters, but for some dastardly material reasons they don’t.
"multiple providers disclose sensitive conversation-derived artifacts — including titles, prompts, and screenshots — to third parties, often alongside persistent user identifiers that enable user attribution. We also find that some providers publicly expose conversation permalinks without access controls, allowing trackers to read the entire conversation."
Security by obscurity -- such an age old anti-pattern!
I believe many AI tools like Gemini generate publicly accessible URLs when we click "Share" on any chat conversation -- and expect users to then own the lifecycle of that link
Depending on how the link gets handled -- by the browser, device OS, any hooks/plugins/extensions, aggressive telemetry, social media url previews, preload/prefetch, wrapping and url shortening, etc as it reaches the intended user -- there are countless ways in which the URL can be indexed and scraped
There was a issue not long ago when Claude artifacts were indexed en-masse by Google and other search engines
This is shockingly lax approach to data security and privacy by design
The same assumptions are true about giving any human that shareable link. They could pass it on to anyone, screenshot it, paste it into their own session. This has been true since before "share with link" permissions on Docs and elsewhere.
If you click "provide a shareable link" you should decide (and behave) as though that made it public.
I'm not saying it's good privacy posture on the side of the companies, but how else do you think that would work if there isn't any authentication step for the person viewing it? Even with authentication, "three may keep a secret, if two of them are dead."
You already have bigger problems then. Most people have given any number of plugins, etc., access to peek at browser history, clipboards, etc and didn't realize it. Run an ad blocker? VPN? Check out the permissions those sorts of things have on your device.
I want to get rid of whatsapp. Well, since years, but now the need increased.
On android it is possible to give permissions "once" "while using the app" "always".
But whatsapp now only accepts the full camera access. If you set "ask every time" to indeed only make a picture once and then no camera access anymore, it refuses and sends you to the permission dialoge.
I'm shocked... SHOCKED! that these companies would sell this data to advertisers and violate the privacy of their users. Who could have seen this coming??
I’m not sure. It could definitely be incompetence. We are talking about an industry rushing as fast as possible in every directions, their systems are very likely a complete mess. I really wouldn’t be surprised if advertisers are getting that data for free just because the AI labs didn’t bother
It's because of people like you that these companies have a free ride. When we're talking about billion dollar companies there's no such thing as a "mistake", because they need to be held accountable for everything that they do. Thinking otherwise let them keep the rewards while getting away with anything wrong they cause to society.
Thank you. I keep saying this and the spin doctors keep using words that imply "oh, my...I am so sorry we had that tiny little privacy issue...we'll get right on that in the next sprint...". And winning the message war.
It's not a 'leak'. It's in the T&Cs you agreed to. This is the business plan.
I have recently noticed that e.g. ChatGPT, when used from a web browser, periodically sends unfinished prompts to their servers, namely to the `conversation/prepare` endpoint, without waiting for the user to actually send it.
This partial prompt data might potentially be used to "pre-warm" some kind of cache.
But it may also be used to track the user's writing cadence, error correction style and evolution of their stub ideas as they are being formulated into a prompt. I would assume that such data could also be sold to the advertisers.
It's the same lesson as the Navier-Stokes credit fight earlier this month. Buckmaster and Alpoge had their unpublished drafts in private Codex sessions and OpenAI says nobody saw them but admits de-identified product data may have improved its models. There it's training data, here it's ad trackers. Either way, prompts and results that should stay private don't.
Thats why even though open models aren't perfect it has to win. You can skip the app and run the model yourself.
A lot of people seem to be very in denial about the fact that OpenAI and co do not give a crap about you. They don't care about the agreements you've signed. You're just a pile of cash to them
You are currently a cost to them¹ - using your data as free training/refinement, and potentially selling², it is a way to offset that a little.
----
[1] https://isaiprofitable.com/ - some of the green bars are creeping up a bit, but not much unless you count the shovel sellers
[2] sorry, leaking³
[3] Though it could of course be incompetence rather than malice, a mistake they are not actually making anything out of, as per Grey's amendment⁴ to Hanlon's razor.
[4] Any sufficiently advanced incompetence is indistinguishable from malice.
Yes, everyone who has an AI boyfriend or girlfriend should make sure that they use only open source models that nobody can take away. Even if a particular cloud service stops hosting them at least you can make other arrangements. And the nice thing is those other arrangements should generally get cheaper over time, and that's nice, when your significant other gets cheaper over time.
In the modern world, it's just critically important that you exert full control over your SO. You wouldn't want them to run off and leak your most intimate secrets to others without your knowledge or control. I'm sure a lot of us could tell stories about our SOs that we didn't fully control and how they ended up leaking a lot of critical information about us. So full control is definitely mandatory in any SO relationship. It's really just prudent
This is a bit surprising to me, considering how much AI companies love to hoard data. Especially since some of these ad companies are direct competitors!
My best guess is that these ad mechanisms are a bit rushed and/or that investor demands for profitability are fighting against company self-interest.
Edit: I guess some data will always need to be leaked for AI chat ads to be most effective. But I imagine AI companies would rather deliver the targeted ads themselves rather than letting competitors do it for them. It would be scary to see AI companies become ad companies too (instead of just hosting them).
I'm taking an onboarding process for an Ai company helping other (much) bigger Ai companies and the amount of vibecoded platforms and documentation is staggering. Like, training process so broken that the platform just doesn't load sometimes, things that have never even been tried are published and you have to use them and they suck so much because it is apparent that no human ever touched that and probably wouldn't want to
It's a part time job hiring contractors, I'm doing this only because they pay would be nice but I know very well I can be fired anytime or when the project is done...
Anyway I'm a student so I'm not looking for something stable. BTW now it's getting better but if I were the client paying for those services I'd be pretty fucking furious for what's going on - guess they'll never know tho
So I only read the abstract, but the question is if the data is leaked, by accident, or if it's deliberately provided. My guess is that we're talking about the first scenario, and that this is an accident.
If that's the case, then I'm not surprised at all. Actually I also wouldn't be surprised if they sold the data, but that's a different story. If we look at OpenAI for instance, they have on multiple occasion shown that they do not have the operational experience or resources to run their services in a safe and secure manor, nor do they frankly have an impressive up reliability (in terms of operational stability).
I'd support your guess that all of this is rushed in an attempt to push for profitabilitet/growth.
the question is if the data is leaked, by accident, or if it's deliberately provided. My guess is that we're talking about the first scenario, and that this is an accident.
Read the T&Cs. If there's even the tiniest bit of "we might provide your data to third parties for the purposes of...", it's not an accident. Virtually all the AI company T&Cs I've looked at had weasel words that open that door, because it's obvious to anyone paying attention that cramming ads into AI products is the next frontier in AI revenue streams.
Well, some data is deliberately provided at least. It's not a situation of other apps managing to grab the data like the facebook-localhost exploit
>The most prevalent third-party services included in CSP headers belong to Google Tag Manager (googletagmanager.com), Google Analytics (google-analytics.com), and Google Ads (googleadservices.com and doubleclick.net). Yet, as Table 8 shows, CSP policies commonly include other prominent actors in the advertising industry, such as TikTok and Meta.
I don’t think it matters much if it is intentional or not, no? In both cases it’s a breach of privacy that should be punished. Though I myself do not believe they are selling, I don’t think they reached that level of sophistication yet, they seem to be way too hacky for that type of scheme. But that will for sure come later on
You should really assume that any information you give to a private model is going directly into a database. These companies don't care about your privacy. Local, open weight models are the only way to truly protect your data.
Prior to this new AI age, your data was calculated and what was inferred about you was "shallow", but as of today. The sort of profile and things that can be known about you is scary especially if you are constantly engaged with cloud AI. IMO, the number one risk of using cloud AI is loss of privacy and loss of freedom. With AI and capabilities, more controls can be placed on people and the more you put yourself out there, the more you are going to lose.
For example, we now have self driving cars, we have cameras everywhere. Based on your chat with a cloud AI, you can automatically trigger an automatic monitoring event that follows and tracks you in the real world with the fleet of cameras, cars, GPU, cell signal. Your tracking due to AI has moved into the real world and eventually, a self driving car will take you in to be "processed" against your will, not even for what you posted in a public forum, but for your private ribbing and chatting with some cloud AI.
So definitely put local AI into the mix and keep personal stuff and thoughts local only.
Ad-tech spent 20 years trying to infer intent from clickstreams. Chat apps now hand over an AI-written one-line summary of intent, labeled and keyed to a cookie. Are we sure the ads business model in AI is about ads in the chat, and not the chat as the targeting signal?
time for somebody to make a quick script to poison your chat history by starting 1000 fake conversations with contradictory identifying details "I worry as a lesbian woman, my tween daughter doesn't blah blah Shabbat blah blah move home to Australia"
I'd be kind of surprised if OpenAI were really doing this deliberately because a whole bunch of their execs come from Meta, and all those guys learned the hard way.
First: You don't want to leak information about your users to advertising networks because it's going to leak, get back to your customers, they're going to figure out you're doing it and get really angry.
But second and more importantly - it's a much better business model to collect that data for yourself, keep it in house and then you control how you use that data to target ads which gives you a massive competitive advantage in selling ads because you have unique targeting data.
The way meta does this now is the model, they don't give the advertiser a list of the people you're going to show the advert to, the advertiser gives you a list of characteristics they want to hit and meta decides who those people are.
> a whole bunch of their execs come from Meta, and all those guys learned the hard way.
Apparently they have not learned, or they learned the wrong lesson. I do not think they leak it intentionally as hoarding is typically more profitable than selling but anyone who has been in the industry for some time knows that the move fast and break things attitude has caused enormous amounts of data leaks.
> First: You don't want to leak information about your users to advertising networks because it's going to leak, get back to your customers, they're going to figure out you're doing it and get really angry.
And then they will keep using your digital drugs like nothing happened and forget about the whole thing. So watch out, executive!
> But second and more importantly - it's a much better business model to collect that data for yourself, keep it in house and then you control how you use that data to target ads which gives you a massive competitive advantage in selling ads because you have unique targeting data.
Perhaps, but getting to the saturated market on this might be something LLM labs simply don’t have time for. They are haemorrhaging money, no path to profitability and OpenAI especially has made ridiculous promises on data centre -spending for the coming year. They need money now.
> I'd be kind of surprised if OpenAI were really doing this deliberately because a whole bunch of their execs come from Meta, and all those guys learned the hard way.
The lesson they've learned is that if the profit from an activity is X, and the sanctions/reputational hit costs less than X, it's full steam forward.
My stance used to be that the invention of the filter bubble combined with targeted advertising is the most dangerous invention for human society of the last 100 years.
The ads ~~industry~~ racket is a cancer upon civilization.
This shit needs to be stamped down by law.
People take up pitchforks and torches against AI data centers,
well how much time, money, and resources have been wasted on advertisements over the centuries?
How many people, including children, have been deceived by ads?
How much privacy has been violated in the name of "sErViNg YoU rElEvAnT aDs"?
Have you ever tried browsing YouTube from a poor connection, and noticed how long it takes to serve ads? How much bandwidth has been wasted on ads so far?
Entire conversations via exposed permalinks. For Grok: trackers receiving the conversation URL could access the full chat because the link lacked access controls.
Screenshots of conversations. TikTok received screenshots of Grok chats during sharing, exposing the actual visible conversation content.
Conversation-derived content tied to persistent identifiers, including prompts and automatically generated chat titles revealing sensitive facts. "Salary 85k NYC: mortgage 280–350k".
In an old Simpsons episode Lisa gets to visit the Teachers room, where all the staff are making fun of the children. Groundskeeper Willie is pantomiming Milhouse “Oh I am Milhouse, I tell all my secrets to Willie since I have no friends!” and the teachers laugh. Later something embarrassing happens to Milhouse and he immediately runs away crying “I have to tell this to Willie!”.
We have all become Milhouse now.
Inequality has eroded trust in society in ~50 years, a lot of the old models (heh) of how we see the world aren't relevant. It's hard to exist when everything around us is adversarial.
Data brokers were around 50 years ago. They just had more limited sources to draw from.
That's like saying my 1980s casio calculator watch and an apple watch are the same. they're not the same.
I love how some people think the scale of a problem doesnt matter. Like yeah we has data brokerw 50 years ago, way less of them and the scope they could collect data about was significantly more limited. That matters a lot
"A difference in degree becomes a difference in kind" seems to escape too many people when it comes to negative social impact, especially those making money in the scaling business.
Most of YC and VC (and the internet) is built on the math of network effects, but you are correct, so many people here don't think scale differentiates things. It might not always be a distinction with a difference, but in some cases, it clearly is. That's why we have the whole darn concept of network effects lol.
Indeed. The Casio calculator watch was a reliable, ergonomic watch, with a reliable, ergonomic (for its size) calculator.
It's not inequality. It's who we've chosen to reward. Adversarial people have eaten a lot of the world and that's because we let them.
People love inequality when it's a celebrity they adore living large. Someone who hasn't scammed them and has demonstrably improved their life and the lives of others in a tangible way. The deeper the con, like Trump and Elon, the more damage to trust.
Eating the world sounds like an, on the balance, unequal outcome.
Yes. So we should stop enriching people who do that.
Yes, the media has conditioned most of us to believe that anyone starting or investing in a tech company and at least a billion dollars in equity is "doing good" and should be emulated.
This just means they're "doing well". Whether or not they're also "doing good" depends on what exactly that company provides to society.
It’s just inequality full stop. Trust is a word by wannabe-Bernays polsci people who would just so dearly want the proles to trust their betters, but for some dastardly material reasons they don’t.
This can't be real I am go gonna ask Willie
But, Willie isn't real?
No, but now you can talk to a model that pretends to be Willie ;)
"multiple providers disclose sensitive conversation-derived artifacts — including titles, prompts, and screenshots — to third parties, often alongside persistent user identifiers that enable user attribution. We also find that some providers publicly expose conversation permalinks without access controls, allowing trackers to read the entire conversation."
Not good at all.
My least favorite trend I’ve noticed with so many AI chat services is they seem to equate a UUID in the url with privacy.
Perplexity does this. Visiting a past perplexity search url exposes your full conversation.
Security by obscurity -- such an age old anti-pattern!
I believe many AI tools like Gemini generate publicly accessible URLs when we click "Share" on any chat conversation -- and expect users to then own the lifecycle of that link
Depending on how the link gets handled -- by the browser, device OS, any hooks/plugins/extensions, aggressive telemetry, social media url previews, preload/prefetch, wrapping and url shortening, etc as it reaches the intended user -- there are countless ways in which the URL can be indexed and scraped
There was a issue not long ago when Claude artifacts were indexed en-masse by Google and other search engines
This is shockingly lax approach to data security and privacy by design
The same assumptions are true about giving any human that shareable link. They could pass it on to anyone, screenshot it, paste it into their own session. This has been true since before "share with link" permissions on Docs and elsewhere.
If you click "provide a shareable link" you should decide (and behave) as though that made it public.
I'm not saying it's good privacy posture on the side of the companies, but how else do you think that would work if there isn't any authentication step for the person viewing it? Even with authentication, "three may keep a secret, if two of them are dead."
Chat UIs are a minefield of “if you accidentally click this your data will be shared or trained without you realizing it!”
Isn't that equivalent to a password? Knowing my password exposes my full data.
You don't store your password in the URL.
I store my session token in a cookie, which is even worse because it's sent with every request.
It's not. The cookie only gets sent to the domains/servers you specify and is not accidentally exposed via browser history or copying a link.
Not in a URL generally, and if it is the only people who can see the full URL are the receiver and the sender if HTTPS is properly enabled.
Genuinely asking: If you don’t share the UUID-based URL yourself, what makes it not privacy-friendly?
It’s not like someone’s gonna guess that URL… right?
Yes, technically, guessing a url is impossible. But browser histories are stored in cleartext and trivially accessible to sketchy actors.
I also accidentally paste random stuff into input boxes all the time.
Good points, thanks.
Although I think at the point of some on-device program reading your browser history against your will, you’re gonna have bigger problems.
You already have bigger problems then. Most people have given any number of plugins, etc., access to peek at browser history, clipboards, etc and didn't realize it. Run an ad blocker? VPN? Check out the permissions those sorts of things have on your device.
I want to get rid of whatsapp. Well, since years, but now the need increased.
On android it is possible to give permissions "once" "while using the app" "always".
But whatsapp now only accepts the full camera access. If you set "ask every time" to indeed only make a picture once and then no camera access anymore, it refuses and sends you to the permission dialoge.
“Screenshots”
The amount of sensitive information that accidentally gets left on screenshots is pretty large. This is a pretty massive security issue
I'm shocked... SHOCKED! that these companies would sell this data to advertisers and violate the privacy of their users. Who could have seen this coming??
No way will your future spouse be an AI entomed robot, with a percentage of your salary endowed to the corporate owner of the hardware.
No, they don't "leak data", data is sold. Leaking data requires a mistake. This is intentional.
I’m not sure. It could definitely be incompetence. We are talking about an industry rushing as fast as possible in every directions, their systems are very likely a complete mess. I really wouldn’t be surprised if advertisers are getting that data for free just because the AI labs didn’t bother
It's because of people like you that these companies have a free ride. When we're talking about billion dollar companies there's no such thing as a "mistake", because they need to be held accountable for everything that they do. Thinking otherwise let them keep the rewards while getting away with anything wrong they cause to society.
Just because something is due to incompetence instead of malice doesn't mean they shouldn't be held accountable.
Thank you. I keep saying this and the spin doctors keep using words that imply "oh, my...I am so sorry we had that tiny little privacy issue...we'll get right on that in the next sprint...". And winning the message war.
It's not a 'leak'. It's in the T&Cs you agreed to. This is the business plan.
Tangential:
I have recently noticed that e.g. ChatGPT, when used from a web browser, periodically sends unfinished prompts to their servers, namely to the `conversation/prepare` endpoint, without waiting for the user to actually send it.
This partial prompt data might potentially be used to "pre-warm" some kind of cache.
But it may also be used to track the user's writing cadence, error correction style and evolution of their stub ideas as they are being formulated into a prompt. I would assume that such data could also be sold to the advertisers.
I wouldn't be surprised if their privacy policy would say "what you send is private" and then it wouldn't apply to unfinished prompts on technicality
It's the same lesson as the Navier-Stokes credit fight earlier this month. Buckmaster and Alpoge had their unpublished drafts in private Codex sessions and OpenAI says nobody saw them but admits de-identified product data may have improved its models. There it's training data, here it's ad trackers. Either way, prompts and results that should stay private don't. Thats why even though open models aren't perfect it has to win. You can skip the app and run the model yourself.
A lot of people seem to be very in denial about the fact that OpenAI and co do not give a crap about you. They don't care about the agreements you've signed. You're just a pile of cash to them
> You're just a pile of cash to them
You are currently a cost to them¹ - using your data as free training/refinement, and potentially selling², it is a way to offset that a little.
----
[1] https://isaiprofitable.com/ - some of the green bars are creeping up a bit, but not much unless you count the shovel sellers
[2] sorry, leaking³
[3] Though it could of course be incompetence rather than malice, a mistake they are not actually making anything out of, as per Grey's amendment⁴ to Hanlon's razor.
[4] Any sufficiently advanced incompetence is indistinguishable from malice.
> A lot of people seem to be very in denial about the fact that OpenAI and co do not give a crap about you
Who thinks OpenAI or any big company for that matter give a crap about them? This isn't a popular sentiment at all, it's just patently false.
All those people with AI friends, boyfriends, girlfriends.
Yes, everyone who has an AI boyfriend or girlfriend should make sure that they use only open source models that nobody can take away. Even if a particular cloud service stops hosting them at least you can make other arrangements. And the nice thing is those other arrangements should generally get cheaper over time, and that's nice, when your significant other gets cheaper over time.
In the modern world, it's just critically important that you exert full control over your SO. You wouldn't want them to run off and leak your most intimate secrets to others without your knowledge or control. I'm sure a lot of us could tell stories about our SOs that we didn't fully control and how they ended up leaking a lot of critical information about us. So full control is definitely mandatory in any SO relationship. It's really just prudent
wait something's gone wrong here
> You're just a pile of cash to them
I thought I was the lazy, irrational, inferior human filth whose job they were replacing.
You're both, which is why none of this makes sense and is yet somehow "inevitable."
That’s not fair, we are also more free training and refining of their models.
This is a bit surprising to me, considering how much AI companies love to hoard data. Especially since some of these ad companies are direct competitors!
My best guess is that these ad mechanisms are a bit rushed and/or that investor demands for profitability are fighting against company self-interest.
Edit: I guess some data will always need to be leaked for AI chat ads to be most effective. But I imagine AI companies would rather deliver the targeted ads themselves rather than letting competitors do it for them. It would be scary to see AI companies become ad companies too (instead of just hosting them).
I'm taking an onboarding process for an Ai company helping other (much) bigger Ai companies and the amount of vibecoded platforms and documentation is staggering. Like, training process so broken that the platform just doesn't load sometimes, things that have never even been tried are published and you have to use them and they suck so much because it is apparent that no human ever touched that and probably wouldn't want to
Doesn’t sound like your new company is going to be very helpful for other AI companies lol
It's a part time job hiring contractors, I'm doing this only because they pay would be nice but I know very well I can be fired anytime or when the project is done...
Anyway I'm a student so I'm not looking for something stable. BTW now it's getting better but if I were the client paying for those services I'd be pretty fucking furious for what's going on - guess they'll never know tho
AI companies rushing an implementation? Surely not :).
No no it’s hyperscaling
hyperscaling == lucratively funded adult summer camp
hyperleaking
So I only read the abstract, but the question is if the data is leaked, by accident, or if it's deliberately provided. My guess is that we're talking about the first scenario, and that this is an accident.
If that's the case, then I'm not surprised at all. Actually I also wouldn't be surprised if they sold the data, but that's a different story. If we look at OpenAI for instance, they have on multiple occasion shown that they do not have the operational experience or resources to run their services in a safe and secure manor, nor do they frankly have an impressive up reliability (in terms of operational stability).
I'd support your guess that all of this is rushed in an attempt to push for profitabilitet/growth.
the question is if the data is leaked, by accident, or if it's deliberately provided. My guess is that we're talking about the first scenario, and that this is an accident.
Read the T&Cs. If there's even the tiniest bit of "we might provide your data to third parties for the purposes of...", it's not an accident. Virtually all the AI company T&Cs I've looked at had weasel words that open that door, because it's obvious to anyone paying attention that cramming ads into AI products is the next frontier in AI revenue streams.
Well, some data is deliberately provided at least. It's not a situation of other apps managing to grab the data like the facebook-localhost exploit
>The most prevalent third-party services included in CSP headers belong to Google Tag Manager (googletagmanager.com), Google Analytics (google-analytics.com), and Google Ads (googleadservices.com and doubleclick.net). Yet, as Table 8 shows, CSP policies commonly include other prominent actors in the advertising industry, such as TikTok and Meta.
I don’t think it matters much if it is intentional or not, no? In both cases it’s a breach of privacy that should be punished. Though I myself do not believe they are selling, I don’t think they reached that level of sophistication yet, they seem to be way too hacky for that type of scheme. But that will for sure come later on
You should really assume that any information you give to a private model is going directly into a database. These companies don't care about your privacy. Local, open weight models are the only way to truly protect your data.
Prior to this new AI age, your data was calculated and what was inferred about you was "shallow", but as of today. The sort of profile and things that can be known about you is scary especially if you are constantly engaged with cloud AI. IMO, the number one risk of using cloud AI is loss of privacy and loss of freedom. With AI and capabilities, more controls can be placed on people and the more you put yourself out there, the more you are going to lose.
For example, we now have self driving cars, we have cameras everywhere. Based on your chat with a cloud AI, you can automatically trigger an automatic monitoring event that follows and tracks you in the real world with the fleet of cameras, cars, GPU, cell signal. Your tracking due to AI has moved into the real world and eventually, a self driving car will take you in to be "processed" against your will, not even for what you posted in a public forum, but for your private ribbing and chatting with some cloud AI.
So definitely put local AI into the mix and keep personal stuff and thoughts local only.
Ad-tech spent 20 years trying to infer intent from clickstreams. Chat apps now hand over an AI-written one-line summary of intent, labeled and keyed to a cookie. Are we sure the ads business model in AI is about ads in the chat, and not the chat as the targeting signal?
It’s not a leak if it’s the business model.
time for somebody to make a quick script to poison your chat history by starting 1000 fake conversations with contradictory identifying details "I worry as a lesbian woman, my tween daughter doesn't blah blah Shabbat blah blah move home to Australia"
ask another llm to generate those prompts for you
I'd be kind of surprised if OpenAI were really doing this deliberately because a whole bunch of their execs come from Meta, and all those guys learned the hard way.
First: You don't want to leak information about your users to advertising networks because it's going to leak, get back to your customers, they're going to figure out you're doing it and get really angry.
But second and more importantly - it's a much better business model to collect that data for yourself, keep it in house and then you control how you use that data to target ads which gives you a massive competitive advantage in selling ads because you have unique targeting data.
The way meta does this now is the model, they don't give the advertiser a list of the people you're going to show the advert to, the advertiser gives you a list of characteristics they want to hit and meta decides who those people are.
> a whole bunch of their execs come from Meta, and all those guys learned the hard way.
Apparently they have not learned, or they learned the wrong lesson. I do not think they leak it intentionally as hoarding is typically more profitable than selling but anyone who has been in the industry for some time knows that the move fast and break things attitude has caused enormous amounts of data leaks.
> First: You don't want to leak information about your users to advertising networks because it's going to leak, get back to your customers, they're going to figure out you're doing it and get really angry.
And then they will keep using your digital drugs like nothing happened and forget about the whole thing. So watch out, executive!
> But second and more importantly - it's a much better business model to collect that data for yourself, keep it in house and then you control how you use that data to target ads which gives you a massive competitive advantage in selling ads because you have unique targeting data.
Perhaps, but getting to the saturated market on this might be something LLM labs simply don’t have time for. They are haemorrhaging money, no path to profitability and OpenAI especially has made ridiculous promises on data centre -spending for the coming year. They need money now.
> And then they will keep using your digital drugs like nothing happened and forget about the whole thing. So watch out, executive!
Not only this, many will continue to help the company by bullying anyone who decides to stop using the drugs.
The only thing they learned from Meta is that this does work!
> I'd be kind of surprised if OpenAI were really doing this deliberately because a whole bunch of their execs come from Meta, and all those guys learned the hard way.
The lesson they've learned is that if the profit from an activity is X, and the sanctions/reputational hit costs less than X, it's full steam forward.
Why do you think they are fighting so hard to dethrone Google?
Google is an ad company.
The word is "sell" not "leak". This title takes away all agency from the thieves selling private data to advertisers.
The personal data fell off the back of a truck, man.
Not surprised. Just wait until they bake the goddamn advertising right into the model.
This is why I built my own chat interface. https://ai.ivx.run/
Access through: https://ai.ivx.run/chat/
You thought... they didn't?
All sells but I saw Chinese models are upfront about that most of the time.
Is it still called a leak if it was the whole point and purpose of the deal?
Someone should have to investigate, but I suppose it's all "legal"?
In the US.
In EU law it is very likely against GDPR.
Yea no shit sherlock!
My stance used to be that the invention of the filter bubble combined with targeted advertising is the most dangerous invention for human society of the last 100 years.
AI turns that up to 11
I never thought this could happen (XD)
Hard to read on mobile. Is there a tldr of how bad each provider is?
I am very surprised.
Insert surprised pikachu meme
my first guess is always Gboard.
The paper doesn't say when the app sends the conversion artifact.
Oh no, what, what happened?
What happened? Oh no!
How terrible! That’s just, that’s just awful!
How terrible! Oh no!
Hanlon's Razor given that these tools are almost certainly vibe coded at this point?
Incompetence and carelessness is rampant in our industry abd with vibe coding it has only gotten worse.
data is sold i think
The ads ~~industry~~ racket is a cancer upon civilization.
This shit needs to be stamped down by law.
People take up pitchforks and torches against AI data centers,
well how much time, money, and resources have been wasted on advertisements over the centuries?
How many people, including children, have been deceived by ads?
How much privacy has been violated in the name of "sErViNg YoU rElEvAnT aDs"?
Have you ever tried browsing YouTube from a poor connection, and noticed how long it takes to serve ads? How much bandwidth has been wasted on ads so far?
Who[m] does it all benefit?
these companies are complete dumpster fires and we just keep giving them more to burn
Entire conversations via exposed permalinks. For Grok: trackers receiving the conversation URL could access the full chat because the link lacked access controls.
Screenshots of conversations. TikTok received screenshots of Grok chats during sharing, exposing the actual visible conversation content.
Conversation-derived content tied to persistent identifiers, including prompts and automatically generated chat titles revealing sensitive facts. "Salary 85k NYC: mortgage 280–350k".
That is a staggering level of incompetence.
what do you want it's the era of vibecoding.
Of course they do. As soon as ads entered the equation this was always going to happen. Nobody should be suprised.