Interesting article. I’ve been curious for a while about how residential proxy IPs are collected too. Many come from shady browser extensions or mobile apps, especially free VPNs (wink wink Hola VPN). People often don’t realize they are turning their device into an exit node.
Some time ago I started to track this as a side project (I work in bot detection and was always surprised by how many residential proxies show up in attacks). It started just out of curiosity. Now I collect proxy IPs, which provider they belong to, and how often they are seen. I also publish stats here:
https://deviceandbrowserinfo.com/proxy-api/stats/proxy-db-30...
For example, in the last 30 days I saw more than 120K IPs from Comcast and nearly 100K from AT&T.
I also maintain an open IP (ranges) blocklist, mostly effective against data center and ISP proxies. Residential IPs are harder since they are often shared with legit users:
https://github.com/antoinevastel/avastel-bot-ips-lists
Even if you can’t block all of them, tracking volume and reuse gives useful signal.
Hola/Luminati rebranded as “Bright Data” and now pays mobile developers to embed their proxy SDK into mobile apps. Apple and Google should put a stop to this practice.
hola vpn is such an interesting case of a money printer, host a simple vpn and present it as free, give the users datacenter ips that are easy to detect. meanwhile you get their precious residential ip's and print millions a month
Thanks for the great read, so much to unpack from that article the click fraud stuff is to be expected, keeping track of everything that goes through their proxy is also expected, but copying files is crazy and this could unravel to a class action
but with that being said, if you are doing something shady/grey area to get ahead you best give everyone a cut of the pie, especially your blood brother
I would add that your chances of having a proxy node increase by 1% with each free app you install these days. We catch them easily at visitorquery.com but the residential proxy business in rampant and probably half are infected devices, android TVs, routers and, ofc, mobile apps.
On the one hand, the guy makes it sound like it 'spawns cmd prompts' which suggests a Windows machine and a bunch of amateurs selling crap to third parties (and to the state), instead of being a state level actor. (which shouldn't be able to gather that much valuable metadata by spying on the network anyways)
On the other hand, 250$ is a suspiciously high number when you can get a dozen people to do it for 50$ in an afternoon.
ps. "top secret" clearing is a not secret club - it's a very big club and its practical purpose is you agreeing to increase legal liability by getting thrown into a different judicial tract if you screw up - eg by installing Russian hardware on your home.
What happens once these residential IPs end up on CSAM or terrorism or IP infringement lists?
I ran a proxy in ~1996 so students could MUD from restricted uni shells, but one weekend I went to visit my parents and there was a knock on the door and a smartly dressed man interrogated me about a plot to assassinate Clinton. (he was Special Branch sent on behalf of the Secret Service and FBI)
Residential proxy botnets have exploded since LLMs became a thing. The amount of DDoS-level scraping we receive from residential IPs has exploded over the last year, one of our sites that typically sees around 10k unique IPs per day jumped to over 2M before we were able to deploy appropriate mitigations. We originally started blocking the IPs, but then we ended up blocking legitimate users as they seem to specifically use ISPs that have very dynamic IPs (i.e. the customer's IP will change even if their router stays on 24/7).
At first they were easily detectable using HTTP header analysis - e.g. pretending to be Chrome but not sending the headers that Chrome always sends. Now it's a combination of TLS / HTTP protocol level analysis and application layer - e.g. we send a cookie on the user's "normal" page view and check it exists on the higher-resource usage pages they might later visit - the bots don't care about normal viewing patterns and try to hit the higher-resource pages on their first visit, so they get blocked.
If you have a product worth buying, it's also worth stealing.
The existence of residential proxies like these is a massive pain if you run free trials or giveaways or host user-generated content (aka a spam/scam opportunity). DSLRoot is only one service of many (see last year's takedown of 911 S5 https://www.scworld.com/news/fbi-takes-down-911-s5-botnet-li... ) and there's plenty of demand for it.
Imagine getting hit by thousands+ of different IP addresses with different user agents, etc. Banning these IPs is not a great option - lots of collateral damage because many real people share IPs, depending on ISP setup.
I work on bot detection involving device fingerprinting - imo this is one of the only ways to defend against residential proxy activity, since you can sniff out the warning flags of automation software and other shared indicators regardless of IP.
>Imagine getting hit by thousands+ of different IP addresses with different user agents, etc.
If I open the gates, I can see oodles of connections from China or Singapore in my server logs, all from different IP addresses but all allegedly (according to their USER_AGENT) from iphones with identical software versions.
Maybe these are infected apps on actual iphones, maybe they are scrapers purporting to be iphones, but one thing is sure: the good old internet isn't any more.
>I work on bot detection involving device fingerprinting
Yikes, this can become a slippery slop towards surveillance state very quickly with these type of authentication or human verification. Kinda like what the invisible pixel thing on steroid, but event more intrusive and harder to evade.
Yes, thanks for bringing this up. We've made product decisions to improve bot detection that also move away from adtech-style tracking - happy to chat about the specifics privately, bchen at stytch dot com.
Related, I have a fairly unusual setup for my personal laptop and that makes many anti-bot products Very Unhappy (same for many of my teammates). It's easy to detect users who dare to run something other than stock Chrome/Safari, but it's disappointing that many services penalize you for it. We designed Intelligent Rate Limiting so that real users on unusual setups aren't blocked: https://stytch.com/docs/fraud/guides/device-fingerprinting/d...
> “When I open the computer, it looks like [they] have some sort of custom application that runs and spawns several cmd prompts,” the Redditor explained. “All I can infer from what I see in them is they are making connections.”
Surprised me that the laptop seemingly wasn't even password protected.
Seems like easy money for slumlords that don't live on site or young people who have roomates the internet is under. Throw a laptop in the attic/basement. Buy it it's own dedicated line from Comcast or whatever. From there you're basically being paid $250/mo to be willing to say "I have no idea, I set it up and forgot about it, I assume it's so Europoors can watch the NFL without paying out the nose" to the cops at some undetermined point in the future.
I've seen way too many crypto adjacent "legal botnets" too and I dont think you could pay me enough to run some sketchy app like that, even if open source, even if fully vlan'd and isolated.
Unfortunately theres a lot of desperate people who will run random apps thinking it'll make them a quick buck.
Interesting article. I’ve been curious for a while about how residential proxy IPs are collected too. Many come from shady browser extensions or mobile apps, especially free VPNs (wink wink Hola VPN). People often don’t realize they are turning their device into an exit node.
Some time ago I started to track this as a side project (I work in bot detection and was always surprised by how many residential proxies show up in attacks). It started just out of curiosity. Now I collect proxy IPs, which provider they belong to, and how often they are seen. I also publish stats here: https://deviceandbrowserinfo.com/proxy-api/stats/proxy-db-30...
For example, in the last 30 days I saw more than 120K IPs from Comcast and nearly 100K from AT&T.
I also maintain an open IP (ranges) blocklist, mostly effective against data center and ISP proxies. Residential IPs are harder since they are often shared with legit users: https://github.com/antoinevastel/avastel-bot-ips-lists
Even if you can’t block all of them, tracking volume and reuse gives useful signal.
Hola/Luminati rebranded as “Bright Data” and now pays mobile developers to embed their proxy SDK into mobile apps. Apple and Google should put a stop to this practice.
they have been paying devs for a good bit now
hola vpn is such an interesting case of a money printer, host a simple vpn and present it as free, give the users datacenter ips that are easy to detect. meanwhile you get their precious residential ip's and print millions a month
The recent feud between founders is bound to reveal more interesting aspects of their business: https://www.haaretz.com/israel-news/tech-news/2021-07-01/ty-... / https://archive.vn/o5ujG
Thanks for the great read, so much to unpack from that article the click fraud stuff is to be expected, keeping track of everything that goes through their proxy is also expected, but copying files is crazy and this could unravel to a class action
but with that being said, if you are doing something shady/grey area to get ahead you best give everyone a cut of the pie, especially your blood brother
I would add that your chances of having a proxy node increase by 1% with each free app you install these days. We catch them easily at visitorquery.com but the residential proxy business in rampant and probably half are infected devices, android TVs, routers and, ofc, mobile apps.
> I work in bot detection and was always surprised by how many residential proxies show up in attacks
Why is that surprising? It seems like it'd be one of the major vectors.
On the one hand, the guy makes it sound like it 'spawns cmd prompts' which suggests a Windows machine and a bunch of amateurs selling crap to third parties (and to the state), instead of being a state level actor. (which shouldn't be able to gather that much valuable metadata by spying on the network anyways)
On the other hand, 250$ is a suspiciously high number when you can get a dozen people to do it for 50$ in an afternoon.
ps. "top secret" clearing is a not secret club - it's a very big club and its practical purpose is you agreeing to increase legal liability by getting thrown into a different judicial tract if you screw up - eg by installing Russian hardware on your home.
What happens once these residential IPs end up on CSAM or terrorism or IP infringement lists?
I ran a proxy in ~1996 so students could MUD from restricted uni shells, but one weekend I went to visit my parents and there was a knock on the door and a smartly dressed man interrogated me about a plot to assassinate Clinton. (he was Special Branch sent on behalf of the Secret Service and FBI)
they query netflow from Team Cymru
Residential proxy botnets have exploded since LLMs became a thing. The amount of DDoS-level scraping we receive from residential IPs has exploded over the last year, one of our sites that typically sees around 10k unique IPs per day jumped to over 2M before we were able to deploy appropriate mitigations. We originally started blocking the IPs, but then we ended up blocking legitimate users as they seem to specifically use ISPs that have very dynamic IPs (i.e. the customer's IP will change even if their router stays on 24/7).
Mind sharing what kind of mitigations you put in place and how well they worked?
At first they were easily detectable using HTTP header analysis - e.g. pretending to be Chrome but not sending the headers that Chrome always sends. Now it's a combination of TLS / HTTP protocol level analysis and application layer - e.g. we send a cookie on the user's "normal" page view and check it exists on the higher-resource usage pages they might later visit - the bots don't care about normal viewing patterns and try to hit the higher-resource pages on their first visit, so they get blocked.
Can you give some examples of these ISPs?
If you have a product worth buying, it's also worth stealing.
The existence of residential proxies like these is a massive pain if you run free trials or giveaways or host user-generated content (aka a spam/scam opportunity). DSLRoot is only one service of many (see last year's takedown of 911 S5 https://www.scworld.com/news/fbi-takes-down-911-s5-botnet-li... ) and there's plenty of demand for it.
Imagine getting hit by thousands+ of different IP addresses with different user agents, etc. Banning these IPs is not a great option - lots of collateral damage because many real people share IPs, depending on ISP setup.
I work on bot detection involving device fingerprinting - imo this is one of the only ways to defend against residential proxy activity, since you can sniff out the warning flags of automation software and other shared indicators regardless of IP.
>Imagine getting hit by thousands+ of different IP addresses with different user agents, etc.
If I open the gates, I can see oodles of connections from China or Singapore in my server logs, all from different IP addresses but all allegedly (according to their USER_AGENT) from iphones with identical software versions.
Maybe these are infected apps on actual iphones, maybe they are scrapers purporting to be iphones, but one thing is sure: the good old internet isn't any more.
>I work on bot detection involving device fingerprinting
Yikes, this can become a slippery slop towards surveillance state very quickly with these type of authentication or human verification. Kinda like what the invisible pixel thing on steroid, but event more intrusive and harder to evade.
"Please drink verification can."
Yes, thanks for bringing this up. We've made product decisions to improve bot detection that also move away from adtech-style tracking - happy to chat about the specifics privately, bchen at stytch dot com.
Related, I have a fairly unusual setup for my personal laptop and that makes many anti-bot products Very Unhappy (same for many of my teammates). It's easy to detect users who dare to run something other than stock Chrome/Safari, but it's disappointing that many services penalize you for it. We designed Intelligent Rate Limiting so that real users on unusual setups aren't blocked: https://stytch.com/docs/fraud/guides/device-fingerprinting/d...
> “When I open the computer, it looks like [they] have some sort of custom application that runs and spawns several cmd prompts,” the Redditor explained. “All I can infer from what I see in them is they are making connections.”
Surprised me that the laptop seemingly wasn't even password protected.
Probably makes it easier if they brick it and lose remote access and have to ask the person running it to enter some commands to fix it.
It's not like a proxy server is anything secret worth protecting.
Maybe. Or the whole story is fake, could also be since it's reddit after all.
They mentioned helping to “trouble shoot connectivity issues when they arise”, which might require access to the laptop UI.
It is so easy to pay a college student to get them to whitelist a MAC address for a GLiNet router you install somewhere in a university.
JSTOR has entered the chat
Seems like easy money for slumlords that don't live on site or young people who have roomates the internet is under. Throw a laptop in the attic/basement. Buy it it's own dedicated line from Comcast or whatever. From there you're basically being paid $250/mo to be willing to say "I have no idea, I set it up and forgot about it, I assume it's so Europoors can watch the NFL without paying out the nose" to the cops at some undetermined point in the future.
I've seen way too many crypto adjacent "legal botnets" too and I dont think you could pay me enough to run some sketchy app like that, even if open source, even if fully vlan'd and isolated.
Unfortunately theres a lot of desperate people who will run random apps thinking it'll make them a quick buck.
They pay developers of popular apps to add their botnet code to the app for money. As a result the app becomes botnet on next update.