Glad that it's published, I'd been following it since ESNI draft days. Was pretty useful back when I was in India since Jio randomly blocked websites, and cloudflare adopted the ESNI draft on its servers as did Firefox client side which made their SNI based blocking easy to bypass.
There was a period where I think both disabled ESNI support as work was made on ECH, which now is pretty far along. I was even able to setup a forked nginx w/ ECH support to build a client(browser) tester[0].
Hopefully now ECH can get more mainstream in HTTPS servers allowing for some fun configs.
A pretty interesting feature of ECH is that the server does not need to validate the public name (it MAY) , so clients can use public_name's that middleboxes (read: censors) approve to connect to other websites. I'm trying to get this added to the RustTLS client[1], now might be a good time to pick that back up.
The server can also advertise a public name that doesn't match any domain it has a TLS certificate for, like example.com or nsa.gov.
I'm not 100% sure it's allowed in the specs, but it works in Chrome.
As I understand it, without this feature it would be pretty useless for small website owners, since they would need to register a separate domain for their ECH public name, which censors could just block.
If i'm not mistaken its because IPs are actually much easier to rotate than domains.
E.g. all the users will remember `example.com` , underlying it doesn't matter what IP it resolves to. If the IP gets "burned" , then the providers can rotate to a new IP (if their provider allows).
Vs. telling your users to use a new domain `example.org` , fake websites etc.
Also sensible ISPs usually don't block IPs since for services behind a CDN it could lead to other websites being blocked, though of course sometimes this is ignored. See also: https://blog.cloudflare.com/consequences-of-ip-blocking/
I wouldn't say you're mistaken, but it's a simplification. In the network world, the capability exists to restrict what BGP advertisements are accepted via RPKI/a peer. Internet providers usually don't because the premium is placed on uptime/connectivity.
If tomorrow, everyone said "we don't want IP's from Frankfurt showing up somewhere in Dubai", you'd have a massive technical problem and rearranging to start with but once that was sorted you could geo-lock. IANA and Network providers simply haven't been doing that.
The reason it doesn't happen is Devs/Stakeholders want uptime from ISPs/Networks and not something they can't abstract. Basically its just a status quo much like the entire internet reverse-proxying through CDNs is a status quo. It wasn't always like that, and it may not always be like that in the future - just depends which way the winds blow over time.
> we don't want IP's from Frankfurt showing up somewhere in Dubai
what do you mean, IPs from Frankfurt?
IP addresses are just IP addresses, they know no geographical boundaries. In RIR DBs you can geolocate them to wherever you want. Which is the entire reason why Geo IP DBs even exist - they triangulate.
They have registration data. Someone could declare they don't want IPs registered to companies from Frankfurt with geofeeds in Frankfurt to be advertised in Dubai.
How do you determine to whom an IP is even registered to? They get sub-leased all the time.
The best you can do is check who has administrative control over the prefixes RIR info, but that doesn’t mean that anyone with control is the factual user of the IPs.
You could check the IRR for the ASN and base it on that, but still.
There's also no way to actually know _where_ an IP actually originates from. Only its AS path.
The DFZ contains all prefixes announced everywhere, for the internet is completely decentralized.
> Was pretty useful back when I was in India since Jio randomly blocked websites
With Jio, you don't really need ECH at all. The blocks are mostly rudimentary and bypassed with encrypted DNS (DoH / DoT / DNSCrypt) and Firefox (which fragments the TLS ClientHello packets into two).
Should've added this was back in like 2018 or so. Setting up DoH was harder than enabling SNI, and from my testing back then they were hard filtering on SNI (e.g. I used OpenSSL CLI to set the SNI to `pornhub.com` and connect to "known good" IPs, it'd still get reset).
Funnily enough, not setting the SNI and connecting the the origin IP, and then requesting the page worked fine.
> Funnily enough, not setting the SNI and connecting the the origin IP, and then requesting the page worked fine.
Such tricks, called "domain fronting" are why ECH exists. The problem is that although domain fronting is effective for the client it's a significant headache for the provider. Big providers involved, such as Cloudflare have always insisted that they want to provide this sort of censorship resisting capability but they don't want to authorize domain fronting because it's a headache for them technically.
Let me explain the headache with an example. Say I'm Grand Corp, a French company with 25 million web sites including both cats-are-great.example and fuck-trump.example. Users discover that although the US government has used Emergency Powers to prohibit access to fuck-trump.example, using domain fronting they can connect to cats-are-great.example and request fuck-trump.example pages anyway and the US government's blocking rules can't stop them.
What they don't know is that I, Grand Corp had been sharding sites 25 ways, so there was only 1-in-25 chance that this worked - it so happened cats-are-great and fuck-trump were in the same shard, On Thursday during routine software upgrade we happen to switch to 32-way sharding and suddenly it stops working - users are outraged, are the French surrendering to Donald Trump?
Or, maybe as a fallback mechanism the other 31 servers can loop back around to fetch your fuck-trump.example pages from the server where they live, but in doing so they double the effective system load. So now my operational costs at Grand Corp for fuck-trump.example doubled because clients were fronting. Ouch.
That's true, usually with domain fronting you provide the (wrong) SNI. But the same strategy is happening here, you were supposed to provide SNI and you didn't to avoid some potential censorship but it's a headache for the provider
They won't have received a certificate for the IP as a name, it's relatively unusual to have those, the main users are things like DoH and DoT servers since their clients may not know the name of the server... historically if you connect to a TLS server without SNI it just picks a name and presents a certificate for that name - if there's a single name for the machine that definitely works, and if not well - domain fronting.
TLS 1.3 even specifies that you must always do SNI and shouldn't expect such tricks to work, because it's such a headache.
> A pretty interesting feature of ECH is that the server does not need to validate the public name (it MAY) , so clients can use public_name's that middleboxes (read: censors) approve to connect to other websites. I'm trying to get this added to the RustTLS client[1], now might be a good time to pick that back up.
Note that it is exactly this type of thing that makes age verification laws reasonable. You're making it technically impossible for even sophisticated parents to censor things without a non-solution like "don't let kids use a computer until they're 18", so naturally the remaining solution is a legal one to put liability on service operators.
You're still ultimately going to get the censorship when the law catches up in whatever jurisdiction, but you'll also provide opacity for malware (e.g. ad and tracking software) to do its thing.
This is exactly reverse of the right idea. If parents need to censor things the solutions are the same as corpos are going to. Put the censors at the device or “mitm” the connection, either actually with a proxy, or maybe with a browser and curated apps - which is again on the device.
This brings us back to "sure you can use my guest wifi, just install my root CA/enroll in MDM".
I do agree though that it should be illegal for device manufacturers or application developers to use encryption that the device owner cannot MitM. The owner should always be able to install their own CA and all applications should be required to respect it.
Why would you want to censor based on network? You don't want to censor based on network, you want to censor based on device. If your 8yo kid is blocked from pornhub, that doesn't mean everyone on your network is blocked from pornhub, and you having the ability to even know if someone on your network is browsing pornhub is a security risk.
How does ECH make it impossible for parents to control their children's access to computers? Sure they can't block sites at the router level, just like your ISP won't be able to block things at the ISP level, but you (the parent) have physical access to the devices in question, and can install client-side software to filter access to the internet.
The only thing this makes impossible is the laziest, and easiest to bypass method of filtering the internet.
Because there are network operators who have mal-intent increasingly no network operators are permitted to exercise network-level control. A parent who wants to filter the network access in their house is the same as a despotic regime practicing surveillance and censorship on their citizens.
Given that it's pretty much the norm that consumer embedded devices don't respect the owner's wishes network level filtering is the best thing a device owner can do on their own network.
It's a mess.
I'd like to see consumer regulation to force manufacturers to allow owners complete control over their devices. Then we could have client side filtering on the devices we own.
I can't imagine that will happen. I suspect what we'll see, instead, is regulation that further removes owner control of their devices in favor of baking ideas like age or identity verification directly into embedded devices.
Then they'll come for the unrestricted general purpose computers.
If you have a device you don't trust, don't allow it on your network, or have an isolated network for such devices. Meanwhile, devices are right to not allow MITMing their traffic and to treat that as a security hole, even if a very tiny fraction of their users might want to MITM it to try to do adblocking on a device they don't trust or fully control, rather than to exploit the device and turn it into a botnet.
Along similar lines, a security hole you can use for jailbreaking is also a security hole that could potentially be exploited by malware. As cute as things like "visit this webpage and it'll jailbreak your iPhone" were, it's good that that doesn't work anymore, because that is also a malware vector.
I'd like to see more devices being sold that give the user control, like the newly announced GrapheneOS phones for instance. I look forward to seeing how those are received.
Network segmentation does nothing for the types of attacks these devices perform (e.g. content recognition for upload to their tracking servers, tracking how you navigate their UI, ad delivery). I'm not worried about them spreading worms on my network. The problem is their propensity to exfiltrate data or relay propaganda. The solution to that is a legal one, or barring that, traffic filtering.
That was my motivation for the "or" (don't allow it on your network, or put it on an isolated network); it depends on your threat model and what the device could do. Some devices (like "smart" TVs) shouldn't have network access at all.
"Sure, you can use my wifi while you're over. Just enroll in MDM real quick".
As brought up in another thread on the topic, you have things like web browsers embedded in the Spotify app that will happily ignore your policy if you're not doing external filtering.
I guess it (network-level filtering) just feels like a dragnet solution that reduces privacy and security for the population at large, when a more targeted and cohesive solution like client-side filtering, having all apps that use web browsers funnel into an OS-level check, etc would accomplish the same goals with improved security.
I think the population at large generally needs to get over their hangups (actually, maybe they have, and it's just techies). No one in a first world country cares if you visit pornhub just like no one cares if you go to amazon. Your ISP has had the ability to see this since the beginning of the web. It does not matter, but we can also have privacy laws restricting their (and everyone else like application/service vendors) ability to record and share that information. If you really want, you can hide it with a VPN or Tor. As long as not everything is opaque, it's easy to block that traffic if you'd like (so e.g. kids can't use it). In a first world country, this works fine since actually no one cares if you're hiding something, so you don't need to blend in. At a societal level, opaque traffic is allowed.
You could have cooperation from everyone to hook into some system (California's solution), which I expect will be a cover for more "we need to block unverified software", or you could allow basic centralized filtering as we've had, and ideally compel commercial OS vendors to make it easy to root and MitM their devices for more effective security.
Yes well some of us live in first world countries that are at risk of declining into third world status, where some states DO actually care what sites you visit and would jump at the chance to further restrict traffic.
Rather than “get over” it I think we need to fight. You seem to insist that monitoring/control is a done deal and we only need to argue about the form it takes, but this is not correct. Centralized monitoring/control can be resisted and broken through a combination of political and technical means. While you may not want this, I do. (And many others are being swayed back in my direction as they start to feel the effects of service enshittification, censorship under the guise of “fighting misinformation”, and media consolidation.)
> "don't let kids use a computer until they're 18"
Ideally you would lock them up in a padded room until then. There is a significant amount of shared real world space that isn't supervised and doesn't require any age verification to enter either.
Notably, explicitly adult spaces like bars and porn shops are not among them, and a significant amount of virtual space would also not require age verification for the same reason.
Rules vary. In Britain it was completely normal for say 15-year old me to be in a bar - it was illegal to buy booze but not a problem to be there. But when I travelled to Austin aged 19 I couldn't meet adult members of my team in the hotel bar because I wasn't old enough even though by then I was legal to drink, to marry, to go to war and so on in my own country.
A little while after that, back in the UK, I drove my young cousin to the seaside. I didn't carry ID - I don't drink and you're not required to carry ID to drive here† so it was never necessary back then, but she did, so I try to buy her booze, they demand ID, I do not have any ID so I can't buy it even though I'm old enough to drink. So, she just orders her own booze, she's under age but they don't ask because she's pretty.
† The law here says police are allowed to ask to see a driving license if you're in charge of a vehicle on a public road, but, since you aren't required to carry it they can require you to attend a police station and show documents within a few days. In practice in 2026 police have network access and so they can very easily go from "Jim Smith, NW1A 4DQ" to a photo and confirmation that you're licensed to drive a bus or whatever if you are co-operative.
My right to access free information, and my global neighbor’s right to read unofficial information without being jailed or killed for it, outweighs your right to let your right use the Internet without supervision.
Sure, and if we want to prioritize your ability to do so despite living in an authoritarian hellhole, those of us in countries that respect their citizens rights will have to put these verification systems in place. It just needs to be understood by technologists building this stuff that this is the tradeoff they're making.
And it's likely a temporary win there until the authoritarian regimes mandate local monitoring software and send you to the gulag if they detect opaque traffic.
I wrote about ECH a couple of months ago, when the specs were still in draft but already approved for publication. It's a short read, if you're not already familiar with ECH and its history:
https://www.feistyduck.com/newsletter/issue_127_encrypted_cl...
In addition to the main RFC 9849, there is also RFC 9848 - "Bootstrapping TLS Encrypted ClientHello with DNS Service Bindings": https://datatracker.ietf.org/doc/rfc9848/
There's an example of how it's used in the article.
Thanks for the writeup, Ivan, I am a great fan of your work!
Now we need to get Qualys to cap SSL Labs ratings at B for servers that don't support ECH. Also those that don't have HSTS and HSTS Preload while we're at it.
Thanks! Sadly, SSL Labs doesn't appear to be actively maintained. I've noticed increasing gaps in its coverage and inspection quality. I left quite a while ago (2016) and can't influence its grading any more, sadly.
Yes, there is! After I left SSL Labs, I built Hardenize, which was an attempt to go wider and handle more of network configuration, not just TLS and PKI. It covers a range of standards, from DNS, over email, TLS and PKI, and application security.
Although Hardenize was a commercial product (it was acquired in 2022 by another company, Red Sift), it has a public report that's always been free. For example:
(Disclosure: I'm a Caddy maintainer), Caddy already supports ECH, leaning on the DNS plugins to automate setting the DNS HTTPS records to wire it up. Here's a lot of technical detail about it https://caddyserver.com/docs/automatic-https#encrypted-clien...
But does it automatically provision the DNS records and rotate the keys?
I'm actually kind of furious at nginx's marketing materials around ECH. They compare with other servers but completely ignore Caddy, saying that they're the only practical path to deploying ECH right now. Total lies: https://x.com/mholt6/status/2029219467482603717
one angle that hasn't come up here yet: ECH basically kills TLS fingerprinting as a bot detection signal
right now tools like Cloudflare Bot Management rely heavily on JA3/JA4 hashes - they fingerprint the ClientHello to identify scrapers vs real browsers. if the ClientHello is encrypted, that whole detection layer collapses. you can still do behavioral analysis and JS challenges, but the pre-HTTP layer that currently catches a huge chunk of naive bots - gone
curious how Cloudflare handles this internally given they're one of the biggest ECH adopters but also one of the biggest bot detection vendors. seems like they're eating their own lunch on this one, or they've already shifted their detection stack to not rely on it as much
Cloudflare can and must decrypt the ClientHello for the sites it serves in order to actually serve the traffic. Using ECH with CF means you use their ECH domain and their keys.
If you control the domain you're fingerprinting clients on, you can decrypt the inner ClientHello and fingerprint on that.
If you're not in control of the domain you're fingerprinting, then ECH is working as intended.
I don't expect naive bots to implement ECH any time soon, though. If a bot can't be bothered to download curl-impersonate, they won't pass any ECH flags either.
the naive bot point is true but the threat model that actually matters is the other end - the sophisticated bots that already do implement TLS spoofing. those are the ones using got-scraping, curl-impersonate, or custom TLS stacks specifically to pass JA3/JA4 checks. they're already past the "can't be bothered" threshold.
for that tier, ECH flips the dynamic a bit. right now detection can use JA3/JA4 as a positive signal - "this fingerprint matches Chrome 120, looks clean". with ECH, if the bot is running behind a CDN that terminates ECH (like Cloudflare), the server sees a decrypted ClientHello that looks like... a real Chrome on Cloudflare's infrastructure. the fingerprint is clean by construction.
so paradoxically ECH might make things harder for the sophisticated bot detection case while doing nothing about the naive case, which is sort of backwards from what you'd expect.
fair point, I should have been more precise. the server (Cloudflare in this case) still decrypts the inner ClientHello and can fingerprint it - jannesan and jeroenhd are right about that.
the part that changes is passive fingerprinting from third parties - network middleboxes, ISPs, DPI systems that have historically been able to read ClientHello parameters in transit and build behavioral profiles. that layer goes away. for bot detection specifically that matters less since detection happens at the server, so your correction stands for that use case.
the Cloudflare paradox I was gesturing at is maybe better framed as: for sites NOT on Cloudflare, ECH makes it harder for Cloudflare (as a network observer) to do pre-connection fingerprinting. but for their own CDN customers, they decrypt it anyway so nothing changes for them. the conflict is more theoretical than practical for their current product.
It depends on the origin and the destination. Their Magic Transit service explicitly allow this, and I assume they have agreements with other AS in case something goes wrong on either side (it often does). You'd have to directly ask them to know specifically but I don't think they would answer since that's proprietary information.
What you're describing is a SNI, not ECH. Those two serve very different purposes.
> Also reverse lookup has nothing to do with hosting own DNS resolver.
It has everything to do with that. Had you used two brain cells, you would've known that they can memorize the IP address and the domain name, and if you connect to that IP in a short period of time, most likely you visited that domain name.
One topic I have not seen discussed is why CDNs, one by one, stopped allowing "domain fronting" yet ECH, developed by people working at CDNs, essentially uses a similar tactic, i.e., two hostnames, only one of them actually needed for a successful HTTP request
In truth ECH sends three: Host header + real SNI + dummy SNI
Every time some security related protocol relies on DNS for its magic (looking at you, ACME), I lament the state of DNS providers. They all have different APIs, with different levels of security. Most at least offer some kind of REST API with API tokens for auth, which is relatively easy to set up.
Many of those (not looking at any particular Germans..) however only offer a single API token across all DNS zones, which is awful when you're managing many zones. One compromised API token = hundreds of compromised zones.
Would be nice if more DNS providers offered granular API tokens, at least on a per-zone basis and ideally on a per-record basis within a zone.
> In verifying the client-facing server certificate, the client MUST interpret the public name as a DNS-based reference identity [RFC9525]. Clients that incorporate DNS names and IP addresses into the same syntax (e.g. Section 7.4 of [RFC3986] and [WHATWG-IPV4]) MUST reject names that would be interpreted as IPv4 addresses.
Aside from apparently not considering the existence of IPv6, why are IP-based certificates explicitly ruled out? This makes the spec entirely meaningless for small servers and basically requires shifting hosting to shared hosts/massive CDNs to provide any protection against SNI snooping.
I think it's saying that you can't make the name look like an IP address; i.e. if the syntax were www.google.com[142.250.117.139] (I'm making this syntax up) you couldn't put 142.250.117.139[142.250.117.139].
The syntax being referred to includes some obscure, outdated addressing formats (IPv4 addresses represented as two or three number groups in dotted notation rather than the normal 4).
However, "DNS-based reference identity [RFC9525]" seems to explicitly disallow IP-based certificates by requiring a DNS name. I can only interpret the sentence I quoted as written to say "make sure you never ever accidentally validate an IP address".
\> This makes the spec entirely meaningless for small servers and basically requires shifting hosting to shared hosts/massive CDNs to provide any protection against SNI snooping.
Actually you can setup ECH on your server, and configure the public_name to be something like `cloudflare-ech.com` , so clients would indeed use that in the OuterSNI, connect to you, without you needing to use CF. And middleboxes might think they are indeed connecting to CF (though CF publishes their IP ranges so this could be checked elsewhere).
IPv6 addresses aren't confusable with a DNS name in these syntaxes AFAIU so it's not that they didn't consider IPv6 but that it's not relevant to the issue.
Yes, "Don't stand out" technologies like ECH aren't useful if you inherently stand out anyway. They're intended to make broad surveillance and similar undirected attacks less effective, they aren't magic invisibility cloaks and won't protect you if you're a singular target.
The colon isn’t a valid character in DNS, so there’s just no risk of confusing IPv6 addresses (which contain at least one colon in all notations I’ve seen).
For IPv4, there’s room for ambiguity.
And how are IP certificates required for small servers?
I can't think of a single numeric TLD, so I don't think anyone is confusing IP literals with domain names, unless they're doing so extremely lazily.
> And how are IP certificates required for small servers?
You need a valid certificate as the outer certificate which contains an SNI that will still be readable. For cloudflare.com and google.com that's easy; you can't tell what website Cloudflare is proxying and whether Google is serving you Youtube, Gmail, or Google Search content.
For an independently-hosted myhumanrightsblog.net, that's not as easy. They'd need another domain reachable on that server to set up the ECH connection to hide the risky TLD. Clients being snooped on still get specific domains logged.
IP certificates work around that issue by validating the security of the underlying connection rather than any specific hostname. Any server could be serving any hostname over an IP-address-validated connection. For snooped-on clients, the IP address is already part of the network traffic anyway, but no domains ever hit the traffic logs at all.
Probably why TLD requires the first character to be an alpha character. com3 is okay, but 3com is not. Unless it's to protect spec against ignoring that requirement I don't see where confusing could surface.
But then your underlying issue is that you're microhosting and can't hide behind a large cloud provider's domain front, so isn't that inherent to anything you might do?
In other words, blocking solutions that know your small blog is hosted exclusively on 1.2.3.4, without any collateral damage to other blogs the blocking government cares about will just block your IP.
Conversely, if you're hosting importedgoodsecommercesitegovernmentofficialslove.com next to myhumanrightsblog.net on the same IP, ECH is for you and solves your problem: Just register mycoolagnostichosting.net and do ECH to that.
"Just buy a second domain exclusively to work around the arbitrary restrictions put onto the protocol" works as a solution, but it's a silly solution that shouldn't be necessary.
ECH doesn't benefit you if you're connecting directly to one IP. Middleboxes can track that you're connecting to this IP.
ECH prevents tracking through routing layers where your ClientHello might contain foo.example.com or bar.example.com but route via the same IP (Cloudflare). A middlebox can see you are using a cloudflare hosted website, but not know what cloudflare website.
There's no benefit encrypting the SNI with 10.20.30.40 if they can see you're connecting to 10
20.30.40 anyway
THe benefit is that the SNI is not being logged. Resolving an IP to a domain name is pretty hard for a small actor who doesn't have a record of all DNS records.
That's a good point. I was thinking more of a "block this list of wrongthink TLDs" use case, but "list all hostnames accessed by person x" is of course also worth considering.
If the client (read: chrome) does support that (and prevent its desactivation), then zscaler and other shitty things are made even more useless than what they are today
TLS Encrypted Client Hello (ECH) standard is another attempt on encrypting plaintext Server Name Indication (SNI). It is very useful for circumventing SNI-based censorship, which is adopted for years by state-backed systems like the Great Firewall (GFW).
The previous attempt of encrypting plaintext SNI is Encrypted Server Name Indication (ESNI), which didn't end well.
>analysis has shown that encrypting only the SNI extension provides incomplete protection. As just one example: during session resumption, the Pre-Shared Key extension could, legally, contain a cleartext copy of exactly the same server name that is encrypted by ESNI. The ESNI approach would require an encrypted variant of every extension with potential privacy implications, and even that exposes the set of extensions advertised. Lastly, real-world use of ESNI has exposed interoperability and deployment challenges that prevented it from being enabled at a wider scale.
China and Russia started to block ESNI before Cloudflare stopped offering it so any argument that ESNI did not work is dubious
IME, ESNI worked for accessing _all_ websites using CF. AFAIK, ECH has never been offered for all websites using CF
ESNI was a bit simpler to use than ECH, e.g., when making HTTP requests with programs like openssl s_client, bssl client, etc. (I don't use popular browsers to make HTTP requests)
When CF ended the ESNI trial, there was nothing to take its place. The public was asked to wait for ECH
It has been roughly five years (correct me if wrong) without any replacement solution for plaintext SNI
As far as I know, ESNI was pushed to a large number of websites in a very fast pace, and soon completely blocked by GFW. Then I didn't hear news about ESNI.
It's an interesting feature, but it's pushing my buttons lately. Specifically on Cloudflare it is on by default, and on the free tier you can't disable it and you need a business plan for it. Which I think is stupid, but never the less it's causing us problems. We are trying some split-dns setup for a company "intranet", and if the site's have be accessed before, the ECH is remembered. So the browser tries and it eventually fails with ECH Error or on Firefox it just hangs like it's loading all the time. And it's so frustrating, because sometimes it works, sometimes it doesn't, you can clear cache and stuff and it still won't work, sometimes it works in Incognito sometimes it doesn't. This is not a real problem, but since we haven't fully switched to the "intranet" and we use some of the WAF features of Cloudflare sometimes it is so frustrating.
Oh snap. Say I'd like to compute some JA4 hashes on my server. I know this can be done as the server has the keys required. But is there a ready made solution that can account for ECH and compute JA4 as a proxy? Please? I saw HAProxy should work and I know that nginx module for JA4 is incomplete, are you please aware of other solutions for self-hosting?
I'm okay advertising ECH as a privacy enhancement, it is, but make no mistake, it will not prevent government tracking or censorship. Governments will request platforms to disable ECH or ask for other signals to be reported, and ban those who don't comply. You can't fight political issues solely with technology.
I know this has nothing to do with the RFC, but I wonder what RFC 10000 will be. I'm just hoping for some sort of joke RFC like TCP over Avian Carriers or the Oops I'm A teapot.
ECH is great from a privacy perspective, but I’m curious how well this will actually work in practice.every time the web encrypts more metadata there’s pushback from middleboxes and network operators.
> I’m curious how well this will actually work in practice
You're experiencing it working in practice. RFC9849 is a published document, the end of a very long process in which the people who make this "actually work in practice" decided how to do this years ago and have deployed it.
This isn't like treaty negotiation where the formal document often creates a new reality, the RFC publication is more like the way the typical modern marriage ceremony is just formalising an existing reality. Like yeah, yesterday Bill and Sarah were legally not married, and today Bill and Sarah are married, but "Bill and Sarah" were a thing five Christmases ago, one of the bridesmaids is their daughter, we're just doing some paperwork and having a party.
The tension is that Security and Dev parts of the stack remove the actual troubleshooting capabilities of the Network layer without opening up the tools that are supposed to replace them.
It's not a problem if Network can still do their job. It's a whole other matter to expect Network to do their job through another layer. You end up with organizations that can't maintain their applications and expect magic fixes.
Orgs that are cooperative probably don't have this issue but there are definitely parts of some organizations that when one part takes capability from another they don't give it back in some sort of weird headcount game despite not really wanting to understand Network to a Network level.
This feels like a recurring pattern in the stack. abstraction removes visibility faster than tooling replaces it.
Encryption and higher-level platforms are great for security and productivity, but the debugging surface keeps shrinking. Eventually when something breaks, nobody actually has the layer-by-layer visibility needed to reason about it.
ECH won't be effective until there's a HSTS-style policy that forces browsers to use it. Otherwise, firewalls will continue to strip parameters and downgrade connections[0].
The Fortigate article proposes that you take a profile in which your end users have said OK, I trust the Fortigate to decide what's allowed, and then you set it to not allow them to use ECH.
Notice that if users don't trust the Fortigate all it can do is IP layer blocks, exactly as intended.
It seems pointless to try to have a policy where people say they trust somebody else (whoever is operating that Fortigate) to override their will but also they don't want their will overridden, that's an incoherent policy, there's no technical problem there, technology can't help.
Well, yes, this is being used in corporate environments but the end user and the system admin aren't on the same page necessarily. Domain blocking doesn't make much sense in my opinion and should be a thing of the past. You already lack admin rights so what is a block on e.g. mullvad.net actually doing other than stopping someone from reading their blog? They can't install the VPN software.
Defense in layers makes sense, but domain blocking was never a "layer" if a hostile actor can just buy a new domain that's not on your blocklist.
I think it'd be good if ECH became more widespread so that we can get away from these antiquated control techniques that just result in frustration with no security benefits.
My understanding is that you can use split mode to only have the load balancer decrypt the server name section, and forward the actual session and key exchange down to the backend without doing double layer encryption.
Only if the attacker has a valid certificate for the domain to complete the handshake with.
Relying on HTTPS and SVCB records will probably allow a downgrade for some attackers, but if browsers roll out something akin to the HSTS preload list, then downgrade attacks become pretty difficult.
DNSSEC can also protect against malicious SVCB/HTTPS records and the spec recommends DoT/DoH against local MitM attacks to prevent this.
DNSSEC can't protect against an ECH downgrade. ECH attackers are all on-path, and selectively blocking lookups is damaging even if you can't forge them. DoH is the answer here, not record integrity.
DNSSEC alone is obviously useless because any attacker interested in SNI hostnames can just as easily monitor DNS traffic.
However, DoH/DoT without record integrity is about as useful as self-signed HTTPS certificates. You need both for the system to work right in every case.
To quote the spec:
> Clearly, DNSSEC (if the client validates and hard fails) is a defense against this form of attack, but encrypted DNS transport is also a defense against DNS attacks by attackers on the local network, which is a common case where ClientHello and SNI encryption are desired. Moreover, as noted in the introduction, SNI encryption is less useful without encryption of DNS queries in transit.
I don't think this is true; I think this misunderstands the ECH threat model. You don't need record integrity to make ECH a strong defense against on-path ISP attackers; you just need to trust the resolver you're DoH'ing to.
If browsers remember which domains do ECH and refuse to downgrade to non-ECH connections after, the way the HSTS cache forces browsers to connect over HTTPS despite direct attempts to load over HTTP, then you only need an entry in the browser database to make downgrade attacks to accomplish SNI-snooping impossible.
For HSTS, browsers come with a preloaded list of known-HTTPS domains that requests are matched against. That means they will never connect over HTTP, rather than connect over HTTP and upgrade+maintain a cache when the HSTS header is present. If ECH comes with a preload list, then browsers connecting to ECH domains will simply fail to connect rather than permit the network to downgrade their connection to non-ECH TLS.
If every server has its own IP then the privacy leak comes from merely connecting to it. It makes ECH useless, but that's not the same thing as making it unnecessary.
Glad that it's published, I'd been following it since ESNI draft days. Was pretty useful back when I was in India since Jio randomly blocked websites, and cloudflare adopted the ESNI draft on its servers as did Firefox client side which made their SNI based blocking easy to bypass.
There was a period where I think both disabled ESNI support as work was made on ECH, which now is pretty far along. I was even able to setup a forked nginx w/ ECH support to build a client(browser) tester[0].
Hopefully now ECH can get more mainstream in HTTPS servers allowing for some fun configs.
A pretty interesting feature of ECH is that the server does not need to validate the public name (it MAY) , so clients can use public_name's that middleboxes (read: censors) approve to connect to other websites. I'm trying to get this added to the RustTLS client[1], now might be a good time to pick that back up.
[0] https://rfc9849.mywaifu.best:3443/ [1] https://github.com/rustls/rustls/issues/2741
The server can also advertise a public name that doesn't match any domain it has a TLS certificate for, like example.com or nsa.gov.
I'm not 100% sure it's allowed in the specs, but it works in Chrome.
As I understand it, without this feature it would be pretty useless for small website owners, since they would need to register a separate domain for their ECH public name, which censors could just block.
I saw this used to obfuscate spam yesterday! Yay?
Why didn't the Indian government block traffics based on IP instead? That would make it much harder to bypass.
If i'm not mistaken its because IPs are actually much easier to rotate than domains.
E.g. all the users will remember `example.com` , underlying it doesn't matter what IP it resolves to. If the IP gets "burned" , then the providers can rotate to a new IP (if their provider allows).
Vs. telling your users to use a new domain `example.org` , fake websites etc.
Also sensible ISPs usually don't block IPs since for services behind a CDN it could lead to other websites being blocked, though of course sometimes this is ignored. See also: https://blog.cloudflare.com/consequences-of-ip-blocking/
I wouldn't say you're mistaken, but it's a simplification. In the network world, the capability exists to restrict what BGP advertisements are accepted via RPKI/a peer. Internet providers usually don't because the premium is placed on uptime/connectivity.
If tomorrow, everyone said "we don't want IP's from Frankfurt showing up somewhere in Dubai", you'd have a massive technical problem and rearranging to start with but once that was sorted you could geo-lock. IANA and Network providers simply haven't been doing that.
The reason it doesn't happen is Devs/Stakeholders want uptime from ISPs/Networks and not something they can't abstract. Basically its just a status quo much like the entire internet reverse-proxying through CDNs is a status quo. It wasn't always like that, and it may not always be like that in the future - just depends which way the winds blow over time.
> we don't want IP's from Frankfurt showing up somewhere in Dubai
what do you mean, IPs from Frankfurt?
IP addresses are just IP addresses, they know no geographical boundaries. In RIR DBs you can geolocate them to wherever you want. Which is the entire reason why Geo IP DBs even exist - they triangulate.
> "we don't want IP's from Frankfurt showing up somewhere in Dubai"
From a network perspective statements like that make no sense. IP addresses don't have any sort of physicality,
They have registration data. Someone could declare they don't want IPs registered to companies from Frankfurt with geofeeds in Frankfurt to be advertised in Dubai.
It’s not how any of it works.
How do you determine to whom an IP is even registered to? They get sub-leased all the time.
The best you can do is check who has administrative control over the prefixes RIR info, but that doesn’t mean that anyone with control is the factual user of the IPs.
You could check the IRR for the ASN and base it on that, but still.
There's also no way to actually know _where_ an IP actually originates from. Only its AS path.
The DFZ contains all prefixes announced everywhere, for the internet is completely decentralized.
> How do you determine to whom an IP is even registered to?
You check the RIR's records.
> They get sub-leased all the time.
With records updated. If not, any consequences from wrong information fall on the lessor and lessee.
> There's also no way to actually know _where_ an IP actually originates from. Only its AS path.
Ping time from different locations on their upstream AS gives a good guess.
> With records updated. If not, any consequences from wrong information fall on the lessor and lessee.
Not always + there are no consequences whatsoever.
Plenty of leasing services will just provide you with IRR & RPKI, without ever touching the actual records.
> Ping time from different locations on their upstream AS gives a good guess.
Upstream AS is meaningless if it's a T1 carrier. Ping AS6939. They are everywhere.
That's why you have a strictly legal domain that enables a convoluted redirect with plausible deniability (not 302)
It'll still eventually stick, but a lot slower
> Was pretty useful back when I was in India since Jio randomly blocked websites
With Jio, you don't really need ECH at all. The blocks are mostly rudimentary and bypassed with encrypted DNS (DoH / DoT / DNSCrypt) and Firefox (which fragments the TLS ClientHello packets into two).
Also: https://news.ycombinator.com/item?id=34232190
Should've added this was back in like 2018 or so. Setting up DoH was harder than enabling SNI, and from my testing back then they were hard filtering on SNI (e.g. I used OpenSSL CLI to set the SNI to `pornhub.com` and connect to "known good" IPs, it'd still get reset).
Funnily enough, not setting the SNI and connecting the the origin IP, and then requesting the page worked fine.
> Funnily enough, not setting the SNI and connecting the the origin IP, and then requesting the page worked fine.
Such tricks, called "domain fronting" are why ECH exists. The problem is that although domain fronting is effective for the client it's a significant headache for the provider. Big providers involved, such as Cloudflare have always insisted that they want to provide this sort of censorship resisting capability but they don't want to authorize domain fronting because it's a headache for them technically.
Let me explain the headache with an example. Say I'm Grand Corp, a French company with 25 million web sites including both cats-are-great.example and fuck-trump.example. Users discover that although the US government has used Emergency Powers to prohibit access to fuck-trump.example, using domain fronting they can connect to cats-are-great.example and request fuck-trump.example pages anyway and the US government's blocking rules can't stop them.
What they don't know is that I, Grand Corp had been sharding sites 25 ways, so there was only 1-in-25 chance that this worked - it so happened cats-are-great and fuck-trump were in the same shard, On Thursday during routine software upgrade we happen to switch to 32-way sharding and suddenly it stops working - users are outraged, are the French surrendering to Donald Trump?
Or, maybe as a fallback mechanism the other 31 servers can loop back around to fetch your fuck-trump.example pages from the server where they live, but in doing so they double the effective system load. So now my operational costs at Grand Corp for fuck-trump.example doubled because clients were fronting. Ouch.
> Such tricks, called "domain fronting"
GP said "not setting SNI"... doing TLS handshake with IP certs don't (need to) set SNI?
That's true, usually with domain fronting you provide the (wrong) SNI. But the same strategy is happening here, you were supposed to provide SNI and you didn't to avoid some potential censorship but it's a headache for the provider
They won't have received a certificate for the IP as a name, it's relatively unusual to have those, the main users are things like DoH and DoT servers since their clients may not know the name of the server... historically if you connect to a TLS server without SNI it just picks a name and presents a certificate for that name - if there's a single name for the machine that definitely works, and if not well - domain fronting.
TLS 1.3 even specifies that you must always do SNI and shouldn't expect such tricks to work, because it's such a headache.
> A pretty interesting feature of ECH is that the server does not need to validate the public name (it MAY) , so clients can use public_name's that middleboxes (read: censors) approve to connect to other websites. I'm trying to get this added to the RustTLS client[1], now might be a good time to pick that back up.
Note that it is exactly this type of thing that makes age verification laws reasonable. You're making it technically impossible for even sophisticated parents to censor things without a non-solution like "don't let kids use a computer until they're 18", so naturally the remaining solution is a legal one to put liability on service operators.
You're still ultimately going to get the censorship when the law catches up in whatever jurisdiction, but you'll also provide opacity for malware (e.g. ad and tracking software) to do its thing.
This is exactly reverse of the right idea. If parents need to censor things the solutions are the same as corpos are going to. Put the censors at the device or “mitm” the connection, either actually with a proxy, or maybe with a browser and curated apps - which is again on the device.
This brings us back to "sure you can use my guest wifi, just install my root CA/enroll in MDM".
I do agree though that it should be illegal for device manufacturers or application developers to use encryption that the device owner cannot MitM. The owner should always be able to install their own CA and all applications should be required to respect it.
Why would you want to censor based on network? You don't want to censor based on network, you want to censor based on device. If your 8yo kid is blocked from pornhub, that doesn't mean everyone on your network is blocked from pornhub, and you having the ability to even know if someone on your network is browsing pornhub is a security risk.
How does ECH make it impossible for parents to control their children's access to computers? Sure they can't block sites at the router level, just like your ISP won't be able to block things at the ISP level, but you (the parent) have physical access to the devices in question, and can install client-side software to filter access to the internet.
The only thing this makes impossible is the laziest, and easiest to bypass method of filtering the internet.
Because there are network operators who have mal-intent increasingly no network operators are permitted to exercise network-level control. A parent who wants to filter the network access in their house is the same as a despotic regime practicing surveillance and censorship on their citizens.
Given that it's pretty much the norm that consumer embedded devices don't respect the owner's wishes network level filtering is the best thing a device owner can do on their own network.
It's a mess.
I'd like to see consumer regulation to force manufacturers to allow owners complete control over their devices. Then we could have client side filtering on the devices we own.
I can't imagine that will happen. I suspect what we'll see, instead, is regulation that further removes owner control of their devices in favor of baking ideas like age or identity verification directly into embedded devices.
Then they'll come for the unrestricted general purpose computers.
If you have a device you don't trust, don't allow it on your network, or have an isolated network for such devices. Meanwhile, devices are right to not allow MITMing their traffic and to treat that as a security hole, even if a very tiny fraction of their users might want to MITM it to try to do adblocking on a device they don't trust or fully control, rather than to exploit the device and turn it into a botnet.
Along similar lines, a security hole you can use for jailbreaking is also a security hole that could potentially be exploited by malware. As cute as things like "visit this webpage and it'll jailbreak your iPhone" were, it's good that that doesn't work anymore, because that is also a malware vector.
I'd like to see more devices being sold that give the user control, like the newly announced GrapheneOS phones for instance. I look forward to seeing how those are received.
Network segmentation does nothing for the types of attacks these devices perform (e.g. content recognition for upload to their tracking servers, tracking how you navigate their UI, ad delivery). I'm not worried about them spreading worms on my network. The problem is their propensity to exfiltrate data or relay propaganda. The solution to that is a legal one, or barring that, traffic filtering.
That was my motivation for the "or" (don't allow it on your network, or put it on an isolated network); it depends on your threat model and what the device could do. Some devices (like "smart" TVs) shouldn't have network access at all.
"Sure, you can use my wifi while you're over. Just enroll in MDM real quick".
As brought up in another thread on the topic, you have things like web browsers embedded in the Spotify app that will happily ignore your policy if you're not doing external filtering.
Fair point.
I guess it (network-level filtering) just feels like a dragnet solution that reduces privacy and security for the population at large, when a more targeted and cohesive solution like client-side filtering, having all apps that use web browsers funnel into an OS-level check, etc would accomplish the same goals with improved security.
I think the population at large generally needs to get over their hangups (actually, maybe they have, and it's just techies). No one in a first world country cares if you visit pornhub just like no one cares if you go to amazon. Your ISP has had the ability to see this since the beginning of the web. It does not matter, but we can also have privacy laws restricting their (and everyone else like application/service vendors) ability to record and share that information. If you really want, you can hide it with a VPN or Tor. As long as not everything is opaque, it's easy to block that traffic if you'd like (so e.g. kids can't use it). In a first world country, this works fine since actually no one cares if you're hiding something, so you don't need to blend in. At a societal level, opaque traffic is allowed.
You could have cooperation from everyone to hook into some system (California's solution), which I expect will be a cover for more "we need to block unverified software", or you could allow basic centralized filtering as we've had, and ideally compel commercial OS vendors to make it easy to root and MitM their devices for more effective security.
Yes well some of us live in first world countries that are at risk of declining into third world status, where some states DO actually care what sites you visit and would jump at the chance to further restrict traffic.
Rather than “get over” it I think we need to fight. You seem to insist that monitoring/control is a done deal and we only need to argue about the form it takes, but this is not correct. Centralized monitoring/control can be resisted and broken through a combination of political and technical means. While you may not want this, I do. (And many others are being swayed back in my direction as they start to feel the effects of service enshittification, censorship under the guise of “fighting misinformation”, and media consolidation.)
There's nothing technical stopping device manufacturers from making this easy for parents to do. They choose not to.
A lot of endpoint protection products rely on SNI sniffing. E.g. Apple's network extensions filters look at TLS handshakes.
Then they would drop the connection with esni
> "don't let kids use a computer until they're 18"
Ideally you would lock them up in a padded room until then. There is a significant amount of shared real world space that isn't supervised and doesn't require any age verification to enter either.
Notably, explicitly adult spaces like bars and porn shops are not among them, and a significant amount of virtual space would also not require age verification for the same reason.
Rules vary. In Britain it was completely normal for say 15-year old me to be in a bar - it was illegal to buy booze but not a problem to be there. But when I travelled to Austin aged 19 I couldn't meet adult members of my team in the hotel bar because I wasn't old enough even though by then I was legal to drink, to marry, to go to war and so on in my own country.
A little while after that, back in the UK, I drove my young cousin to the seaside. I didn't carry ID - I don't drink and you're not required to carry ID to drive here† so it was never necessary back then, but she did, so I try to buy her booze, they demand ID, I do not have any ID so I can't buy it even though I'm old enough to drink. So, she just orders her own booze, she's under age but they don't ask because she's pretty.
† The law here says police are allowed to ask to see a driving license if you're in charge of a vehicle on a public road, but, since you aren't required to carry it they can require you to attend a police station and show documents within a few days. In practice in 2026 police have network access and so they can very easily go from "Jim Smith, NW1A 4DQ" to a photo and confirmation that you're licensed to drive a bus or whatever if you are co-operative.
My right to access free information, and my global neighbor’s right to read unofficial information without being jailed or killed for it, outweighs your right to let your right use the Internet without supervision.
Sure, and if we want to prioritize your ability to do so despite living in an authoritarian hellhole, those of us in countries that respect their citizens rights will have to put these verification systems in place. It just needs to be understood by technologists building this stuff that this is the tradeoff they're making.
And it's likely a temporary win there until the authoritarian regimes mandate local monitoring software and send you to the gulag if they detect opaque traffic.
Ironically, or perhaps not, I think we’re both posting from the US. I am.
I wrote about ECH a couple of months ago, when the specs were still in draft but already approved for publication. It's a short read, if you're not already familiar with ECH and its history: https://www.feistyduck.com/newsletter/issue_127_encrypted_cl...
In addition to the main RFC 9849, there is also RFC 9848 - "Bootstrapping TLS Encrypted ClientHello with DNS Service Bindings": https://datatracker.ietf.org/doc/rfc9848/
There's an example of how it's used in the article.
Thanks for the writeup, Ivan, I am a great fan of your work!
Now we need to get Qualys to cap SSL Labs ratings at B for servers that don't support ECH. Also those that don't have HSTS and HSTS Preload while we're at it.
Thanks! Sadly, SSL Labs doesn't appear to be actively maintained. I've noticed increasing gaps in its coverage and inspection quality. I left quite a while ago (2016) and can't influence its grading any more, sadly.
Is there a well-maintained alternative to SSL Labs you can recommend?
Yes, there is! After I left SSL Labs, I built Hardenize, which was an attempt to go wider and handle more of network configuration, not just TLS and PKI. It covers a range of standards, from DNS, over email, TLS and PKI, and application security.
Although Hardenize was a commercial product (it was acquired in 2022 by another company, Red Sift), it has a public report that's always been free. For example:
https://www.hardenize.com/report/feistyduck.com
The CSP inspection in Hardenize could use a refresh, but the TLS and PKI aspects are well maintained [at the time of writing].
I use testssl.sh [1] mostly because I can test things not publicly accessible.
[1] - https://github.com/testssl/testssl.sh
> There's an example of how it's used in the article
A bit tricky in Go, but nothing too complicated. We implemented ECH in Aug 2024 for our DNS Android app and it has worked nicely since: https://github.com/celzero/firestack/blob/09b26631a2eac2cf9c...
(Disclosure: I'm a Caddy maintainer), Caddy already supports ECH, leaning on the DNS plugins to automate setting the DNS HTTPS records to wire it up. Here's a lot of technical detail about it https://caddyserver.com/docs/automatic-https#encrypted-clien...
Nginx also supports ECH now, since the December release.
But does it automatically provision the DNS records and rotate the keys?
I'm actually kind of furious at nginx's marketing materials around ECH. They compare with other servers but completely ignore Caddy, saying that they're the only practical path to deploying ECH right now. Total lies: https://x.com/mholt6/status/2029219467482603717
one angle that hasn't come up here yet: ECH basically kills TLS fingerprinting as a bot detection signal
right now tools like Cloudflare Bot Management rely heavily on JA3/JA4 hashes - they fingerprint the ClientHello to identify scrapers vs real browsers. if the ClientHello is encrypted, that whole detection layer collapses. you can still do behavioral analysis and JS challenges, but the pre-HTTP layer that currently catches a huge chunk of naive bots - gone
curious how Cloudflare handles this internally given they're one of the biggest ECH adopters but also one of the biggest bot detection vendors. seems like they're eating their own lunch on this one, or they've already shifted their detection stack to not rely on it as much
Cloudflare can and must decrypt the ClientHello for the sites it serves in order to actually serve the traffic. Using ECH with CF means you use their ECH domain and their keys.
If you control the domain you're fingerprinting clients on, you can decrypt the inner ClientHello and fingerprint on that.
If you're not in control of the domain you're fingerprinting, then ECH is working as intended.
I don't expect naive bots to implement ECH any time soon, though. If a bot can't be bothered to download curl-impersonate, they won't pass any ECH flags either.
the naive bot point is true but the threat model that actually matters is the other end - the sophisticated bots that already do implement TLS spoofing. those are the ones using got-scraping, curl-impersonate, or custom TLS stacks specifically to pass JA3/JA4 checks. they're already past the "can't be bothered" threshold.
for that tier, ECH flips the dynamic a bit. right now detection can use JA3/JA4 as a positive signal - "this fingerprint matches Chrome 120, looks clean". with ECH, if the bot is running behind a CDN that terminates ECH (like Cloudflare), the server sees a decrypted ClientHello that looks like... a real Chrome on Cloudflare's infrastructure. the fingerprint is clean by construction.
so paradoxically ECH might make things harder for the sophisticated bot detection case while doing nothing about the naive case, which is sort of backwards from what you'd expect.
It doesn't prevent fingerprinting, stop spreading misinformation. It only prevents your ISP from knowing what website you're connecting to.
fair point, I should have been more precise. the server (Cloudflare in this case) still decrypts the inner ClientHello and can fingerprint it - jannesan and jeroenhd are right about that.
the part that changes is passive fingerprinting from third parties - network middleboxes, ISPs, DPI systems that have historically been able to read ClientHello parameters in transit and build behavioral profiles. that layer goes away. for bot detection specifically that matters less since detection happens at the server, so your correction stands for that use case.
the Cloudflare paradox I was gesturing at is maybe better framed as: for sites NOT on Cloudflare, ECH makes it harder for Cloudflare (as a network observer) to do pre-connection fingerprinting. but for their own CDN customers, they decrypt it anyway so nothing changes for them. the conflict is more theoretical than practical for their current product.
> the part that changes is passive fingerprinting from third parties
That's exactly what I said:
> It only prevents your ISP from knowing what website you're connecting to.
Why would Clownflare ever see traffic to sites not on Clownflare?
They do routing. Even if you're connecting to a non Cloudflare server, the traffic may still be routed through their servers.
Why would they want to peek traffic? Most likely for statistics (most frequently visited websites etc).
Can you give an example of a BGP route or traceroute to a site not on Clownflare that was routed through Clownflare?
It depends on the origin and the destination. Their Magic Transit service explicitly allow this, and I assume they have agreements with other AS in case something goes wrong on either side (it often does). You'd have to directly ask them to know specifically but I don't think they would answer since that's proprietary information.
Since most ISPs also maintain their own DNS resolver, they could always reverse lookup the IP address AFAIK.
The whole idea behind ECH is one IP hosts tons of sites (eg. CDN) so you have no idea which one it is.
Also reverse lookup has nothing to do with hosting own DNS resolver.
What you're describing is a SNI, not ECH. Those two serve very different purposes.
> Also reverse lookup has nothing to do with hosting own DNS resolver.
It has everything to do with that. Had you used two brain cells, you would've known that they can memorize the IP address and the domain name, and if you connect to that IP in a short period of time, most likely you visited that domain name.
SNI is unencrypted, so your ISP can see it. ECH encrypts it.
How does this relate to my comment?
True. ECH is useless if you're using plain DNS. DNS over TLS or HTTPS is the way to go.
What OP wrote seems correct:
> ECH basically kills TLS fingerprinting as a bot detection signal
They are not talking about fingerprinting in general. Please elaborate how else TLS fingerprinting can be done.
I am talking about TLS fingerprinting, not JS fingerprinting.
> Please elaborate how else TLS fingerprinting can be done.
By doing everything as it is right now?
How would you (an arbitrary web server) fingerprint a TLS connection if the Client Hello is encrypted?
The website owner (or cloudflare in this case) has the keys to decrypt the client hello. That's necessary for routing information.
You're right, sorry! I got confused myself.
By decrypting it? I don't think you know how TLS, or E2E works in general. ISP doesn't perform the fingerprinting, the server does.
Of course! My bad, thanks for engaging.
One topic I have not seen discussed is why CDNs, one by one, stopped allowing "domain fronting" yet ECH, developed by people working at CDNs, essentially uses a similar tactic, i.e., two hostnames, only one of them actually needed for a successful HTTP request
In truth ECH sends three: Host header + real SNI + dummy SNI
Every time some security related protocol relies on DNS for its magic (looking at you, ACME), I lament the state of DNS providers. They all have different APIs, with different levels of security. Most at least offer some kind of REST API with API tokens for auth, which is relatively easy to set up.
Many of those (not looking at any particular Germans..) however only offer a single API token across all DNS zones, which is awful when you're managing many zones. One compromised API token = hundreds of compromised zones.
Would be nice if more DNS providers offered granular API tokens, at least on a per-zone basis and ideally on a per-record basis within a zone.
Something that puzzles me about ECH:
> In verifying the client-facing server certificate, the client MUST interpret the public name as a DNS-based reference identity [RFC9525]. Clients that incorporate DNS names and IP addresses into the same syntax (e.g. Section 7.4 of [RFC3986] and [WHATWG-IPV4]) MUST reject names that would be interpreted as IPv4 addresses.
Aside from apparently not considering the existence of IPv6, why are IP-based certificates explicitly ruled out? This makes the spec entirely meaningless for small servers and basically requires shifting hosting to shared hosts/massive CDNs to provide any protection against SNI snooping.
I think it's saying that you can't make the name look like an IP address; i.e. if the syntax were www.google.com[142.250.117.139] (I'm making this syntax up) you couldn't put 142.250.117.139[142.250.117.139].
The syntax being referred to includes some obscure, outdated addressing formats (IPv4 addresses represented as two or three number groups in dotted notation rather than the normal 4).
However, "DNS-based reference identity [RFC9525]" seems to explicitly disallow IP-based certificates by requiring a DNS name. I can only interpret the sentence I quoted as written to say "make sure you never ever accidentally validate an IP address".
I don't think your interpretation is right. If it were,
> Clients that incorporate DNS names and IP addresses into the same syntax
They wouldn't mention the IP addresses at all. Also, notice the word "and".
\> This makes the spec entirely meaningless for small servers and basically requires shifting hosting to shared hosts/massive CDNs to provide any protection against SNI snooping.
Actually you can setup ECH on your server, and configure the public_name to be something like `cloudflare-ech.com` , so clients would indeed use that in the OuterSNI, connect to you, without you needing to use CF. And middleboxes might think they are indeed connecting to CF (though CF publishes their IP ranges so this could be checked elsewhere).
IPv6 addresses aren't confusable with a DNS name in these syntaxes AFAIU so it's not that they didn't consider IPv6 but that it's not relevant to the issue.
Yes, "Don't stand out" technologies like ECH aren't useful if you inherently stand out anyway. They're intended to make broad surveillance and similar undirected attacks less effective, they aren't magic invisibility cloaks and won't protect you if you're a singular target.
The colon isn’t a valid character in DNS, so there’s just no risk of confusing IPv6 addresses (which contain at least one colon in all notations I’ve seen).
For IPv4, there’s room for ambiguity.
And how are IP certificates required for small servers?
> For IPv4, there’s room for ambiguity.
I can't think of a single numeric TLD, so I don't think anyone is confusing IP literals with domain names, unless they're doing so extremely lazily.
> And how are IP certificates required for small servers?
You need a valid certificate as the outer certificate which contains an SNI that will still be readable. For cloudflare.com and google.com that's easy; you can't tell what website Cloudflare is proxying and whether Google is serving you Youtube, Gmail, or Google Search content.
For an independently-hosted myhumanrightsblog.net, that's not as easy. They'd need another domain reachable on that server to set up the ECH connection to hide the risky TLD. Clients being snooped on still get specific domains logged.
IP certificates work around that issue by validating the security of the underlying connection rather than any specific hostname. Any server could be serving any hostname over an IP-address-validated connection. For snooped-on clients, the IP address is already part of the network traffic anyway, but no domains ever hit the traffic logs at all.
Probably why TLD requires the first character to be an alpha character. com3 is okay, but 3com is not. Unless it's to protect spec against ignoring that requirement I don't see where confusing could surface.
But then your underlying issue is that you're microhosting and can't hide behind a large cloud provider's domain front, so isn't that inherent to anything you might do?
In other words, blocking solutions that know your small blog is hosted exclusively on 1.2.3.4, without any collateral damage to other blogs the blocking government cares about will just block your IP.
Conversely, if you're hosting importedgoodsecommercesitegovernmentofficialslove.com next to myhumanrightsblog.net on the same IP, ECH is for you and solves your problem: Just register mycoolagnostichosting.net and do ECH to that.
"Just buy a second domain exclusively to work around the arbitrary restrictions put onto the protocol" works as a solution, but it's a silly solution that shouldn't be necessary.
ECH doesn't benefit you if you're connecting directly to one IP. Middleboxes can track that you're connecting to this IP.
ECH prevents tracking through routing layers where your ClientHello might contain foo.example.com or bar.example.com but route via the same IP (Cloudflare). A middlebox can see you are using a cloudflare hosted website, but not know what cloudflare website.
There's no benefit encrypting the SNI with 10.20.30.40 if they can see you're connecting to 10 20.30.40 anyway
THe benefit is that the SNI is not being logged. Resolving an IP to a domain name is pretty hard for a small actor who doesn't have a record of all DNS records.
That's a good point. I was thinking more of a "block this list of wrongthink TLDs" use case, but "list all hostnames accessed by person x" is of course also worth considering.
Will the adoption of this RFC prevent corporate MITM attacks like Zscaler TLS inspection?
Hopefully: yes
If the client (read: chrome) does support that (and prevent its desactivation), then zscaler and other shitty things are made even more useless than what they are today
I've written a visual article about TLS ECH, in case anyone doesn't have time to read this RFC: https://growingswe.com/blog/tls-ech
TLS Encrypted Client Hello (ECH) standard is another attempt on encrypting plaintext Server Name Indication (SNI). It is very useful for circumventing SNI-based censorship, which is adopted for years by state-backed systems like the Great Firewall (GFW).
The previous attempt of encrypting plaintext SNI is Encrypted Server Name Indication (ESNI), which didn't end well.
As someone hasn’t followed this, what was wrong with ESNI?
I was curious as well, https://blog.mozilla.org/security/2021/01/07/encrypted-clien... has some info
>analysis has shown that encrypting only the SNI extension provides incomplete protection. As just one example: during session resumption, the Pre-Shared Key extension could, legally, contain a cleartext copy of exactly the same server name that is encrypted by ESNI. The ESNI approach would require an encrypted variant of every extension with potential privacy implications, and even that exposes the set of extensions advertised. Lastly, real-world use of ESNI has exposed interoperability and deployment challenges that prevented it from being enabled at a wider scale.
China and Russia started to block ESNI before Cloudflare stopped offering it so any argument that ESNI did not work is dubious
IME, ESNI worked for accessing _all_ websites using CF. AFAIK, ECH has never been offered for all websites using CF
ESNI was a bit simpler to use than ECH, e.g., when making HTTP requests with programs like openssl s_client, bssl client, etc. (I don't use popular browsers to make HTTP requests)
When CF ended the ESNI trial, there was nothing to take its place. The public was asked to wait for ECH
It has been roughly five years (correct me if wrong) without any replacement solution for plaintext SNI
ECH is available on a few test sites, e.g.,
https://test.defo.ie
But software support for ECH makes little practical difference for www users if major CDNs still don't support it
And as far as a solution that applies to CDNs other than CF, there has been no solution at all
Plaintext SNI is everywhere. It more or less defeats the stated purpose of "encrypted DNS"
As far as I know, ESNI was pushed to a large number of websites in a very fast pace, and soon completely blocked by GFW. Then I didn't hear news about ESNI.
It's an interesting feature, but it's pushing my buttons lately. Specifically on Cloudflare it is on by default, and on the free tier you can't disable it and you need a business plan for it. Which I think is stupid, but never the less it's causing us problems. We are trying some split-dns setup for a company "intranet", and if the site's have be accessed before, the ECH is remembered. So the browser tries and it eventually fails with ECH Error or on Firefox it just hangs like it's loading all the time. And it's so frustrating, because sometimes it works, sometimes it doesn't, you can clear cache and stuff and it still won't work, sometimes it works in Incognito sometimes it doesn't. This is not a real problem, but since we haven't fully switched to the "intranet" and we use some of the WAF features of Cloudflare sometimes it is so frustrating.
Split dns is such an ugly hack, it causes no end of hard to debug problems.
It is yeah, but it is a temporary solution while we are working out the setup.
This would kill SNI proxies, correct?
Oh snap. Say I'd like to compute some JA4 hashes on my server. I know this can be done as the server has the keys required. But is there a ready made solution that can account for ECH and compute JA4 as a proxy? Please? I saw HAProxy should work and I know that nginx module for JA4 is incomplete, are you please aware of other solutions for self-hosting?
I'm okay advertising ECH as a privacy enhancement, it is, but make no mistake, it will not prevent government tracking or censorship. Governments will request platforms to disable ECH or ask for other signals to be reported, and ban those who don't comply. You can't fight political issues solely with technology.
I know this has nothing to do with the RFC, but I wonder what RFC 10000 will be. I'm just hoping for some sort of joke RFC like TCP over Avian Carriers or the Oops I'm A teapot.
ECH is great from a privacy perspective, but I’m curious how well this will actually work in practice.every time the web encrypts more metadata there’s pushback from middleboxes and network operators.
> I’m curious how well this will actually work in practice
You're experiencing it working in practice. RFC9849 is a published document, the end of a very long process in which the people who make this "actually work in practice" decided how to do this years ago and have deployed it.
This isn't like treaty negotiation where the formal document often creates a new reality, the RFC publication is more like the way the typical modern marriage ceremony is just formalising an existing reality. Like yeah, yesterday Bill and Sarah were legally not married, and today Bill and Sarah are married, but "Bill and Sarah" were a thing five Christmases ago, one of the bridesmaids is their daughter, we're just doing some paperwork and having a party.
The tension is that Security and Dev parts of the stack remove the actual troubleshooting capabilities of the Network layer without opening up the tools that are supposed to replace them.
It's not a problem if Network can still do their job. It's a whole other matter to expect Network to do their job through another layer. You end up with organizations that can't maintain their applications and expect magic fixes.
Orgs that are cooperative probably don't have this issue but there are definitely parts of some organizations that when one part takes capability from another they don't give it back in some sort of weird headcount game despite not really wanting to understand Network to a Network level.
This feels like a recurring pattern in the stack. abstraction removes visibility faster than tooling replaces it.
Encryption and higher-level platforms are great for security and productivity, but the debugging surface keeps shrinking. Eventually when something breaks, nobody actually has the layer-by-layer visibility needed to reason about it.
ECH won't be effective until there's a HSTS-style policy that forces browsers to use it. Otherwise, firewalls will continue to strip parameters and downgrade connections[0].
0: https://community.fortinet.com/t5/FortiGate/Technical-Tip-Ho...
The Fortigate article proposes that you take a profile in which your end users have said OK, I trust the Fortigate to decide what's allowed, and then you set it to not allow them to use ECH.
Notice that if users don't trust the Fortigate all it can do is IP layer blocks, exactly as intended.
It seems pointless to try to have a policy where people say they trust somebody else (whoever is operating that Fortigate) to override their will but also they don't want their will overridden, that's an incoherent policy, there's no technical problem there, technology can't help.
Well, yes, this is being used in corporate environments but the end user and the system admin aren't on the same page necessarily. Domain blocking doesn't make much sense in my opinion and should be a thing of the past. You already lack admin rights so what is a block on e.g. mullvad.net actually doing other than stopping someone from reading their blog? They can't install the VPN software.
Defense in layers makes sense, but domain blocking was never a "layer" if a hostile actor can just buy a new domain that's not on your blocklist.
I think it'd be good if ECH became more widespread so that we can get away from these antiquated control techniques that just result in frustration with no security benefits.
Will this have an impact on Loadbalancers? Like does one have to do client side load balancing like in gRPC?
My understanding is that you can use split mode to only have the load balancer decrypt the server name section, and forward the actual session and key exchange down to the backend without doing double layer encryption.
If your load balanced doesn't support ECH, don't tell clients to use ECH.
The loadbalancer can force a downgrade .
If the load balancer can force a downgrade, an attacker can do it as well.
Only if the attacker has a valid certificate for the domain to complete the handshake with.
Relying on HTTPS and SVCB records will probably allow a downgrade for some attackers, but if browsers roll out something akin to the HSTS preload list, then downgrade attacks become pretty difficult.
DNSSEC can also protect against malicious SVCB/HTTPS records and the spec recommends DoT/DoH against local MitM attacks to prevent this.
DNSSEC can't protect against an ECH downgrade. ECH attackers are all on-path, and selectively blocking lookups is damaging even if you can't forge them. DoH is the answer here, not record integrity.
DNSSEC alone is obviously useless because any attacker interested in SNI hostnames can just as easily monitor DNS traffic.
However, DoH/DoT without record integrity is about as useful as self-signed HTTPS certificates. You need both for the system to work right in every case.
To quote the spec:
> Clearly, DNSSEC (if the client validates and hard fails) is a defense against this form of attack, but encrypted DNS transport is also a defense against DNS attacks by attackers on the local network, which is a common case where ClientHello and SNI encryption are desired. Moreover, as noted in the introduction, SNI encryption is less useful without encryption of DNS queries in transit.
I don't think this is true; I think this misunderstands the ECH threat model. You don't need record integrity to make ECH a strong defense against on-path ISP attackers; you just need to trust the resolver you're DoH'ing to.
> but if browsers roll out something akin to the HSTS preload list, then downgrade attacks become pretty difficult.
Can you explain why, considering it is at the client's side ("browsers")?
If browsers remember which domains do ECH and refuse to downgrade to non-ECH connections after, the way the HSTS cache forces browsers to connect over HTTPS despite direct attempts to load over HTTP, then you only need an entry in the browser database to make downgrade attacks to accomplish SNI-snooping impossible.
For HSTS, browsers come with a preloaded list of known-HTTPS domains that requests are matched against. That means they will never connect over HTTP, rather than connect over HTTP and upgrade+maintain a cache when the HSTS header is present. If ECH comes with a preload list, then browsers connecting to ECH domains will simply fail to connect rather than permit the network to downgrade their connection to non-ECH TLS.
If only we had some technology that would relieve us of the need to share IP addresses among multiple servers, this might have been unnecessary.
If every server has its own IP then the privacy leak comes from merely connecting to it. It makes ECH useless, but that's not the same thing as making it unnecessary.