Nginx mainline 1.29.x supports it.
So once you get that and also the openssl version on your system, good to go.
Likely too late for ubuntu 26.04, maybe in debian 14 next year, or of course rolling release distros / containers.
But, in a personal/single website server, ech does not really add privacy, adversaries can still observe the IP metadata and compare what's hosted there. The real benefits are on huge cloud hosting platforms.
FWIW Nginx 1.30 [1] just released and supports it so most distributions will have support as soon as those responsible for builds and testing builds push it forward.
"Nginx 1.30 incorporates all of the changes from the Nginx 1.29.x mainline branch to provide a lot of new functionality like Multipath TCP (MPTCP)."
"Nginx 1.30 also adds HTTP/2 to backend and Encrypted Client Hello (ECH), sticky sessions support for upstreams, and the default proxy HTTP version being set to HTTP/1.1 with Keep-Alive enabled."
But, in a personal/single website server, ech does not really add privacy, adversaries can still observe the IP metadata and compare what's hosted there
I don't quite follow. I have dozens of throw-away silly hobby domains. I can use any of them as the outer-SNI. How is someone observing the traffic going to know the inner-SNI domain unless someone builds a massive database of all known inner+outer combinations which can be changed on a whim? ECH requires DOH so unless the ISP has tricked the user into using their DOH end-point they can't see the HTTPS resource record.
It's not that adversaries can directly see the domain name; this doesn't have anything to do with domain fronting. The issue is that ECH doesn't hide the server's IP address, so it's mostly useless for privacy if that IP address uniquely identifies that server. The situation where it helps is if the server shares that IP address with lots of other people, i.e., if it's behind a big cloud CDN that supports ECH (AFAIK that's currently just Cloudflare). But if that's the case, it doesn't matter whether Nginx or whatever other web server you run supports ECH, because your users' TLS negotiations aren't with that server, they're with Cloudflare.
I can't speak for anyone else but I think I can work around that by moving the site around to different VPS nodes from time to time. I get bored with my silly hobby sites all the time and nuke the VM's then fire them up later which gives them a new IP. I don't know what others might do if anything.
If I had a long running site I could do the same thing by having multiple font-end caching nodes using HAProxy or NGinx that come and go but I acknowledge others may not have the time to do that and most probably would not.
That's not quite it. The issue is that there's no other traffic bound to that IP - ECH doesn't buy you any security, because an observer doesn't even need to look at the content of the traffic to know where it's headed.
Maybe it will be more useful for outbound from NGinx or HAProxy to the origin server using ECH so the destination ISP has no idea what sites are on the origin assuming that traffic is not passing over a VPN already.
Anyone who wants to track your users can just follow the IP changes as they occur in real time.
That's cool. I only make my own mini-CDN's.
There is always the option to put sites on a .onion domain but I don't host anything nearly exciting or controversial enough. For text that's probably a good option. I don't know if Tor is fast enough for binary or streaming sites yet. No idea how many here even know how to access a .onion site.
I will test out your theory and see if anyone bothers to track my IP addresses and does anything with them. I probably need to come up with something edgy that people would want to block. Idea's for something edgy?
Doesn't matter, I (not OP, but also operating VPS) still want to support this, so the clients can eventually assume all correctly configured servers support it.
TLS (the IETF Working Group not the protocol family named for them) have long experience with the fact that if you specify how B is compatible with A based on how you specified A and ship B what you did won't work because the middleboxes are all cost optimized and don't implement what you specified but instead whatever got the sale for the least investment.
So e.g. they'd work for exactly the way you use say TLS 1.0 in the Netscape 4 web browser which was popular when the middlebox was first marketed, or maybe they cope with exactly the features used in Safari but since Safari never sets this bit flag here they reject all connections with that flag.
What TLS learned is summarized as "have one joint and keep it well oiled" and they invented a technique to provide that oiling for one working joint in TLS, GREASE, Generate Random Extensions And Sustain Extensibility. The idea of GREASE is, if a popular client (say, the Chrome web browser) just insists on uttering random nonsense extensions then to survive in the world where that happens you must not freak out when there are extensions you do not understand. If your middlebox firmware freaks out when seeing this happen, your customers say "This middlebox I bought last week is broken, I want my money back" so you have to spend a few cents more to never do that.
But, since random nonsense is now OK, we can ship a new feature and the middleboxes won't freak out, so long as our feature looks similar enough to GREASE.
ECH achieves the same idea, when a participating client connects to a server which does not support ECH as far as it knows, it acts exactly the same as it would for ECH except, since it has neither a "real" name to hide nor a key to encrypt that name it fills the space where those would fit with random gibberish. As a server, you get this ECH extension you don't understand, and it is filled with random gibberish you also don't understand, this seems fine because you didn't understand any of it (or maybe you've switched it off, either way it's not relevant to you).
But for a middlebox this ensures they can't tell whether you're doing ECH. So, either they reject every client which could do ECH, which again that's how you get a bunch of angry customers, or, they accept such clients and so ECH works.
Just be aware any reasonable network will block this.
Russia blocked it for Cloudflare because the outer SNI was obviously just for ECH but that won't stop anyone from using generic or throw-away domains as the outer SNI. As for reasonable I don't quite follow. Only censorious countries or ISP's would do such a thing.
I can foresee Firewall vendors possibly adding a category for known outer-SNI domains used for ECH but at some point that list would be quite cumbersome and may run into the same problems as blocking CDN IP addresses.
You'd say incorrectly, firewalls have an implicit deny rule, so any case ICMP traverses a firewall, someone wanted it to. Obviously large hosting providers tend to find value in ICMP being enabled.
But for example, our firewall at work responds to ICMP but all of the endpoints which aren't meant for public use do not. That is less because ICMP is a problem and more because everything works fine without it and least privilege is good design.
ICMP is also more than just ping, and some parts of ICMP are considered a vulnerability if exposed to the public internet by some scanning services.
That kind of cargo culted tradition is how you end up with weird packet loss and VPNs that flat-out refuse to work.
I could be convinced to block inbound pings. Anything past that and I'd want solid evidence that it wouldn't break anything, with the expectation that it would.
Any "reasonable" network just sees a regular Client Hello, the rest is encrypted. They designed it with your very concern in mind to obscure that the ECH even happens.
Eventually these blocks won't be viable when big sites only support ECH. It's a stopgap solution that's delaying the inevitable death of SNI filtering.
How is OpenSSl these days? I vaguely remember the big ruckus a while back, was it Heartbleed? where everyone to their horror realized it was maybe 1 or 2 people trying to maintain OpenSSL, and the OpenBSD people then throwing manpower at it to clear up a lot of old outstanding bugs. It seems like it is on firmer/more organized footing these days?
Compared to OpenSSL 3 this transition has been very smooth. Only dropping of "Engines" was a problem at all, and in Fedora most of those dependencies have been changed.
Manual opt out processes are becoming a major friction point. It's interesting how these tools only improve their defaults after a community backlash. Trust is so hard to build but so easy to burn in this space
> libcrypto no longer cleans up globally allocated data via atexit().
> OPENSSL_cleanup() now runs in a global destructor, or not at all
by default.
Oh oh. Heartbleed 2.0 incoming.
I really do hope that they broke APIs specifically throwing errors or race conditions so that devs are forced to cleanup. Otherwise this is going to be a nightmare to find out in terms of maintenance and audits.
I mean it's a new major release so it's a valid design change. But I hope they're thinking of providing and migration/update guide or a checklist to reduce usage errata.
(I'm heavily in favor of deprecating the fixed version method names)
Good to see const more prevalent. Too often I have to add that in to libraries for embedded. Possibly I believe in const by default but it is what it is at this point
86 comments
What if he wanted to use it for requesting blog.cloudflare.com
Where are the ECH keysFor example,
or It's true one can "use it today". One could use it for the past several years as well. The software has been around for a whileBut ECH has never been consistently enabled for the general public beyond a small number of test sites that are only for testing ECH
But, in a personal/single website server, ech does not really add privacy, adversaries can still observe the IP metadata and compare what's hosted there. The real benefits are on huge cloud hosting platforms.
"Nginx 1.30 incorporates all of the changes from the Nginx 1.29.x mainline branch to provide a lot of new functionality like Multipath TCP (MPTCP)."
"Nginx 1.30 also adds HTTP/2 to backend and Encrypted Client Hello (ECH), sticky sessions support for upstreams, and the default proxy HTTP version being set to HTTP/1.1 with Keep-Alive enabled."
But, in a personal/single website server, ech does not really add privacy, adversaries can still observe the IP metadata and compare what's hosted there
I don't quite follow. I have dozens of throw-away silly hobby domains. I can use any of them as the outer-SNI. How is someone observing the traffic going to know the inner-SNI domain unless someone builds a massive database of all known inner+outer combinations which can be changed on a whim? ECH requires DOH so unless the ISP has tricked the user into using their DOH end-point they can't see the HTTPS resource record.
[1] - https://news.ycombinator.com/item?id=47770007
If I had a long running site I could do the same thing by having multiple font-end caching nodes using HAProxy or NGinx that come and go but I acknowledge others may not have the time to do that and most probably would not.
That's cool. I only make my own mini-CDN's.
There is always the option to put sites on a .onion domain but I don't host anything nearly exciting or controversial enough. For text that's probably a good option. I don't know if Tor is fast enough for binary or streaming sites yet. No idea how many here even know how to access a .onion site.
I will test out your theory and see if anyone bothers to track my IP addresses and does anything with them. I probably need to come up with something edgy that people would want to block. Idea's for something edgy?
So e.g. they'd work for exactly the way you use say TLS 1.0 in the Netscape 4 web browser which was popular when the middlebox was first marketed, or maybe they cope with exactly the features used in Safari but since Safari never sets this bit flag here they reject all connections with that flag.
What TLS learned is summarized as "have one joint and keep it well oiled" and they invented a technique to provide that oiling for one working joint in TLS, GREASE, Generate Random Extensions And Sustain Extensibility. The idea of GREASE is, if a popular client (say, the Chrome web browser) just insists on uttering random nonsense extensions then to survive in the world where that happens you must not freak out when there are extensions you do not understand. If your middlebox firmware freaks out when seeing this happen, your customers say "This middlebox I bought last week is broken, I want my money back" so you have to spend a few cents more to never do that.
But, since random nonsense is now OK, we can ship a new feature and the middleboxes won't freak out, so long as our feature looks similar enough to GREASE.
ECH achieves the same idea, when a participating client connects to a server which does not support ECH as far as it knows, it acts exactly the same as it would for ECH except, since it has neither a "real" name to hide nor a key to encrypt that name it fills the space where those would fit with random gibberish. As a server, you get this ECH extension you don't understand, and it is filled with random gibberish you also don't understand, this seems fine because you didn't understand any of it (or maybe you've switched it off, either way it's not relevant to you).
But for a middlebox this ensures they can't tell whether you're doing ECH. So, either they reject every client which could do ECH, which again that's how you get a bunch of angry customers, or, they accept such clients and so ECH works.
Russia blocked it for Cloudflare because the outer SNI was obviously just for ECH but that won't stop anyone from using generic or throw-away domains as the outer SNI. As for reasonable I don't quite follow. Only censorious countries or ISP's would do such a thing.
I can foresee Firewall vendors possibly adding a category for known outer-SNI domains used for ECH but at some point that list would be quite cumbersome and may run into the same problems as blocking CDN IP addresses.
They were wrong then, of course, and they're still wrong now.
Because I can ping almost any public server on the internet and they will reply. I can ping your website just fine and it replies to me!
But for example, our firewall at work responds to ICMP but all of the endpoints which aren't meant for public use do not. That is less because ICMP is a problem and more because everything works fine without it and least privilege is good design.
ICMP is also more than just ping, and some parts of ICMP are considered a vulnerability if exposed to the public internet by some scanning services.
I could be convinced to block inbound pings. Anything past that and I'd want solid evidence that it wouldn't break anything, with the expectation that it would.
Eventually these blocks won't be viable when big sites only support ECH. It's a stopgap solution that's delaying the inevitable death of SNI filtering.
According to this one should not be using v3 at all..
On the one hand, looks like decent cleanup. (IIRC, engines in particular will not be missed).
On the other hand, breaking compatibility is always a tradeoff, and I still remember 3.x being... not universally loved.
> libcrypto no longer cleans up globally allocated data via atexit().
> OPENSSL_cleanup() now runs in a global destructor, or not at all by default.
Oh oh. Heartbleed 2.0 incoming.
I really do hope that they broke APIs specifically throwing errors or race conditions so that devs are forced to cleanup. Otherwise this is going to be a nightmare to find out in terms of maintenance and audits.
I mean it's a new major release so it's a valid design change. But I hope they're thinking of providing and migration/update guide or a checklist to reduce usage errata.
(I'm heavily in favor of deprecating the fixed version method names)
From what I remember hearing, the move from 2 to 3 was hard.