I don't like this post's negativity towards ARP. ARP is the reason we can have IP networking on a LAN without a router. The default gateway just becomes a special case of general IP networking on a LAN.
Otherwise, the networking history part of this post is amazing. I haven't gotten to the IPv6 part yet.
But it's not the only way to tackle the problem of resolving layer 2 addresses, and you can do so without introducing the layering violations and expansive broadcast traffic that ARP implies (along with the consequent problems with WiFi and such).
For instance, IPv6's NDP is built on actual IPv6 packets (ICMPv6), rather than some spoofed IP-lookalike thing. No layering violation, and, thanks to multicasting, no need to dump a bunch of broadcast traffic on the layer 2 network.
What is this article even on about? The stuff on my network assigns itself ipv6 addresses based on their mac address? That's how you can do stateless ipv6?
Regardless, ipv6 was to have more IP addresses because of ipv4 exhaustion and NAT?
My Xbox tells me my network sucks because it doesn't have ipv6, but this is a very North-American perspective regardless.
> The stuff on my network assigns itself ipv6 addresses based on their mac address? That's how you can do stateless ipv6?
Nit: per RFC8064[0], most modern, non-server devices do/should configure their addresses with "semantically opaque interface identifiers"[1] rather than using their MAC address/EUI64. That stable address gets used for inbound traffic, and then outbound traffic can use temporary/privacy addresses that are randomized and change over time.
Statelessness is accomplished simply by virtue of devices self-assigning addresses using SLAAC, rather than some centralized configuration thing like DHCPv6.
> My Xbox tells me my network sucks because it doesn't have ipv6
Pretty sure that it's complaining about lack of upnp. Which, yes, would not be an issue if we had ipv6... but ironically consoles typically have been slow to adopt ipv6 support themselves, so I'm curious if the xbox even supports it..
Steam and Meta Quest are both terrible at ipv6. At least from a year or so back. My home netowkr supported good ipv6 networking on two providers. Steam games would mess up constantly and Quest would take minutes to load.
Steam having issues makes sense given its been around ages. Meta Quest is all new OS and code yet they managed to bork ipv6. Super annoying.
There's one point I don't really get and I would be glad if someone could clarify it for me. When the author says that even over wifi, the CSMDA/CD protocol is not used anymore. Then how does it actually work?
Discussing this, the author explains:
> If you have two wifi stations connected to the same access point, they don't talk to each other directly, even when they can hear each other just fine.
So, each station still has to decide at some point if what its hearing is for them or not, as it could be another station talking to the AP, or the AP talking to another station. How is that done if not using CSMA/CD (or something very similar at least)?
Thanks. So the author's point in the linked article is wrong, it's the opposite of what they wrote. Contrary to what they say, it's indeed a bus, and it isn't the case that CSMA/CD is useless, it's that isn't enough to deal with the situation, so additions have been made to it.
Thanks for your link that helped clarifying this for me!
When you have switches that link two nodes together, for only the duration of one-way transmission you don't need CSMA/CD. We literally have no use for it. We will never have two computers transmit onto the same Ethernet wire anymore.
WiFi is different of course. However as the author wrote, your WiFi devices always go through the access point where they use 802.11 RTS/CTS messages to request and receive permission to send packets. All nodes can see CTS being broadcasted so they know that somebody is sending something. So even CSMA/CA is getting less useful.
Yes I'm only talking about wifi networks. I get that CSMA/CD itself is getting less useful, but it's because something else is doing its job, not because what it did is useless (that's why I wrote "or something similar" when I asked). Wifi is still, necessarily, a common bus where everyone talks.
> Now imagine that X changes addresses to Q. It still sends out packets tagged with (uuid,80), to IP address Y, but now those packets come from address Q. On machine Y, it receives the packet and matches it to the socket associated with (uuid), notes that the packets for that socket are now coming from address Q, and updates its cache. Its return packets can now be sent, tagged as (uuid), back to Q instead of X. Everything works! (Modulo some care to prevent connection hijacking by impostors.2)
And how the fuck anything in-between knows where to route it ? The article glows a blazing beacon of ignorance about everything in-between.
The whole entire problem with mobile IP is "how we get intermediate devices to know where to go?" we're back to
> The problem with ethernet addresses is they're assigned sequentially at the factory, so they can't be hierarchical.
Which author hinted at then forgot. We can't have globally routable, unique, random-esque ID precisely because it has to be hierarchical. Keeping connection flow ID at L4 instead of L3+L4 changes very little, yeah, you can technically roam the client except how the fuck server would know where to send the packet back when L3 address changes ? It would have to get client packet with updated L3 address and until then all packets would go to void.
But hey, at least it's some progress ? NOPE, nothing at protocol layer can be trusted before authentication, it would make DoS attacks far easier (just flood the host in a bunch of random uuids), and you would still end up doing it QUIC way of just re-implementing all of that stuff after encryption of the insides
> And how the fuck anything in-between knows where to route it ? The article glows a blazing beacon of ignorance about everything in-between.
Because the IP address changed, so classic routing still works. Their point is about identifying a session with something non-constant (the IP of the client), rather than a session token.
Instead of identifying the "TCP" socket with (src ip, src port, dst ip, dst port), they use (src uuid, dst uuid) which allows flows to keep working when you change IP addresses. Just like you can change networks and still have your browser still logged in to most websites.
The packets carrying those UUIDs still are regular old IP packets, UDP in the case of QUIC. Only the server needs to track anything, and only has to change the dst ip of outgoing packets.
As for flooding and DDoS, that’s what handshakes are for, and QUIC already does it (disclaimer: never dug deep in how QUIC works so I can’t explain the mechanism here).
> We can't have globally routable, unique, random-esque ID precisely because it has to be hierarchical
This is not, technically, true. We could have globally-routable, unique, random-esque IDs if every routing device in the network had the capacity to store and switch on a full table of those IDs.
I'm not saying this is feasible, mind you, just that it's not impossible.
I guess it depends on what you mean by "impossible." If you only mean that it's theoretically possible, then sure, one can imagine a world where that is done. But even with IPv4's meager 32 bits of address space, it would explode TCAM requirements on routers if routers would start accepting announcements of /32s instead of the /24s that are accepted now. And /64 (or, jesus, /128) for IPv6? That's impossible.
That was said in my comment. The routing devices need to both be able to store the full table, and also to switch packets by looking up entries within it.
This is one of my favourite blog posts ever. For those unaware (or who didn't read right to the bottom), the author is the CEO of Tailscale.
One of the problems we have is when we're born we don't question anything. It just is the way it is. This, of course, lets us do things in the world much more quickly than if we had to learn everything from basic principles, but it's a disadvantage too. It means we get stuck in these local optima and can't get out. Each successive generation only finally learns enough to change anything fundamental once they're already too old and set in their ways doing the standard thing.
How I wish we could have a new generation of network engineers who just say "fuck this shit" and build their own internet.
> One of the problems we have is when we're born we don't question anything
I don't know about you personally but every grade-school, high-school, & college level instructor I ever had would probably vehemently disagree with this statement about me. I remember at least 70 year old college instructor becoming visibly irritated that I would ask what research supported the assertions he made
The source and destination addresses don't change. If a bomb takes out a router in-between (the military scenario DARPA had in mind), it is NOT IP (L3) or TCP (L4) that handles it. Rather it is a dynamic routing protocol that informs all affected routers of the changed route. Since the early days of the Internet, that's been the job of routing protocols.
For smaller internets, protocols such as RIP (limited to 16 hops) broadcast routing information from each still-working router to other routers. Each router built a picture of the internet (simplifying a bit here, RIP and similar protocols used "distance vector" routing, but other more advanced routing protocols did have each a picture of the internet). So when a packet arrived at its router, that router can forward the pack towards the destination. Such protocols are "interior" routing protocols, used within an ISP's network.
The Internet is too big for such automatic routing and uses an "exterior" routing protocol called BGP. This protocol routes packets from one ISP to the next, using route and connectivity information input by humans. (Again I'm simplifying a bit.)
Wifi uses entirely different protocols to route packets between cells.
Fun fact: wifi is not an acronym for anything, the inventors simply liked how it sounded.
Moving running computers around and maintaining connection would have required large trucks and very long cables at the time the internet was invented.
Mobile IPv6 is a thing. I could compile it into the kernel on my mobile machines, main reason to not do it is that I'm currently using a phone as a WiFi hotspot and it doesn't have Mobile IPv6 support.
the mobility in context of article means "changing IP within same TCP connection".
IP + some dynamic routing handles the situation of "the connection site got nuked and we need to route around it", it's just not in the protocol, it's additional layer on top of it
But there's now multipath TCP handover? Weird behaviour to want different network interfaces on different network share the same IP, and pass it along like a volleyball?
Wi-Fi and ethernet also have different IPs. And what if you also add Wi-Fi peer-to-peer (Airdrop-ish), Wi-Fi Tunneled Direct Link Setup (literally Chromecast)?
If a vendor implemented simultaneous Dual Band (DBDC) Wi-Fi, that means it can connect to both 2.4ghz and 5ghz at the same time, each with their own mac & ip, because you're trying to connect to the same network on a different band. Or route packages from a 'wan' Wi-Fi to a 'lan' Wi-Fi (share internet on (BSS) infrastructure Wi-Fi A to a new (IBSS) ad-hoc Wi-Fi network B with your smartphone as the gateway on Android.
There's also 802.11 the IEEE 802.11 standard to add wireless access in vehicular environments (WAVE) and EV chargers or IP over the CCS protocol, etc. If all cars need to be 'connected' and 'have a unique address' NAT / CGNAT also isn't cutting it.
There's also IoT. Thread is ipv6 because it's the alternative to routing whatever between wan / lan / zigbee / Z-Wave / etc with a specific gateway at a remote point in the mesh network.
And how about the new DHCP / DNS specs for ipv6, you can now share encrypted DNS servers, DHCP client-ID, unique OUID, etc etc.
It's an infuriating post really. As if IP was only designed for a small scale VPN / overlay network service such as Tailscale.
> But there's now multipath TCP handover? Weird behaviour to want different network interfaces on different network share the same IP, and pass it along like a volleyball?
Mobile IP actually wanted to do this, it just never took off (not the least because both endpoints need to understand it to get route optimization). I think some Windows versions actually had partial Mobile IPv6 support.
I don't like this post's negativity towards ARP. ARP is the reason we can have IP networking on a LAN without a router. The default gateway just becomes a special case of general IP networking on a LAN.
Otherwise, the networking history part of this post is amazing. I haven't gotten to the IPv6 part yet.
But it's not the only way to tackle the problem of resolving layer 2 addresses, and you can do so without introducing the layering violations and expansive broadcast traffic that ARP implies (along with the consequent problems with WiFi and such).
For instance, IPv6's NDP is built on actual IPv6 packets (ICMPv6), rather than some spoofed IP-lookalike thing. No layering violation, and, thanks to multicasting, no need to dump a bunch of broadcast traffic on the layer 2 network.
What is this article even on about? The stuff on my network assigns itself ipv6 addresses based on their mac address? That's how you can do stateless ipv6?
Regardless, ipv6 was to have more IP addresses because of ipv4 exhaustion and NAT?
My Xbox tells me my network sucks because it doesn't have ipv6, but this is a very North-American perspective regardless.
> The stuff on my network assigns itself ipv6 addresses based on their mac address? That's how you can do stateless ipv6?
Nit: per RFC8064[0], most modern, non-server devices do/should configure their addresses with "semantically opaque interface identifiers"[1] rather than using their MAC address/EUI64. That stable address gets used for inbound traffic, and then outbound traffic can use temporary/privacy addresses that are randomized and change over time.
Statelessness is accomplished simply by virtue of devices self-assigning addresses using SLAAC, rather than some centralized configuration thing like DHCPv6.
[0] https://datatracker.ietf.org/doc/rfc8064/ [1] https://datatracker.ietf.org/doc/rfc7217/
> My Xbox tells me my network sucks because it doesn't have ipv6
Pretty sure that it's complaining about lack of upnp. Which, yes, would not be an issue if we had ipv6... but ironically consoles typically have been slow to adopt ipv6 support themselves, so I'm curious if the xbox even supports it..
Steam and Meta Quest are both terrible at ipv6. At least from a year or so back. My home netowkr supported good ipv6 networking on two providers. Steam games would mess up constantly and Quest would take minutes to load.
Steam having issues makes sense given its been around ages. Meta Quest is all new OS and code yet they managed to bork ipv6. Super annoying.
Ironic, considering that Meta is one of the more notable companies to run IPv6-only internally.
Does Xbox work if there’s no IPv4? Because my corporate windows device doesn’t and I know other gaming consoles don’t.
It’d be interesting to compile a list of hardware/software that works fine in an ipv6 only network.
Thanks for sharing this very interesting read.
There's one point I don't really get and I would be glad if someone could clarify it for me. When the author says that even over wifi, the CSMDA/CD protocol is not used anymore. Then how does it actually work?
Discussing this, the author explains:
> If you have two wifi stations connected to the same access point, they don't talk to each other directly, even when they can hear each other just fine.
So, each station still has to decide at some point if what its hearing is for them or not, as it could be another station talking to the AP, or the AP talking to another station. How is that done if not using CSMA/CD (or something very similar at least)?
> How is that done if not using CSMA/CD (or something very similar at least)?
AFAIK, WiFi has always been doing CSMA/CA and starting with the 802.11ax standard also OFDMA. See https://en.wikipedia.org/wiki/Hidden_node_problem#Background
Thanks. So the author's point in the linked article is wrong, it's the opposite of what they wrote. Contrary to what they say, it's indeed a bus, and it isn't the case that CSMA/CD is useless, it's that isn't enough to deal with the situation, so additions have been made to it.
Thanks for your link that helped clarifying this for me!
When you have switches that link two nodes together, for only the duration of one-way transmission you don't need CSMA/CD. We literally have no use for it. We will never have two computers transmit onto the same Ethernet wire anymore.
WiFi is different of course. However as the author wrote, your WiFi devices always go through the access point where they use 802.11 RTS/CTS messages to request and receive permission to send packets. All nodes can see CTS being broadcasted so they know that somebody is sending something. So even CSMA/CA is getting less useful.
Yes I'm only talking about wifi networks. I get that CSMA/CD itself is getting less useful, but it's because something else is doing its job, not because what it did is useless (that's why I wrote "or something similar" when I asked). Wifi is still, necessarily, a common bus where everyone talks.
Not really. Wifi does not do CSMA/CD. It does CSMA/CA, something quite different.
Wifi is in any case not considered a bus network, rather a star topology network.
Very interesting post. I never considered the fact that IPv6 was going to be more than just a bigger IPv4.
Also funny it was made in 1990 and it only recently reached 50% adoption.
> Now imagine that X changes addresses to Q. It still sends out packets tagged with (uuid,80), to IP address Y, but now those packets come from address Q. On machine Y, it receives the packet and matches it to the socket associated with (uuid), notes that the packets for that socket are now coming from address Q, and updates its cache. Its return packets can now be sent, tagged as (uuid), back to Q instead of X. Everything works! (Modulo some care to prevent connection hijacking by impostors.2)
And how the fuck anything in-between knows where to route it ? The article glows a blazing beacon of ignorance about everything in-between.
The whole entire problem with mobile IP is "how we get intermediate devices to know where to go?" we're back to
> The problem with ethernet addresses is they're assigned sequentially at the factory, so they can't be hierarchical.
Which author hinted at then forgot. We can't have globally routable, unique, random-esque ID precisely because it has to be hierarchical. Keeping connection flow ID at L4 instead of L3+L4 changes very little, yeah, you can technically roam the client except how the fuck server would know where to send the packet back when L3 address changes ? It would have to get client packet with updated L3 address and until then all packets would go to void.
But hey, at least it's some progress ? NOPE, nothing at protocol layer can be trusted before authentication, it would make DoS attacks far easier (just flood the host in a bunch of random uuids), and you would still end up doing it QUIC way of just re-implementing all of that stuff after encryption of the insides
> And how the fuck anything in-between knows where to route it ? The article glows a blazing beacon of ignorance about everything in-between.
Because the IP address changed, so classic routing still works. Their point is about identifying a session with something non-constant (the IP of the client), rather than a session token.
Instead of identifying the "TCP" socket with (src ip, src port, dst ip, dst port), they use (src uuid, dst uuid) which allows flows to keep working when you change IP addresses. Just like you can change networks and still have your browser still logged in to most websites.
The packets carrying those UUIDs still are regular old IP packets, UDP in the case of QUIC. Only the server needs to track anything, and only has to change the dst ip of outgoing packets.
As for flooding and DDoS, that’s what handshakes are for, and QUIC already does it (disclaimer: never dug deep in how QUIC works so I can’t explain the mechanism here).
> We can't have globally routable, unique, random-esque ID precisely because it has to be hierarchical
This is not, technically, true. We could have globally-routable, unique, random-esque IDs if every routing device in the network had the capacity to store and switch on a full table of those IDs.
I'm not saying this is feasible, mind you, just that it's not impossible.
I guess it depends on what you mean by "impossible." If you only mean that it's theoretically possible, then sure, one can imagine a world where that is done. But even with IPv4's meager 32 bits of address space, it would explode TCAM requirements on routers if routers would start accepting announcements of /32s instead of the /24s that are accepted now. And /64 (or, jesus, /128) for IPv6? That's impossible.
It isn't just storage - lookup in those tables needs to be usefully fast.
That was said in my comment. The routing devices need to both be able to store the full table, and also to switch packets by looking up entries within it.
[flagged]
The article is from 2017?
This is one of my favourite blog posts ever. For those unaware (or who didn't read right to the bottom), the author is the CEO of Tailscale.
One of the problems we have is when we're born we don't question anything. It just is the way it is. This, of course, lets us do things in the world much more quickly than if we had to learn everything from basic principles, but it's a disadvantage too. It means we get stuck in these local optima and can't get out. Each successive generation only finally learns enough to change anything fundamental once they're already too old and set in their ways doing the standard thing.
How I wish we could have a new generation of network engineers who just say "fuck this shit" and build their own internet.
> One of the problems we have is when we're born we don't question anything
I don't know about you personally but every grade-school, high-school, & college level instructor I ever had would probably vehemently disagree with this statement about me. I remember at least 70 year old college instructor becoming visibly irritated that I would ask what research supported the assertions he made
(2017)
Previous discussions:
https://news.ycombinator.com/item?id=14986324 (2017)
https://news.ycombinator.com/item?id=20167686 (2019)
https://news.ycombinator.com/item?id=25568766 (2020)
https://news.ycombinator.com/item?id=37116487 (2023)
Thanks! Macroexpanded...
The world in which IPv6 was a good design (2017) - https://news.ycombinator.com/item?id=37116487 - Aug 2023 (306 comments)
The world in which IPv6 was a good design (2017) - https://news.ycombinator.com/item?id=25568766 - Dec 2020 (131 comments)
The world in which IPv6 was a good design (2017) - https://news.ycombinator.com/item?id=20167686 - June 2019 (238 comments)
The world in which IPv6 was a good design - https://news.ycombinator.com/item?id=14986324 - Aug 2017 (191 comments)
Why repeating the old news?
> Internet routing can't handle mobility - at all.
so all the fairy tales about IP invented for nuclear war was a lie? the moment military started moving around, IP became useless?
The source and destination addresses don't change. If a bomb takes out a router in-between (the military scenario DARPA had in mind), it is NOT IP (L3) or TCP (L4) that handles it. Rather it is a dynamic routing protocol that informs all affected routers of the changed route. Since the early days of the Internet, that's been the job of routing protocols.
For smaller internets, protocols such as RIP (limited to 16 hops) broadcast routing information from each still-working router to other routers. Each router built a picture of the internet (simplifying a bit here, RIP and similar protocols used "distance vector" routing, but other more advanced routing protocols did have each a picture of the internet). So when a packet arrived at its router, that router can forward the pack towards the destination. Such protocols are "interior" routing protocols, used within an ISP's network.
The Internet is too big for such automatic routing and uses an "exterior" routing protocol called BGP. This protocol routes packets from one ISP to the next, using route and connectivity information input by humans. (Again I'm simplifying a bit.)
Wifi uses entirely different protocols to route packets between cells.
Fun fact: wifi is not an acronym for anything, the inventors simply liked how it sounded.
> Fun fact: wifi is not an acronym for anything, the inventors simply liked how it sounded.
Most certainly it's a reference to "Sci-Fi" or "Hi-Fi".
I always thought Wi-Fi meant wireless fidelity? (Or wireless fiction since in the end, everything is wired).
Moving running computers around and maintaining connection would have required large trucks and very long cables at the time the internet was invented.
Mobile IPv6 is a thing. I could compile it into the kernel on my mobile machines, main reason to not do it is that I'm currently using a phone as a WiFi hotspot and it doesn't have Mobile IPv6 support.
the mobility in context of article means "changing IP within same TCP connection".
IP + some dynamic routing handles the situation of "the connection site got nuked and we need to route around it", it's just not in the protocol, it's additional layer on top of it
But there's now multipath TCP handover? Weird behaviour to want different network interfaces on different network share the same IP, and pass it along like a volleyball?
Wi-Fi and ethernet also have different IPs. And what if you also add Wi-Fi peer-to-peer (Airdrop-ish), Wi-Fi Tunneled Direct Link Setup (literally Chromecast)?
If a vendor implemented simultaneous Dual Band (DBDC) Wi-Fi, that means it can connect to both 2.4ghz and 5ghz at the same time, each with their own mac & ip, because you're trying to connect to the same network on a different band. Or route packages from a 'wan' Wi-Fi to a 'lan' Wi-Fi (share internet on (BSS) infrastructure Wi-Fi A to a new (IBSS) ad-hoc Wi-Fi network B with your smartphone as the gateway on Android.
There's also 802.11 the IEEE 802.11 standard to add wireless access in vehicular environments (WAVE) and EV chargers or IP over the CCS protocol, etc. If all cars need to be 'connected' and 'have a unique address' NAT / CGNAT also isn't cutting it.
There's also IoT. Thread is ipv6 because it's the alternative to routing whatever between wan / lan / zigbee / Z-Wave / etc with a specific gateway at a remote point in the mesh network.
And how about the new DHCP / DNS specs for ipv6, you can now share encrypted DNS servers, DHCP client-ID, unique OUID, etc etc.
It's an infuriating post really. As if IP was only designed for a small scale VPN / overlay network service such as Tailscale.
> But there's now multipath TCP handover? Weird behaviour to want different network interfaces on different network share the same IP, and pass it along like a volleyball?
Mobile IP actually wanted to do this, it just never took off (not the least because both endpoints need to understand it to get route optimization). I think some Windows versions actually had partial Mobile IPv6 support.