I am not sure I understand why Headscale was excluded. As far as I know, it is made by people not related to Tailscale.
It would be like complaining Vaultwarden is bad because the Bitwarden project is not fully open source even though Vaultwarden is fully open source and has most of the features implemented.
And Headscale kind of ticks all the other boxes mentioned, except "not headscale", because:
* p2p mesh network - it is a mesh network. And even when mesh is blocked, you can use multiple relay servers (derp) which will relay to the mesh from closest location. And you can host your own derp servers.
* Open source and selfhosted - check
* Not Wireguard (Signature-based blocking) - in cases where wireguard is blocked, the derp relay servers run over https and are usually not blocked based on signatures. For example, I use it with Traefik proxy in TCP mode so I could run derp and other http services on the same 443 port and it works great. So - check?
Packaged in nixpkgs - check
On top of that, if you add Headplane admin UI you get nice graphical management, very similar to the one of Tailscale.
How did you manage to put the derper behind Traefik? It seems to be not supported at least officially:
“ The DERP protocol does a protocol switch inside TLS from HTTP to a custom bidirectional binary protocol. It is thus incompatible with many HTTP proxies. Do not put derper behind another HTTP proxy.”
I did it mostly by religious reason, but also because everyone writes about Headscale and I didn't want to write about things that everyone already knows.
FWIW; Tinc has been my workhorse between my various cloud providers, some on prem remote access for a few offices I consult with, and my home + workstations.
I keep telling myself I should switch it to a wireguard mesh, but the configuration of tinc + the "right defaults" make it pretty neat. It's fun to watch as you roll out a configuration to one node in the mesh, and ping times drop suddenly once it can make a direct connection with the optimal route.
I keep my provisioning script + the public keys in git; so deploying it to a new machine is a git pull, generate key, push public key, and pull on the other nodes. I have about 17+ hosts on it; not using it for very high bandwidth, but I couldn't do what I do without it.
Nailed it. Long time Tinc user here as well for all my personal stuff. Besides the fact that onboarding individual machines is a garbage process, it's my second best "set it and forget it thing" I use; the first being Syncthing which came later (and additionally, the two work very nicely together through weird edge cases like work VPNS and suc.)
That is sadly, the biggest worry about it. Once I collect a few round tuit's, I wanted to see if I could build up a tool to build up the same mesh + routing, perhaps confined to a linux namespace so as to keep it contained, and try to reproduce the ease of configuration, on top of much better maintained core linux tools.
But the key has to be -- the configuration has to be just as simple as tinc to be effective. Like, almost just parse the configuration files and build it up with WG tunnels.
https://github.com/m13253/VxWireguard-Generator <-- this is what I use with a more serious "internal business" use case, so I know wireguard is in the sensitive security/encryption loop. It uses wireguard tunnels to make a broadcast/mesh Layer2 network with vxlan connections, and you then run OSPF/babel/your-favorite-routing-protocol over that.
I am not sure what the author thought was so pompous about nebula. It's unfortunate because it seems to have prevented them from reading the documentation. The ssh interface is used as a debug tool not a configuration tool:
https://nebula.defined.net/docs/guides/debug-ssh-commands/
I really think of ways on how I can use this which seems an amazing almost uncensorable-ish tech on how to connect two pc's. Like sometimes my brain just thinks "pipes (piping server) just for email". Its a bit of an obsession...
Something like a VPN could theoretically be created where the piping server well "pipes" it in an encrypted manner through internet.
I would love to create something like this just for the funzies but what I am more interested about is the transport layer.
Like I want something which can be independent of udp or whatever and the only thing I am worried about is how I will transport them to the other pc and then I can then lets say send them over piping server, send them over matrix or signal if need be too idk.
Is there any foss projects that can help me just hook up into things in a similar manner as to what I am asking?
I want a implementation independent-ish transport layer so that I can experiment with things which I can just pipe if I can be really really honest.
I also want more people to look into it as I use sometimes piping-server as a way to transporting files between podman containers even though its a bit slow just to try it out and honestly, just having the fun of installing curl and then being ready to go makes it so much more easier to transport files out of the box... and I want to experiment more with it, its been an obsession for almost an year on and off thinking about piping servers and how elegant they are. I used them of sorts to break an intel nat once, but since then we got some better options if somebody wants to know how to break any nats without any root without any emulation but maybe I want to create a blog post about it someday but I am lazy.
> And also Nebula's interface is absolutely shit.
Instead of a normal CLI, you need to configure an internal sshd and connect via SSH to localhost.
Maybe it's more secure, but it's utterly disgusting.
This seems to be a strong misunderstanding? The ssh interface is for debugging only. You can disable it and configuration is solely handled by the daemon configuration file.
I operate a small (few dozen hosts) network on Nebula with mostly NixOS hosts, so I have some applicable experience. Nebula was primarily chosen because it allows me to, among other things, assign fixed prefixes to hosts and have a full declarative config.
You don't configure a host via a CLI, instead you provide it a signed cert for the privkey + CA cert + private key + lighthouses and that's it. The daemon listens on the IPs from the cert and the lighthouses offer a public exchange where peers advertise their IPs (and associated endpoints for p2p).
The CA approach is also an important part here as all your peers effectively have the CA cert in their config file and use it to verify other peers. The CA signs a cert for each host that contains the IP prefixes that a peer may handle packets for. The only CLI you actually regularly use is here, for creating keys and signing certs.
> And Amnezia VPN has made their own fork of Wireguard, specifically for breaking through government censorship. But the main problem with obfuscation is the reduction of effective packet MTU
The "obfuscation" in Amnezia's fork does not shrink the available MTU (important for QUIC as it requires a minimum MTU of 1280 while WireGuard itself needs +80 bytes or so for route encapsulation). Amnezia's fork modifies the 4 WireGuard header values (which must be pre-agreed between peers) & occassionally appends (to handshake packets) or sends randomly generated "junk" data.
I am not sure I understand why Headscale was excluded. As far as I know, it is made by people not related to Tailscale.
It would be like complaining Vaultwarden is bad because the Bitwarden project is not fully open source even though Vaultwarden is fully open source and has most of the features implemented.
And Headscale kind of ticks all the other boxes mentioned, except "not headscale", because:
* p2p mesh network - it is a mesh network. And even when mesh is blocked, you can use multiple relay servers (derp) which will relay to the mesh from closest location. And you can host your own derp servers.
* Open source and selfhosted - check
* Not Wireguard (Signature-based blocking) - in cases where wireguard is blocked, the derp relay servers run over https and are usually not blocked based on signatures. For example, I use it with Traefik proxy in TCP mode so I could run derp and other http services on the same 443 port and it works great. So - check?
Packaged in nixpkgs - check
On top of that, if you add Headplane admin UI you get nice graphical management, very similar to the one of Tailscale.
How did you manage to put the derper behind Traefik? It seems to be not supported at least officially:
“ The DERP protocol does a protocol switch inside TLS from HTTP to a custom bidirectional binary protocol. It is thus incompatible with many HTTP proxies. Do not put derper behind another HTTP proxy.”
I did it mostly by religious reason, but also because everyone writes about Headscale and I didn't want to write about things that everyone already knows.
It's also not hindered. It works completely fine, but it doesn't have users (self hosted competitor).
It uses the same plain WG which doesn't pass across borders by the rules of this experiment.
FWIW; Tinc has been my workhorse between my various cloud providers, some on prem remote access for a few offices I consult with, and my home + workstations.
I keep telling myself I should switch it to a wireguard mesh, but the configuration of tinc + the "right defaults" make it pretty neat. It's fun to watch as you roll out a configuration to one node in the mesh, and ping times drop suddenly once it can make a direct connection with the optimal route.
I keep my provisioning script + the public keys in git; so deploying it to a new machine is a git pull, generate key, push public key, and pull on the other nodes. I have about 17+ hosts on it; not using it for very high bandwidth, but I couldn't do what I do without it.
Nailed it. Long time Tinc user here as well for all my personal stuff. Besides the fact that onboarding individual machines is a garbage process, it's my second best "set it and forget it thing" I use; the first being Syncthing which came later (and additionally, the two work very nicely together through weird edge cases like work VPNS and suc.)
Tinc sounds pretty awesome, but based on the repo activity and a post from the author[1], it looks to be unmaintained.
[1]: https://github.com/gsliepen/tinc/issues/443#issuecomment-184...
That is sadly, the biggest worry about it. Once I collect a few round tuit's, I wanted to see if I could build up a tool to build up the same mesh + routing, perhaps confined to a linux namespace so as to keep it contained, and try to reproduce the ease of configuration, on top of much better maintained core linux tools.
But the key has to be -- the configuration has to be just as simple as tinc to be effective. Like, almost just parse the configuration files and build it up with WG tunnels.
https://github.com/m13253/VxWireguard-Generator <-- this is what I use with a more serious "internal business" use case, so I know wireguard is in the sensitive security/encryption loop. It uses wireguard tunnels to make a broadcast/mesh Layer2 network with vxlan connections, and you then run OSPF/babel/your-favorite-routing-protocol over that.
I am not sure what the author thought was so pompous about nebula. It's unfortunate because it seems to have prevented them from reading the documentation. The ssh interface is used as a debug tool not a configuration tool: https://nebula.defined.net/docs/guides/debug-ssh-commands/
This is interesting! I've been using mosh, Xpra, WireGuard, ZeroTier, and Git (which I know is a stretch), but mostly plain old ssh -L.
Hey amazing post. I have a really interesting project which I want to share which I really obsess about which is called piping server.
https://github.com/nwtgck/piping-server
I really think of ways on how I can use this which seems an amazing almost uncensorable-ish tech on how to connect two pc's. Like sometimes my brain just thinks "pipes (piping server) just for email". Its a bit of an obsession...
Something like a VPN could theoretically be created where the piping server well "pipes" it in an encrypted manner through internet.
I would love to create something like this just for the funzies but what I am more interested about is the transport layer.
Like I want something which can be independent of udp or whatever and the only thing I am worried about is how I will transport them to the other pc and then I can then lets say send them over piping server, send them over matrix or signal if need be too idk.
Is there any foss projects that can help me just hook up into things in a similar manner as to what I am asking?
I want a implementation independent-ish transport layer so that I can experiment with things which I can just pipe if I can be really really honest.
I also want more people to look into it as I use sometimes piping-server as a way to transporting files between podman containers even though its a bit slow just to try it out and honestly, just having the fun of installing curl and then being ready to go makes it so much more easier to transport files out of the box... and I want to experiment more with it, its been an obsession for almost an year on and off thinking about piping servers and how elegant they are. I used them of sorts to break an intel nat once, but since then we got some better options if somebody wants to know how to break any nats without any root without any emulation but maybe I want to create a blog post about it someday but I am lazy.
Sometimes you don't need a P2P VPN, but rather a P2P stream manager (eg. BitTorrent).
A somewhat nice solution for that is Iroh (QUIC P2P w/ hole punching): https://www.iroh.computer
They also provide a solution to discoverability: https://www.iroh.computer/docs/concepts/discovery
Which boils down to storing ECC signed arbitrary data on the mainline DHT.
Two showcase Iroh utilities that are actually useful in practice:
https://github.com/n0-computer/dumbpipe
https://github.com/n0-computer/sendme
There used to be many p2p vpn (full mesh) solutions, which disappeared into obscurity.
Social VPN, Remobo, NeoRouter, GBridge, Wippien, PeerVPN. Remember any of these?
Just checked — none of the domains are working.
Yep, lots of VPN options: https://gist.github.com/mrbluecoat/e725474483dbd81b6195bd3b9...
I'll need to add EasyTier - https://github.com/EasyTier/EasyTier
I'm wondering if a VPN in layer 2 mode would work better for such cases.
- Make all participants keep a connection with each other, each connection equivalent to an ethernet cable, and each participant a network switch.
- Just like real switches, use spanning-tree to decide which connection is used for data and which is kept as redundancy.
- Bring your own router/dhcp/etc
I think that would be quite robust.
A yggdrasil private mesh [0] might be worth evaluating too.
- [0] https://www.complete.org/using-yggdrasil-as-an-automatic-mes...
> And also Nebula's interface is absolutely shit. Instead of a normal CLI, you need to configure an internal sshd and connect via SSH to localhost. Maybe it's more secure, but it's utterly disgusting.
This seems to be a strong misunderstanding? The ssh interface is for debugging only. You can disable it and configuration is solely handled by the daemon configuration file.
I operate a small (few dozen hosts) network on Nebula with mostly NixOS hosts, so I have some applicable experience. Nebula was primarily chosen because it allows me to, among other things, assign fixed prefixes to hosts and have a full declarative config.
You don't configure a host via a CLI, instead you provide it a signed cert for the privkey + CA cert + private key + lighthouses and that's it. The daemon listens on the IPs from the cert and the lighthouses offer a public exchange where peers advertise their IPs (and associated endpoints for p2p).
The CA approach is also an important part here as all your peers effectively have the CA cert in their config file and use it to verify other peers. The CA signs a cert for each host that contains the IP prefixes that a peer may handle packets for. The only CLI you actually regularly use is here, for creating keys and signing certs.
> And Amnezia VPN has made their own fork of Wireguard, specifically for breaking through government censorship. But the main problem with obfuscation is the reduction of effective packet MTU
The "obfuscation" in Amnezia's fork does not shrink the available MTU (important for QUIC as it requires a minimum MTU of 1280 while WireGuard itself needs +80 bytes or so for route encapsulation). Amnezia's fork modifies the 4 WireGuard header values (which must be pre-agreed between peers) & occassionally appends (to handshake packets) or sends randomly generated "junk" data.
Yes, this is just a bad writing. I wanted to say something about Amnezia, but didn't find a good place for that
Another one to look at is vpncloud: https://github.com/dswd/vpncloud. It's written in Rust. I've used it in production for several years now.
It seems abandoned.