SELF-HOSTING behind CGNAT for fun and IPv6 transition
Vložit
- čas přidán 13. 06. 2024
- This video started as the answer to a simple question - how can I self-host a service for my friends and family, behind cgnat, without requiring them to install any apps (like tunnels)? This video turned into a bunch of different ways to proxy IPv4 to IPv6, so you can receive IPv6 traffic natively and bring in legacy traffic from a VPS which does have public IPv4.
I cover several different methods:
- Using IPv6 only
- Using Cloudflare Tunnels
- Using SNID (sni daemon) - github.com/AGWA/snid
- Using HAProxy to forward TCP and TLS traffic
- Using Tayga to port-forward from IPv4 to IPv6
- Using Wireguard along with port-forwarding and NAT across the tunnel
The installation instructions for snid, haproxy, tayga, and wireguard are all in my blog post, along with a comparison table:
www.apalrd.net/posts/2024/net...
Support me on Ko-Fi if you enjoy my content and find it useful:
ko-fi.com/apalrd
Feel free to chat about my upcoming projects on Discord!
/ discord
Timestamps:
00:00 - Introduction
00:43 - Goals
03:36 - Use IPv6 Only
06:00 - Cloudflare Tunnels
09:02 - SNID
16:50 - HAProxy
20:53 - Tayga
27:52 - Wireguard
33:35 - Bonus!
#networking #ipv6 #homelab #selfhosted - Věda a technologie
ROFLOL! Telnet to the IPv4 address to get rickroll'd, telnet to the IPv6 address to get Star Wars. Nice one!
There’s also a sad no ipv6 animation you might randomly get
that's weird, I got the proper star wars without an ipv6. though I did get the "no ipv6" at first
Hey man! Just found some of your older ipv6 videos yesterday. Loving this content and your presentation style! Thanks so much for sharing all this hard won knowledge!
You can also use a Hurricane Electric Tunnelbroker 6to4 tunnel, among some other providers - another solution that doesn't require you install any software. HE will give you a whole routed /48 block for free if you do their IPv6 certification (also free), and they even have a ton of config examples for every router OS you can think of. I have both native IPv6 and IPv4 service thankfully, but I still use a HE tunnel for public facing things just for the sake of having it on a separate IPv6 block. (+ I also like having the extra /48 to do with as I please.) I use it with both MikroTik RouterOS and a Debian router and both were easy to configure to use the tunnel.
Only problem with HE tunnels is they use IPIP and you need a public v4 for the tunnel to work (this is a protocol limitation, not policy).
@@apalrdsadventures Yeah, I forgot your use case here. I got it backwards, like you had IPv4 but no proper IPv6.
I have been running a modified version of your last option using nebula instead of wireguard. Nebula has the punchy.respond option that keeps the tunnel alive so I didn't have to roll my own solution. Sadly there is still no IPv6 support in nebula but it it's robust enough for all of Slack's infrastructure it's probably good enough for me. Great video as always! Keep up the good work!
I use Nebula as well, but as a purely closed network (no routing / traffic flow outside of the Nebula subnet).
besides this video, thank you for the RSS on your site, much appreciated
No problem!
was gonna like the video but then I got rickrolled love it
Run 464 xlat, basically a private cgnat, but you control the exit port, so you can control the port forwarding. You can use this to derive your own IPv4 exit.
You can use jool to implement this see siit-dc-xlat. You can then port forward on your virtual private server gateway.
Jool is kinda a pain to work with because of how traffic bypasses a lot of the normal linux Netfilter, and doesn't work for translating packets from the local system.
@apalrdsadventures there is also cgnatd that can do the local traffic. But you can implement it on a router or in VMs too. The big thing is it gives you a non cgnat ipv4 tunnel over ipv6.
I have 2 different ISPs offering gigabit FTH (on separate networks even! there are reports of shenanigans of the new one "accidentally" cutting the fiber then sending a sales person and an installer behind them to sell people new service), and neither has any support for IPv6. At some point I'll have to figure out an IPv6 tunnel or something...
Hey man! been following alot of your guides for Proxmox setups, and i stumpled upon your Immich setup on your Blog, however i'm running into some issues trying to set that up do you plan on releasing any video soon to complement that blog post? I would really appreciate that!
I (kind of) use the Wireguard method but instead of allowing the VPS acces to my whole local subnet I set up tunnels on every server and configure the port forwarding with nginx (stream directive) since I always had problems with iptables. Failover (or load balancing) come as a nice bonus, and I can use both tcp and udp, and if I ever have to route to a website, I can set up a normal http/s proxy with caching which will also allow for easier certificate management and the ability keep the incoming ip addresses in x-real-ip parameter. The configuration is not that big of a deal to do manually once a month but I may make some webui to automatically rebuild the configuration at some point.
I am currently running a more complicated version of the wireguard setup: A VPS acts as NAT for the wireguard network, but I then use policy routing combined with connection tracking on my home router, which allows me to not mangle the source addresses; It behaves as a multi-isp setup. But, debugging incorrect policy routing is annoyingly difficult; I haven't yet really discovered a good way to attack it.
I believe using an overlay network like Nebula or tail scale for your self hosted services and a reverse proxy back to them from the Internet, is an option. It requires you to have a public VPS as the reverse proxy for your publicly available service (i.e. - jellyfin) but you don't have to forward any ports (if you were lucky enough to have a public IP from your ISP). All traffic from the reverse proxy (which is also a node on your overlay network) back to your home network is on the overlay VPN.
Commented before reading the rest of the comments. Y'all already knew this. 🤦🏻♂️ 😛
You can also use SSH port forwarding for port forwarding. A benefit is that it requires no special configuration on the public server. Just write a service on your origin server that SSHs into the public server and everything will just work even if your home network changes. Many of the solutions in your video break if your home prefix changes.
This is a good solution for SSH or other low bandwidth protocols, but not a good solution for generic hosting due to the head-of-line blocking problem when multiplexing multiple TCP streams over a single TCP socket (the SSH session).
I love your videos; thank you. Do you have a video showing how to make the public IPv6 devices behind the firewall accessible from outside?
In OPNsense you just add a rule on the firewall to allow the traffic
It depends on your router. But in general it should be an “allow” rule.
Currently, its a UDM-Pro, but I will switch to Opnsense soon. I did create one rule (Internet IPV6) to allow incoming traffic to a device behind the firewall, but I can't ping it yet from the outside, not even the lan public ipv6 from the firewall (with another rule); I am also not able to ping it. Its a CGNAT. I was thinking maybe I need another rule elsewhere
The UDM should be able to do it, but I don't have one to test with.
Remember if using a CF tunnel for media streaming, you're breaking their ToS.
They removed section 2.8 from their ToS some time ago, so not anymore, you just have to set a rule to not cache your streamed content if your site doesn't have it in headers already.
They made it more vague, but still don't really want to deal with media streaming.
Remember if you're pretending to be an expert who can teach the teacher on CZcams while you have no idea what you're talking about, you're breaking the entire world's TOS, and probably need to go outside and ride a bike or something. Godspeed.
@@Lenny3669 ok boomer
@@Lenny3669 I was just trying to let everyone know he careful if using with media streaming. Even before it was in their ToS years ago I got my account locked because of media streaming. Just letting everyone know. I asked them why and they said it was against their terms, but couldn't show me where.
I have a VPS with a GRE tunnel to the IPv6 address of my home router, then enable ip_forward and set the next hop of the IPv4 address of the VPS to my router through the GRE tunnel. The advantages of this is less overhead so you can host it on a dirt cheap VPS, and it's even possible to assign the public IP directly to a server on your LAN if you use a GRETAP tunnel.
All of this is configured without installing any extra software on the VPS since it only uses the "ip" command in Linux.
I did setup a prototype with IPIP (GIF on BSD) and found that OPNsense wasn't properly configuring GIF for IPv4 in IPv6 (it's normally used for the opposite, IPv6 in IPv4 for Hurricane Electric).
Thank you for this nice list of explanations!
SNID made me curious but I did not find any comparison between them, but maybe I missed something and there is none to make? For me it look a lot like the socat commande that we can easily run as a service to redirect trafic on a port to another network (v4 or v6), port, a washing machine, a file ... is there a difference between it and SNID?
snid reads the TLS SNI, then does a DNS AAAA lookup on the resulting name, and if the name is within the backend cidr range it redirects the session at layer 4 (TCP).
So the big difference is pulling out the SNI so you don't have to configure snid itself.
Another difference is that SNID codes the IPv6 source address with the IPv4 source of the origin, which is why it uses a /96 instead of a single address. That way, the backend server can get the IPv4 source address in their logs without needing to use PROXY protocol.
@@apalrdsadventures Oooh, I see, thank you for the explanation! I think I do not know enough about TLS SNI to full get the first part, but that's a good reason to learn about it! Even more since it is not the first time I heard about it.
That point about know the source IP is really interesting too, in my last iteration of server and service I planned to have a host only running LXC and passing the needed 80/443 ports via socat to a container with HAProxy inside to route the request to other containers, but I never looked at the logs yet since it's still in setup, I may have ruined most of them by erasing the source IP 🤔
Maybe it's time to go back on that idea and leave HAProxy on the host instead...
TLS sends the ClientHello immediately after opening the tcp socket. This contains proposals for crypto algorithms to use, and also the domain name of the server requested. The server responds with the ServerHello which includes the servers certificate. They do this so the server knows which certificate to present to the client, but as a result, the server name is sent unencrypted.
Ahh ya got me!
could you please explain a scenario, when the ISP is IPv4-only CGNAT and doesn't provide IPv6 at all?
Hey Apalrd, what are you using to draw that diagram? Thanks!
draw.io
@@apalrdsadventures Thank you!
I'm trying to follow the last one since that's my only option but I'm really confused, 30:04 how do you set you home network endpoint using a domain if its behind a CGNAT?
You can leave it blank.
I used Wireguard in the end whem I ended up behind CGNAT. I get IPv6 but it's a dynamic /56. I dont know if/when it'll change :(
Does the dynamic /56 actually change though? Or is it effectively static?
Any of these methods would work with a MQTT self hosted broker?
MQTT uses TCP transport, so any of the TCP examples. MQTTS will also work with snid if you listen on 8883 (you can -listen multiple times).
by the way, on cloudflare if you only add a AAAA record (with "orange cloud" enabled), cloudflare automatically proxies ipv4 connections if you don't add an A record
also I didn't have to provide a credit card to use cloudflare tunnels (though the proxy dies for an hour sometimes and I've read that happens on the free tier so maybe that's it?)
With Orange Cloud, it will not actually add either record to 'real' DNS. 'Real' DNS will point both A and AAAA to Cloudflare's CDN (if you add either an A or AAAA), the record you 'add' just tells CF where the origin is.
What is your network drawing program
draw.io
I use a cloudflare tunnel in addition to my site SSL configured in Full (Strict) encryption mode. This means I still maintain a cert on my origin server and Cloudflare re-encrypts all traffic between them and me. In addition, I only use that tunnel for web traffic. I still port-forward for media streaming, so you can mix both.
But cloudflare can still read anything
@@AWildLeonhow can they read everything while my server is using my own SSL cert and I haven't given my private key to anyone?
(I'm also using the full (strict) setup)
@@Lenny3669 He's correct. Cloudflare still decrypts traffic in transit using their cert, then re-encrypts it using your cert before it passes along to your origin. They need to do this to enable the majority of their services, despite the privacy risk it creates. That's a decision I chose knowingly.
@@Lenny3669 not sure how/why my original reply got deleted, but @AWildLeon is correct. Cloudflare presents their cert for your domain to users. They decrypt the traffic in transit, and re-encrypt it with your cert when they proxy it to your origin server. They have to do this to enable the majority of their services despite the privacy risk. I chose to use the tunnel knowing this.
My replies keep getting instantly deleted, but AWildLeon is correct. Cloudflare presents their cert for your domain to users. They decrypt the traffic in transit, and re-encrypt it with your cert when they proxy it to your origin server. They have to do this to enable the majority of their services despite the privacy risk. I chose to use the tunnel knowing this.
@apalrdsadventures, I have tried to use the setup wizard on my opnsense router for wire guard but it never works. Do you have a particular video or guide you followed?
I just made a VPN video last week! It includes Wireguard.
@@apalrdsadventures I will be diving into that then!
Now, how's this for a joke. My ISP has IPv6 by default, but.. If I rent IPv4, they cut off my IPv6 access.
And the most hilarious thing is their excuse saying that if a client requests IPv4 address, there is a need to use it as a static IP for auth reasons in 3rd party services and if they were to leave IPv6 access routers would prefer v6 over dedicated v4 and the whole auth thing would fail. So they a preferring to break transport so that some Chad renting IPv4 for auth reasons would not bother their dysfunctional support.
IPv6 😢
I was so smart and left to latam and had my homelab server running behind my firewall in Germany. There it's a ds lite so same problem as you are describing here.. but in latam we have our own IPv4 but no IPv6.. it was a pain in the ass.. if anyone cares how I solved that let me know
Any reason for "not trusting Oracle at all"?
Their history of being very litigious at the slightest hint of license violations
..or setup headscale on your vps and be done with it :)
That would require your friends to install Tailscale though
Headscale is a godsend.
I've found tailscale breaks IPv6 so that my Mac doesn't resolve AAAA records. Even with MagicDNS disabled and not connected to a trailnet. As soon as I close tailscale things start working
@@apalrdsadventures not necessarily you can always use your vps public ip (ipv4 and ipv6) and then use tailscale / headscale túnel tp “proxy “/ route to where you have CGNAT. Basically the same what you did with WireGuard just simple to setup and no hole punch needed on firewalls. The advantage of headscale is that you can use whatever ipv4 subnet you want for the tailscale network so doesn’t conflict with CGNAT or don’t use ipv4 at all
+1
At 4:40 you say that IPv6 doesnt have NAT.
Well, it does. It's called NAT66.
The private IPv6 address space is «FC00::/7».
every time someone uses NAT66 a fairy dies
There's also NPT as the lesser evil... still not great though.
Headscale with 2fa. Make a video about that buddie.
small pp
how do you make your own wifi with fiber can you make a video about it sooo i want get because networking than i have fast wifi @apalrdsadventures i realy learnt much from you video s