How To Guide For HAProxy and Let's Encrypt on pfSense: Detailed Steps for Setting Up Reverse Proxy
Vložit
- čas přidán 20. 05. 2024
- lawrence.video/pfsense
Connecting With Us
---------------------------------------------------
+ Hire Us For A Project: lawrencesystems.com/hire-us/
+ Tom Twitter 🐦 / tomlawrencetech
+ Our Web Site www.lawrencesystems.com/
+ Our Forums forums.lawrencesystems.com/
+ Instagram / lawrencesystems
+ Facebook / lawrencesystems
+ GitHub github.com/lawrencesystems/
+ Discord / discord
Lawrence Systems Shirts and Swag
---------------------------------------------------
►👕 lawrence.video/swag/
AFFILIATES & REFERRAL LINKS
---------------------------------------------------
Amazon Affiliate Store
🛒 www.amazon.com/shop/lawrences...
UniFi Affiliate Link
🛒 store.ui.com?a_aid=LTS
All Of Our Affiliates that help us out and can get you discounts!
🛒 lawrencesystems.com/partners-...
Gear we use on Kit
🛒 kit.co/lawrencesystems
Use OfferCode LTSERVICES to get 10% off your order at
🛒 www.techsupplydirect.com?aff=2
Digital Ocean Offer Code
🛒 m.do.co/c/85de8d181725
HostiFi UniFi Cloud Hosting Service
🛒 hostifi.net/?via=lawrencesystems
Protect you privacy with a VPN from Private Internet Access
🛒 www.privateinternetaccess.com...
Patreon
💰 / lawrencesystems
⏱️Time Stamps ⏱️
00:00 HAProxy on pfsense
02:50 How The HAProxy Reverse Proxy Works
06:46 pfsene packages and WebConfigurator settings
07:28 ACME Let's Encrypt Setup
10:40 Setting Up HAProxy General Settings
11:47 Creating HAProxy Backend
12:50 Creating HAProxy Frontend
14:45 DNS Settings & Host Override Setup
#pfsense #firewall #networking - Věda a technologie
This channel is one of the most valuable sysadmin channels on CZcams. I refer back here routinely. Your presentation is clear and accurate.
Wow, thank you!
Thanks for dropping the updated video link on the old one.
Tom, let me just tell you how amazing your content is! Thank you for all your hard work and willingness to share your knowledge with us simple folk. :)
Thank you as always Tom! I previously missed the part about having to set the record to the router's IP. After fixing that (thank you for being so mindful about speaking on it) I got HAProxy working perfectly!
I set up my HA proxy 2 years ago based on these videos, it's great to get a refresher since the system has needed very little maintenance not sure I remember how to set it up!
Thanks Tom .. i already have it setup and working externally with my home automation system.. but never did get it working internally.. time to take a second look @ it and get it setup .. appreciate all your hard work
Very important tutorial tks
Thank you again Tom!
I got this working for our internal network, and now have no more annoying SSL warnings when I am using our servers/services. So nice for those of us with OCD about this stuff
Next hurdle, getting this to resolve for remote workers over OpenVPN.
This is great, thanks for going into the details on this, best video on the subject i've seen
Great video as always!
Excellent tutorial. THANK YOU for making this process clear. I have been using certbot for securing my web-services for years, but I never figured out how to get haproxy to host my cert for making even lan-only services accessible with a letsencrypt cert. This made that painless and simple.
This was an amazing guide. For me it was important to disable the monitoring on the backend, otherwise it wouldn't work. But I got it working! Thanks so much!
I do not think that the importance of this video can be overstated. I've done already, and I wish this video was available then.
Do you use HAproxy just for web based http/https stuff or also for other protocols? I am interested in getting as many protocols as possible on the same adres on the same port. This is more or less a challenge i have for myself. Maybe you have something i could add to my setup.
Other protocols not supported you will need a generic router.
Ok, so this worked. I'm pretty shocked this solution was on my pfsense the whole time. I wish I had known this before the invested time in learning Nginx. Thanks for the very clear guide!
I'm looking forward to content in 2024 plus information on the new DHCP server backend.
You're a hero bro!
This timing is insane I just setup HAProxy from the old video yesterday
@11:41 For anyone doing this and getting an error starting haproxy "Errors found while starting haproxy" for me it was was that I was on an old version of PF 2.7.0 updating to 2.7.2 skipping 2.7.1 fixed it for me, it seemed to have to do with dynamic pages not config'd for HTTP status codes. Also when you try to upgrade PF if it fails they you might have to execute the shell command "certctl rehash' in Diagnostics/command prompt. Hope this helps someone! Also @11:29 the syslog port is implied so just the IP is needed.
Thanks Tom,
Okay, so this answered all the questions I had from other videos on HAproxy.. Thanks so much Tom
Great to hear!
Thank you Tom 🙏
thank you - this will work really well to deal with apple not accepting self signed certs for things like local jupyter notebooks.
Great video, thank you very much.
Was just watching your previous video on this topic. Glad its now updated! Thanks, Tom
very cool
This is one of the best videos I’ve seen this year. Short, snappy and very important. Even sat at home sick as a dog with COVID I was still engaged throughout. I would love to see a follow up video which adds a 2FA authentication layer to this setup too (mainly for the external access use case) using an app such as Authelia. Great work Tom.
Great Video. You make the same assumption on this video as your last one. "Host Matches" only works if the frontend port is 443. If you use a different port such as 10443, then you need to use "Host Contains". I spent way too long debugging that one! Thank you for everything you publish.
Loved the last video and happy to see an updated one. Leaving a comment and Like' to help the algorithm put this in front of more eyeballs. Thanks as always Tom and team.
What a great channel. Thank you for providing such useful content in an easy to understand manner.
You're very welcome!
Hello, after following the guide and ensuring that I perform all the steps correctly, I couldn't access the servers locally by selecting the LAN interface in the Frontend. Instead, by selecting the WAN interface, I was able to access externally without any issues.
I tried testing by selecting both WAN and LAN interfaces in the Frontend, and the same thing happened - it worked externally but not locally. Every time I tried locally, I got an ERR_CONNECTION_REFUSED.
So, I decided to try selecting ANY as the Frontend interface... and surprise... it works both externally and locally!
What could be the problem? Why does selecting WAN always work, LAN never works, and ANY also works? Something is escaping me...
@LAWRENCESYSTEMS, as always your tutorials are great and really detailed!. I'm using Pfsense and I have many clients set in static lease and also adresses set in host override. I followed the tutorial multiple times, trying to have a wildcard working for the adresses set in static lease or override, however it doesn't work for me. I presume that there is something wrong with the 'order of routing'. At launching the url request, the step via ssl certificate on pfsense is just bypassed. When launching in a browser the full host+domain, the self-signed certificate of the application behind is showing up instead of the one set in pfSense. Is this issue related to the static lease or host override that are set?
I am having the same issue. I have watched the video a few thinking I missed something. I think I will just try with nginx-proxy manager and see if it works.
Hi , I was just wondering what were you using to make your topology presentation at the beginning of the video ?
czcams.com/video/mpF1i9sfEJ0/video.html
AGAIN! Your guides have helped me replace my rickety nginx on-a-pi solution with something far more expandible.
Quick question: I have a frontend for public traffic from Cloudflare and a frontend to catch internal traffic and all resolving to the same backend. I thought there was one frontend configuration to rule them all when I started. Is my concept valid or am I missing a NAT reflection component to make the single frontend usable?
Thank you for your in-depth tutorials on these tools. I was able to learn a lot of new material in this project.
You're still my hero!
HAProxy can be one front end for all, but I am not sure about using Cloudflare with it as well.
Nice update. I was able to use the previous walk-though and after resolving a few fat finger issues and misunderstandings, I got everything working. The update should help those getting started - good job!
Thank you Tom, I love these "recipe" type videos... Wonderful presentation.
Have completed the backend entries for HAproxy. But haven't changed the NAT settings on pfSense. Port 443 is in use there. I guess I can just turn off those NAT things that would interfere... Port 443 and Port80 that is. Any suggestions on the migration ? Thanks for what you do. If you lived in Eastern NC, I'd hire you.
Very nice. I’ve been using nginx proxy manager and a certificate authority. This looks much cleaner. Only problem is that I don’t use pfsense…yet. You’re slowly changing my mind
I also struggled with that SNI part... I think the GUI could be organised to be a little more self explanatory
Hey Tom. in the frontend portion when adding the external ip table. if i don't have a specific VLAN set up for my server, what would i choose in that dropdown? ----- also my issue, EVERYTHING works to point my server to my domain/subdomain but it is not showing secure. Followed every instruction SPECIFICALLY except for this one. Would that be causing it to not pull the certificate we made.?
I'm also having an issue here as well.
Greetings,
I have watched your video about 5 times thinking I was doing something wrong. HA-Proxy only work for a once sub-domain at the time. At first, I used a wildcard certs then I created a certs for each sub-domains but still nothing. Only the first one in the ACL list works. I am only using HA proxy internally. Any thoughts?
Hi and thanks for a great channel. I´ve followed all of the steps in your guide but still it uses the old slef signed cert and I get a warning that its not secure. In pfsense my wildcardcert is issued correctly from lets encrypt. Any suggestions ?
It works now :) Thanks again. I dont know what it was wrong. Maybe some caching.
thanks Tom, I followed this excellent video to setup my nextcloud server, unfortunately its returning "400 Bad Request The plain HTTP request was sent to HTTPS port". I wonder why?
Hey man. This video was more in detail and covered more of the basics than the other one so appreciate the time it took to make it! I do have a quick question. If I'm opening one of the my servers that is already behind HA Proxy. In the WAN rules would I still create a rule like normal (aka WAN to X Server) or wouldn't I just need to make a general (WAN to HA Proxy) rule and just let HAProxy sort it out? Basically what is the best way to do WAN to LAN Rules when using HA Proxy?
If you open WAN to HAProxy you do not need a port forward rule to go from WAN to a server behind pfsense.
@@LAWRENCESYSTEMSI'm still having issues getting to to reach the site through HAPProxy after I disable the default 443 rules and test with WAN to HAPRoxy, didnt work so I tried to manually create new one for HAPRoxy on 443 to {this firewall). I'm using Cloudflare and it says it cant reach the host While HAPRoxy is enabled. Is it possible that since Cloudflare also adds their own certificate on the front end thats is conflicting with certificates from HAProxy? I tried with offloading on and off and also tried to SSL Encryption on/off and no change. I even made a new subdomain in IIS and didn't apply a certificate to it, updated back and front end connectors in HAProxy and it still cant reach host with HAProxy enabled. Its kind of acting like it cant reach the site because the ports are closed but enabling WAN to HAProxy should have allowed it correct? Any other ideas? And thank you!
I have never tried mixing HAProxy with Cloudflare tunnels.
@@LAWRENCESYSTEMSAh gotcha. Well I was able to get it working with Host Overrides on but the issue is the root domain wont resolve, only sub domains work. I tried using a Domain Override but that didnt work. Kinda odd that subdomains work but root doesnt : / Anyhow, thanks for your help. Ill keep messing around with it.
Not that i need it myself but i had hoped for more examples on HA-proxy. Like the ability to stack everything you want to publish on one adres one port. And how you can filter the requests to go to the correct endpoints when doing so. That you can do extra entry checks like client certificates, so you have a secure SSL connection with some extra access checks eliminates the use for vpn in some usecases. Most people do not understand how powerfull this is don't think this basic demonstration created a lot of new users. Can you please do another more advanced one?
For those that didn't know haproxy existed, this flies through way too much info to understand how to set any of this up. He's touching on way too many concepts, in under 20 minutes. I like his videos but this one should have been a two part or atleast much longer video.
Thanks for the fantastic video! You mentioned utilizing HAProxy for LAN access. Is it simply a matter of setting up a frontend on the LAN address? For instance, with TrueNAS, I’d like to access it within the LAN but not externally, while still using my domain with a valid certificate.
Yes, binding it to LAN works.
@@LAWRENCESYSTEMS I did try to create an new frontend 443, new backend for the internal server. Updated the DNS to resolve the hostname to the LAN-address of the pfSense/HAproxy... then I got ERR_TOO_MANY_REDIRECTS... any idea on why?
nice vid. i prefer nginx proxy manger though. cool that pfsense includes this feature with their firewall nonetheless.
Wow. Didn't know about HAProxy until now. I understand the concept due to experience with F5 load balancers. Thank you for the information.
I saw a guide about using a virtual IP for haproxy then forward 443 to the virtual IP which enables you to keep pfsense webgui on 443. I have been using that setup for over a year now. Is it advisable to use the virtual IP?
Not sure if it is wrong but I don't use it that way.
@lawrencesystems If you do this to a FREENAS server, the NAS IP will now point to the proxy, providing a correctly encrypted web UI. But the DNS has been updated, so won't SCP/NFS/iSCSI traffic also get sent to HAProxy, and fail because nothing is listening on those ports?
That's why you should have a separate DNS entry for the web interface versus the other services
My WAN address is actually my public IPv4 address.. Do I need to port forward 443 to my pfsense firewall?
HAProxy is one of the most useful tools in existence. Nice video!
What about firewall rules that you are using to get this working? Looks like the HAProxy sits in its own VLAN and then you redirect the sites to a separate VLAN / LAN? Or am I overthinking it?
this video is only about serving inside the home network.
the firewall rules are done in the old video
is there a way to ask for a certificate that works for both the star and flat domain? i mean, 1 certificate which covers both domain.something and *.domain.something
thanks
Not sure why you would do that, but I guess you could do that.
Changed port to 10443, now I cant access the GUI typing ip:10443… did an nmap scan and only port 53 shows open..
Get: 400 Bad Request
The plain HTTP request was sent to HTTPS port
nginx
When trying to access on port 10443.
Do I need to reset the whole thing?
Thanks for the updated tutorial Tom! DNS was indeed the issue when I was having trouble setting it up the first time back in the day. 😅
I encountered something strange when setting up a front- and backend for my HP network printer: it throws errors with password fields. Logging in works fine, but changing a password throws an error. The unprotected site works fine.
It's similar for the Mikrotik router login page, those won't accept the password. So it seems HAProxy does something more than just relay the page? 🤔
Can you do a video just like this, but with Squid proxy/reverse proxy instead? I figured it out by referring your HAProxy video a long time ago, but would love to see if I missed anything.
No, we don't use Squid because it's a headache and breaks things.
@@LAWRENCESYSTEMS ironically, here I am, back again, following your tutorial after Squid has been deprecated, lol
UPDATE: Worked like a charm!
Hi. What tool do you use for the animated diagrams?
czcams.com/video/9YQJF1sTtC0/video.html
I already have several public domain names. Will the one I use for haproxy be "lost". Just want a heads up so I can order another if need be.
You could make a subdomain as well.
I heart your videos.
On my setup the DNS server and listening address is the same since since all traffic comes through one interface. So when a host looks up the IP for the address, it points to the listening address ,which is the same as the DNS server. If i try to assign a custom IP to the listening server, i get an error. Any way around this ?
HAProxy can be on the same IP as the DNS server.
@@LAWRENCESYSTEMS thanks for the reply (and video). Well i guess its something else wrong then, i will check it all again :)
I use Traefik now, switching back to HAProxy would feel like a step backwards for me.
Can we do HAproxy without a Cert?
Yes
Tom
very much appreciate your work (as you correct or explain yourself if you get to many questions, its not so easy to find the right way how to starten and how to explain in our Job ....), one of best videos I have ever seen so far of an very valuable Channel ...
but one question? Is there an option to do 2FA an HAproxy before the Application with PFsense?
regards
Oliver
not really but you can do some basic authentication. Not something I have ever tested or have a use case for.
How do you secure webbased Services? all via VPN only ?@@LAWRENCESYSTEMS
is anyone on the East Coast near Morehead City NC ?
What is your dns for the vlan.
pfsense
I can't resolve the domain internally. My DNS is the gateway of the VLAN, but.....
Hi
Right now I am using pfsense and I have installed haproxy and it is working properly as a reverse proxy for websites.
the aim is to use a subdomain as syslog receiver.
1- I have created a subdomain on Cloudflare and send the traffic to pfsense valid IP
2- on pfsens haproxy is listening on 443 port (and it is okay, I have tested it )
3 -I have created a backend to forward traffic to 514 (the receiver port on the log server is tcp/udp both)
4- I have created a frontend and send the traffic to the backend
but it is not working, What did I do wrong ?? (When I switch the 514 to for example 80 I can see the admin console that proves the haproxy and others are working but it doesn't get the syslog messages)
I don't think you can proxy syslog traffic
In version 2.3, HAProxy introduced a feature for receiving Syslog messages and forwarding them to another server.
but I couldn't find a way to implement it on pfsense
@LAWRENCESYSTEMS
any idea?
@@hamidfathi6252 nope, not something I have tried or plan to try.
@@LAWRENCESYSTEMS so any idea how to keep syslog server whit domain behind a pf sense firewall ?
so this is simple and easy setup for a homelab where the annoyance is mostly the browser itself, but how secure is this really? what would pentesters say about not checking the cert between nginx and local backend - is there any room for potential abuse?
As far as HAProxy goes this is a secure setup. As for not validating the SSL connection made on the back end that could be abused because someone could replace the server and HAProxy would still connect to it. But if someone has the level of access required to replace a server on your network you have some bigger issues.
@@LAWRENCESYSTEMS I do fully agree that I have a much bigger problem, my question was rather directed towards - can this be used to gain credentials/access once the attacker is inside my network? Like in this case using the truenas - once i log in as an admin to the console
@@Destroyer954 I mean if an attacker takes over HAProxy they could see the traffic if that is what you are asking.
Hi! I was having trouble with Octoprint. I had to add the following in the Frontend - Under Advanced settings, advanced pass thru: http-request set-header X-Forwarded-Proto https if { ssl_fc }
Works now!
14:32 Not seeing a certificate in that list. Any ideas as to why not?
you need to click on ssl offload
If one has dual WAN, they can have the frontend(s) listen to both WAN interfaces and create DNS records for both WAN IPs to get load balancing for free ^ ^
Having 2 DNS entries should work for that BUT would more like failover
from where to get the DO API for certs,, we are using NOIP DDNS
You need a domain with the DNS being managed by Digital Ocean.
currently we are using ddns service by no-ip, and a cname rec (which will be our weeb service fqdn) is added in our DNS at AWS which is pointing to no-ip ddns record@@LAWRENCESYSTEMS
I did this but ended up junking it because trying to use my truenas servers dns name for setting up file shares also resolved to pfsense, unless I'm missing something here...
As I said in the video, you need to have different names and DNS entries.
I have a DDNS will that work as well as a DNS?
It should work
I’m not understanding the logic of binding HA proxy to different interfaces.
If LAN1 is my trusted network and LAN2 is my untrusted but I want LAN2 to access my TrueNAS what difference does it make if HA proxy is only listening on LAN1. DNS will still have it pointed to my LAN1 address and you still need a firewall rule for LAN2.
Bit confusing/misleading in the last bit of the video but overall it’s good
You still have to have rules that will allow LAN2 to talk to LAN1
@@LAWRENCESYSTEMS that’s my point entirely. It’s towards the end of your video that you make the use case about placing HA proxy on an untrusted vlan. In truth it doesn’t matter where the proxy listens because at any time you will have to allow flow from untrusted to trusted
Its working from local network like a charm. But not from outside network.
You use two different ip and say they are both the ip of the truenas server. Which is it?
As I show in the diagram 172.16.16.5
Hi , how can I do with pfsense to manage multiple lets'encrypt certificates and then upload them to haproxy?
The ACME certs plugin allows for multiple Let's Encrypt setups
@@LAWRENCESYSTEMS Thanks Lawrence , but having active on the pfsense a certificate generated with a key and host name e.g. mypfsense.duckdns , I have 3 other host names always duckddns on the pfsense . Can I add them in the section where the active one already exists? I just have to generate a new certiifcato duckdns always with the same token but different fdqn name?
Not clear on your ask and I have not used DuckDNS before.
@@LAWRENCESYSTEMS Thank you, I will try again with the English Italian translation ... At the moment thank you for answering .
@@LAWRENCESYSTEMSThanks Lawrence, even if you didn't use duckdns I did it now, so it's possible as you say to upload or rather request other LE certificates (with the duckdns hostnames configured in the pfsense). I performed the procedure as in the first and LE generated me all the certificates I wanted. Through Ha proxy I selected in the front end mainly the use of the other certificates, so now when from the wan I type the url of my lan server or rather of the servers, the certificates are loaded perfectly. I just have a problem with another server that has its own LE certificate in its webroot and at the moment I don't know how to upload this certificate to the pfsense to be able to manage it from him . Thank you .
👍🏻Golden Content as usual, thanks Tom !!!!
And remember......it's always DNS😂
is there any anti_ransomware tool worked pfsense, not point you setup pfsense perfectly but still get hit by ransomware, so what is best configuration get protect pfsense understand pfsense does not have any ransomware tools or plugin..can come with a good video to setup pfsense with any third party ransomare...
Firewalls don't really do much for ransomware.
Did not go over required firewall :(
You don't NEED to open any ports for this to work, but if want it publicly accessible then you CAN open up the WAS IP that you bound it to.
since your not going to show me how to point my domain to my local ip no need to watch this video hey but at least you got one more comment.
Lol, your shirt is RAID is not a backup... no, you have to turn on shadow coppies too. Now you have backup!!!! Hahaha
Joking for anyone that took that serious.
New shirt idea: "Shadow Copies Are Not A Backup"
the question is, how to force 10G DUPLEX in TRUENAS SCALE? And then I have a problem with the switch, if I restart the server, then it will work at a speed of 1gb/ s, I have to restart the switch so that the server works 10gb /s with
TP-LINK support, they advised me to force 10GB DUPLEX on both sides, it's already out of the box on the switch, but TRUENAS, I do not know how
I don't have any issue with my TrueNAS systems auto negotiating 10G
You lose me at the frontend listen address of LABVLAN1313 address. What is that? What if I am not running VLANs?
Those are all interfaces in pfsense for each network.
Thank you. I don't have any other interfaces so when I set the frontend to use the LAN address (this is all for internal usage) then I lose the pfsense WebGUI. This is where I get stuck. Do I have to create a virtual IP for HAProxy to listen on? @@LAWRENCESYSTEMS
@@LAWRENCESYSTEMS its difficult to follow when you have everything set up already. I fortunately understood what had to be done but can understand why others would be lost. I have several vlnas and most my services are one one vlan so easy enough to set up. just 2 others are on a different vlan so just have a different front end for them and the dns points to their default gateway ip.
It was not explained why do I need this for…
This is a tutorial for people that need a reverse proxy using HAProxy, not why you need one.
xoxoxo
First
I was 'informed' by a self proclaimed cybersecurity expert keyboard warrior that HAProxy is unsafe and should never be used.
I got tired of his only solution is to have everything on a VPN (which he wouldn't tell me if he hosted or used a service). You VPN is only good if you 100% trust the provider and they never have a breach.
Idk, I have HAProxy setup, secure ssl and passes ssl labs testing. Nothing is ever perfectly secure, but I don't understand the issue here this guy has. It's not just a proxy or open port. Got tired of his silliness
I have read via CVE that nginx proxy manger have some serious vulnerabilities which still aren't fixed due to a very small group of developers with limited time to fix and test things. HAProxy been around alot longer and very well vested. Just like anything if you misconfigure it you will have a bad time.
Hi Tom. Thanks for the great tutorial. I am just trying to understand one thing. If I have two vlans, trusted/untrusted and lets say I wanted to allow Truenas on my trusted vlan to a specific IP/device on the untrusted vlan only and not the entire subnet. How can I do this with HAProxy? Because if I add a host override pointing to the Truenas subdomain for the untrusted vlan, then all devices on that subnet can now access Truenas.
You can create rules that are per IP in the firewall.