Self Host 101 - Run Multiple Apps with Caddy | DNS, Static Sites, Reverse Proxies and Let's Encrypt
Vložit
- čas přidán 5. 06. 2024
- In this video CJ shows you how to setup DNS records for domains and subdomains with Cloudflare and how to set up, configure and host multiple types of applications with Caddy Server.
Watch the 1st video on setting up a VPS here: • Self Host 101 - Set up...
Watch the 2nd video where CJ answers your questions here: • Self Host 101 - Office...
View all the Self Host 101 videos here: • Self Host 101
00:00 Intro
00:56 Domain Purchase and DNS Setup
02:43 Creating DNS Records
04:32 Query DNS Records with dig
07:50 www subdomain record, CNAME records and TEXT records
10:19 Email forwarding and a catch-all address
11:23 DNS Resources
12:13 SSH to your VPS with a Domain Name
13:08 Responding to HTTP Requests from our Server
16:27 Web Server Software
18:12 Why do we need a Web Server?
19:05 How To Navigate The Caddy Docs
21:32 Caddy Architecture / reloading configuration
22:35 Install Caddy
24:17 Set up https for the root domain
25:39 Redirect IP to the root domain
27:21 Multiple Caddyfile Setup
31:41 Redirect www subdomain to root domain
34:15 Set up /var/www folder
39:05 Transfer files to the server with scp
40:35 Sync a folder with rsync
42:31 Clone a public git repo
44:44 Hide files from Caddy file_server
46:08 handle_errors in the Caddyfile
47:46 Clone private git repos with an SSH deploy key
52:16 Install node.js with nvm
54:03 Set up Holidays API Example
56:37 Create a reverse_proxy with Caddy
1:00:45 Manage long running processes with pm2
1:04:07 Run a TypeScript node.js API
1:09:52 Run a static website that has a build process
1:14:44 Caddyfile concepts and Logging
1:17:12 Common Caddyfile patterns including SPA redirects
1:18:15 Thanks!
DNS on Wikipedia - en.wikipedia.org/wiki/Domain_...
Cloudflare DNS - www.cloudflare.com/applicatio...
dig (command) - en.wikipedia.org/wiki/Dig_(co...)
Cloudflare Email Routing - www.cloudflare.com/developer-...
How DNS Works Zine - wizardzines.com/zines/dns/
Mess with DNS - messwithdns.net/
a simple DNS lookup tool - dns-lookup.jvns.ca/
netcat - en.wikipedia.org/wiki/Netcat
Comparison of web server software - en.wikipedia.org/wiki/Compari...
Apache HTTP - httpd.apache.org/
nginx - nginx.org/en/
traefik proxy - traefik.io/traefik/
Caddy Server - caddyserver.com/
Listen to Syntax Episode 340 - syntax.fm/show/340/servers-wi...
Caddyfile Reference - caddyserver.com/docs/caddyfile
Caddy Architecture - caddyserver.com/docs/architec...
Keep Caddy Running - caddyserver.com/docs/running#...
Install Caddy - caddyserver.com/docs/install
Caddyfile root - caddyserver.com/docs/caddyfil...
Caddyfile file_server - caddyserver.com/docs/caddyfil...
Caddyfile redir - caddyserver.com/docs/caddyfil...
Caddyfile Common Patterns redirect www - caddyserver.com/docs/caddyfil...
ln (command) - en.wikipedia.org/wiki/Ln_(Unix)
chown command - en.wikipedia.org/wiki/Chown
Caddyfile file_server hide - caddyserver.com/docs/caddyfil...
Caddyfile handle_errors - caddyserver.com/docs/caddyfil...
Generate an SSH key - docs.github.com/en/authentica...
Github deploy keys - docs.github.com/en/authentica...
Node Version Manager - github.com/nvm-sh/nvm
Caddyfile reverse_proxy - caddyserver.com/docs/caddyfil...
Run node.js with systemd - nodesource.com/blog/running-y...
pm2 - pm2.keymetrics.io/docs/usage/...
Caddyfile concepts - caddyserver.com/docs/caddyfil...
How logging works in Caddy - caddyserver.com/docs/logging
Caddyfile SPA redirect - caddyserver.com/docs/caddyfil...
Example sites:
notlocalhost.net - github.com/w3cj/notlocalhost....
Holidays API - github.com/codinggarden/holidays
Express API Starter TS - github.com/w3cj/express-api-s...
Big Timer - github.com/CodingGarden/react...
------------------------------------------------------------------------------
Hit us up on Socials!
www.syntax.fm/links
Brought to you by Sentry - Use code "tastytreats" to get 2 months free - sentry.io/syntax
#vps #webdevelopment #selfhosted - Věda a technologie
00:56 Domain Purchase and DNS Setup
02:43 Creating DNS Records
04:32 Query DNS Records with dig
07:50 www subdomain record, CNAME records and TEXT records
10:19 Email forwarding and a catch-all address
11:23 DNS Resources
12:13 SSH to your VPS with a Domain Name
13:08 Responding to HTTP Requests from our Server
16:27 Web Server Software
18:12 Why do we need a Web Server?
19:05 How To Navigate The Caddy Docs
21:32 Caddy Architecture / reloading configuration
22:35 Install Caddy
24:17 Set up https for the root domain
25:39 Redirect IP to the root domain
27:21 Multiple Caddyfile Setup
31:41 Redirect www subdomain to root domain
34:15 Set up /var/www folder
39:05 Transfer files to the server with scp
40:35 Sync a folder with rsync
42:31 Clone a public git repo
44:44 Hide files from Caddy file_server
46:08 handle_errors in the Caddyfile
47:46 Clone private git repos with an SSH deploy key
52:16 Install node.js with nvm
54:03 Set up Holidays API Example
56:37 Create a reverse_proxy with Caddy
1:00:45 Manage long running processes with pm2
1:04:07 Run a TypeScript node.js API
1:09:52 Run a static website that has a build process
1:14:44 Caddyfile concepts and Logging
1:17:12 Common Caddyfile patterns including SPA redirects
1:18:15 Thanks!
This is seriously top quality content on CZcams. Amazing work as always CJ and the Syntax team!
This is what I need. Please for the next video, guide for securing your server from DDOS, etc. Thanks CJ
Probably just add a cloudflare protection layer between your DNS
You can do this easily with Cloudflare. I'll touch on it in a future video.
You can read about it here:
developers.cloudflare.com/learning-paths/prevent-ddos-attacks/baseline/proxy-dns-records/
developers.cloudflare.com/dns/manage-dns-records/reference/proxied-dns-records/
@@syntaxfm oooh! Never knew about this learning paths from cf. Thanks for sharing CJ
Please, please, please continue this series! Subjects I'd love to see:
- How to configure things like logging and errors for "globally", so you don't need to configure it separately for each and every site
- Any way to limit repetition in the setup? Saw someone mention something with the @-character in a comment?
- Safe and easy ways to auto-deploy from git when pushing to a branch? Both for plain static sites, ones requiring a build step, and custom (e.g. node) servers
Most comprehensive Caddy tutorial so far
I don't think I've ever watch a clearer, easy to understand and to digest youtube video. Ive loved every bit of it. Il be following you everywhere!
Awesome video! Followup video idea: using Caddy as a reverse proxy for docker containers. 👌🏻
now we need a video of docker + Caddy and setting up a remote build server with automated deploys instead of having to git clone.
I plan to cover this in a future video. -CJ
would love to see example of setting up webhook to listen to changes too (not only changes, but maybe also external ones like headless CMS publish). I planned to use caddy-exec ojust a simple node server to do that.
@@syntaxfm The goat
@@syntaxfm thank you for this. Dockerising it as well as setting up a local env would be amazing!
Gotta say, these are amazing videos. It's so hard to pick the right level of abstraction to learn these topics. CJ's approach to "We're learning here at this level to move the tutorial forward, but if you wanted to dig deeper, here's 30 seconds on DNS records" is an amazing way to teach. Also, the netcat trick is so fun and a good illustration of how two computers "talk" to each other. The HTTP protocol is *just text*.
This is great!! I feel like no one has made decent content about this thank you!!
I have been doing this type of work for years, yet I learned so many little tricks on this video, insane quality
these videos are just GOLD, thanks CJ !
Thank you, CJ! This video helps demystify this process.
CJ is the teacher I wished I had in my entire time in university! Thank you, CJ
This series makes me feel powerful
Thanks for the video CJ, super in depth. Also thanks for the shirt, should be here in a few days
CJ is the best! Thanks for sharing your knownledge!
Love the series!
Looking forward to seeing solutions for better isolation between the apps. E.g. nested virtual servers, Docker, etc.
This was a thorough explanation! Thank you!
Thank you so much guys, amazing content.
Great videos. Thank you so much. I have watched both and worked through the first on my own VPS. Will be working through this one as well. I would like to see a setup on some of the other dynamic websites like next.js and ruby on rails. Also would be interesting in see how an email server could be setup.
These video's are amazing! Great job :) would love to learn more about how you would approach ci/cd and I'm very curious to see Coolify in action.
Thanks CJ for this awesome content! I've read you're planning on releasing a video on setting up a build server. Will you mention Coolify to do that? Coolify now seems to work with a Caddy config as well.
This is awesome! I would love to see how you would set up and manage docker and a db on the vps!
CJ you are the best!!
Ty for the video you're a life saver 😄 learnt a lot
Exactly 100 times DUBDUBDUB WAS MENTIONED🙃
Great as always 🥳
thanks Cj comment before watching after the end I came back for questions hhh
Keep going CJ!
this is super good thank you CJ , next can we talk about github actions so its will auto setup and update websites
Wow this video is a goldmine. If we have a video about anything like this video, we can achieve anything. Thank a lot CJ. You are the goat.
I have a question that not relevant to this video. Do you guys know any video similar to this one about React webpack and babel, i just want to learn about more low level of React?
sudo !! … I have NEVER seen that and even in a video about VPS an not primarly linux I learned something new
Excellent quality! Thank you for this! Question: How would I set up automatic deployment in my VPS?
There are a few ways to do this:
1. Clone a repo to the VPS and automatically keep it in sync with a remote repo - Checkout github.com/kubernetes/git-sync and github.com/GitJournal/git-auto-sync
2. Create a webhook on github and have an application running on your VPS that pulls the latest changes and restarts the app when a new commit is merged to main
Learn about github webhooks - docs.github.com/en/webhooks/using-webhooks/creating-webhooks
This article shows a way to setup the webhook on the VPS: maximorlov.com/automated-deployments-from-github-with-webhook/
3. Use a github action to pull the changes into the VPS - you can use the ssh-action to ssh into the VPS and perform any command - github.com/appleboy/ssh-action
4. Use a tool like Coolify which supports auto deploy - coolify.io/ - coolify.io/docs/knowledge-base/git/github/integration
-CJ
Thank you so much for the great content. I am actually following along, as a non-programmer.
I am setting all this up on my spare laptop, and my isp requires me to user certain ports, which means that I can't use the standard ports for HTTP and https. This means it's not as easy to get the auto https unfortunately. But looks like it's still possible
Because you're using full paths to files, you can do it from any folder on the system. As in, you don't have to go to `/etc/caddy/sites-enabled` to do it. eg. `cd var` then run `sudo ln -s /etc/caddy/sites-available/Caddyfile /etc/caddy/sites-enabled/Caddyfile`
Hah, something I already do. Caddy is great, and I use it with reverse-proxy to for all sorts of self hosted bits (often anything I can, be it api or services)
Good instructional content. I skipped through to see if I could pick up any tips, especially on Caddy which I've only been using for about a year. Gratified that I organise things in /etc/caddy in a similar fashion. Curious if there is a reason you don't use CNAME instead of A for the www entries in your DNS?
CNAME for www records is a good idea to prevent having to update in 2 places.
For this tutorial I wanted to really drive home the point of IP -> domain
-CJ
🤤 great this video super delicious for my mind thanks cj
Wow! That was a great video, full of content! Quick question, when setting up the DNS Records for various websites, why not use CNAMEs with the '@' target? And also why not use the Cloudflare proxy to mask the vps IP? Isn't this common practice?
Yes you are right. You can use the root domain as the CNAME value to prevent having to update the IP in multiple places. I really just wanted to drive home the idea of domain -> IP address in this tutorial for beginners.
I disabled the Cloudflare proxy because I wanted to make sure this video is useful for those who are not using Cloudflare for their DNS as well. I will cover Cloudflare features in a future video.
-CJ
@@syntaxfm Cool, thx CJ!
@@syntaxfm Would be great with another follow-up video explaining how to do whatever that @ thing is, the Cloudflare thing, and other improvements that are common and useful to do 😊
It would be interesting to see if we can automatically link running apps in Docker to Caddy ports. This could eliminate the need for explicitly mapping ports in the reverse proxy setup.
I have not used it yet, but there is this popular project that does just that: github.com/lucaslorentz/caddy-docker-proxy
-CJ
Hi CJ, thanks for this amazing tutorial as always perfect,
please can you make a tutorial about Nginx because when we try to get a job everyone asks for Nginx
I don't have any plans to cover nginx anytime soon, but it is very similar to Caddy, they just have a different config file format.
The way I setup the sites-enabled and sites-available folder is very similar to how nginx sets things up, though newer installations might use a single conf.d folder instead.
You can learn how to install it on Ubuntu here: ubuntu.com/tutorials/install-and-configure-nginx
Learn the basics here: nginx.org/en/docs/beginners_guide.html
Test out your configs here: nginx-playground.wizardzines.com/
To get TLS certificates, you can use certbot. They have an nginx plugin that works pretty seamlessly. As long as you have the site set up as http already, certbot will automatically add the https config: www.nginx.com/blog/using-free-ssltls-certificates-from-lets-encrypt-with-nginx/
-CJ
CJ this is amazing. I have a request please can you create automatic deployments and preview deployments on push to github for full stack apps like next.js or nuxt. That would be just awesome :)
I have been learning Coolify and will cover it in a future video. It has preview deployments built in: coolify.io/docs/resources/introduction#preview-deployments
Are you planning to make a video about hosting a node.js app in VPS (Hetzner+ Coolify)? Would love to see that 😊
I've been spending some time learning Coolify and will cover it in a future video. -CJ
@@syntaxfm Yay! 😊 Looking forward to it 🤩
This is very nice, and I learned a lot.
Another way deploy vite and others statically built applications is to built them with github actions and then copy them to the VPS. I never tried, but I've seen examples online.
I love pm2 and all the reports it gives. Is there another to monitor the application without being sshed into the vps?
pm2 plus is a paid offering from the creators of pm2. It has a web dashboard to see all your server stats: pm2.io/docs/plus/overview/
It looks like some people have made open source dashboards as well. I have not tried any of them though: github.com/search?utf8=%E2%9C%93&q=pm2+web+ui&type=repositories
-CJ
Keep going please
That was a great video, can you please dive into how to setup kubernetes on VPS?
I don't have much experience with kubernetes, but if I can find the time to learn it, I'll try to do a video on it as well. -CJ
@@syntaxfm Thank you
Hi CJ, is it possible to run both caddy and coolify on the same vps?
can cooliy be installed on a directory in our vps?
Coolify has Caddy built in as a proxy and allows for auto assigning of domains via docker labels, I recommend you use it if you can. You can disable this proxy though. In coolify under Server -> Settings -> Proxy - stop the proxy. From there you can use any software you want to listen on port 80/443 and reverse proxy to any running application. This will however be much more involved because you will need to map the ports of the running Coolify applications to the host OS OR determine the docker network address of each of your Coolify applications.
-CJ
@@syntaxfm Alright. Thanks
CJ I think by disabling the cloud on cloudflare we are missing a lot of cloudflare features like caching and security. Like we can hide our original ip address behind the cloudflare proxy
Hi, I don't understand it either, I watched it twice and I still don't understand why cloudflare proxy CJ is disabling.
Personally, I'm currently testing Cloudflare tunnels in production with applications running in Docker, which allows me to tighten the environment even more
I disabled the Cloudflare proxy because I wanted to make sure this video is useful for those who are not using Cloudflare for their DNS as well.
I will cover Cloudflare features in a future video.
In the meantime you can read about it here:
developers.cloudflare.com/learning-paths/prevent-ddos-attacks/baseline/proxy-dns-records/
developers.cloudflare.com/dns/manage-dns-records/reference/proxied-dns-records/
Thanks for the tutorial but i followed every step got namecheap domain then did cloudflare DNS setup but I cant access the server via the domain is there an extra step missing?
after hours of debugging i had to open port 443 aswell not only 80 on the server after doing so everything works fine
Glad you figured it out. Sorry I did not make it clear that port 443 needed to be open as well. I hinted at it earlier in the video but I could have been more explicit about this when we enabled https on the domain. -CJ
I found the form! But I might not be the first, lol
You are the first!!! I'll send the coupon code on Monday when I'm back at work. -CJ
@@syntaxfm Yay!!! 🎉
i added my cloudfare nameservers to my namecheap domain but the site is still unreachable. what am I doing wrong?
It can take up to 48 hours for the initial name server change to be propagated.
what a gr3eat channel name | cheerz
Next video should be CI/CD with GitHub Actions
I will put this on the list of things to show. I have configured github actions to deploy to a VPS in the past. -CJ