Video není dostupné.
Omlouváme se.
This homelab setup is my favorite one yet.
Vložit
- čas přidán 20. 08. 2024
- Hosting my own services using a homelab has been an absolute dream. However, my first homelab setup had some mistakes, and so I decided to rebuild it from scratch, and share with you how I did so.
Links to my hardware:
Beelink EQ12: amzn.to/3zK4Pbu
32GB Memory: amzn.to/3Lkotgy
2TB SSD: amzn.to/4eV117n
Ubiquiti Switch: amzn.to/3W5sJWp
Desk Whiteboard: amzn.to/3LogIGw
Keyboard: ZSA Voyager
The above are Amazon Affiliate Links which means I get a commission if you decide to make a purchase through them. This comes at no additional cost to you and helps to support the channel.
Other Links:
Github Repo: github.com/dre...
NixOS Installer: nixos.org/down...
NixOS Anywhere: github.com/nix...
Discord Community: / discord
i did not understand anything past nixos but i still watched until the end
`!!`
literally me haha
same
Yup
I hear I'm supposed to like NixOS but I still don't understand it
We are really interested in a crash course about homelab computing. How is a cluster useful? What is the purpose of a DNS? Load balancer? Namespace? All of those questions are a search away, but a "getting started" video would blow up for sure.
I shall add this to my backlog!
Second this, I work in devops so I have some understanding of what those things are (I'm still a junior) and how they are useful in a company. But for personal use? I can envision a usecase for hosting your own movies and maybe website.
tbh, very limited you can learn from video like this, it's like a showcase of how he setup his home lab, to achieve that you need to learn lot of knowledge to has foundation to start to do same setup. From your question, can see that you're lack of very basic knownledge on computer networking, and it's more nightmare if you put your feet on thing like IaC as use Helm, complex system like Kubernetes.
@@ChaiRuou That's why, a getting started video could cover the standard concepts without using advanced tools
@@Otakutaru try look at devops road map 🙂
Devops engineer here. Ubuntu server + Ansible is how I roll, both for homelab and at work.
I'm envious! I've never been able to get into the mindset for ansible. Now I've got a spare cluster (my old one) I may find some time to give it a go
@@dreamsofautonomy To be honest, even though it is simpler and easier to use, I feel like Ansible is a downgrade from NixOS, in regards to the control you have over the state of the machine. Nix allows you to rollback and specify the exact state of the machine, while Ansible only allows you to declare what you want to be performed on the machine, it doesn't give you any assurances that it is the state you wished for.
@@obvionaoe Aren't these different tools? I am not sure you can achieve the same ideas ansible can bring to a bunch of a machines while having good compatibility.
But please explain if it's possible, I am intrigued to know.
@@areweevensomeoneoutthere the benefits of nixos being a declarative system, rather than a declarative tool used with a normal system, are the full system reproducibility (if you set it up right) and freedom from dependency hell (every program and library is installed to its own directory, allowing multiple installs of the same program/library)
ansible is a great tool for reproducibility, but nixos can replicate all of ansibles functionality and a bit more
@obvionaoe nix is great for the initial set up phase but ansible is good for controlling multiple servers
It's funny this video found me. I just purchased 3 of those Mini-Pc's 3 days ago, and now this is sitting in my video feed. I guess the algorithms are on the job.
You got the beelink? Which model did you get?
Budget tip #574: Buy 8GB version of Beelink EQ12 + 32GB RAM stick. You can save some money especially when buying more nodes. Reselling the 16GB stick is not easy (and not worth it).
I have 6 EQ12s in a cluster, fantastic little machines to play with!
the main question i have is, what do you use it for? hosting websites? hosting game servers?
unless you are renting out these in form of k8 compute, its really hard to justify the cost.
@@DogeMultiverse Three control-plane nodes, three compute-nodes? It's always good to have options to try out different things.
Do I need a justification for a hobby?
I use them to test different technologies: hypervisors, hyperconverged storage, high availability.
Currently running vSAN cluster, purely for learning purposes. When I’m done, I’ll tear it down and build something else.
@@TheSoberPirate That's the same reasoning I used to justify a 4U storage-server with 160TiB usable storage and dual 10G fiber.
@@SnorreSelmer hahahah
neat to see some examples of k8s based homelabbing/self hosting!
I did learn nothing from this video - too complex 😅 but it did spark interest for trying kubernetes stuff myself
I'll definitely go through concepts in more detail! This one was much for of a devlog/homelab log. I'm glad it inspired some interest though! Let me know if there's anything you'd like me to dive in more about and I shall do so :)
@@dreamsofautonomy what are your thoughts on k3s vs Proxmox?
Relieved to see that I am not the only one.
@@dreamsofautonomy I thought longhorn was the Win Vista codename, but here it is what exactly?
what makes it a bit "complex" is the fact that he is using kubernetes heavily, but in the end its all the same, DNS is DNS, TCP/IP and networking are the same, its just kubernetes introduces some new concepts to people not familliar but its not hard to learn and understand..
i would recommend some videos on kuberentes and helm on youtube then return to this video.
Nice to see more NixOS around! I am using colmena to keep a config of my nodes, so I don't need to SSH into them to apply changes to my NixOS nodes.
Keep it coming!
Great video, just note. You should never writte a secret in a nix things even if you don't use git. Almost everything you put in secret will end up in the nix stores so beware of leaving behind secret there
so where do you put it?
@@John-x3l7m I can't explain here the rabbit hole of nix encryption but basically you need to store in the nix store already encrypted secrets files and in your NixOS configuration a module that can at the start of your server decipher your encrypted secrets and expose them in some runtime temporary file only specific program can have access.
Personally I use nix sops and sops to do that but like I said it's a rabbit hole and there more tools for the same jobs but here the ideas.
This was excellent. Although I would love to have seen the HA feature testdriven; perhaps builidng out the k3s stack and THEN upgrading the RAM & storage iteratively. I would also love to see cert-manager integration, as well as acme/letsencrypt.
Last week in office cleanup I was asking for 2-3 cpus and two guys from leadership gave me their own old cpus.
I was thinking about setting a kubernetes cluster. Lo and behold your video comes now I know what to do this weekend.
How did it go?
Great in-depth and detailed explanation for setting up a cluster from scratch, especially nice that in the end you explained why you needed two additional charts to get the final pieces of the puzzle working. Thanks!
I still need to learn more about how Kubernetes works, but I really appreciate your approach of setting up part of it, encountering an error, then fixing that error. Moving along in small steps, showing the error and how to fix it, I find really helps me to understand how the pieces fit together, rather than just giving a config file that works for you and leaving it up to me to figure out what I need to change for my environment.
Tried a similar setup about a year ago for some light homelabe stuff I wanted highly available. This would have made the exact thing I was trying, much simpler. I got flash backs multiple times in this video. The iscusi dependancy abd fix, metal lb crds with nginx as the ingress, yea identical and this is cleaner still. Add cert manager and you'll have everything I needed at the time.
If you plan on continuing with this project and series I'd be interested to see, meanwhile I'll take a look at you're other videos. Good stuff, nice work.
Thanks! I'll definitely be continuing with this setup and have added in cert-manager etc as well 😁. Should be more content coming out soon!
never heard of helmfile until today, thanks.
It's pretty great for managing helm files without setting up any gitops or similar!
me neither! flux and argocd are everywhere but normally a bit much
This is a really nice channel.
I hope it grows.
The rest of the video was good enough to excuse not saying "i-skuzzy". Took me a second to realize that MetalLB was required for DHCP since its layer 2 and would not be routed in or out of the cluster without an outside relay.
I didn't understand everything in this video, but it was good to watch, and certainly something to refer back to when i want to learn more about setting up my own home lab. I subscribed to see more videos about this and what you do with this home lab in the future.
I manage some Kubernetes clusters in the cloud and I learned a few things. Very interesting video. I bumped into your video 2 days ago because I did some searches on mini-pcs with n100 Intel cpu. I wanted to setup a Proxmox server with a Home Assistant VM as a starter. I also already copied some of your zsh config and moved my prompt from PowerLevel10Kto oh-my-posh (I already used it for powershell with the default config).
To test some Kubernetes configurations without deploying to the cloud I use K3D (in WSL on my laptop). Nodes are docker containers. It worked well for my use cases, I managed to validate some istio (service mesh) egress load balancing setup. But yes it is not a permanent home lab setup...
Great video! As a follow-up, I'd be interested in more specifics on HA and disaster recovery, especially regarding Longhorn.
this is so awesome and exactly the kind of video i was hoping for from you. i'm here for all the homelab content all day.
i'm impressed by the clear and concise instructions you give in this video. to be honest, the whole kubernetes with helm with helm charts with helm values with ingresscontroller etc, sounds like a lot of work to setup. it doesn't give me the feeling this setup is stable, but don't really know any details beyond that first look.
It would be interesting to hear about the usual breakage scenario's, and how updates look like.
You can use avahi to broadcast hostnames to the network so you can refer to all nodes by their hostname and not have to remember ips.
Great Homelab Setup Video, will also implement in my Homelab system
Bookmarks for the video would be a *really* nice thing to have. Such an awesome setup! Thank you for sharing it!
Great video and you have a talent to very cleary explain complicated concepts in a simple way. As a devops engineer working with this stuff daily I can say your setup is very nice and clean. But honestly I would not spend that much just to run a few containers! $200 worth of used laptops would be good enough for 10 times this workload. But I can also understand the desire to have a perfect setup. Looking forward to videos where you utilise that horsepower, maybe with Frigate and local LLMs?
I mostly enjoyed how clean your UI looks.
The DNS automation is huge! Thank you.
Can't wait to see your video on Talos Linux
😮 ngl the way the tooling is presented reminds me of the Microservices skit
I just got my framework 16 recently. Loving it so far :). ⚙️
I understood maybe 10% of that, but enjoyed every second of it!
I'm glad! Is there anything you'd like me to do a deeper dive on?
@@dreamsofautonomy I could watch a deeper dive on any of the individual services that you covered. Would love to have a better understanding of longhorn for example.
Thank you so much for this video! This pretty much exactly encapsulates my vision of what i am currently in the process of building! The helmfile and kustomize were two Lego bricks i felt like i was missing in my current attempt.
I'm excited to see more!
I understood everything you just did and that’s honestly a fantastic process you laid out but I do NOT feel confident that I could do it myself. You ran into extremely difficult issues that I don’t know how I would have figured out on my own. Great video though.
Who all saw yesterdays stream,
nice thumbnail!
That was a fun stream! Crowd sourced design!
Great video as always, boss
Great video! love how detailed it is
Your automation tools and workflow is amazing, really interested if you make like crash courses for the tools you used.
This is the first time your channel has been recommended to me, and I subbed before I even watched the video.
Great video, thanks for making! 🙏
Am curious to learn more about nix in upcoming videos
This is the tutorial I wish i had a couple of years ago trying to figure all this out
This seems fun, once i move to a bigger place i will definitely try to do something like this
Good video!
Im so jealous you had the knowledge to be able to execute this project. I would love to make your acquaintance and learn your background.
That's some overkill setup for a pihole ;)
Been there, done it.
I would NOT use really small PCs, unless they are Enterprise grade (used, cheap) so you never have to worry about dangling cables.
Highly relevant to my interests! I was dabbling with RKE2 but I'm not really in that mindset yet, so I think I'll go back to K3S. My nodes are running on RK3588 CMs, so I'm not sure if I can use Nix though. Will investigate further.
excellent video! I appreciate your CLI-to-video approach and you cover interesting content.
Wow, I learned a few things and it gives me some ideas to try in my clusters
Excellent! Thank you very much for sharing!
Wow, that's brilliant !
Overcomplexifying things that could be done with bachelor's degree networking knowledge.
Do tell!
You can use sops nix to point to the k3s tokenFile in a secure way. The problem lies in setting up sops-nix to play well with nixos-anywhere.
We are rebuilding our Kubernetes clusters at work to use Talos, highly recommend a look! Its the first time servers i deployed on-prem truly felt lile cattle instead of pets.
Terraform + Talos config + Helmfiles + Gitlab CI pipelines is life changing lol. I can deploy a full production ready cluster from Gitlab in a matter of minutes
I'll definitely be giving it a go!
We use tf and talos for our ka8s cluster as well. talos was definitely a bit of a learning curve, as it is a bit different than a more traditional distro.
@@solido888 yea learning to maintain the nodes in an API driven way is a learning curve for sure. But having essentially two config files to configure the nodes with a single command is wild. Plus the immutability drastically cuts down on maintenance required, and upgrading is as simple as changing the image in the file, and applying the config.
It's great
MORE HOMELAB PLZ!!
01:08 British accent and American plug. Brain went "huh?" 😀
Nix mentioned 🙏
Talos definitely would have been a quicker way of getting to a Kubernetes cluster. My firewall provides DHCP, which sets the hostname for each of my Talos nodes. This means I can use a single config for all nodes, only diverting controlplane and worker nodes. If you take a look at the quick start docs - yes, it is actually this easy.
But kudos for going the long way, it builds experience ;)
Great video!
I would love to see a kube install of jellyfin with hardware acceleration. This is easy on raw-docker, but not so straightforward on kube.
I have it set up! If you join my discord and drop me a message I'll send you over my configuration.
This is the most alt way Ive seem of setting up high availability. To to it from scratch like this seem overly complicated but fun
Loved the video after nixos anywhere I am lost but still saw the whole video at least I will sound smart when i will use the terms I learned here in front of my noobs friends. Hope to see many more videos in homelab series. ❤
Let me know if there's anything I can break down easier and I'll do some more dedicated videos!
Thanks for the video! And bonus points for having the same laptop and keyboard as I do
👍
It's a great keyboard!
Going to start calling it Kuberneddies from now on
Pretty dope! But just some remarks: with Talos you'd have spend much less time setting up your cluster (I totally get why you had done this, but still maintenance is a lot better on Talos), also you should check out the piraeus storage operator, it is muuuuch better than longhorn in terms of draining nodes, speed and maintainability as It's using drbd kernel module under the hood. (Talos has an extension for the kmod btw). Still great setup, I'm pretty jealous. Maybe try out kubevirt and hosting some VMs;) but you'll probably want to use piraeus/drbd for the storage then 😅, Longhorn is just hella slow
Definitely looking forward to trying out Talos in a future video! I'll be decommissioning my old cluster and using it for dedicated videos once I get everything migrated which should be end of this week!
Kubevirt is also something I want to check out as well! So much fun k8s stuff to jump into.
@@dreamsofautonomy awesome! But really do try out Piraeus on that test cluster, it is much much faster and I am pretty confused why nobody is taking about it. Everyone's just using Longhorn and many complain about it's performance but still choose and use it
@@LampJustin Longhorn has been a thing for longer than Piraeus, that's definitely the reason. Most people prefer stability of a known tool over faster performance or extra features, especially when running services with high SLOs
@@obvionaoe while it's true that Piraeus is a younger project, the technology is not drbd is pretty dang old at this point and very much proven. Longhorn is known to have problems with lagging volumes that are out of sync, so slo is definitely worse. I also didn't mean for all people to talk about it, but I've seen no CZcamsr talk about Piraeus.
Just get a bunch of used optiplex with 10th gen intel for fraction of the cost. up-to 64GB ram. Can even upgrade to 2.5/10/25/40 GbE anytime.
This video is like ASMR to me.
i had a *fun time setting up pihole's recursive DNS server via docker on my NAS. in hindsight, recursive dns was a nice to have but not worth it for the extra container instability/troubleshooting during a reboot.
been into homelab for about a year. now wfh is never been the same
or even everywhere thanks to cloudflare tunnel
What do you use cloud flare tunnel for currently? I've been seeing some good things about it! Might try to find an excuse to add it to my homelab.
what do you do for wfh?
@@dreamsofautonomy I use it for ssh to my computer. So wherever I work I can just ssh to my computer and use my fast computer with tmux+neovim instead of 8GB laptop provided by my company lol
@@DogeMultiverse I make docker swarm cluster with my 2 server to replicate my office stacks (postresql, redis, timeseries db, prometheus, kafka, etc) and use them instead of try to run them all in my main working pc/laptop
Man NixOS kicked my ass the first time I confidently decided to install it , had a small issue , no biggie ill just check the wiki ... 2 hours later and 30 tabs open I knew I was in over my head ! i will try again sometime though.
I just use a basic server with SystemD, worked fine for years now.
Awesome video. I do have a question: Why do you use Kustomize for a file that doesn't need any templating? Just use kubectl apply -f file.yaml instead of using kustomize and adding more complexity.
NixOS is sooooooooo goooood!
This is awesome. I also have own the Beelink EQ12. Really fantastic device.
A thing to note is that you'll need at least a 6.6.x kernel to get the wireless interface(s) functioning. I wanted to use Ubuntu Server, but of course I purchased the device just before 24.04 dropped, so that meant more work. Instead I went with Pop! OS ... Definitely not ideal, but the machine is running pretty well.
Next up, I would love to replicate what you've done here. We both need the rack-mount tray thing that "Techno Tim" used for his mini pc build. Though, he went with much more powerful Intel NUC machines.
Ohhh that's awesome I'm gonna check out the rack mount, currently I have both clusters in operation as I'm migrating and my tiny shelf is a mess.
Good to know on the kernel as well! I'm pretty spoiled for ethernet runs so I forget to test out wifi capabilities!
Great video, and a lot of cool ideas for me to now try over the next month or so. I setup a k3s "cluster" just this morning, the timing for this video could not be any more perfect. All I wanted was a postgres database for one of my side projects, so I just use an old rpi 3b that I had lying around and installed k3s on it just for fun. helmfile looks interesting though, I might give it a shot. I just made 5 different yaml configs to deploy postgres, followed a guide from digital ocean. A beelink eq12 cluster seems a little overkill for my use case, but I might add some more PIs to my cluster and then add kubernetes to my resume.
Another hardware to consider is the Odroid H3 or H4 mini boards. They can be more affordable depending on where you live (edit: spelling)
Oh awesome! I'll add this to my backlog and take a look.
I just want you to make actual in-depth courses, workshops and alike, to really push this sort of stuff forward. I'd be first on queue.
Soon! I promise. I'm looking for video editors currently so I can spend more time creating :)
Nice video! It’s nice to see some alternative choices.
You could also check out rancher and harvester. I’m currently building a new homelab with rancher, Traefik (feel the pain), cert-manager, longhorn and metallb.
I'll check it out!
Exactly what I was looking to do on my homelab (except I'm using debian/terraform/ansible, instead of Nix/helmfile). Two small things missing I think : `ingressClassName: "nginx-internal"` in `pihole.yaml`, and remembering to configure the router to use pihole as the default DNS. Otherwise, thanks for the base config!
The last time I heard about home lab was from breaking bad
Great video! I know it's a bit off-topic, but I'd love to see a video of you showing how you setup your Hyprland, it looks so nice and clean at 5:00 😊
Absolutely!
Talos user here. If all you are interested in is k8s, Talos is a perfect set and (almost) forget solution (remember to update it!)
I also run Talos on production clusters, I can vouch for the ease and stability.
Awesome! I'm really looking forward to giving Talos a go.
I've been thinking about using nixos on the server side as well for a while. But I felt like it was more suited for clients. But after seeing this, I may should try it on the serverside as well..
I've got a couple of plans for server side nixOS videos coming soon!
Maybe try installing home assistant on this cluster.
I’m looking to do something similar but the n100 is not that adequate for running anything you throw at it. While capable, they are similar in performance to 8th gen laptop CPUs.
N100 is perfect for a NAS or pfsense box but it’s a stretch for this application (depending on services hosted).
I'd love to see how to get HTTPS in Kubernetes with local auto renewal. Like Kubernetes Cert-manager or something!
You just got a new subscriber 👍
Glad to have you on board!
You should look into CUE and Timoni as a replacement for Helm. Very new projects but have tons of promise (and type safety + helm interop)
Adding both on to my backlog to take a look at! Thank you for the suggestions
Isn't the point of high availability to have the nodes different locations (eg: having all nodes in one location is still a single point of failure), from what i see the benefit here is the replication of data into three different nodes, but a local DAS or NAS would have done the trick!
Appreciate you documenting your homelab renovation! Any chance you have a public facing repo with some of the IaC shown here? ty!
I should do in the description! At least I hope so.
Great video! Interested about your jellyfin setup
Great video, would love to see the next episode. I think I prefer Ubuntu server though
Awesome video. Can you kindly explain your terminal setup? Especially what terminal based file explorer that was? I know about and use zen but your configuration was 👌
Dude, you inspired me to get my Lab inorder :D
Got two of those Belkins together with some HP Mini G3s I have
One thing thought, Would you consider making a video abt NixOS?
I am running MacOS with Nix installed ( dont know if I can do what you suggesting with nixOS anywhere )
Pretty decent lab you have there !
Thanks! You can do nixos anywhere with macOS and nix!
I have a couple of videos planned for nix! Going to start with one looking at Nix Darwin and then on nixos.
It'd be great if you add wireguard or any other VPN into the setup
Please make a tutorial on home lab
Hi, loved the video. I might be wrong, but I believe you have your cluster set up such that you have all three nodes as part of both the control plane and hosting worker pods. Is this correct? If so, are you not concerned about a possible container escape?
Nice! I just added a Beelink SER5 to a RPi4 K3s cluster and was thinking I might want to add 1 more Beelink, thinking about the EQ!
Personally, go EQ12. The new EQ13 is really cool, but they dropped the 2.5 Gbit Ethernet.
Hmm. To be honest. 32 gb ram is bit overkill when deploying services on slow cpu. 20w is nice selling factor though. At the beginning, I often spend time thinking about how many nodes running and what are the specs? Overtime, I realize having a solid storage strategy is more important.
Absolutely! I make use of a NAS for more long term storage. Will be doing a follow up video on how that works as well
neatly explained, nice setup!
one question - aren't you creating potential bottleneck/inconvenience by using DRAM-less QLC SSD? compromising both performance and lifespan while not really saving much money or power draw?
You are lucky that you have all that shipping options available on amazon 😂😂
An honest question: what's the advantage of having multi-nodes vs say using a tower pc for all services? I can only think of possibly redundancy and it being more modular. A larger tower pc also has the advantage of being able to use spinning hard disks, lower total idle power consumption, and is easier to administer. It also saves the headache from networking complexities.
Redundancy is the big one. I also think power draw tends to be better on mini PCs as they use laptop hardware which is generally more efficient that desktop parts.
As for spinning disks. I actually use a NAS for this which sort of acts as the desktop PC! Ill do a video on this soon and how it works.
If you only have a tower PC available, then I'd recommend using that tbh, you do get the ability to use GPUs which is a benefit.