Proxmox Virtual Environment Complete Course Part 13 - Networking
Vložit
- čas přidán 22. 05. 2024
- Setting up a separate network for your virtual machines is a great idea, especially since that allows you to segregate your management network. In class 13 of LearnLinuxTV's Proxmox series, we'll set up a new network and you'll see how to set up your virtual machines to use it.
Support Learn Linux TV
Note: Commission may be earned for any and all links presented here.
- Get your own cloud server with Akamai Connected Cloud ➜ learnlinux.link/akamai
- Support me on Patreon and get early access to new content! ➜ learnlinux.link/patron
- Check out jay's latest book, Mastering Ubuntu Server 3rd Edition ➜ ubuntuserverbook.com
- Affiliate store for Linux compatible hardware/accessories ➜ learnlinux.link/amazon
- Check out the Tiny Pilot KVM for your Homelab ➜ learnlinux.link/tinypilot
#*Time Codes*
00:00 - Intro
00:59 - Quick comment about hardware requirements
02:36 - Looking at the available network interfaces
05:00 - Adding a network (Linux bridge)
07:35 - Creating a virtual machine on a separate network
Other Proxmox episodes:
Class 01 - Getting Started: linux.video/pve1
Class 02 - Installation Process: linux.video/pve2
Class 03 - Web Console Overview: linux.video/pve3
Class 04 - Containers vs Virtual Machines: linux.video/pve4
Class 05 - Launching a Virtual Machine: linux.video/pve5
Class 06 - Setting up Virtual Machine Templates: linux.video/pve6
Class 07 - Creating Containers: linux.video/pve7
Class 08 - Setting up Container Templates: linux.video/pve8
Class 09 - User Management: linux.video/pve9
Class 10 - Backups and Snapshots: linux.video/pve10
Class 11 - Integrated Firewall: linux.video/pve11
Class 12 - Command-line Interface: linux.video/pve12
Class 13 - Networking: This video
Class 14 - Shared storage: linux.video/pve14
Class 15 - Clustering: linux.video/pve15
Class 16 - High availability: linux.video/pve16
Bonus video 1 - Launching a Windows VM: linux.video/pve-win
Bonus video 2 - Getting started with Proxmox Backup Server: linux.video/pbs
Bonus video 3 - Proxmox VE - How to build an Ubuntu 22.04 Template (Updated Method): linux.video/pve-2204
Bonus video 4 - Build a Kubernetes Cluster on Proxmox: linux.video/proxmox-k8s
Full Courses from Learn Linux TV
• Linux Essentials Certification Workshop ➜ learnlinux.link/lpi-course
• Linux Crash Course series ➜ linux.video/cc
• Learn how to use tmux ➜ linux.video/tmux
• Learn how to use vim ➜ linux.video/vim
• Bash Scripting Series ➜ linux.video/bash
• Proxmox VE Cluster Full Course ➜ linux.video/pve
• Learn Ansible ➜ linux.video/ansible
Official *Learn Linux TV on the Web*
• Main site ➜ www.learnlinux.tv
• Community ➜ community.learnlinux.tv
ABOUT THE HOST
- Personal blog ➜ www.jaylacroix.com
- Twitter ➜ learnlinux.link/twitter
Learn Linux TV provides technical content that will hopefully be helpful to you and teach you something new. However, this content is provided without any warranty (expressed or implied). Learn Linux TV is not responsible for any damages that may arise from any use of this content. The person viewing Learn Linux TV's content is expected to follow their best judgement and to make their best decisions while working with any related technology. Always make sure you have written permission before working with any infrastructure. Also, be sure that you're compliant with all company rules, change control procedures, and local laws.
#Proxmox #ProxmoxVE #Virtualization - Věda a technologie
A nice thing about your videos is you slow down your speaking and it makes it easier to follow along. Thank you for all your work on these videos!
you dont want to add an IP address to the bridge network that you setup for the VMs to use.. with that IP address defined... VMs can reach your pve webgui
Great series. Thanks for your efforts! This is the one video I was looking forward to the most. I think there’s probably a lot potential for a further video on this topic.
You should have done an update to that linux VM at the end to show that it actually had net connection (which it hadn't). I cant get why you skipped that.
Thanks for this series. This video helped me find the answer to a problem that has been plaguing my server.
I set it up on a desktop machine and all was working well to start. Then I decided to throw in a four-part NIC I had laying around because I thought it could come in handy at some point as opposed to sitting in a desk drawer.
After a reboot, I realized I had no ability to connect to the server any longer. It took a bitvgorbthis noob to figure out that all the network ports were now set to manual start for some reason. I got eno1 set to start automagically and gave it the original static IP it had. Eh violá, I can get back in! Only now I find out that my servers have no network connection. I was stumped again, but knew I'd find the answer in this series somewhere.
The part where you showed the original network bridge was tied to the port you had assigned during the proxmox install made me realize I needed to go tie my bridge 0 to the eno1. I'm gonna try it when I get home and see if that solves my problem.
Thanks for the work you do. By the way, I love the mastering Ubuntu server book. It was very well written.
After I saw this video I said out loud "where can I give this guy some money?" new patron alert!
u are segacious, I like it. this is the first time i write comment on youtub. you deserve that man
All the best
Pretty cool Jay. Hardware passthrough is awesome too.
So glad I stumbled upon your channel. Great videos. Thank you
Great video as always, Jay!
you saved me soooo much time, please continue doing videos, thanks !!!
Muito bom Jay, obrigado!
very well explained series. thank you
Great video and series. Very helpful. One question to the community, I had issues with the bridge until I added a VLAN tag when creating the VM. Once I added the tag it started using the correct IP subnet. Is this typical or because how I setup the network from my router?
Excellent video. Hopefully we can see more of it. One question: how can i separate management interface and VMs through vlan aware or OVS. I did plenty of researching over internet about but still couldn't get to understand it completely. In cases where your proxmox server has only one nic how can I separate both interfaces?? Thanks in advance
Can you still access the webserver from your laptop in your management network now that it's on the vm network? Can the webservers still reach the outside world (for updates, for example?)
In Proxmox you didn't use a second gateway but split out the interfaces and subnets. Are you simply running a flat subnet, is Proxmox supplying the DHCP somewhere? This made me have a lot of questions as a network engineer. I would expect a whole other interface set to be segmented from the primary interface. Unless it were a part of the same bridge. How do you perform bonding and use say lacp? I have not had a chance to play around with Proxmox yet. Wanted to RTFM a bit before I started and your class looked decent so I spent some time here. Will say I love it so far for starter information. Your firewall covering was nice. Thanks for the quality information.
Thank you it's not just me! Watching this made go crazy. Does the second port have a cable attached? How does all this work?
@@bjornarsimonsen7592 I was also wondering how the vm got it's ip, what is serving dhcp? He must have connected the second port with a cable to another router but regretfully forgot to mention that.
Great video! I created a TrueNAS Scale VM inside of Proxmox VE 8 and I assigned a new bridge vmbr1 to it. However, I noticed that the MAC address of the port I assigned to the bridge and the MAC address seen inside of TrueNAS are different. Is this an inherent property of a bridge? How do you view the actual MAC address of the new bridge inside of Proxmox? I need this in order to set static IPs in my router...
When separating the networks like this is there anything special for the physical layer? Like would you have to connect 2 ethernet ports to that one node or is this done through the firewall/proxmox.
thanks for the guide, I was able to separate my TrueNAS LAN connection
Why have a 10Gb connection for the management layer? Is that for running containers?
Hello, thank you and congrats for your really good and helpful videos!
I am new to proxmox and i want to test it before start using it in production environment. But to do that, i would really need your advise, if that's possible.
What i do now in my production environment, im using windows server as hypervisor, with only one physical network interface getting public ip via dhcp.
Im using NIC Teaming to create an additional virtual network adapter so I can connect it to my provider's vSwitch, using vlan.
When NIC Teaming is ready, i have two virtual adapters, and then I am creating two vSwitches inside Windows HyperV Manager Settings
(one vSwitch for local ip - vlan and one vSwitch for public ip).
That way my VMs have one public IP (which i buy from my provider)
and one local IP using VLAN, so my VMs can connect with rest of my local network.
What can I do to have a similar setup inside proxmox? Thank you in advance.
Hi, can u explain how to limit bandwidth of VMs inside the proxmox, please
I did exactly that but they don't have any wan access. It's for two windows vms to practice active directory. Ended up spending more time setting up the network than practicing for class 😢. Anyone got any tips or website to look at id appreciate it.
Hi, can you please do an updated video where OPNsense is used with 3 physical nic's one for management one for WAN and one for LAN that will then have vLAN's. Then be able to use the SDWAN feature in ProxMox 8.1? Thank you.
Thanks for the video
That dual port 10Gbps Network card, can you show how to setup LACP in Proxmox ???
for Best Practice please use 1Gbps Nic for Management and 10Gbps for share/external storage & VM.
By choosing to use another bridge the ip change on the vm can give some problems so the ideal situation is to do that on a fresh install
Great video! I only have a NIC with only one rj45 port, can a make this work?
Great videos Jay. May I suggest a topic for a future video in the Proxmox sessions? " web hosting from a VM in Proxmox". I have been searching for help on this topic. Nothing helpful for a novice exists at this time.
I’d love to see some tutorials on the new SDN features in proxmox
I know little about networking and I want to learn it, but for some reason there is no hands-on course or tutorial teaching you how to learn networking just like building mobile apps or web apps; it is all theory which I don't like. Thank you Jay for showing us this.
That's because you should know the underlying principles if you're going to properly learn networking.
Thanks sir!
Is there a way to find out what port is for what card/device? I have a chassis with 4 internal ports and 2 add in cards with 2 ports each, how do I determine what's add in card 1 and card 2 or builtin? I currently have wires in all ports but proxmox only has the one active from setup, but I don't know what port/card it is.
Secondary question (don't know what video to look for) how can I tell proxmox to assign bays x-z of a multibay chassis (sc846, single hba card to a sas2 expansion backplane) to a certain vm so regardless of what drives are in said bays they're passed through for example 0-11 to an unraid vm and 13-23 to a Truenas vm.
Perfect timing as always! Found a second Asus Z9pe-d16/2l with dual 10 gig ports!!! VM's for everything.
I am just curious where can I buy cheap networking equipment to practice on? ebay?
Good video,, but if possible can you expand on network on proxmox? How to do lacp bond and creating a trunk. So that we can have multiple vlans over the bond?
I'd also like to see a video like this!
Great video and guides - I did install proxmox but I think i need GPU acces sometimes to use on peertube and other stuf doing encoding how you make some guides for this also ;)
What would be the settings for pfsense with this setup, I can't get my vms to show the lan ip and when they do there is no internet
What i fI have two networks created on my router Mikrotik and both have DHSP servers and act as a gateway. Can I add a gateway for a second network too? And wouldn't that allow me to access Proxmox console though other networks too?
Thanks!
NIC Teaming and failover config would be nice for production environments. Is Proxmox supporting NIC failover etc?
To my knowledge it supports bonding/link aggregation, which does that.
Hi,
How to configure port aggregation or bond scenario.
Thank You so much. Do you mind to share a book recommendation for internetworking on linux - advanced -.
I would like this as well, possibly a course or tutorial.
Unfortunately my machine does not have two interfaces and I had to follow the video and write down the steps so when I manage to have several interfaces, I thought it was a very good video, congratulations!
There is one small thing that I can't understand, if your PC has 172 and the Proxmox machines have 10, how can you connect without having display problems? That is, apart from this exercise, have you had to change the router's subnet mask or something so that you can connect to 10, right?
I don't understand the point of having 2 different subnets and physically having a connection without having to modify the masks or routers.
Great video!
wow, such cool explanation!! 👏
so, how do you configure the network for a VM can be reached from outside the host? 🤔
Same question
I think you’ll need an public ip for that
how would you do a second vlan? would you have to use a third network port or could u run multiple vlans off the second port
Make vmbr1 VLAN Aware and designate which VLAN you want in the VLAN tag field. You can have all your VLANs over that one vmbr.
@@fredtheilig9636 Thank you Fred!
This may be a dumb question or misunderstanding on my part, but do you plug in both of your NICs using a lan cable? For me when I try this, I have a cable going to my mobo port, from which I can connect to my LAN, and then I have a card with another port that I am trying to use as a bridge. I was assuming that if I have the mobo port connected, that the bridge made to the card port would be able to work and any container built from that should connect to my LAN. Am I missing something?
tldr: have two bridges, vmbr0 and vmbr1. Vmbr0 is plugged into a network switch and this allows me to hit the command GUI. Building a container from this lets me access the LAN and WAN. Vmbr1 is NOT connected via a cable and building a container from this does NOT connect to either LAN or WAN (it does hit the IP of the bridge itself but that is it).
But how do VMs on the 2nd network access the outside world if there is no gateway? And how to get it from another host not on that 2nd network?
Does the 10.0 network have internet access?
Nice video but couple comments: (1) You added a subnet on the vmbr1 instead of an IP address. When I configure a new network bridge I add an IP address that I can ping on that interface. On the switch side the port needs to be configured as Access Port on the specific VLAN that you want to use to make it work (and match the subnet configured on the vmbr1). (2) You can use the same vmbr0 or vmbr1 and enable VLAN aware and define the VLAN that the VM will use. In that case the uplink port on the switch needs to be configured as Trunk Port that allows to pass multiple VLANs.
he's trying to separate the VM network and the ProxMox web management. If a hacker got on the VM network, say your webserver, it would be harder to access the proxmox server itself.
exactly. Option number 2 makes more sense as you wont be limited to just one VLAN/subnet (per bridge)
I see some people are confused by the lack of gateway on the VM subnet. Yeah, I was too at first. The gateway is for the HOST OS, and not the VMs. You can only have one default gateway on Linux. For each GUEST/Container VM, they'll have their own default gateway, which would be, in this case, on his 10 dot (vm) network.
Thank you. I still struggle a bit with network in proxmox.
So let say a vmbr is like a switch
Why would I need to bind that linux bridge to a port (ensp ...) ?
If gateway doesn't matter, how to make the VMs on one bridge get access to internet if I need to?
I will be glad if I find answers to these questions.
Thanks
@@digiground7613 The name says it all ... DEFAULT gateway, there's only 1.
Consider a default gateway the front door of your house that opens up to the world. You can however have multiple gateways (just 1 default), like having a back door openening up to the world as well, just using a different road.
Every OS uses its default gateway to send traffic over the internet, but that is defined in the OS or by DHCP, not on proxmox. Proxmox just has 1 default gateway (but you can define secondary (or more) gateways who will have a lower priority. This is for example used when providers have a fast and a slow link for backup.
Imagine a computer has 5 network cards and every card has a default gateway defined. How would the OS keep track of any outgoing or incoming traffic when it's being sent over a different network card each time? Instead it just uses its one and only default gateway to send traffic over. But you can set local routing rules that overrule the use of the default gateway for certain traffic. This is where networking gets interesting... for the geeks anyway.
vmbr just means that your network card is in bridged mode. It's like a layer or a mask you pull over enp5s0f0 where you can define the properties of the bridge, On a virtual host, you usually have several important network modes:
- host only: this network connection only allows network traffic between the proxmox host and the VM
- "normal" (i forget the name lol): Your proxmox host is going to behave as the router for this network card, using NAT. From the outside, this network is protected, like how your home router protects your private lan.
- bridge: this mode gives "full access" to the network card by the VM, bypassing the VM host and exposing itself fully to the network. This scenario is the least restrictive but also the most dangerous to use as your VM sits wide in the open and requires additional security measures. It is a valid choice in most situations to avoid needing to do double-NAT, which can cause issues for some applications and services. Just be aware of the dangers and the necessity to implement additional security.
Depending on your virtual software, you might see some more modes but these are probably the most used situations.
And always keep in mind what you are talking about, a nic on a proxmox or a nic on a VM are very different things. You define the bridge in your host. IP address assignment, DNS, router/gatewat etc is all defined in the VMs OS. It has no clue what mode the NIC is defined as on Proxmox. The VM just knows there's a working network card for it to use.
Random question, which is outside the scope of this video I think... but when it comes to best practice... should my idrac/iLO's be on a separate network than the management networks?
Hi - the way I see this, and I do this on our network is that ILO/iDRAC interfaces are in fact management interfaces and as such will fall into the management network IP addressing scheme.
yes
This is very helpful coming from the esxi world. What is not clear is what is providing DHCP services.. What did I miss?
The DHCP server on my network handles that, the same DHCP server that all my other stuff uses.
@@LearnLinuxTV Thanks :) I had a fantasy that proxmox would do that also! :)
how did your vm get the ip address? Do you have a dhcp server running on that 10.10.10/x/24 net?
I'm not sure I follow the gateway explanation. You have a gateway. It is for Proxmox and the management network. You create another bridge without a gateway for the VMs. However, if the VMs need to get to the Internet, they are going to use the gateway for the management network? I thought the whole purpose was to have separate networks. In my lab, pfsense+unifi+proxmox, I built separate networks on pfsense and created bridges in proxmox with the respective vlans.Any VM connected to that bridge goes out the VLAN gateway I created in pfsense. Same as yours, I did not have to specify a gateway on the bridge but because that bridge is on that vlan, it got the dhcp scope with gateway for the vlan. Anyway, I guess we can all have it network differently. Thanks for the lesson.
Why does the IP end in .0? You don't assign an actual IP address to the bridge?
Doesn't the first IP address indicate the scope of the pool, just as the last indicates the broadcast?
I'm confused because usually we assign an address to an interface in order for it to be reachable in the network.
Same question here. Adding to that: I have another problem. When I create a new bridge the management GUI transfers over to the new bridge.... I HAVE NO CLUE WHY! Any ideas?
Thanks for the valuable series. It is really appreciated. I am not sure if I lost something within the video. My question: How do you assign a static IP to a specific VM? I mean, my PVE has IP 192.x.x.100, by default my VM was assigned with 192.x.x.192 but I need that VM to become 192.x.x.80. Thanks for helping oaths.
in the network interface configuration of VM. In debian you have /etc/network/interfaces
@@vadvenurt Thank you
So, if I have only one network card (which is in my laptop), can I still make separate network with management network? I really new in networking (and quite bad in it) so mind if you could help me with this? thanks!
You're a wizard
You configured the vmbr1 with a subnet, why? I thought you need to set a static ip in ordner to give your host an ip adress in this network. What's the point of giving the proxmox interface a network instead of a static ip? Or is proxmox literally creating this subnet on that bridge?
Great question ! When I do the same thing as in the video, it doesn't work. The VM is not accessible .
Hey. I also like this video series, but as few others mentioned here, there are some confusions about the actual IP of the vmbr1. I presume those VMs/LXCs will require the GW, but there is no mention on it in the video. I'm also lacking any response from Jay in this comment sections. :-(
I find it easier to create a bond with 2 x 10Gb ports, configure the switch end as a trunk port and send all your vlans down it. You can create separate vlans on the Proxmox end for management, storage etc. Dont forget to config the vlan id for the guest.
This also works well for XCP-NG but is a bit of a pain to get setup.
Create a vm-gw, enable IP routing on it. You can also use a fw such as pfsense. Add two nics on it, one connected to the internal network and one connected to the management network. Set the default routes on other vms to this gw
So please correct me if I'm wrong, the management interface is now tied to the same ip-address that points to the gateway. Isn't this the very thing we're trying to avoid here? Edit: Also, my bridge automatically created during install. is "vmbr0", tied to "eno1". "eno2-4" are on the same network card. Is this bad from a security standpoint or is the physical separation of ports already enough?
As long as your firewall separates your vlans properly, the same gateway won't matter. As for the 2nd part of your question, this also boils down to your firewall. You can have separate vlans all on the same physical port and be completely secure as long as your firewall separates them from each other.
How to add subnet to network?
Hi, how about accessing the internet on VMs from the second bridge you just created, Port-forwarding and so on? It's worth to show all the moving parts regarding networking in Proxmox. This is my suggestion.
Exactly, Jay doesn't show being able to connect to the internet, but I'm a newbie so maybe I'm missing something here.
@@leviholman Yeah, same problem. Followed this, and can't connect to internet from my VMs. Still trying to figure it out.
@@PeterBoling Welcome to the club
Yeah, you need NAT or something similar to allow VMs from separated network to access Internet. I don't think this separation is worth except for mission critical systems. It is just another layer of complication that barks and bites you every time you want to do something useful. The argument "if someone breaches into your webserver VM, he can't access your VE" is strange; because if someone access your VE from your VM that means that your VE does not have proper security policies set up, which should not be the case.
Nice tutorial , but how l can create bond ?
Wow!... There's not one CZcams Proxmox networking tutorial that covers the OVS aspect of networking. I've been searching forever and it seems that the creators of Proxmox tutorial videos either don't like it or don't know it. Why is that?
0:20 Networking is currently one of my least favorite topics right now, I've installed proxmox in a datacenter. i managed that the test-ct got an ip address but whatever i'm doing, it can't ping the Google DNS server. From the host, I can ping the DNS server with ease. Hope that Video helps me in that regard. And i have to say that i'm currently very upset about proxmox, for a software, which seems like to be designed like "have a gui for everything" the network configuration so far needed a lot of CLI so far. Seems not complete in that regard IMHO. So as you might notice: I'm very frustrated about that :(
10:07 "great", this video showed me also not what i needed to know: How do VMs reach the "outside". Cause they don't, they can't even Ping anything else besides the Host itself :(
Did you get it resolved? Could be an issue with the Proxmox firewall (if it's enabled), or there could also be a route issue going on inside Proxmox. If you've made a lot of CLI edits to the networking interfaces something probably got messed up along the way. This video is pretty introductory, and even though he mentioned VLANs at the beginning, he didn't cover them. I have lots of VLANs and went to other sources to understand how to set them up in Proxmox.
can use network teaming or bond and used by containers and VMs plz
This is a great series about proxmox. I have a small questions. I need to add bluetooth i mean motherboard`s bluetooth. I have hp prodesk 400 g6. Could you please help me. Thanks so much.
I highly DO NOT RECOMMEND the process shown in video! I tried twice and not only It did not work but I also lost connection to my management IP / bridged interface. I had to twice completely reinstall Proxmox sever. 😞
From where does it get the IP? Why didn't you define an IP for the bridge, but only a subnet instead? How does it know how to route the traffic without a gateway? Is there an actual cable connected to that extra port? Everything in this video is almost useless without this info.
I think it works that way: Since you didn't assign an valid IP-Adress, your interface can't be accessed. But traffic can still be routed through that interface.
And since your VMs have configured the required gateway, the traffic can reach the suitable gateway.
No need to mention vlan I'd for vm ?
Everything is good but I can't understand why youtubers like you totally forget the IPv6 on their vidoes..
It's a shame that you only explain thins which already mentioned in Proxmox's documantation..
It will be great if you do something on IPv6 as well, specilally the routed network with IPv6
Please upgrade VLAN it is video
I cant even assign a port/slaves from vmbr0 to eno1 it says “no acces” or i get “acces denied” or it says “already have default port” 🫠