XCP NG VS Proxmox 2022 And Why I Chose...
Vložit
- čas přidán 10. 05. 2024
- lawrence.video/xcp-ng
Lawrence Systems Forum Post
forums.lawrencesystems.com/t/...
Lawrence Systems tutorials on XCP-NG
lawrence.technology/xcp-ng-an...
Learn Linux TV Proxmox course
www.learnlinux.tv/proxmox-vir...
Learn Linux TV Proxmox Backup
www.learnlinux.tv/proxmox-bac...
Connecting With Us
---------------------------------------------------
+ Hire Us For A Project: lawrencesystems.com/hire-us/
+ Tom Twitter 🐦 / tomlawrencetech
+ Our Web Site www.lawrencesystems.com/
+ Our Forums forums.lawrencesystems.com/
+ Instagram / lawrencesystems
+ Facebook / lawrencesystems
+ GitHub github.com/lawrencesystems/
+ Discord / discord
Lawrence Systems Shirts and Swag
---------------------------------------------------
►👕 lawrence.video/swag
AFFILIATES & REFERRAL LINKS
---------------------------------------------------
Amazon Affiliate Store
🛒 www.amazon.com/shop/lawrences...
UniFi Affiliate Link
🛒 store.ui.com?a_aid=LTS
All Of Our Affiliates that help us out and can get you discounts!
🛒 lawrencesystems.com/partners-...
Gear we use on Kit
🛒 kit.co/lawrencesystems
Use OfferCode LTSERVICES to get 10% off your order at
🛒 lawrence.video/techsupplydirect
Digital Ocean Offer Code
🛒 m.do.co/c/85de8d181725
HostiFi UniFi Cloud Hosting Service
🛒 hostifi.net/?via=lawrencesystems
Protect you privacy with a VPN from Private Internet Access
🛒 www.privateinternetaccess.com...
Patreon
💰 / lawrencesystems
⏱️ Timestamps ⏱️
00:00 XCP-NG VS Proxmox 2022
02:30 Features Comparison
03:09 Explanining Xen Orchestra
08:15 Max Support VM Disk
11:30 Proxmox interface
13:57 XCP-NG Xen Orchstra Interface
#XCPNG #proxmox #hypervisor - Věda a technologie
Here is an excellent post by Olivier Lambert who runs Vates, the company behind XCP-NG & Xen Orchestra covering some very important details regarding the history and architecture of the hypervisors
forums.lawrencesystems.com/t/xen-vs-xenserver-vs-kvm-vs-proxmox/14256/2
awrence Systems Forum Post
forums.lawrencesystems.com/t/xcp-ng-vs-proxmox-2022/14200
Lawrence Systems tutorials on XCP-NG
lawrence.technology/xcp-ng-and-xen-orchestra-tutorials/
Learn Linux TV Proxmox course
www.learnlinux.tv/proxmox-virtual-environment-full-course-class-1-getting-started/
Learn Linux TV Proxmox Backup
www.learnlinux.tv/proxmox-backup-server-full-getting-started-guide/
⏱ Timestamps ⏱
00:00 XCP-NG VS Proxmox 2022
02:30 Features Comparison
03:09 Explanining Xen Orchestra
08:15 Max Support VM Disk
11:30 Proxmox interface
13:57 XCP-NG Xen Orchstra Interface
I also have used both and both are very powerful. However, I have mostly used Proxmox and have used it for the last 13 year in large scale and it really is a solid enterprise solution, especially together with CEPH. I know every corner of it, it can’t surprise me with anything I can’t do or fix. I believe Lawrence have the same knowledge about XCP, there is nothing that will take him by surprise that he can't do or fix, and this is really the right reason to choose one or the other. Choose one and stick with it, be a god with it and it will never fail you.
How/Did do you solve the IaC? Terraform? Ansible? Other?
How to get cheap addon for whmcs with Proxmox？
I find Proxmox interface much easier: this is just to reiterate it is a matter of preferences between two very good products.
Proxmox is simpler interface however I can see why Lawrence likes Orchstra.
@@wayland7150 prox is dated on the interface but it’s ok it works. I still prefer xcp
Thanks for this. It is helpful to see the comparisons.
Nice video and a good try to do the VS thing but 2.5 mins Proxmox against the rest of the video ( 10+ mins ) isn't really a Versus video. I understand you are more at home with XCP and therefor can demonstrate more features. This would be the reverse with someone using Proxmox daily.
Some clarification though:
Backups do not need the Proxmox backup application. There is backup build-in and has options to include/exclude disks, so big disks do not have to be an issue.
I love the comparison, so thank you Tom, and this gives some really terrific aspects of XO and XCP-NG. I'd love to find out if XCP-NG plans to support LXC / LXD Containerization int he future. It's just a no brainer to want to use those for much of the VM work these days due to the far lower resource needs for a ton of the use cases out there.
Containers would seem to be out of scope for XCP-NG, which I think is trying to just be a solid, easily administered secure hypervisor. Proxmox is meant more as a home lab all in one server, so host OS containers make sense, but I expect that a "correct" implementation of LXC on XCP would be inside a guest OS.
Great video, I've been using clustered Proxmox but really like the XO interface you showed.
Love that you have Jays Proxmox video's linked.
Great episode with interesting looks into both systems.
Thank you for putting this comparison together.
I've been looking at the two platforms, and I think that one of the ways for me to experience it would just be to try it.
I think that it IS important to mention though, that Proxmox does not have an easy/default way to import OVA appliances (so if you are say, migrating from VirtualBox or some other virtualisation ecosystem -- I would mention that).
Also, I would mention, in your comparison chart, that XOSAN is a ***PAID*** feature.
Therefore; if someone wants to use hyperconverged features and Proxmox allows you to do it for free (even if the hyperconvergence feature set is limited with the free option), that might also be worthwhile to note for some of your audience members.
Proxmox does OVM? Oh!
@@wayland7150
I don't think that I can import OVAs into Proxmox easily/readily like I am supposed to be able to with XCP-ng/XenOrchestra.
But I guess that I'll find out!
We shall see.
Great video that answered quite a few questions about the two systems, with the VM ESXI drama going on right now, I bet this will start to get more views as people start looking for other options.
This is informational and really well done, Tom. I've worked with so many Hypervisers through the years I couldn't tell you my opinion of each of them; when it comes to Citrix (the heart of where Xen is) I always remember going back to Citrix terminal servers in the late 90s off of NT4.0 to enable dumb terminals at a hospital client.. I digress. Most of those I work with are VMWare, or, in some cases, Microsoft Hyper-V under a setup where the root server cannot be virtualized due to specific hardware passthroughs/key security issues.
In this sense, I've always preferred VMWare. Once VMWare 7 came about, it put a stop to a lot of the things I really liked about VMWare, and I've looked at other options. Their are a few things here; I am not opposed to cost; I have always preferred as a backup solution Acronis for varying reasons - most recently native endpoint web management and full security modes, but also easy export to review for test conversion (because if you aren't testing a backup on a schedule it's worthless)
This is where I have issues with XCP. I like the core concept of it; but the way that XO is structured really boils down to a "pay to play" $85/mo*12 year for features included in ProxMox, which has a newer, and more frequently updated Hyperviser in KVM, and Xen is absolutely showing it's aging when I look at the IOPs I see...
I think what makes XCP really solid is that XO connects to multisite in a way that really only Microsoft's manager compares to in node control for Hyper-V, in a different way; but if you're using Hyper-V it is is, again, often for transition purposes, specific hardware pass-throughs, or for hardware supports.
If you're a homelab, ProxMox might be more likely to provide you exactly what you are after; the interface is instantly there, many of the items that are removed from XO without $ are present (backup), and easy management capability works. XO's handling of XOSAN and backups is a great feature, but it is very important to mention they are -not- free or included by default, as people who walk through the default install will find they can't get there.
Your videos are really informational. Every hyper viser I've used has it's strengths and weaknesses.. It boils down to what is the end goal of the Hyperviser and what am I trying to accomplish
Brief add on: screw my auto-correct; I do know grammar well enough to recognize the correct version, and it should be "THERE"..
Firstly great video not less from you Tom. I am newbie to OpenSource applications. Been to long brain washed with those high end applications. A friend introduce me to Proxmox I have installed it on 3 server and clustered them as well. Yes was really easy install and setup. I am looking at migrating a number of VM from Hyper-V to Proxmox or XCP NG. I have not installed XCP NG but after this video I will be formating the Promox lab setup. My goal is provide easy management for my staff knowing who has done what but also key factor migrating from Hyper-V.
Very informative. Thank you Tom.
This was pretty nice, but I think it would be much more interesting to have a proper comparison of strengths and weaknesses between XCP NG and Proxmox with maybe some guest youtuber who knows Proxmox well. Would be even nicer if you could do some more advanced scenarios such as Ceph vs. XOSAN.
Looking at discussion forums after Broadcom's announcement to buy VMWare, sysadmins' choice to replace VMWare seem to be leaning more toward Proxmox while XCP NG isn't often even mentioned. Might be because Proxmox has been around much longer than XCP NG though.
Would love to see the performance differences between Ceph and XOSAN. XOSANv2 is coming out soon so rather wait on that.
I have used both and settled on Proxmox. I really didn't care for the XO interface, and the VM console is very limited. I want to be able to break out the console to its own window, and Proxmox can use SPICE which is vastly superior to what XCP-NG offers. Granted, I haven't used XCP-NG for over a year so maybe things have changed in that regard.
I rarely ever use the console except to setup a VM. Once configured I use either ssh or RDP.
@@LAWRENCESYSTEMS For sure. I thought I saw Olivier say they were working on SPICE integration. That might be enough for me to try XCP-NG again.
+1
The lack of SPICE support is an argument for not choosing XCP-ng.
I've used both also. XO was cripple-ware when I last used it (certain features were turned off unless you paid). Proxmox has ZFS, SPICE, it's 100% free, has replication, ZFS snapshots, HA, etc. Really no comparison. Proxmox is the way to go for sure.
with the container management I had asked during a live stream and they will be implementing RunX which should allow for similar container management on the hypervisor.
Excellent point about large storage volumes and VM's, although I am not sure it really matters. Worth a look, I think.
XCP NG looks beautiful, I'd love to delve into it if I had the need but for my small enterprise Proxmox + TrueNAS iSCSI is bulletproof. It's easy to train my staff to use it as its paradigm mirrors real hardware and snapshots and replication are painless...
I have a question for you regarding Proxmox and TrueNAS iSCSI setup. I ran into an issue with getting proxmox to connect my iSCSI storage on both hosts (2 nodes). Each node connects to the TrueNAS server via a direct cable connection and on a different ip that is on a different subnet, no switch as I have dual 10GB nics in each host (proxmox and TruNAS systems). What I found was I can add the storage but only input one of the two ip addresses the TrueNAS server is answering on. The fist proxmox host sees the storage but the second hos sees the storage as off line. I did some research into multipath configuration but that didn't work for me. Any thoughts?
@@bdunigan80 You can only have 1 default gateway on TrueNAS and it is usually for the management interface. All other NICs (for iSCSI/NFS) on TrueNAS for sharing are gateway-less as your servers that are consuming them have NICs on the same (dedicated) network segment. Maybe you just have a misconfigured NIC on either TrueNAS or servers that prevent you font seeing it?
If you are using two different subnets check that you have a portal on each subnet. Or get a 10Gb switch and keep both servers on the same subnet and portal. Good luck.
Certainly does look nice, for my homelab I'm probably steering to Proxmox, with iSCSI storage. However XCP-NG mmm...
The affero license is a non-starter. Anything that talks to proxmox becomes subject to it. Are you prepared to provide the source code for Windows?
Proxmox can somewhat do the tagging features with their pools. You add resources to a pool and you can have some actions target everything in the pool.
I use pools in ProxMox as well to help figure out which group of VMs are using TrueNAS storage so I can shut those VMs down before rebooting it due to updates.
I love both options but for my homelab I went with proxmox because it has just worked so easy.
I love your open. I use in my home lab libvirtd on openSUSE, Proxmox and VMWare ESXi, if I had another server I would tried some thing. I don't use Citrix Xen because there is a limit on the free versions. Great Video.
Good comparison at a glance as a good starting point, thank you. Slight typo in the "Shared Storage", for Proxmox you put "CefRBD" instead of CephRBD. Also "stroage" and "packacge".
Thanks, spelling is always a challenge for me, maybe I need to hire a proof reader.
great video!! thanks for the comparison. I think I will still keep my proxmox setup, due to the LXC containers I have. And I wont have so many VMs haha. Also, I was not able to find much information about GPU/PCI passthrough for xcpng, and I use the VMs for gaming. thanks again!
It is not supported through the UI but you can do pass through
@@LAWRENCESYSTEMS care to explain more?
I’m not rushing away from vSphere immediately but this was interesting. Thanks Tom.
I'd like to see a vSphere vs Prpxymox comparison, it seem that proxmox has a reasonable attempt at the feel of vSphere. Mind you it has been two years since so perhaps I'm mistakened.
An awesome teacher. This channel is a must watch for me and has long been in my toolbox.
Thank you Tom
Thanks for the video, lots of great info. That guy Tom that keeps showing up in the logs seems suspicious though.
He's the biggest risk to having a stable system.
Thanks for this comparison. A few typos in your matrix: 'suported', 'stoage', 'packacge'. I'm waiting on a Beelink GTR5 to spin up a virtualization lab and looked at XCP-NG but I don't like it as compared to Proxmox. I'll be running quite a few Linux VMs so the LXC container support in Proxmos will really come in handy.
I run a Storj node as a VM in one of my servers. The virtual disk attached to it is 27TB right now (I expand it as it gets filled up). I also used over 2TB virtual disks when I was mining chia and I have a 3TB virtual disk on a VM that is backing up my other physical machines.
The 2TB limit is quite small to be honest, I know some VMs our clients have that have more than 2TB virtual disks.
Of course, with a limited virtual disk size it would still be possible to just attach multiple virtual disks to the VM and then combine them using zfs or software RAID inside the VM, but that feels like more complicated setup for no real gain.
I like backing up using zfs snapshots (on the host) and doing "incremental forever" with only one "full" backup.
Does XCP-NG have any integration or offload for storage like VAAI?
Tom is right. It depends on use case and your familiarity to the particular tool you intend to use.
Once you master any of the tool, you can invent the work around its shortcomings.
proxmox ceph storage is far better than xosan (very expensive) , but at the end overt is also interesting...and good.....
cannot agree more
I use some (older) Infiniband hardware for my storage backbone and having the OS not hinder my ability to install what I need to get it working is kind of a big deal.
For me the "custom", and rather limited, when it comes to hardware support, CentOS image was a big strike against XCP-NG.
Similar issue with TrueNAS Scale where they actively block the user from installing additional packages.
Doesn't necessarily matter for professional use-case, but it does for homelabbing.
Yeah I also had some older motherboards where xcp-NG would just panic and crash, the same motherboards work fine on Windows or with a modern Linux system (or proxmox/Ovirt)
Its kinda silly that people think just because you dont use something its bad. The main reason apart from the second is i like the XO interface look and if im spinning up stuff all the time i at least want the ui to look sleek but on a different note for me XCP-ng turned out to fit my use case because of the meany to one relationship my lab scales up and down systems get added and tour down so having the signal pain of glass setup were i can just add servers and remove live or shutdown migration of VMs is really powerful
@Lawrence Systems Kindly advise me how I can compile xen orchestra to have the version of the Premium so I can play around with all that features. Thank you in advance.
I tried both xcp-ng + XO and Proxmox and decided to go with the latter one for my homelab. It feels more solid for me personally and I didn't have any troubles compared to xcp-ng + XO. For example, XO doesn't allow you to create more than ~6 or 7 interfaces, but it's possible to create them manually through the xcp-ng console. Also, as a person who is new to virtualization-I'm a student-I find Proxmox's web UI a bit more intuitive. Both tools are good and powerful, so each person should try to play with each of them and decide which one to use.
I use LXC containers a lot. Basically all my services run on Debian CTs. It works really well in Proxmox.
Agreed. I know that this isn't a priority for xcp-ng and that there's a myriad of alternative options you can pursue, but there's something just _nice_ about having your VMs and CTs 'colocated' in the same workspace/GUI. Currently using xcp-ng but considering moving back to Proxmox for this reason (wasted memory by having docker running in VMs)
@@transatlant1c Yes. It is quite nice. The only thing Proxmox doesn't have is native docker containers. But that is mostly because it is supposed to run LXC CTs. It is possible to run Docker cts inside a VM or even a n LXC CT, but it is not trivial, it will have another virtualization layer and it can be somewhat buggy. I try to avoid docker into proxmox.
@@gustavinus yep that’s right, when I was going down that path I found setting the CT as unrestricted and enabling nesting, along with a turnkey Linux image was successful but results may vary!
@@transatlant1c yep... with those settings it runs pretty much natively on the Proxmox kernel. I tend to use Debian for those CTs because Proxmox is also base on Debian.
Can Zen Orchestra be hosted on the same hardware as the XCP-NG hypervisor? That is have a VM that runs that stack, or even install the package within the Debian environment it is built on? I have a small home lab with only one node that does all the things.
Yes, you install it as a VM on the main host
What do you use for UPS graceful shutdown on XCP-NG? USB passthrough to a VM, or just install nut on the host?
Proxmox is what I've using. I haven't tried XP-NG or XO before. I will delve into it in the future.
When you do things at scale, you do infrastructure as code ;-) For that reason, oVirt's terraform provider is pretty compelling, integrated with CI/CD of sorts like drone, pxe/matchbox and ansible (rundeck), you then get ability to manage infrastructure of basically any scale, and that's on-prem, in the cloud and in hybrid environment - simple and great for lab or bigger production project.
Yes, that is why they added Terraform support to XCP-NG registry.terraform.io/providers/terra-farm/xenorchestra/latest/docs
Excellent evidence-based presentation. Quality tech advice does not get much better than this.
Thank you
If I follow the guide & compile a full copy of Xen Orchestra myself.. will updates via web console work or will I have to compile another version each time?
Hey Tom, what are your thoughts on Harvester? It's a super simple ha cloud native hyper converged cluster. Been using instead of proxmox and xcp and it's nice!
Have not used it and not on my list to test
Nice overview, but what about GPU pass through for example ? We use that a lot in our work vms and proxmox has been able to do it (albiet not flawlessly)
They can be done but not through the UI
For small team and self managed VM server, I feel like proxmox has more online videos and tutorials to learn from, XCP-NG seems to be much less. Thus, I consider proxmox has a easier learning curve. Not sure if my impression is right.
Two factors at play. Learning curve and system capabilities. Obviously you want an easy learning curve to get started but if the thing does not have the features you need then it gets increasingly difficult as you implement workarounds..
The most important question is whether understanding Xen can be an asset on a professional level. I'm not worried about Proxmox because KVM is the most widely used hypervisor on the market along with Hyper-V and ESXI but Xen, there aren't many job postings about it.
That might be the most important question to you. It is not to me when choosing a platform.
Just started to play with xcp-ng / xo because of your videos. What I haven't found yet is a counterpart to the physical disk view of the proxmox server. Is this missing? Where do you see in what state your hdds and ssds are?
You can see the storage status under "Home -> Storage" but they don't show the individual disk status via XO.
@@LAWRENCESYSTEMS but how do you monitor (upcoming) hardware failure? Via iLO or similar? Sorry for asking this, but I'm a bit used to Proxmox and ZFS...
Via iDrac and via syslog
Afaik you can also use tags in Proxmox, but I think they work a little different
Great video on a topic that is now pretty much at a junction of which way to move off ESXi. One question I have is - If I were to run XCP-NG free edition with no support for say my development or "not so important" workloads then could I still manage these nodes with a paid version of XOA?
Yes
Just a bit of clarification that was not evidently clearbinnthe video. Hyperconverged storage using XOSAN is not available with Xen Orchestra from sources.
It would be nice if you have the chance to share how do you migrate the machines from esxi, for example I'm able to export to ova/ovf but when I Import them on XPC it just says importing then says nothing and do nothing, I tested very small vm's with no luck.
Best regards
Use clonezilla or your back up software of choice
I'm using Proxmox since 2 weeks installed it on i7 875K (2010 plarform) with 16Gb RAM and I love it. I run Openmediavault RAID5 as VM (HDD passthrough) and 2 minecraft servers - one in VM docker on Debian Stable second in LXC archlinux container. Perfomance is stability are great. I had problem tho with True nas and ZFS RAIDZ but I gues it was too demanding for that PC to handle.
IaC - It became standard approach with clouds. Why it's so neglected feature among hypervisors and their comparisons?
I tried both, but I like proxmox more. The Webinterface is IMO far more intuitive, it runs perfeclty stable, most people already know kvm/qemu and I don't need a VM just to have a webinterface.
But that doesn't mean XCP NG is bad.
Also, at least for Samba and NFS the used storage graph works fine
Is it possible to use both proxmox and virt-manager to manage the same qemu/kvm-VMs or is it asking for trouble or simply won't work?
And for containers, could I still use the console lxc tool along side with proxmox? (for those same containers, that is)
If you are running Proxmox use their UI for both, not sure what wold happen if you added more management to it.
proxmox and virt-manager both use the same qemu instance but store their files in different locations. You'd have to test by moving the files from one under the others vm config storage location.
Who says you hate Proxmox ?
Dude i've found out about Proxmox 5.X from you and i've been watching your channel when you switched from Proxmox to xcp-ng.
I love your videos about Proxmox, XCP-NG, pfSense, TrueNAS. (not so much Synology and Unifi)
Does adding nodes to the xen orchestra needs any kind of quorum so you need to have a specific number of nodes and above?
Does afterwards deleting any of these or shutting down any of those nodes floods you with messages?
If you already have VMs installed on the node can it be added?
Adding nodes requires them to have no VM or extra network settings.
@@LAWRENCESYSTEMS ok what if they hadn t, you joined them created the appropriate configuration and afterwards you want to remove it? Does that require many steps? Do you need to update .cong files outside the gui and stuff?
settled on Proxmox, KVM is easy to set up when testing things like beta versions of MacOS or Windows, VMs are basically set and forget :)
Proxmox is deffo my favorite.
Tom, have to go back to commuting. is there a way to convert the CZcams video to MP3, so i can listen on the train? (Preferably a free converter)
I’m using CZcams app on iPhone and there’s a Download button, but haven’t used/tried it.
@@rreveler6705 thanks for the reply. i have an old "Sansa" and you need to load it with the MP3 files. i have tried my Samsung phone but only screwed up the phone after a few times. i think i should stick with the "sansa".thanks.
What backup tools do people use with these open source solutions, or does it require running agents on all the virtual machines, vs using something like Veeam backing up or Replicating the clients on the Hypervisors directly,
With XCP-NG I use the backup system that comes built into Xen Orchestra.
Would oyu consider doing a deeper dive into XOSAN?
Eventually
4:22 Get all features by downloading the source code. (Just has No Support added to the interface).
Even more interesting is the new kids on the block. OpenShift/OKD Virtualization and Harvester. Leveraging Kubernetes and kube-virt. Red Hat is stopping support for RHV/oVirt (which XCP reminds me of) in favor of OpenShift Virtualization.
Stopping? No. they are not stopping anything. Oracle/Rocky,Alma will support for many years ahead oVirt.
@@levskilevov4888 I didn't say that oVirt itself was stopping. Just that one of their biggest contributors is stopping support for them. That already small community is about to get even smaller.
Can you cover how to backup or protect Xen Orchestra (HA diff sites) or just backup restore process.
I have all the tutorials for that and more here lawrence.technology/xcp-ng-and-xen-orchestra-tutorials/
I do have an old Dell server that is incompatible with VMware so it is running on Proxmox but I will try XCP-NG to see if it is compatible and probably move the 4 servers I have to XCP-NG if it is compatible as you can have all servers at one place. Right now VMware you have to open the GUI to be accessible with VCenter. I seen both videos of Proxmox and VMware comparison with XCP-NG and the Proxi looks promising. I do really love to see each videos you do but I do really think I need to look foward for a move to have all my servers to be accessible at the same place. The big test as I said will be my old Dell R320 that is old but still working good for some applications. I did moved him from KVM to Proxmox as KVM was a pain in the ass for windows VM still if the VM vas Light. Did any one tryed to give a GPU dedicated directly to a VM in XCP-NG ? The best way for me was VMware with modification in configuration. Thanks for all the infos!!!
I love proxmox but I run into a challenge performing differential/incremental backups on proxmox. It performed full backups each time. If I recall correctly this affected windows VMs most. However on XCP-NG the backups were flawless. Both differential and full. Has anyone experienced such an issue. I am strongly considering reverting to XCP-NG because of this
-- I've tried both, and while Proxmox looks nice for a one-server student lab, it doesn't scale nearly as well as Xen/XCP-ng/XO. For large-scale deployments, the Xen/ng/XO stack is fairly reliable and the Xen Hypervisor is a lot more secure and quicker
To me, the real issue is Xen vs Qemu/KVM. Xen is pretty old and Qemu/KVM is clearly setting up to be the dominant Linux hypervisor of the future. Even AWS switched over to Qemu/KVM from Xen...
I do like a lot of the feature-set of Xcp-Ng and even Xen Orchestra, but if one is choosing to standardize around Qemu/KVM, then unfortunately both are really non starters. 😔
The majority of AWS is running on Xenserver, it's part of the Linux Foundation, and it's still very actively being developed.
@@LAWRENCESYSTEMS All new AWS instance types released from ~2017 forwards (ie anything on AWS' Nitro system) are using KVM, including all Graviton products. So... Unless you're sure the majority of currently running AWS' instances are using older instance types, that may not be a factually correct statement. 😉
Not trying to give you a hard time... But enterprise-scale cloud DevOps is my day to day, so it's an area of the market I track quite closely.
Xen will no doubt be around for quite some time; speaking of AWS they even built a compatibility layer for it, so older Xen-only OS deployments don't break with KVM-based instances. But if you look at market share trajectory over time, feature velocity, and enterprise backing, it's clear KVM will eventually become the standard for HV in the Linux space.
@@shammyh Linode is also using Qemu/KVM in their infrastructure. I've created a few test Linux VMs on their platform and took me a few mins to figure out what their host systems are running which are in fact using Qemu/KVM. Wouldn't be surprised if they are using ProxMox to manage the VMs and Host as it can be fully scripted to provision the VMs.
Hello, thank you for your video. It's possibile use multiple computers with xcp-ng, and make them looks like one in a virtuale machine with (eg.) Windows Server? Because, my intection is to create a super computer for rendering activity. Are xcp-ng & Xen Orchestra a valid solution, for my purpose?
No, you should look into applications that support distributed computing to get something like that to work.
@@LAWRENCESYSTEMS Thank you for your answer, I have immagined that wasn't appropriate, I will try with DragonflyBSD.
One imporant fact: XCP-NG is based on Xen vs. Proxmox on KVM. The former is gradually losing the battle against the latter in the commercial virtualization market. This may be not quite noticeable to a home lab user but personally I would be more inclined to stand for the future.
Doesn't Citrix use Xen?
@@EarthStarzYes. XCP-ng was originally a fork of Citrix XenServer, which is now just called XenServer.
Thanks Tom.
@Lawrence Systems I seem to have a passion for virtualization and automation. Can you please guide me how to install setup xen and xcpng noting that I am a complete noob and have worked with vmware esxi single implementation the free one. (After that my dell t7600 crashed stuck in reboot loop lol)
We have tutorials here lawrence.technology/xcp-ng-and-xen-orchestra-tutorials/
I've been running Proxmox for about 5 years now, and for my homelab its been great. I would however love a solid solution that's FreeBSD and Byhve based.
But why? The last time I used Bhyve it was REALLY bad.
@@tobimai4843 I like FreeBSD. I find it more a real server OS. I found no performance issues when using Bhyve in FreeNAS/TrueNAS.
The `tags` option is nice to have, on Proxmox also have it but only from console. no LXC support also is a disappointed
Many thanks for the excellent video. I have to confess I'm a Proxmox fan and love that system, but I was very impressed with XCP-NG in your video. It has a very nice web UI, unlike vSphere which looks nice but is buggy, and seems more capable at a very large scale.
As I read, both have issues with iSCSI multipath. That's reason why I would probably move to VMware, even it is most expensive. In small business, I need a solution which is easy to administer and maintain.
I have never tried iSCSI multipath in Proxmox, but it works fine in XCP-NG
Bonjour, je me pose la question quel est le mieux, est-ce VMware Workstation Pro ou Proxmox. Le but est d'utiliser aussi la carte graphique et de faire facilement un copier/coller, pour installer des jeux ou programmes.
Both VMWare and Proxmox support GPU passthrough, so you shouldn't have any issues with either platform. For disk management, it depends on what you want to achieve. If you want to allow multiple people to play the same games simultaneously, you might be butter off with a multiseat solution such as Windows MultiPoint Server (sadly defunct as of 2012), MultiPoint Services for Windows Server 2015 or later (sadly also defunct as of 2018), or Remote Desktop Services for Windows Server 2016 or later.
If you want to run Linux instead of Windows, search for apalrd's video on setting up a Linux terminal server, and/or look at the Linux Terminal Server Project.
Does XPC NG or Proxmox support NVMeof or NVMe-IP? I know with VMWare and Pure Storage you can enable NVMe over fiber to improve IO speeds. Also I saw some forum conversations and blogs on XCP-ng which talked about being able to use LVM command line to extend a volume beyond the 2TB limit, was wondering what the pros/cons of that are.
NVMe-IP is not something I have tested.
What's the better backup solution IE delta snap etc for vms, proxmox or XO?
I prefer delta via XO.
I am a fan of XCP-NG but it does have a gap in enterprise environments, or even a home lab. Try to load/import enterprise VMs such as Palo Alto firewalls, Pulse Secure and a many others. They will not work. These will run fine on VMware and Proxmox. Palo Alto does support Xen ( have virtuals deployed in AWS on their customized Xen ) but the Palo Alto Xen image can't be imported into XCP. So, if you only need Windows/Linux/BSD VM support, this product works really well. For other enterprise solutions, you may have to stick with VMware/KVM/HyperV.
Personally I cannot get over the ackward XOA web gui interface.. It just is very unintuitive (to me). XOA-center isn't really an option since it only has limited features. Proxmox has a lot of problems in that area too. However I temporarily switched back to proxmox because XOA kept throwing errors at me which made no sense.... basically the log kept filling up with erors. It seemed every 1 out of 5 actions I tried to perform, gave me errors.. I've been around IT systems for 40 years and they don't make sense.... anyways... Of course all of this is very subjective to personal taste.
One thing that bothers me about xen is that the opensource guest tools are developed/maintained by 1 overburdoned guy and the offical ones are being a login wall now.. (new ones anyway). Without those guest tools you might as well go back to T2 virtualization..
Funny because I find xen orchestra to have the superior ui compared to proxmox but to each their own. Also Having only 1 guy responsible for the open source guest tool is alarming though.
@@bthegawd8113 It is really a user preference. I myself actually find Proxmox UI more odd ball when comparing with Xen.
@@TanKianW79 yeah the proxmox ui looks like a first year college web design project lol.
Yeah... I am now running XCP-NG inside a proxmox VM and I have to say... It may also be just a bit of knowing how to do things the right way.. Some of the gripes I had with it are probably mostly due to my own inept skills with it... Keep on learning :) Thanks to Tom's videos I've learned more about virtualization and firewalls than in the 20 years before that :) btw @tom ... I've just been watching some of your 2016 on-site videos and they are amazing... Thorougly impressed with what you did there and are still doing..
I actually agree about the XOA interface.
For my homelab I still regularly use XCP-NG center (the former Xencenter app) on Windows.
Ok, I hate you 😜 On VMware and now Azure we have big machines, some disks around 16TB, some machines up to 36TB, and we back them up ok - although with huge disks you really must use CBT (change block tracking). But yes, best to avoid hugh disks/VMs as restoring them could be challenging. We did use VMware SRM, and now Azure ASR, so possible to failover to another replicated copy, in another location, quickly, within your RPO and RTO. Just installed Proxmox, great video as always, I will have a look at XCP.
How much of the functionality you showed from the xen side is xcpng vs xen orchestra?
Without XO you would me managing XCP-NG from the command line.
@@LAWRENCESYSTEMS so all of the automation, HA, and tagging are things xo is doing to the xen server running on xcpng rather than being things baked into xcpng that xo just pokes at like an API, correct?
Yes, most things are done by XO poking the AI
wow, i really like the interface compared to proxmox, especially the tags. But also something simple as ip adresses, which i really mis in proxmox
As far as I'm concern Proxmox is now more stable then few years ago. KVM seems to be faster then XEN too. But 3-2-1 backup with it's backup solution is atrociously inflexible. Orchestra is not perfect either as it not support compressed incremental backups, but still is far more intuitive and simple in that regard.
I still find disk image limitation in XCP-ng a problem, as unfortunately I do have some images that big.
Just use proxmox backup server for backups. Works good, incremental, retention shcedules, etc
@@jgould30 No, no, no... Proxmox backup is more of a storage solution. It enables server do all that staff. It cannot create disk image out of backup, cannot extract files form backup and basically is useless without a Server itself. That a problem if you keep backup on remote location with network as bottleneck.
Moreover if I lose my online infrastructure due to unforeseen circumstances I need to rebuild it as it was. I know that would not be possible in short time. Meanwhile offline off-premise backup will be useless to me in that period. Very inflexible backup solution when it comes to 3-2-1.
@@wielkiptok For your use-case i would recommend to install both PVE and PBS on your off-premise server. It will give you all features you mentioned and an ability to run some important VMs off-site during disaster recovery.
@@mvs3246 When server is on the same site as backup, then backup is no longer offsite. But I get what you mean. I just do not have enough redundant infrastructure offsite for that to work (and money is the factor).
In that kind situation I prefer that backup solution would operate on disk images not unknown chunks, that all. It makes stuff lot easier and would let me panic in more organized fashion.
A big take away here is managing 10k instances of plan 9 or linux then go with XCP NG. Managing K8s cluster, XCP-NG with extra steps. Typical /r/homelab or /r/selfhosted user then use proxmox All else use openstack or a cloud provider.
Do wonder how this stacks up to myb (bhyve).
2TB disk size is a deal killer. Backup solutions are virtualized now a days... and say I have a Veeam backup server backing up several machines. 2TB gets burned up really quick. Even if you're on shared storage such as a SAN, 2TB limit is horribly low.
I got the impression Lawrence was saying use a backup server rather than a 2TB attached hard drive. The server can be whatever capacity you need.
@@wayland7150 Veeam is the backup server. VMware supports up to 62TB per disk.
Say you wanted to backup a 2TB file server, which contains large engineering files. Your backup server is Veeam as VM. With this statement, Veeam wouldn’t be able to backup that 2TB file server.
does Xen Orchestra and xcp-ng have a max number of hosts for the free version?
Nope, no limitations on that.
Stupid question -- so if Xen Orchestra runs on its own, separate, VM to manage everything -- are you able to "clone" that management VM so that the two can work together as a high availability pair, especially if the "cloned" VM runs on another physical server?
I'm just thinking of the question: what if the management server dies?
What happens to the management of the VMs?
Can you make a HA pair in Proxmox? (so that the clone can manage the original set of VMs?)
Or if the management host dies, does all of the VMs that are controlled by that manager, dies along with it?
You can use a continuous replication job or install XO on shared storage to mitigate the host failure issues.
I think the management VM works the same way as vmware's vcenter meaning the host servers keep the VMs running without issues. Just they won't be able to move between hosts since that is handled by the management VM.
@@LAWRENCESYSTEMS
But that would mean that if the management VM dies or gets corrupted; to bring the management VM back up online (which, presumably also means that if a VM hasn't started already, then you can't start any VM that's isn't already running until Xen Orchestra is back up and running, or you will have to start it manually via CLI or something along those lines), right?
i.e. even with continuous replication, that's not the same as a high availability/automatic failover in case your Xen Orchestra VM fails/gets corrupted/dies, correct?
Like, if the Xen Orchestra VM gets corrupted, how would you roll back to a previous replication/snapshot before it got corrupted?
I assume that that wouldn't be a trivial task, correct?
@@Darkk6969
"I think the management VM works the same way as vmware's vcenter meaning the host servers keep the VMs running without issues. "
Right, but that's my question: what happens when vcenter dies and/or gets corrupted?
What happens to the VM?
Are you able to have a pair of systems/baremetal servers work together in unison to form a high availability cluster for vcenter such that if one of the two servers goes down/goes offline, the other can still manage all of your VMs?
Or is it that once vcenter goes down, so too, does all of your VMs go down with that ship as well?
You can set XO to be HA and it will automatically restart on the next host, the HA is handled by the pool members, no XO.
In xen-orchestra from source you will have not all features. Example is Proxy. It is not available in XO from source. XOSAN not available in XO from source
I've had problems with building ProxMox VMs on iSCSI. The drive no longer works after a reboot.
Not sure what you are doing wrong for that to happen, try posting in their forums.
Did you considered oVirt as an alternative?
Nope, it does not have the same number of features of XEN.
Prox Orchstra ?
Is it just me, or does proxmox seem much easier to manage? I just tried splitting my local hard disk with xcp-ng and tried managing it with xen orchestra, but I wasn't able to figure it out.
Yes, that might be you. Check with the man pages or ask away in the forums. You'll find it 😊
I’m getting off ESXI for my home environment and your videos lately have been leaning me more towards XCP, plus it’s gorgeous
Neither is great with terraform, first I chose xen because it seemed more fleshed out. Having tested both the proxmox provider have worked much better in reality. There are definitely very frustrating issues still however.
Does XCP support gpu or pcie pass through
Not via the UI
Used XCP-NG for the past 2 years. Lost entire environment when some kind of file corruption was introduced and it would NOT let me export VM's. Wiped entire environment, started over. Was experimenting with backing stuff up, tried to export VM's so I could do extensive testing. Exporting failed so early on it didn't even create the config files for the VM's. 100% unable to export VM's again. Wiped environment *again* and went bare-metal. I can't keep having to wipe my whole environment because XCP-NG was failing with 2 different fresh builds to export VM's. Any VM. Windows, Linux, didn't matter. No errors thrown in GUI.
Not an issue I have encountered
@@LAWRENCESYSTEMS I can understand the file corruption - but not having file corruption and being unable to export VM's was a show-stopper for me. I wasn't too crazy about Proxmox, and, seeing as how all of my hosting was in docker containers, I decided to go bare-metal. I had to give up my PXE boot server though - I'm not too comfortable yet with building that in a docker container.
@@hiddenfromyourview Have also experienced corruption on XCP where we copied a VM to a different pool and on trying to start it, it wouldn't come up. The most frustrating thing was that even the original VM wouldn't come up. It had a disk size of 1TB and on checking on the storage itself the parent vhd file was merely 3.2MBs