4-Gamer 1-CPU PC for $1000?!?
Vložit
- čas přidán 14. 03. 2022
- Thanks to Linode for sponsoring this video. Visit linode.com/CraftComputing to get a $100 60-Day credit when signing up for a new account.
I get it... GPUs are unobtainable and expensive right now. But there are Enterprise GPU Accelerators you can get on the cheap, and virtualization allows you to split that power between multiple users at the same time. Today, we're going to build a Proxmox server with two nVidia Tesla M40 GPUs to run Four Gaming PCs inside a single box.
But first... What am I drinking???
Lucy you, it's a Two-fer... New Image Brewing come the Hella Phantastic NEIPA + Ex Novo Brewing Fanciful Name West Coast IPA.
Links to items below may be affiliate links for which I may be compensated
Check out the parts from today's build (cheaper options included)
Intel Xeon E5-2690 v3 12-Core: ebay.us/ewszQG
Jingsha/Bewinner x99 ATX Motherboard: amzn.to/36lDlKm
OLOy DDR4 32GB 3600MHz: amzn.to/3tdfhlS
16GB DDR4 ECC-REG (Buy 4x): ebay.us/lBdn7x
Tesla M40 12GB: ebay.us/2OD6VL or ebay.us/40kdCU
Tesla M40 24GB: ebay.us/FW9GWQ
be quiet! Pure Base 600: amzn.to/34K6JJO
be quiet! Shadow Rock 3: amzn.to/3I99wde
be quiet! Silent Wings 3 140mm: amzn.to/3isPosl
Fractal Design Ion Gold 850W: amzn.to/3q514FG
Silicon Power 1TB NVMe: amzn.to/3IdVIOH
Delta BFB1012EH Blower Fan: amzn.to/3w4XRKa
nVidia Tesla Cooling Shroud: www.thingiverse.com/thing:474...
My previous Tesla M40 Benchmark video: • $220 for Titan X Perfo...
vgpu_unlock Github: github.com/DualCoder/vgpu_unlock
vgpu_unlock-RS Github: github.com/mbilker/vgpu_unloc...
Both repos needed for this process. Tutorial coming shortly!
Follow me on Twitter @CraftComputing
Support me on Patreon or Floatplane and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
www.floatplane.com/channel/Cr...
Music:
On Hold For You by Kevin MacLeod
Link: incompetech.filmmusic.io/song...
License: filmmusic.io/standard-license - Věda a technologie
Cool, now I just need 3 more friends!
Hi fran
Hi brodi
I was thinking the same thing 🤣🤣🤣
On the upside you got a gaming PC for 1000
I'd be happy with 1
this setup for two gamers at once would be sick. just do 32GB of ram to save some cash and it is solid. I can imagine a techy dad building something like this for their two kids or something to play games.
could run the VMs from a pair of Pi400s and secondhand monitors or cheap tvs. keep the budget real low and still have a pretty solid gaming experience.
@@catalystguitarguy maybe use the mandatory school chromebooks depending on where you live.
You could also probably realistically use phones and plug them in a monitor.
Dad here, I want to do that (like a thin client), how do I connect to the VMs?
@@aliasname5518 look into parsec, it was designed for remote gaming, and should work well for a setup like this.
Keep in mind that if you do that, you still want four sticks of Ram. Losing quad channel on that set up would be a huge problem
Absolutely fantastic build, Jeff. Your videos are always excellent and informative. Keep up the awesome work.
Fantastic camera work. Oh and it doesn’t matter how you apply thermal paste in a video, it’s always wrong. 😂
The kraken g12 aio bracket is a great cooling solution for these cards, I left the backplate with the thermal pads on for my m40 and didn't even have to use heatsinks on the vrm. Runs super cool with no problems
Best solution. I can’t take noise anymore. Good to see someone already tried a quieter method :)
Nice video....the format of live assembly is very interesting!
It works! Great to see this setup working well. Awesome for a cave setup for small lan parties
Well done with an amazing Build. it makes me drool and at same time gives me tears
Very cool project And i approve of the custom shroud for the 2 cards. With a high static pressure 140mm fan you could be good if its sealed up good.
Fantastic video Jeff!
Great build! I'd use something like this to set gaming machines up for my nephews to run from their chromebooks since we don't have a lot of room for full desktops in our place at the moment.
Need a dedicated section for craft beer! Thank you for the review!🍻
I hate broccoli. You said to leave our hate in the comments. Love the video, been wondering how well similar combo would work (add some cheap laptops and android boxes for remote). Would really love it if it wasn't so darn loud.
Jeff, Great video, to get the M40's to be quieter, I would suggest trying to use a fan controller with thermal probes such as NZXT's....I use one on my M40 12GB and it's very Vega 56 like versus being loud 24/7
Would you happen to have a little write up on this?
Vega 56? What?? It's way more powerful than the Vega 56.
@@Austin-cx2xe They likely meant sound-wise
This setup is sick Jeff!
Solid VM content. Excited about the tutorial. I wish the gaming giants were not so picky about emulation, would love to see a few of these used for retro consoles. The 960-1050 level GPU is perfect match even for some 1080p Gamecube/Wii and even some WiiU
The thing is you can use even lower end GPUs for gamecube and wii emulation. My htpc has a 750ti and its more than capable of 1080p emulation
you know it's gonna be a good video when there's a chinese xeon board & tesla cards in the thumbnail
Thanks for the video, Goodnight!
Amazing - just amazing. Looking forward to see some Radeon/ryzen variation. Subscribed
Great video, love all the Bequiet fans, and immediately overpowered by blowymatron.
Also 2nd shirt is fantastic.
This sounds like quite a good beast for a games room, I'd be keen to play some multiplayer games on this and see how it goes.
Yay, the first video with a cameraman! There is some echo though in that part so you might want to ask your little helper to add some sound dampening panels
Nice setup; you can try to mount the 120mm fans where the PSU sits to circulate the air in the whole case. since hot air is normally pushed out via the rear part of the case; I believe that should help
I like your thermal paste application. I'm gonna follow suit ☮️❤️
would like to see this in a server case to rack mount. I would also install SSD Drive for each VM, and passthrough disks to the VM. Want to build something like to but in NZ.
Someone needs a height adjustable desk. My back hurts just by watching this video. Nice video, god job.
Great stuff, Jeff, thank you!
Are the other (gamers) computers connecting with ethernet or wirelessly? Are you running Win 10 VM or proxmox?
Not sure how to spell it but Rett is doing a fantastic job if I do say so myself.
And Jeff your killin it as always bud.
Keep these gems coming!
Awesome haircut 👍
🤜🤛
The boss comeback!
Awesome video, one question though, after the split do you still need the Nvidia License Server and vGrid CC License?
there's another way to cool those cards , if u disassemble the shroud you can remove the clear part and get a expansion slot cooler it makes it alot quieter but takes up a extra 1-1.5 slots, its what im doing rn
I did a very similar build, but for a dedicated AI Rig, and you were about the only person using P40's, so thanks for the videos. They really helped! My rig ran only about ~$810 all in and I just repurposed my old T5500 (was about $90 on eBay at the time barebone with poor CPU & RAM), 192GB RAM ($20), Xeon X5690 ($10), 80 Plus 800W PSU ($0, included), 2x Nvidia P40's ($390) + Nvidia 1030 ($20) for Graphics output, 1TB SSD (hardware doesn't support NVME boot drive $35), 18TB HDD ($220), and adapters, fans, misc. ($25).
The tower is interesting as it supports up to 4 GPU's if you can squeeze them in. Also, it supports adding a riser card for another physical CPU + 6 DIMM's of RAM. I opted not to do this, and just use the extra CPU 4-pin for the second P40 GPU (power).
Another options would be the T5810 that supports the E5-2600 series CPU's (at about 2x performance) and supports up to 256GB of DD4 Memory (but based on my quick check that would cost $1/GB+, which is considerably more than the dirt cheap ebay bundle I got).
And I designed and 3D printed a dual Shroud and used one 92mm x 38mm fan to cool both and I have been impressed by the performance for the cost, and the sound. I used some rubber and spacing for noise dampening on it, and for how much air it moves it's amazing, and was only ~$16 on Amazon.
While the CPU is outdated, for my purposes, it honestly seems to run like a champ.
I'm interested in doing something similar. We're you and you aggregate the P40s and the 1080 on the same machine and use all the vram for ML? Can you give a bit more info to explain how you did it because here he is splitting the cards into 4vm, so I'm not sure the difference. Thanks
@@RandomShart Yes, I was able to get this working with all VRAM but I modified the low profile 1030 and removed the fan (because it was a tight fit, sandwiched in-between the two P40s, and I wanted the airflow to have somewhere to go, and not just into the other card). As for using all the VRAM, I think it depends more on the software you are using. I've used LM Studio, Stable Diffusion and a few others without problems (other than your normal debug and setup stuff).
@@michealkinney6205 Thanks for the reply, and which driver are you running that covers the P40s and the 1030 together?
@@RandomShart I'm not sure how to answer, because it'll likely be specific to your system. I'm running on Ubuntu Mate myself. The 1030 is just a generic display and I believe the driver I'm using is the the Data Center / Tesla Driver For Linux 64-bit (latest) with CUDA toolkit/bindings. If on Windows I wouldn't be able to tell you but I'd probably recommend getting and extracting the drivers only (without all of the other NVIDIA stuff) and pointing your device to it (in Device Manager to install the specific drivers).
What ram was 20 buck for 192GB? DDR2?
Nice server Jeff, yeah like it. If it were mine I'd use it for folding at home on its down time.
Nice project. I would have taken a different motherboard. Like the Huananzhi X99-F8. You have a extra PCI X16 slots that runs 4x, but you can install a 10Gbit card which improves latency. Also it supports quad channel memory, I'm not sure if this Jinghsha does, it looks like dual channel memory.
You can also a variation with 1 Tesla GPU and use the 2nd slot for a PCIe to 4x M2 adapter. Then you can run 2 VM's with their own SSD's.
What I would do for cooling is get some acrylic or polystyrene sheet and pipe air from two fans in the front (something like a nidec beta 5 TA450 which can have great static pressure but also can run silent may be my choice)
I haven’t looked at my M40 12GB for a while, my feeling was that the memory profiles were the problem. I don’t think I’d seen the newer script or hadn’t appreciated that it had more granular control if I had. This is going to be the ideal end of month project.
Yes, the issue with the M40 in vGPU has always been the 8GB memory cap for Maxwell cards. You couldn't allocate more than 8GB before. VGPU-Unlock-RS allows you to customize the profiles, and use all available memory.
For the fan mods on the Tesla cards, do a push/pull. one in front of the case as you mentioned, and another outside the case at the back, as there are no vidoe ports.
i went super aggressive . took the aluminum fins and separated each individual apart slightly then ziptied two 120mm fans to the front . temps are great
Bbbeeeaaaauuuutttiiiifffffuuuuulllllll!
Disclaimer: I may be biased.
great video! Wonder if it's possible to pass the GPU through windows the same way laptops do, with all of it going through the iGPU.
Yes, you can do this with the Tesla M40 as well. In fact, that's what my previous video on the card focused on: czcams.com/video/TQ9B67T5qWs/video.html
Love the gpu virtualization videos. But isn’t playing via Parsec always just a bit laggy?
How does it sound with the side panels on? does it dampen noise in any significant way? also this is so awesome. I may try to do something similar on a dell precision dual socket system with some v4 xeons.
Nice build...how many KASM instances could I run on it?
Am I the only one that was screaming eps over and over until I saw the surgical precision of the screwdriver plugging method lol 😂 been there
Hmm, interesting setup
While I find your use case interesting and plan on experimenting with VM GPU sharing, mostly I want to use these older cards like the M40 for things like stable diffusion. Appreciate your content.
This would be the perfect use for zfs deduplication, because every game would be installed 4x
Jeff, it seems the M40 has the same layout as the GTX 980 ti. A friend of mine fitted a 3rd party cooler with 3 fans from that card and fitted it on the M40. It works great, full load he gets max 75°c.
DL580 Gen8, here I come!
Have you tried a P40 yet? I am curious if you can do the same workarounds that you can do with the M40 to get it working as a gaming card.
Thank for your video to pass-trough a GPU to multiple Hyper-V's, I also have success with passthrough a displaylink DL3900 adapter with USB Eltima gateway. I think multiple USB3.0 docks will also be possible. No GPU hdmi/dp is needed because the docks have them self. Also no GPU encoding.
Just saw the video description... the tutorial you have coming up, is it on how to set it all up or just on the vGPU unlock stuff? I know I'm going to watch it and find out, so this comment is useless but I am hoping it's a full tutorial.
Amazing video, Jeff! A while back, I was working on a silent cooling solution for Tesla K80s in desktop PCs. I left off at two Noctua 60mm hooked up to a Y-connection splitter, so that it could be controlled by a single PWM header. You can have them run at 100% load, and still be quieter than the CPU cooler on idle. A proof-of-concept is on my channel (card is not powered, but fans are at 100% load). Happy to send you the STL file if you wanted to print it and give it a shot.
In hindsight, I would have actually just scrapped the concept of using a blower-style cooler configuration, and instead opt for an open-air configuration. Some desktop Tesla users found that slapping on one or two fans directly on the heatsinks could dissipate heat just as effectively and much quieter than using a blower. For further improvement in thermals, you might also want to consider repasting the cards, as many Teslas were manufactured ~10 years ago (thermal paste is likely underperforming by now).
The other alternative is watercooling, and there are a few CNC-machined waterblocks on Ebay for the Tesla K80/M40. I haven't tested those, but they seem like a promising (albeit a bit costly) solution for quieter cooling.
I'd be interested. I got myself a k80 a while back but haven't figured out a way to cool it without going deaf
@@mitlanderson As a quick hack, might be worth just slapping 2 noctua 80mm fans, straight onto the heat fins (can unscrew the top cover pretty easily, held by a few hex bolts on the side), and see if that can provide any feasible amount of thermal dissipation. Unfortunately, I gave my K80 to a colleague, but that would have been my next idea for a noctua-based cooler
I found that if you have a Corsair commander fan hub, you can connect the blower fans to it and use the iCue software to tune the fan curve for those turbos and assign them to the gpu temp sensors as well, it’s a lot quieter during light video tasks; however I just need to know if someone have validated whether these m40s can work in sli mode or similar with the igpu output from motherboard?
Itd be interesting to see how nis or fsr through lossless scaling helps. Also can I do this with a ryzen pc?
This is amazing
One thing I'd like to see is all the other things required for this type of setup. Would it work well on 1GB ethernet or would you need to add cards and hardware to run 2.5GB+? What kind of upload speed would you need at home to make this a viable experience away from home? I know these things have been mentioned before, but I don't have time to go back and find it. Would be nice to have a short section for all of these videos with little details like that. Basically, how fast does my network need to be to handle the different loads? 1GB should work fine for 1 VM, but what about 2? 4?
At what point would I need to upgrade the networking? At 4 VMs? 6?
Even just a breakdown of bandwidth requirements for a single connection and we can do the math from there.
Love the content, just moving and trying to figure out what infrastructure I'd need to run in order for something like this to work.
hey Jeff, can you make a video on how you did the configuration of multiple VMs. A step by step guide would be great.
That's literally next 😉
I’ve been wanting to do a sick lan rig. Hoping I can do this but but with output on the cards
When will you do a rendering focused build?
Is it possible to remove the gpu shrouds completely and jerry-rig a tower cooler to them to run with a normal fan? Or possibly water cooling with a custom block?
Really...nice
Came here for the tutorial and evidence that this is possible, left to hit the beer distributor for that Hella Phantastic
I wonder if you can use LVM copy on write so you only have to keep one instance updated for all the installs to save time and storage, even better than deduplication
I would love to setup something similar to this, one VM for my personal gaming desktop and another for a friend to use, although i doubt this would work over the internet even with some networking wizardry, at least until i get real fibre with 1Gbps upload.
How did you get this to work? I assume ProxMox for the Hypervisor, but wouldn't any hypervisor require physical hardware to run? Curious as I have some hardware I'd like to use to turn into a dual-gaming pc
Music was too loud when you were talking in the intro. (Juat a heads up)
I love these multiple virtual gaming machines with gpu passthrough videos.
Also take the next step, have this server serve TFTP boot configuration for thinclients that automatically open the session from the VM on boot!
Man, I’m loving your videos, I’m learning a lot! I could get my hands on a couple of M60, I saw you use it with a different approach (nvidia grid I think), and the results wasn’t good. How do you think will that model work with this method?
I run a pair of M60s in my main server using this method, and they are incredible. If you can get them, I say go for it!
@@CraftComputing that’s great news! Thanks a lot!
Could you use the Noctua industrial fans as front intake to remove blowymatron?
Hi. Can you do a performance test on davinci resolve over the virtualization? I wonder if it would be better than my 1060. Thanks.
Thanks! I ordered a couple 12GB cards so my kids can play with their neighborhood friends. I'm going to put them in a PowerEdge R720 with a pair of e5-2667 v2 processors. Hopefully, the case will keep them cool enough not to require a shroud as I don't have a 3D printer.
That's basically the setup I run here for neighborhood friends :-)
Yup, they will stay cool enough, but the idrac fan controller will jump to noisy levels by default unless you override it with ipmi commands.
I'd be curious to see if you could create enough positive pressure in the case to cool the GPU's. Even loaded up with Noctua industrial 3000 rpm fans running full boar would still sound way sexier than those blowers. Seal every gap except the GPU vents and put all fans pulling air into the case. Please and thank you
How you managed video outputs? How use it how wire it, how separate peripherials?
Great video, i still have one of these gpus for a project like this. However, have you already or can you show how to host this service and share the VM, maybe through a VPN? I'd love to have a family gaming server that my nephews can hop on from their non-gaming pcs from home.
Jeff to your knowledge do most those Chinese motherboards have above 4g decoding? Or at least the x99 ones.
Can't you use one of those in-line resistors that will slow down the turbine fan?
did you ever do the video you said you'd link at 12:16?
Hey! Im looking to build a gaming server for some friends, would the M40 be good in todays day, i know this is from a year ago! any other enterprise cards you recommend?
Was wondering if those GPU cores would OC under water cooling. If I can find a block i will try
Which gpu did you use for video passthrough?
Hey Jeff, please is it possible to pass through one vgpu to a lxc container running jellyfin or Plex for hardware transcoding
Hey. I know I’m late to the party on this topic, but can you use the p40 for streaming and gpu work loads? I wanted to use it with a 3060 and I want to know if it’s a good investment
I was considering building a dual xeon 10 core 20 thread system and then break it out to 4 VM's each that can be used as a video editor/render machine running DeVinci. Any suggestions, comments or thoughts are always welcome.
I run my first Gen EPYC system with a Tesla M60, and use this same setup for remote video editing.
The M40s would also be a great option, especially the 24GB models from this video. I'd spring for a pair of 12-Core or 18-core chips... E5-2690v3 or 2696v3. That would give you plenty of power for 4x editing VMs.
What kind of cable powers the gpu's ? I've heard you dont use a standard pci-e gpu 8 pin cable but a motherboard power 8 pin cable . Is that correct? If so, is there a converter cable that will switch a standard power supply pcie to this?
How do you hook the monitors and peripherals on it?
The dad noises were strong in this one
Is this possible in Hyper-V on a W10Pro host or do I need to look into a different hypervisor?
So how did you connect a monitor to this, let alone four monitors, one for each player?
So, can I team up 2 of these on a single VM for rendering, modelling, etc?
Could you do 4 VR headsets on 2 Tesla P40s? How would that setup look like?
Sounds like the start of a LAN-party.
The e5 2690V3 can be run at all core tubo without generating much extra heat, if the X99 you have is bios modable
I'm going to sound like a total n00b, but that's because I am... so, after you build the system physically, what's the next step to get this set up? Like, how did you install the software without any video out and what did you install? A video on that would be cool (you might already have one, I don't know) or just some links to some tutorials would be much appreciated. 😃 As always, you're Jeff, and thanks for the awesome content!
No idea how he did it, but Linus Tech Tips have used a separate gpu with video out to set everything up (including getting it to run without a monitor hooked up), then removing the temp gpu and voilà! Not much help if you don’t have a gpu with video out lying around though.
I'd make CZcams videos about it lol I'm glad you clearly mentioned the cons. Seems some eBay sellers get stuck on some CZcamsr saying how "amazing" something is and he pricing gets stupid. I think the M40 shouldn't go for even $100. I did buy one to try to sort out using it for a local machine, unfortunately it died with plenty of other parts when 12 volts arced across a USB connection. Thanks to a past video of yours, I used a virtual UB monitor to get games running that wouldn't before on a headless machine and streamed through Steam. Was mostly fun as a hobby experiment. A couple games I play would run without too much hassle, but vsync is typically broken, often can't change resolution (sometimes the resolution shows as "undefined" etc.), and games that seemed to get updates to run better were apparently just fixed by game ready drivers that the Teslas don't get without the setup fuss that makes these a harder sellllll. Neat/cool stuff, but yeah...
to be fair, in the current GPU climate, 980Ti/1070 level performance for $140 is a legitimate bargain when new 1050 Tis are still being pushed for $200 and 1650s for $300. I suppose it has entirely to do with the complete lack of video ports and active cooling that really prevent these cards from being readily repurposed by desperate gamers
@@bojinglebells well yes, ure right but 1650s and 1050tis are pushed due to not avaibility and demand for em
@@bojinglebells Cooling is easy (if loud is fine), display out is THE problem, the end. If they had even one, then it'd be fine... and people would probably try getting $250-300 or maybe even more for them.
If I had a system like this I would use it for Touchdesigner or Magic Music Visuals art installations.
How well does parsec free run if you have 3 pc's streaming off this server ? Ie. 1 parsec free on each vm streaming to 3 different rigs. Or does parsec prevent this on the free version ?
The BE QUIET!!1! rig :)
WHAT??!?
Would love to see a build like this with a newer Xeon Platinum 8124m with how cheap they have gotten
Now find a cheap LGA3647 motherboard :)
Always on the lookout for one 😉
I used these exact cases for my Chinesium X79 ESXi Hosts... I came to wish I had bought one of those for my personal rig instead of NZXT.