The PERFECT 2U Cloud Gaming Server - Asus ESC4000 G3
Vložit
- čas přidán 25. 04. 2024
- Thanks to Maximum Settings Cloud Gaming for sponsoring today's video. Get started with your very own Cloud Gaming VM or Bare Metal Machine at bit.ly/3wxfUuB!
Get the all new BMF Minimalist Wallet at craftcomputing.store
It's no secret I have a deep obsession with Enterprise Graphics Cards. It seems I have an ever-growing collection of them, but one issue persists... how do you actually install them into a server AND keep them cool.
After years of searching for an affordable option, I've come up with this, the Asus ESC4000 G3, a purpose-built GPU server that should be perfect for hosting a Cloud Gaming vGPU box, or deploying a self-hosted AI/LLM server.
But first... What am I drinking???
South Pacific IPA, Pfriem Brewing (Hood River, OR) 6.8%
Links to items below may be affiliate links for which I may be compensated
Asus ESC4000 G3 GPU Server: ebay.us/ngCIDO ($175 + shipping!) or ebay.us/hAThwr
Asus ESC4000 G3 Sliding Rails: www.neobits.com/asus_tool_les...
Intel Xeon E5-2697 v4 18-Core: ebay.us/6nlzsD
8x32GB DDR4-2400 REG-ECC Memory: ebay.us/4DNJ00
1.92TB Patriot Burst Elite SATA SSD: amzn.to/4b8Q9jc
Nvidia P100 16GB: ebay.us/OYEgUn
Nvidia P40 24GB: ebay.us/1FurAp
Follow me on Mastodon @Craftcomputing@hostux.social
Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
"Space Jazz" Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
creativecommons.org/licenses/b... - Věda a technologie
![tattoo printer tutorial[제품단순선물]](http://i.ytimg.com/vi/g40_GwEJak0/mqdefault.jpg)
I'm not salivating, you are salivating
I am very sorry to say CZcams did not show me your channel for years till 6+ months ago...... I almost forgot you, I am ashamed of that.
Laziness is king. We should have a plung-in that shows us the channels we like/love no matter what G/Tube does... We are here because of people like you, not the video quality/stream/access.
Cheers,
- 1 more beer with Ben
Don't rely on your recommended feed. On the left bar, click on Subscriptions to see what your favorite channels upload (and nothing else).
Would be nice if CZcams had an option to set sub page as default
@@Krrillin678 ImprovedTube is a good browser plugin to have a better experience on CZcams. You can set the default landing page to your subscriptions page instead of CZcams's recommendations to solve that specific problem.
"ah shit here we go again"
Thank you for always letting me live vicariously through you with these builds. I'm much excite
Server days are the best days!
Great build, glad you are doing this for us to see. Really interested in the power draw once its all done.
Great video. I have G2 & G3 of this GPU server, and both came with their GPU cable extensions!
Man it's so cool. i'm technology savvy but not so much to understand server stuff, unraid alone gave me headaches. You're a smart guy, waiting for the continuation video of this beast!
I keep drooling over these hardware setups even though I'm doing with the two proxmox boxes I already have.
this is a pretty rad build, I cant wait to see how it works out for you.
You should follow Wendel's posts on SR-IOV with Intel hardware, he is teasing a Arc A770 'hack' for enabling it (making it work as a Flex 170). He doesn't do any cloud-gaming setups, he just does the groundwork for it, and he is very technical about it. If he find a way to get the A770 to run multiple gpu-accelerated virtual instances, you should try building a gaming cloud setup with it!! Something like a LAN-ready gaming cloud which can run multiple instances of esport titles (or games on that level of demand at 1080)! That would be very neat!
Oh, I'm well aware. We've been talking behind the scenes. I have my eye on some A770s for this box....
@@CraftComputingthis will turn into one of these projects you will use to justify blacking out with a bottle of whiskey on a work day.
Im really looking forward to it. I mean i am looking forward to the server content.
We call that 'Tuesday' around here
@@CraftComputing Oh man I want sr-iov more than anything in tech right now. I hope you guys can come through with the a770. I'm really hoping battlemage just supports a few sr-iov pfs by default. That'd make me switch to team blue in an instant.
@@CraftComputing re: SR-IOV virtual functions with a consumer card.. Sadly the stability isn't acceptable for an 8-player machine as then it's crashing 8x as often and affecting 8x as many people = effectively 64x as flaky.
Where the bleep is this $239 on ebay?! All I see are $500+ machines!
$235 is what I paid for mine, though I only found out when it arrived it was missing the GPU cables. I'm into it ~$400 with cables and rails for the barebones chassis.
@@CraftComputing Thanks for the reply! I predict the "Jeff" effect on eBay pricing for items like this :)
@@zeroforkgiven Jeff is smart and has alerts set for low prices on server parts, he is not actively looking for every single piece, but he gets notified with rare opportunities when they happen, we should all do the same and not overpay for stuff!
Seriously. Ebay must already be ahead of the curve here because even barebone machines are $499 on eBay
Edit -- I was able to get a very barebones version (w/ PSU at least) for ~$160 shipped on ebay. TBD on how it will perform on arrival. Anxiously awaiting the next video for a solution on the GPU power cable issue.
awesome project!!! I am very tempted to do something similar.. This is a lot of temptation!
NOOOOOOOOOOOOOOO you are not allowed to end this video ...Continueeeee. I'm excited for part 2
Moss in Pulp Fiction! Brilliant!
Very interested to see how this project turns out!
love your videos keep up the good work
Wow ,this is the best 2u chassis form factor which i want to find!
these v4 systems are just at the sweet spot right now for the used market....the HP Z840's are awesome and cheap as well, great for a cheap PVE server
I put together a spreadsheet of available CPUs for my cloud gaming server/homelab server and ended up choosing the same pair! Running a single P40, ESXi/vCenter (been testing on multiple trial licenses). Your chassis may have the fan power, but I highly recommend these for your CPU cooling. They work great. I found out the "A" on the end of the processor name means it needs active cooling.
Dynatron R25 Cooling Fan/heatsink - 1 X 3.15 - 7000 Rpm - Dual Ball Bearing - Socket R Lga-2011 Compatible Processor Socket - Copper
Jef is that friend you need for your enterprise toys.
YAY New Server day!!!
I too finally got my perfect gaming setup. Really great gaming desktop with Sunshine and moonlight clients on every TV in the house.
Or VPN in from anywhere.
I tried this, but wanted 120Hz VRR and HDR...so I decided to run Bullet Train AOC 48Gbps HDMI cables from my office to each TV spot. For input I use a Digi Anywhere 2 Plus for USB over IP. However I still do use Sunshine/Moonlight and Steam Streaming for handhelds and/or Parsec.
Fun fact about VPNs: most mikrotik home consumer-level routers can host Wireguard directly on-device, if you update your software
@@rezenclowd3 yeah I originally had a 50 ft HDMI cable I ran for my office with a gaming PC is to my living room downstairs.
Work great with a GTX 1080. But with the 7800 XTX I guess the new HDMI standard only supports 25 ft cables?
But either way it was kind of a hassle to set up. Having to go into windows and enable disabled displays and change to 4K and everything... And for some reason Windows loves to open windows on the opposite display of whatever you're looking at. 😂🤣😂.
So much simpler to just send a weight gone land request from moonlight. Then have it automagically configure everything.
Latency is not bad at all. Played most of cyberpunk this way. Definitely wouldn't play a competitive first-person shooter with it though. But everything else is fair game.
@hotrodhunk7389 about 10ft for copper hdmi. For fiber...over 100ft. My cables were $750 ea. There are cheaper but these bullet train cables are used by commercial installers. In my use case I just walk to pc and unplug my 6 monitor setup and plug in the remote tv of choice.
im interested to see how something like this would pair with a Steam-Linked phone with...say a backbone for controls. Excited to see what comes of this!
U got this Jeff!!
Hairs looking nice!
I'm actually attempting to grow it out. I think I'm almost through that awkward in-between length.
Looking forward to the next video. ;)
Great stuff, interesting idea
I'm about to place an order for this server with a K80 gpu. Hope the cable video comes soon
Editing the cable video today. Should be up inside 24h.
At 7:00
CPU prices....
$50 / 18 cores = $2.78 per core....
$150 / 24 cores = $6.25 per core....
That's a steep rise in price per core....too bad you can't stick one of each in there to mitigate some of the price difference.
Good video so far, now I'll go finish watching it.
Im here for the beer review
Querry:
Have you thought of publishing your favorite beers somewhere, along with your description of them??
Thank you for your time;
Monte
The GOAT is back 🎉
I checked the ebay links in the description. They're around 200 USD. But the shipping is at 400 USD (thanks to me living in asia 😂). Time to see if there are good deals locally. But from what I've seen, mostly are Dell poweredge, which I got an R820 already.
X99 machines always intrigue me!. That is why I got three!.
I know there is always more hardware to buy, but if it's something that's within your production budget, please take a look at the Intel Arc Pro A40, single slot, bus powered, for transcoding, or other background gfx tasks, I'd really like to see what it has to offer but it's a relatively unknown card.
Thanks for the video.
Was wondering are you going to be putting 8 dual slot gpus in this rack? Do not know if I missed something in the video.
The server will support (4) Dual Slot or (8) Single Slot.
@@CraftComputing That is what I thought. When you said you wanted 8 gaming machine on bare metal and showed a stack of what looks like dual slot card, was wondering how it would work out.
Then was wondering what single card would give you 1080 med, but then again I know not much about server hardware.
Thanks for the reply, will watch and see what you do with the server.
@@ehiebert1297the P4 is available on both single slot and low profile. The big limitation is 8gb of vram, but If you have 8 slots...
Eventually I am going to attempt 6-7 P4's for a home gaming server. I have tested it and gotten it working with two dual slot cards I had laying around.
Hope your G3 is more reliable than the ESC4000 G4 that decided to be my nemesis at work. Regularly has issues of "losing" a GPU and crashing during the lengthy UEFI boot-up, meaning I have to drive 60 minutes from the office just to make sure it reboots cleanly, while kind of losing my mind while staring at the "in search of incredible" tag line. I have a Supermicro barebone lined up for a replacement which does a bit less GPU, and once replaced I will make sure to reseat the CPUs and hope it is more reliable when it resides in the rack within the office.
Hey Jeff, what PCI card are you using for the dual NVMEs for the OS? Thanks!
Could you do a video on the client side too? It would be cool to have a cheap client system for 8 people in one area.
I've done a comparison of Parsec + Sunshine/Moonlight before on the channel. Might be worth revisiting once this box is up and running though.
@@CraftComputing I was thinking if there's cheap thin clients available to stream to. Gamer house vibes.
How well does the p40 work, i have been looking into building a cloud gaming server myself.
Ive got 2x 2690v4's 1tb of ram at 2400mhz, nvme storage, and quad 10gb networking. Its been working great. I jammed a 1060 to test with, which also works great for what little performance it can dish out.
But now i am looking into the gpu's im down to p40's vs p100's any advice? Ive been leaning towards the p40's. Both are the same price, but there is not much info on building game servers from them.
Also those eps cables are available on amazon if its the same pinout as a dell, lookup "r730 to m60" or "r730 to p40"
Just becareful some of these "eps" ports are eps in formfactor but not quite eps in pin out. Dont fry anything while your testing this stuff, looking forward to seeing what you make..
Those cpu's can do cluster on die, it would be interesting to see its impact on benchmarks
Have you looked into the Dell t630? I’ve got that going right now with p40s and it’s great. I just can’t figure out how to pass though the p40s to a vm with ProxMox.
I'm really thinking about this one, but curious question. How loud are the fans? I've been trying to come up with a mostly quiet solution as this would go in a dry basement.
Can I just say I love your videos!
Awww, thanks!
i dread to think on the state of the electric bill for this monster! very impressive!
Could you use truenas scale to hypervise 8 windows vm's and use gpu isolation for each of the vm's ?
what nvme adapter did you use? I've got one on aliexpress x8 to two nvme but it did't worked and burned down one drive :(((
awsome video!!! :D
After I noticed that there is light reflection in the lower right corner and it starts to blink whenever Jeff raised his hands, I could not look away anymore and lost focus on what Jeff was saying in that moment.
Luckily it went away in the middle of the video, when he got to the part I found interesting.
hello ADHD, I'm dad
Man you must have a TON of rack mounted servers around the house. I have to wonder where your spare servers go? Do you even HAVE spare servers, or do you USE all of them to some degree?
I have *some* unused gear. Mostly, I buy servers I'm interested in, run them for 8-12 months, then upgrade to something else I'm interested in. Old gear gets sold.
Does anyone know what dual-slot NVMe adapter he's using? I bought a cheap one to try but it only shows one of the SSDs, and I suspect it has something to do with not using Intel VROC after looking at the listing again lol
i wonder how this would compare to 8 amd apu mini pcs in a 3d printed rack mount carrier
What nvme pcie adapter did you use?
In reaction to your sponsor video. Did you just use an abxylute? How is it? I'm looking for a cloud gaming handheld that doesn't break the bank but is still usable. For instance for running moonlight.
The handheld in the video is an Ayn Odin Lite.l, and worked great in my testing. Obviously with cloud gaming, your mileage may vary depending on your internet latency, distance from the cloud servers, and your own internal network. I'm almost 2000mi away from Max, and still had a great experience.
Ok I beg, you please do a ProxMox tutorial in passthrough of the Tesla p40. I feel like I’ve tried everything. I can passthrough my other cards. But the p40s won’t work. They show up in LSPCI but I can’t get them to work.
back in 2005 my masters project was on using virtualization and I use the example of a home gaming server with dumb clients - my professor thought it was a good example case but didn't think there would ever be a market for this use-case
Still really isn’t at scale
Have you ever tried doing a multiseat setup with one Linux install instead of VMs?
I'd be curious to know how well that works out.
I plan on testing out both Windows and Linux VMs on this server.
@@CraftComputing I meant without VM's at all.
A single linux install can support multiple users at the same time each with their own keyboard, mouse, and GPU without the overhead of running the OS multiple times.
But I've never seen anyone do it with good hardware, it's always some "The 2nd one uses the iGPU for my little brother / girlfriend / etc"
i want a 1u rack audio production pc but dont really know where to get started
I could be wrong but I believe that server can actually support up to 2933 ecc ram. It’s just not officially supported.
Ohhh I want a video on the P40 please !! to see if i can have it for a VM for video edition and machine learning i got the 11900 erying MB , maybe that can be a killer VM server
P40 will absolutely be included in benchmarks. Stay tuned.
Is it Server Sfriday already?
I wonder what do you think about dell precision R series workstations like dell R7610 R7910 R7920. They also have big risers, also decent amount of X16 slots, etc. Personally I have dell R5500 and upgrading currently to R7610. I used dell r5500 under proxmox for vGPU on Tesla P4 and GTX 1660 Ti and experience was very good. I am upgrading to R7610 mainly for UEFI and above 4g decoding support.
My Cloud Gaming videos literally started on an R7610 :-) Solid box.
@@CraftComputing I didn't know I am interested in all this virtualization stuff like for a year.
What are you using for the remote access client to be able to game on this cloud gaming server?
(I tried Parsec, but the connection STILL wasn't stable.)
Parsec has been remarkably stable for me. What is your client WiFi like? You need a VER stable stream for any low-latency remote access client. I've also used Sunshine/Moonlight with some great success.
@@CraftComputing
Not using WiFi.
Using GbE hard wire on both ends (and outbound WAN traffic as well).
I joined Sunshine's discord channel, and posted my issue there but there was never any resolution (that I've seen, for my post).
Can each pcie slot use full slot power?
Today's beer is the kinda beer they use to make beer goggles, since it's so clear.
Wish it had 12 front drive bays and was a dell. 😉 Also I don't get why the AMD MI25 are going up in price.
my dream is a 3u server case, but i cant find one with enough cpu 8-pins / gpu 8-pins.
While cloud gaming is cool on multi-VM GPU servers, I'm always more interested in the potential to use hubs or thunderbolt to create essentially a LAN center or multi-user PC setup!
Come on, when are you going to make a beer uploading server so the channel members can download some beer!
Still waiting for my Beerinator from @45Drives
What are your thoughts on the G4 model?
Too expensive still, but definitely on my radar to pick up.
Bro pls try gaming on this machine . Want to get a performance aspect
...that's literally the whole idea behind this project.
I had initially considered this for an AI project. I went with another option that is similar in architecture but not in footprint.
Same with the P100s but decided I needed RTX class cards. My heart sunk when I saw you cover the tesla card so decided to change scope.
Those x99 systems run their best when you can employ 4 or 8 channel memory and 2s processors
Xeon has "pure" cores, not threads?
How do you get your setups to work with Anti Cheat? Almost all the games that I try running on my setup tells me they do not permit running in VMs? Games like PUBG, Fortnight etc
There are some ways to disguise VMs, but it is a crapshoot when it comes to anticheat.
@@CraftComputing I’m sure, I’ve tried quite a few things but I haven’t been able to get it to work.
I want the server hardware but I have no actual use for it what so ever.
So Ill just make a cloud gaming PC out of it and then boom it has a use.
they sell Asus ESC4000 G3 CTO only to USA country , not to Europe etc :( is an some similar product?
Jees u guys got it lucky these are 900 over here for just the chassis.
Be very careful on the power connectors, specifically the positive and negative voltages.
There are two types for enterprise systems, and they work with each other.... so you can short the system & gpu
( talking from real life experience here )
Voltmeter is going to be used to verify any connections before plugging into a GPU.
Doing these kinds of jobs requires a cheap voltmeter to check every pin so you know what is 12v and what is G and what is whatever else. Servers have proprietary stuff so you can't trust anything.
Why didn't you go with the E5 2699 v4. It's supported?
Cost. The 2697 v4 is $50/ea. 2699 v4 s are still trending $150-200 each. The extra cost would only get me ONE extra core per VM, making little to no difference in performance.
sure $175 in america, $700 if you arent there though just for the empty server
It must be noted that Jeff's luck improved after he finally accepted Linux as his daily driver.
There are currently zero Asus ESC4000 G3 servers for sale in Europe/France. it's getting really annoying 😢
Did you forgot to unmute the music track? :D So weird without any songs in the background
He is doing all the editing on Linux now so maybe a bug?
Nah, I just haven't been including backing tracks... going on over a year now.
@@Americancosworth how dare you blame Linux
Didn't even notice. Wouldn't have noticed if he had included one either.
Man, just me that saw the "Beer" coaster and shrugged it off as a Doom disk? I had to double take. Kudos Sir. :D
Available at craftcomputing.store 😁
Whelp, there goes my wallet, becoming empty again. 😂
good
How many times have I heard this one? 😬
I'm not upgrading from my pair of e5-2690v4 and 128 gigs of ram
How about the noise?
Honestly not terrible. It's the loudest server in my rack right now, but those are 2U low power or 4U NAS servers with Noctua fans. It's definitely not obscene.
It's a rack server, even "not terrible" is still "I wouldn't want this near me" kind of loud
👍👍👍
Make your own power connector and provide them for sale.
There's another 2U chassis that supports 8 dual slot full height graphics cards, but I need them so I will gatekeep them.
Jeff, no! 😭 You could've gotten yourself a Gigabyte G292-Z20 with an EPYC 7402P for ~700-800 USD
You would still have to get RAM ofc, but 7402P vs. those ancient Xeons is night and day
That's 800 vs 235 for the base server. Trust me, I have my eye on the Z20 as well, but I'm also not made of money :-)
And at minimum for a single CPU, I'd need a 48-Core Rome for that to make sense. Have you priced those lately? The 2697v4 were 110 for the pair.
@@CraftComputing Fair enough. No I haven't looked at the 48 core romes.
Rip the hs9215 server video ;P
RIP it, buddy :-/
And yea Left Us Hanging on The Full Run Tests...Hmmmm....
Hopefully ASUS doesn't have a non-standard pin layout using a standard plug. Looking at you HP and Dell.
I'm going to take a multimeter to it before committing ;-)
I have the front of an HP server running just as a disk shelf. HP used the EPS connector but with only 6 wires used and 4 of those flipped.
Are your beer reviews sponsored
no he buys all the beer.
Correct. The beer is either bought by me, or gifted from fans.
You assassin of my money you!