Twelve Node Cloud Gaming Server - AsRock Rack 4U12G BC-250
Vložit
- čas přidán 21. 03. 2024
- Thanks to Maximum Settings Cloud Gaming for sponsoring today's video. Get started with your very own Cloud Gaming VM or Bare Metal Machine at bit.ly/3wxfUuB!
Grab yourself a Rocks Glass at craftcomputing.store
When hardware deals sound too good to be true, they usually are. This is the AsRock Rack 4U12G BC-250, a 12-Node server stuffed to the gills with Zen 2 Cores and RDNA2 graphics... It's the same APU that powers the Playstation 5, so surely it should be able to play games, right?
Right???
But first... What am I drinking???
12 Failures means I needed a 12-year scotch. Highland Park 12 was the only logical option.
Hardware Haven - AMD 4700S Kit: • This is the WEIRDEST P...
Chips and Cheese - PS5s Nerfed Zen 2 Cores: chipsandcheese.com/2024/03/20...
Follow me on Mastodon @Craftcomputing@hostux.social
Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing - Věda a technologie
@carbogger shared an updated BIOS. Here is v5 for those looking for it - bit.ly/3Pv2GFx
I might give one final crack at this after updating. If there's no second video, you already know why.
Amibios has some interesting options in there as well. Might help out. Not to mention you can set a custom fan curve in the bios of the rightmost card if you're looking at the side with the connectors. I did that as soon as I could. Then I replaced the 80mm fans with 3x140mm bgears b blasters (900cfm total) now I can survive with it In my office lol. AC infinity has some nice triple 120mm faceplates that can be modified or some people reportedly use 4u blanks. Gotta drill the rivets out to remove the front faceplate though. Worth it lol
Bios editor that is. They do support pxe too.
excited to see if anything ends up coming of this, being able to salvage these things with a bios update would be great for making a bunch of mining ewaste into something useful
"Not everyone's crazy enough to have a server rack out in their garage" always makes me smile. Mine's in my bedroom.
maintaining a remote gaming rig isn't the hassle, it's paying for the gpu.
My server racks are in my bedroom as well 😂
Racks are way too loud to have in your bedroom.
You mad lad, how tf do you live? 🤣🤣🤣
@@kellymoses8566 true but I manage
"Oh hey, each of these compute units is more powerful than my main machine, and there's twel....Oh Jeff has scotch, maybe i shouldn't buy this"
Those who know look for a bottle before 'adding to cart' 😁
@@CraftComputingI never noticed this before... interesting
@@CraftComputing I certainly don't love my power company enough to give them this level of donations. I don't blame you for quietly (or even loudly) fuming over this one.
Yep I too saw the scotch and knew what to expect.
"The only thing from preventing me from loosing a finger..." I won't lie, I was expecting a quick cut to Jay chopping carrots on a Delta Fan
I considered that clip!
Never stop doing this series... im actually setting one of these up and your videos are quite helpful
7:35 that fan ramp-up.. i feel like it should culminate with a laser blast and large detonation in the far off distance.
Or at least acceleration and takeoff.
WHATT????????
“That’s no moon…”
it's a little loud....
The last sound heard on Alderaan.
Call Wendell... I've been waiting for a collaboration for years... Both of you have the same mindset and passion....
That intermezzo with the jet engine right after "what could go wrong" is just soooo fun to watch😂
"highland park 12, because, f*** this thing."
I'm surprised there's any left in the bottle with this one.
Man, I've been following for years. Not having the space or money for server's myself- I live through you. Never stop ♥️
Awesome vid as im watching it thru so far, and good to see Hardware Haven mentioned/shown too!
Dude beat me to the punch on this useless chip :-D
It's quite a journey when you have to break out the good old stuff to deal with the troubles a project put you through. Thx for the chuckle at the end.
Thanks for taking one for the team. I have to admit, the specs caught my attention. True to form your experience just saved me $700. Thanks again!
So glad you did this Jeff, just because. What a cool crazy piece.
Good of you to mention the Chips and Cheese article. Definitely a bright spot in tech journalism.
I learned of that servers existence a few days ago. Now I’ve got a video to watch about it! Perfect timing!
Jeff you may want to check out the v5 bios and see if that behaves any differently. Most units ship with v3 or even v2. As for mining, they seem to have big driver issues with later linux kernels and we default to 5.15.0 with amd drivers 22.20.5 for ocl support. I picked one up with my thought about these as some sort of ai related workload.
Where might one find this v5 bios?
I haven't been able to get any bioses for these, where are they?
Just write it out (https) web.site/link
Most links are immediately flagged. I don't even have control over it. I would be interested in seeing what the differences are, and if it would help with driver installs.
@@carbogger there are several of us that DO have the hardware to test. If you would provide the link, we would greatly appreciate it. 👍
@carbogger Got it, and approved the comment. It should show up here. I'll also be rehosting the file and linking it in the description. Thanks for sharing!
It's not even just that the fans are loud... that frequency seems particularly brutal.
Really enjoying the last few videos like this.
6:20 Sony doesn't use Vulkan, they use a proprietary internal API called AGC on PS5. The only game console with Vulkan available to the developer is the Nintendo Switch, and that is also very rarely used in light of NVN (a proprietary NVIDIA API).
Man. What an insane server. There is so much potential in that box.
This was hilarious to watch. I can't even tell you my favorite part as there was so many. I think it was a nice touch to hear those fans just getting higher and higher in frequency, while watching your face so that's at least in the top three. I am so sorry "for your loss" (more e-waste, 3-days of hacking...) but THANK YOU for the entertainment and also for me being able to watch a really smart dude at work. Treat yourself to Highland Park 18 (not 12, not 25). It's better than what you have as a 2kw "garage space heater".
If you look carefully, my hair is blowing in the wind.
""what could go wrong" - famous last words in the IT world
What makes this so sad, is the difficulty of repurposing these machines, so they become e-waste, was a deliberate act by everyone in the manufacturing process.
Mad respect, Jeff. It takes a lot to showcase a fail. Excellent video, on every level. I want those fans.
really enjoyed this video, the whole vibe of this video reminds me of so many (quite minor compared to this) "great bargain" projects I've gotten into. 👍
Since they have unified memory, they might be useful as a cheap LLM server, if you can change the ratio between the CPU/GPU, that's nearly 192 GB of VRAM. OpenCL might be good enough to run the current state of the art LLMs on.
Only problem is that they're limited to 1gb ethernet for communicating between nodes. I looked into this a while back, and the latency between nodes would just kill the performance
But they are physically separate chunks of memory. Not sure how efficient would it be to run a distributed model if possible at all with 1Gbps interlink.
@@iraqigeek8363even the lowest latency 100gbe interconnects would be a major bottleneck so a plain 1 gig most definitely won't cut it.
@@iraqigeek8363 a 10G m.2 could be added.
@@iraqigeek8363using a 10GbE m.2 adapter could fix that.
Amazing video. Even if nothing came from it your explanation of the steps you took and your analysis of the situation made it fascinating. Consider me subscribed.
Everytime I watch one of your videos, I always get the small thought that you're Cody's Lab as a well maintained/classy version
Thank you for the review, even though it was a failure! I would have considered buying such a rig and would have attempted the same thing!
I think the loudest bit of server kit I worked on was an HP C7000 blade enclosure, when you first turned it on, it was like a small jet engine. Then the management agents kicked in, and slowed the chassis fans down.....🙂
It also gets louder under a full load. Also if you cut the metal mesh off the fans, in nulls the sound. I know because I have 5 of them
@CraftComputing as far as I remember, you can still copy DXVK's dll files on Windows and they should translate your DirectX calls to Vulkan the same as they do on the Linux side! I tried that with some games (LoL and 2015ish Star Wars) and they worked better with my RX 5700 XT under this Vulkan translation layer than DirectX11 using Windows 10, maybe try it out in a next video!
Was going to say the exact same thing. DXVK dll files work fine in windows, though best copying the to individual game folders rather than using them system wide. And yes, DXVK on windows is often faster the DirectX11 due to multi-threading.
@@crayzeape2230the thing is the PS5 doesn’t actually use Vulkan, so I don’t know if that would even work.
@@LtdJorge The GPU supports Vulkan 1.2 according to the video 6:17
The GPU can manage 10TF of fp32 compute, so across all 12 you have 120TF, which is one and a half 4090s. You also have more memory than the 4090. So this could be a good box for some HPC workload with very high parallelism. Maybe Monte Carlo simulation?
One more thing- the shape of the board would enable easy submersed cooling (in oil), with plugs facing upwards :-)
I am sure glad you did this video. I almost bought 1 of those cards on eBay for 85 dollars. Thank you.
"A couple of Haswell Xeons..." just made em smile. I got a triple of those, two 14cores and a 12 cores. Two running my DSM Nases and one as the main machine. All for a neat $600.
Ok... that outtake at the end got me lol!
I used to live next to the highland park distillery in Orkney. Try the 18 year old, it costs way more than the 12 but it well worth the additional cost!
It would make a great heater with the bonus feature of a noise generator. :)
one blade might look pretty sick on your shelf
This video was a rollercoaster :D
Would like to see a part 2 with some of the suggestions.
Just to lower that TOC for that CAPEX report.
The BC250 is a cut down version with 6 cores 12 threads. It is made of the APUs that didn't pass QC for a PS5
Almost all servers and enterprise storage arrays are that loud or louder. You can’t imagine what a cold start to a data center sounds like. I think a 747 is quieter.
I've heard my fair share of screamers.
I'm dumb enough to buy enterprise networking gear and use it until neighbors complain.
But a buddy of mine also bought one of these and it is really loud. Like, more than expected given our experience.
The only thing I can compare these fans to are EDFs or mid-sized quadcopter motors.
@@Prophes0roh gawd
Oh bet, I once had a gigabyte 2U Epyc server had its fans stuck on full. 4 of them were PFM0812HE-01s... Rated 7A max!
Lol I saw the literal same listing a month back and basically had the same monologue as you in the intro. I guess I dodged a bullet XD
If you want more bad investment advice - there's also a similar mining chassis from Sapphire with nerfed and passively cooled RX 6700
Link?
Woo love watching Jeff and Colton save Ewaste from dumpsters to watch it turn into a dumpster fire! I honestly live vicariously through yall as it's everything I'd love to do 😂
Literally spitted my coffee when the sound started lmao
Thank you for spending time/money on dumb experiments so we don't have to. Your service is appreciated.
There are a few of us out there that ARE crazy enough to have server racks in their house. When you turned on your server, it immediately went into what's called "panic mode". That's why it went screaming off. In some racks it even makes a difference whether the case lid is on or not. They actually measure air pressure going through the rack. After a very short period of time, those fans would spin down to a fairly quiet mode, pending how hard you are pushing the blades. Those servers sound like they could be used as headless Linux servers for gaming emulators, such as Everquest, World Of Warcraft, Star Wars Galaxies, etc... All you need to build servers for those kinds of games is a stable server version of Linux and a few separate network adapters installed in the rack/server.
I may own a bunch of these units. The fans start off at 40% and only get worse from there. Even with the case lid on. They are intolerable at their lowest speeds. - Reversing the fan direction helps soften the shrillness of it all but they’re still terribly loud.
Nice to have your own, home based jet engine. :)
My garage is fully detached and uninsulated. And 100 years old. Just dealing with the spiders would be a pain in the ass.😅
This is one of those looks too good to be true, because it IS too good to be true things.
Depreciation is one thing, but dropping from $15K to $700 is a massive red flag.
One thing to keep in mind is that more than any time before, the people designing building and selling stuff like this are in our age group, they know we're always scouting for deals on the sly, and exactly how to thwart that. How hard they go on it is largely down to how locked down their legal department wants to be on things. Those folks are also onto us, only they're not under any pressure or financial incentive to reduce e-waste, resulting in dead ends like that power hungry banshee. Any wins you get after jumping through hoops to get stuff to work are because it's not worth it to them to make it near impossible.
Those fans are hilarious though. Send them to Linus or Jay so they can do stupid things with them.
If they'd included DX support but disabled Open CL so as to appease Sony, I could see a small shop located somewhere with cheap power using this for remote terminals. Use whatever you want to remote into one of those nodes for a full and secure desktop experience. Setup an inexpensive storage server to house their data data and you're good.
I mean why would you want someone selling the guts of your main gaming product? I totally get that, and it's 100% reasonable. You wouldn't even need to emulate. What I would like to see is that the agreement would have an expiration date where say once the PS 6 came out and it reached some sales threshold, AMD can unlock whatever was disabled and release drivers, even if it means leaving something gimped to block running PlayStation software on it. It addresses the e-waste issue. Then you could have a more reasonable chassis with bigger quieter fans and maybe half the nodes so it doesn't need it's own substation.
I don't like that ASRock or anyone else made crap like this to support mining, but I can't really fault them for making some quick and easy cash off of mining scum.
I find that 24K RPM fans tend to deal with spiders quite easily.
Reminds me of my misadventures into the HP C-7000 with 16 blades.
We're starting with the bottle on the table already and in the thumbnail. I'm a simple man. I see that in a Craft thumbnail, I'm already invested. When you turned it on, did the dogs three blocks over start just howling? I think one of those fans could cool a small apartment. So much hardware for so cheap but... it's e-waste that's power hungry. I'd really love it if there were a 'fix' to use this in some way other than crypto.
I'm working on using a BC-250 as a standalone computer, and this video has plenty of information that I haven't found anywhere else simply because of how obscure this server is. Hopefully this video helps bring more attention to this device because it's got a ton of untapped potential!
Did you succed?
Did you get 3d acceleration working?
I need these fans in my life!
The devil's in the details. Nice overview of some interesting hardware, thanks for the video.
Thank you for your efforts. I wish this would have worked, because it seems like a great opportunity to run a local "cloud" gaming cluster, but clearly, it wasn't meant to be. Hopefully something pans out eventually on this front.
Now I see why Zeuxs will heat his garage with this unit
Dang, sorry you couldn't get this running. I was all excited at the price point, to use for BOINC.
The ramping up like a jet going for takeoff made me giggle.
Well done, failure yes but your work to get there was the enjoyment. Keep the fans!
7:31 if NASCAR ever switches over to all-electric vehicles, I suspect it will sound like this.
If NASCAR ever switched to electric, the drivers would all be able to go take a nap during pit stops.
Damn that would have been cool as hell. I would have watched your 3 days of struggle, but I understand the frustration of editing that much stuff only to have to take the L. You'll get'm next time Jeff.
companion Craft Computing, a concrete commitment, more cloud gaming for everyone✊😅
Interesting to see tekelec frame style servers in a small case like that. Just their number porting solution took up an entire rack or two.
Would be a killer option for test machines in the homelab, or little web servers maybe... I would definitely have to modify the cooling system though...
Having dual NICs would be nice, which you can get in similar multi-node systems like a SuperMicro MicroCloud.
Plenty of things this server can still be used for. The question isn't what can it be used for, but what can it be used for to justify the power required on the machine.
There are other 2u4node or 4u8-12node servers available on Ebay for much more and much less than this that would work better for such usecases
@Ziggy Correct. Like I said, performance was somewhere around Zen 1, but at 90W power draw. A 16-core Haswell would destroy this with similar idle and peak draw.
Honestly still tempted lol.
I don't think I will be able to do any better by any means. It will take a village of community support devs. And will people donate their time? Who knows.
There's T-Shirt design for you "Uhh.... Noise is going to be a factor" with a picture of that thing. I'd probably buy it.
It sounds like it was taking off!
Hahaha. The noise is amazing! That’s fracking awesome. You could repel mosquitos and loitering teenagers with that. It reminds me why I didn’t put bitcoin ASICS in my house. Even though this was a failure, I am glad to hear about your experiment.
Great adventure
A nutty project I've been kicking the idea around was to get my hands on an 8 socket server. I don't know what I would do with it, or where I would put the thing, or why I need that much redundancy, but I want it
That’s not redundancy, tho
@@LtdJorge some 8 socket servers allow you to hot swap processors and memory as they are frequently on carrier boards
this was awesome
Holy hell dude!
I'm sorry, Jeff, did you say something?
Lol. I giggled in your suffering when those fans hit full tilt.
It would make a Fantastic paperweight, or a step.
7:30 * Jeff's hair starts giggling as if he's in a shampoo commercial *
There are kits that use (3) 140mm fans instead of the small ones. It really helps with the noise
Looks like a good candidate for a high availability cluster.
but HA of WHAT. Nothing works on this...
@@Vatharian High availability heating, with this power draw.
Thinking of blades, it would be nice to see a pc-on-a-pcie-card where the pcie interface functioned as the network. CommVision had these in the 1990's running 386 "edge servers". Its easier to run long DP/HDMI and USB than 100G fibre around the house.
Did you try using this as a blender render, transcoding or plex server? I'm assuming with the comment earlier about the 1FPS streaming means a no but might be good to check.
I got some older games to boot in windows on one., no sound device found though. Did you run into any sound problems?
As a software entusiast i would love to play around building clusters on that one. But my server rack is behind me so for it to work the fans needs replacing.
I have a couple of opencl projects that could be interesting to but power draw is an issue. Somethings might be built for crazy people trying to make a buck on crypto.
Keep up the great work.
@CraftComputing perhaps the Linux kernel or OpenGl driver package just needs a few lines f code changed to recognise the gpu as it cannot be architecturally different form normal Radeon GPU so just a case of pci id being updated somewhere?
would this work for rendering stuff? you could set it to be wake on lan and to only turn on to render something, upload it to a NAS and then turn off again. that would mean even though it uses alot of power it would not have the issue of idle power. That would only use one or two nodes though but you could probably just use one and shove it in a 1u server chasses and be done with it.
The noise can be mitigated by replacing the fans with bigger ones to get the same air flow. This might require a front plate replacement. I did not see in the video what was the stock mining OS that might be your best bet for making any use of the unit.
1. Holy shit that fan noise! I'm sure I wasn't the only one that immediately reached for the volume dial on their headset! :P
2. If it wasn't for that high power draw, that might actually make for a decent Folding server box.
Spins like a quadcoptor is taking off 😂
Failures are just as entertaining as wins when it comes to computing!
And, lets not forget, some of your past failures DID end up in wins later down the line!
"Tis but a scratch!"
@7.55 🤣 - supply the family with good quality ear guards!
It would make a grate heater for the winter just set it up to mine any thing it makes will reduce your heating bill
Now that this board is seen the light of the day in an accessible way I imagine that the support will start to stabilize soon enough, also if the gpu side works like steam deck with a minimum plus dynamic allocated memory, it will serve really well for ML/DL, 96GB (up to 192GB) of GPU accessible memory to run/train your LLM (chatGPT)? Really Amazing! And all of it for just U$700??? OMG, pack me 2 dozens of it for travel... The power consumption is a little concerning, but if each node could provide 10TF/FP32 (20TF/FP16), this would be 240TF/FP16 for 2.5KW, this would be 2x performance of the RX 7900 XTX with 4-8 times more memory while cheaper than a RX 7900 XT (not XTX)... You could take the difference in price (of a theoretically equivalent performance system, like 2x 7900XTX) and buy some solar panels to compensate the downside XD
Of course, this is only IF it could really works, when you are using ML/DL/LLM you are already suffering so much when using supported hardware, investing in something just for the hope (not even promise) of a functional platform is simply insane...
You are so going to have to try out submersing this whole server in non-conductive coolant!!
Would these nodes be good for something like tdarr? I'd imagine that kind of core density would be great for transcoding tasks.
Time to turn into a giant door wedge, so it doesn't go to e-waste, 🤦.
Goog video, interesting project. Mayel be next time, you will hit the jack pot🤞
7:45 - That was clearly made and setup for mining, you definitely should be able to modify it to be a lot more quite, though it won't be really quite, you still would need high pressure fans to push through these heatsinks, but like, 5k rpm fans would be still sufficient
Have you tried swapping the node you tried this all on? It could have been that specific node being very much broken?
What runway is that thing taking off from?
Must say I got really excited when you mentioned the price and then less excited later on😢. Wish this worked.
Minus the power draw could it still be useful for a vdi clister of sorts?
Would be tempted to try hashcat and ffmpeg if possible.