128-Core AMD Epyc Rome Server Tear-Down, ft. Level1Techs
Vložit
- čas přidán 13. 05. 2024
- In this video, we work with Wendell from Level1Techs to tear-down one of the world's fastest computers: A 128-core Epyc build with support up to 4TB of RAM.
Sponsor: Get Linode Cloud Computing (geni.us/NehI9e - code GNEXUS20 for $20 credit)
Subscribe to Level1Techs here: / @level1techs
This AMD Epyc server is a dual-socket configuration in a Gigabyte chassis with capacity for up to 4TB of memory, 24 NVMe drives, and plenty more. The server has numerous world records and is semi-overclockable, and in our video, we'll take it apart (as much as we're allowed) to look closer at the base components of this monster server. These cost tens of thousands of dollars to configure, mostly depending on the expansion devices chosen. It's probably the highest-end system we'll have in our video set in some time.
We will work with Eden Reforestation Projects to plant at least 10 trees PER ITEM purchased from the GN store during November: store.gamersnexus.net/
Like our content? Please consider becoming our Patron to support us: / gamersnexus
** Please like, comment, and subscribe for more! **
Follow us in these locations for more gaming and hardware updates:
t: / gamersnexus
f: / gamersnexus
w: www.gamersnexus.net/
Host: Steve Burke
Guest: Wendell from Level1Techs
Video: Andrew Coleman
Links to Amazon and Newegg are typically monetized on our channel (affiliate links) and may return a commission of sales to us from the retailer. This is unrelated to the product manufacturer. Any advertisements or sponsorships are disclosed within the video ("this video is brought to you by") and above the fold in the description. We do not ever produce paid content or "sponsored content" (meaning that the content is our idea and is not funded externally aside from whatever ad placement is in the beginning) and we do not ever charge manufacturers for coverage. - Hry
![Linus Torvalds: Why Linux Mascot is a Penguin 🐧 ?! [No Ads]](http://i.ytimg.com/vi/0BiVDLTo-04/mqdefault.jpg)
Subscribe to Level1Techs here: czcams.com/channels/4w1YQAJMWOz4qtxinq55LQ.html
Watch our server build with Wendell! Pt 1 here (and Pt 2 is now live on the channel!): czcams.com/video/Hix0l8cFaMw/video.html
We will work with Eden Reforestation Projects to plant at least 10 trees PER ITEM purchased from the GN store during November: store.gamersnexus.net/
Yeah those fans are faster than you think - they are Delta fans and I know they are typically over 10K RPM. I saw the part number so I looked them up ( www.mouser.com/ProductDetail/Delta-Electronics/PFM0812HE-01BFY?qs=AvgxZrxb8y1mKi%252BBJXKU5Q== ) - and yup - they are rated for up 16300 RPM - oh and 77 dBA - mmmmm... quiet! ;)
Hey guys. Sorry for the off topic. Does anyone happen to know if Navi has PSP/Secure Processor built it like Vega did? I can't seem to find any proof of either if that's the case or not.
@@curt8806 Space and reliability there isnt much space to fit a clc and for a open loop its complicated and leaks can be catastrophic espetially in the way servers are stacked . and noise isnt a big problem in data centers
16:15 Do they also have ghost recon?
@@SnowTheBard and another thing is that they can pressurize the air and normal fans don't.
New fan testing methodology must include a steve hair test XD
I was worried that his hair was going to get sucked into the fan and rip a chunk out of his head.
4000 feet / minute = 45.5 miles / hour
@@iankirk3537 oh common one of these can push your car just off of wind power alone
wrr
When he put his hand there, i literally thought you better put your hair to work, and he did 😂😂😂
3:15 technician "hey did that fan just die"
Server: "well better start the 747 for a transatlantic flight"
It sounds very realistic
Like when takeoff
it moves to get a new fan
"I think these fans do 3000-3500 rpm"
It's actually 16300 rpm lol
Yeah as soon as he said that
I have one if these fans, just randomly
The 1U Supermicro servers we have are 23k RPM fans.
Zachary Sandberg oof
maybe he was thinking of the idle speed? ahahahah
"Can they hear this on the mic?"
I can hear it in my soul.
I still hear while the video is at 13:54 atm... Jesus
The really interesting part is just how loud those fans are. The video doesn't do it justice. My server is easily 90dBa+ when the fans are full speed. That's louder than a vacuum cleaner. Loud pipes on motorcycles are typically ~100dBa.
When Steve hair in front of fans looks like PC master race.
An awful subreddit
Peach r/ihavereddit
@@----------------------------- why
I was slightly worried for a second, that some hair might fall in front of the fans. ^_^
"We taught sand to do billions of operations per second..."
Great job, everybody.
CodgerFace Im gonna caligraphe that and then frame it
@@user-iy8gi7fj6c 26:03 Wish I could take credit, but I can't live this unintended lie anymore.
Hardware is amazing, but the fact that software can ever function at all is mind-blowing. Flipping billions of gates packed into a square inch or so billions of times per second in perfect synchrony where a single mistake will cascade and take the whole house of cards down... and doing it by using programming languages a dozen or more layers of abstraction away from the hardware, essentially like building a space shuttle with chopsticks from the moon. By all rights, NONE of it should work.
@@DustinRodriguez1_0 There is some redundancy there especially in corporate with ECC memory for example, but I get your point.
That's intense
The Jensen ghost was HILARIOUS!!!!!!!!!!!!!!!!!!!!!!!!!!
yeah and the he went poof was brilliant
actually laughed out loud
so not funny
Got a timestamp?
@@Batman-bh6vw 16.17
This is why they tell you to wear hearing protection when going into a pod in a data center. Imagine 100s of those things going off plus air handling units. Fun times.
Oh, the container of earplugs by the door is not just for OH&S compliance?
As much as I’d want one for home .. that decibel level though.
We have a couple rows of 44U racks with 38U or more fully occupied with 1U servers. They're a bit loud. I've grown to dislike 40mm fans.
@@NewmanOnGaming go buy 3 delta 120mm fans with their highest amp draw hahaha
@@mndlessdrwer That's an excellent way to get tinnitus. Done that and all I got was this constant white noise in my ears.
"that's pretty blowy"
-Tech Jesus
"That's pretty blowy"
-Bill Clinton
Z80
ll.q
f
3:15 "Control tower this is Epyc-128256 requesting for take-off"
Permission granted, Runway 2A, recommended engine adjustment to higher power
Mav: "Tower this is Maverick requesting a fly-by."
Tower: "No."
Mav: *Revs up to 16k rpm*
This surpasses Hot Air Balloon technology!
Supersonic Servers.
@@InternetEntity Use Quad intel 56 core xeons and you got your hot air part right there!
Homemade Lemonade Runway 2 A? you mean Runway 24 Right
one of the function of this server is also , transatlantic passenger service, fasten your seat belt
It sounds like jet engine @ 30.000RPM
I was literally crying with tears of laughter at 3:12, the reaction from Steve is pure gold all the while Wendell is laughing. This alone made my year :-)
the sever runs f1 2019 audio benchmark as soon as it is on
As some people added a real engine to racing arcade, here you just need 300W of fans :)
🚥🏎🏁 L😂L
Wendell was one of the last Tech you tubers I’ve come across. Become one of my favorites and regulars to watch. Paired with Tech Jesus, it’s something i look forward to all the time.
Angel Bunny You weren’t around in the raisethew0rld days then I assume. Tech 30
@@More_Row razetheworld. Those were the egdy days.
He's been around for a long time, but used to just be the top of a face behind a monitor.
@@putinstea True, i kinda enjoyed the dynamic though before shit hit the fan.
My opinion? Pistol poisoned the business, creating rifts between the partners.
*best CZcamsr ever. Where's Wendell?
when you see someone geek out like wendell you understand the passsion, and know that something awesome will come
Definetly need more collab with Wendell in future. It's so heart warming to see Wendell completely geek out about stuff like this.
Also, who wouldn't want an air raid siren to their server room?
I'm loving the server content on this channel. Especially unRAID and server CPU content. Keep it up Steve (and Wendell)
LOL that name and avatar :D Party Time Garth!
I can’t stop hearing “Cores for sale!” In my head. Love this dude!
I love Wendell's enthusiasm about this stuff.
My grandfather ran around screaming "the germans are comming!!" after this. sounds like air raid sirens apparently so well done you activated his ptsd
I was just about to say it sounded like an air raid siren spinning up lol.
The Germans are Coming to save you
hahahahahahahahahahah
IHRE PAPIERE BITTE!!!
@Imperfect Servant The Germans just can't stop trying to conquer Europe. I'm sure they'll find a way to lose again. Probably when the EU breaks up.
I don't know why but I love the jet engine start up sounds
I remember when I first powered up a workstation that was marketed as quiet only to be greeted by the sound of a jet engine powering up. Turned out it was tuned to run all fans full power on power on, only backing them off once the BIOS had initialized. After that it was actually very easy to live with, as long as you didn't turn it off or rebooted that is. Everyone in the building could hear when the Windows updates trickled in...
@@blahorgaslisk7763 I can Imagine when the windows updated itself. It's like "System reboot! system reboot! guys wheres my earmuff!"
"we taught sand to do billions of operations per second" - Wendell, 2019
This video kind of gives the "enterprise" space a bit of mystique, which to me is always a little funny. Truth is, 90% of mission critical corporate infrastructure is running on unpatched servers running unlicensed software which is at least 5 years past end-of-life. The challenges of enterprise IT are largely bureaucratic.
Kord Martin Going through an MS audit myself rn... FML
@@matthewducote8442 oof. my condolences. I'm managing a VoIP transition from a POTS/PBX solution and I'm arguing with 6 different vendors of who is supposed to do what. Very little of corporate IT has anything to do with technology.
Money and more money is the main thing. My boss bought himself new Audi rather than splash on upgrade of our data center. Because he has no knowledge about tech but wants to impress his golf buddies.
I get that from skulking around r/sysadmin and homelab and other areas on the internet where IT guys hangout. Lot of complaining about laypeople just thinking enterprise gear is like a toaster - 20 year old toasters still work if the toast can come out not on fire. Not realizing the extra work and risk and how computer tech advances yearly and 5 years is 2 decades in other industries.
@@hi_tech_reptiles I started years ago in prototyping company with Sandy Bridge high-end stuff. Than company folded and for some time, I did odd-jobs. But one retail company offered me sysadmin job for good money. And we are still running Sandy Bridge, but the most low-end spec. With network and security wiring done by Homer Simpson, pretty much.
"Duh, cable doesn't fit. Must use adaptor. I don't have suitable adaptor. Duh, I use two."
Ridiculous.
As he said the server room was traditionally 68 degrees I was like "holy shit people are getting backed when working in there I thought server rooms where traditionally at 20°".
Then I remembered they are from the US and are probably talking in freedom units...
yeah snowflakes and imperial units are a big reason of mistakes in science and technology.
Vierax i don’t think you understand how units work if you really think that, unless you’re referring to the mars orbiter
Server rooms are traditionally lower than 20C. How much exactly varies datacenter to datacenter, and sometimes even hall to hall.
Nothing free about the american gulag
@@master74200 I run mine at 24C - you don't gain anything from running them below 20C other than higher power bills for the cooling.
Remember: You NEED mechanical HDDs to run at about 35-45C for best longevity and the fans will spin up if the rear section heats up anyway (it will under heavy load)
It's not like the bad old days of systems pulling 3-400W idling. These things are drawing a couple hundred watts when twiddling their thumbs
3:41 I admit i replayed that sequence like 4 times LOL
That's it guys, this is were they peaked. No way they can best this moment xD
same, I started @3:42 tho
I would have loved to see Steve's face when Wendell showed up with this, after intentionally not getting linus involved to keep the server reasonable
you mean you can laser cut the top of the 2U chassis and get some RGB in there? :'(
Probably thinking; "Oh my god he is crazier than Linus, I like it!" Lol
2 subs with no picture to be honest, I would've preferred to see what kind of monstrosity Linus would have come up with. Too bad that Steve is a bit averse to "extreme overkill".
RGB and kids friendly videos? Hell no, thanks you Tech Jesus!
linus would drop it ofc
Enterprise-E! Sorry, I couldn't help myself!
Engage! *fans spin up and just bolts*
14:18 I see you guys are following The Verge's advice with that Swiss Army knife.
I've really enjoyed these collaboration videos between you guys. Great content!
Fiiiinallllyyyy been waiting for this video forever. Release it again please.
Been watching the content over at Level1Techs since the server build vid. Really cool content.
I found some specs for these fans. They are a dual ball bearing design, move air at 129.4 CFM, run at a rated 77 dB (A), use 51.6 watts of energy, and have a 16,300 RPM rating at + - 10%. These fans are pure insanity, but make a lot of sense in a server application like this.
This is amazing content, I could watch this all day
"this is fine, this aint so bad"
**AIRRAID SIREN STARTS UP**
My next PC for flight simulation for sure!!! I felt I was in a jet... Great video, guys. Thank you for uploading it!
This is refreshingly nice topic. No nonsense, no RGB, pure computing power.
It's always a good day with Wendell
17:06 his phone clicks gave me flashbacks of the old psp 3000 menu
the Spectre of Jensen 16:14 🤣🤣🤣🤣
the spectre, meltdown, fluffy bunny and all the other intel snafu's ;) lolz. im amazed how they got the 2x 64 core parts because if you are in the enterprise space, there is a waiting list. AMD just cant make enough of these parts.
I'm glad you do a video with Level1Techs. They are great guys.
Great content!
The hardware is extremely cool, but personally I found the conversation and questions aspect of it all to be just as interesting. I learnt a lot, and all of the demonstrations were also very cool to see as it's not something that is really ever shown off on enterprise hardware videos of the newer stuff.
Well done to both Steve + the Gamers Nexus Team & Wendell + Level1Techs Team for producing such great content for us to gaze at :)
> gets a ridiculously expensive server
> Steve immediately shoves his hair into it
Ridiculously expensive?
You've never worked in Enterprise.
I have a similar Haswell server in my living room. His config is pretty tame other than the drives, which compared to SSDs are still cheap.
Awesome video. The pinnacle of technology is always interesting, especially when they do it while mitigating all other factors.
Idk why but I absolutely love seeing how careful and aware you are with the equipment. Compared to how you treat the products you own.
That was the coolest thing i have seen so far... Epyc!
i'm having a hard time coming up with reasons to justify this purchase to my wife
Maybe remove the Wife from the equation? You'll have a new wife anyway when you get one of these, and it can blow just as good without any nagging hahaha.....
@@longnamedude3947 She won't *whine* a bit unlike the older wife!
AND you can do *hot* stuff with the new wife.
She'll make your *pullout game* quiet good too!
@@longnamedude3947 was that too much? XD
Don't need her to justify it, just buy it when she bitches turn it on you won't have to hear her.
After you got "this baby" there might not be a wife around anymore.
This is really awesome content, puts you a cut above the other techtubers. I really appreciate the holistic industry view you have - really top to bottom stuff; from screw and heatpipe manufacturing all the way to ultra high end server technology. You just don't see that anywhere else
I love seeing this data center/enterprise content. I work on software that is for DC hardware monitoring/alarming and it's always interesting to watch/listen to content that is directly related to the work I do.
Also, can confirm that DC's are loud af lol
2:09 *BEGINNING LAUNCH SEQUENCE...*
As someone that works on HPE servers like this every day, it greatly amused me seeing your reaction to how loud those fans can get lol.
Work on Dell servers every day, had same reaction :d
Wait till they hear an M1000e blade enclosure on startup.
Nice to see GN getting into the RC jet enthusiast scene I can not wait to see what you guys turn this bad boy into.
This was awesome! Wendel's laughs in the background as the fans ramp up is like a little devil admiring his miscreation! And finally, it all makes sense! Steve's long hair is a built-in anemometer and that's why he's so careful about fan and case testing!
I love how steve is showing hes lack of knowledge. He could have easly asked all the question before and then just report them to us. But props that he shows it this way
Wendell's evil laugh is now my new favorite thing in this world
Great video guys! What a treat it was to see this kind of Epicness!
Love the sound. I have mine ready soon. Excited to play with this bad boy very soon. :)
Wendell's stuff is cool, thanks for having him on. I subscribe to his channel but this is extra fun. Maybe you could describe how to get an iso image of windows and show how to create a VM to run it in. I don't quite understand that but would love to learn.
You can download a iso right from Microsoft.
@@norm1233 Thanks, I forgot that option when downloading for USB or CD
@@johngermain5146 don't forget to download some ram while their!! 😂😂
@@norm1233 Thanks, I'll need it. Dilbert says Mauve takes more memory.
@@johngermain5146 haha
For those of us who want something we are alittle more comfortable with for comparison, 4000 linear feet per minute is equal to 45.45 mph
It was the year 2019 when you commented this, imperial units where already obsolete since over 100 years, for very good reasons. Please use units that make more sense like m/s or km/h.
@@happygimp0 were*
Also m^3/s would be useful too.
@@xXDESTINYMBXx *cubic* meters/second??? How...
@@shanedoesyoutube8001 flow rate for water and other volumes
@@xXDESTINYMBXx oh... Ok
Oh wait, for fuk sake, there's CFM (cubic feet per minute) for case fan airflow, aaaargh damn it
Way out of my technical envelope. Still watching ♥️
FML how did i miss this vid, i love Wendell! you guys rock!
3:40 - Wendell, master of evil!!! Love him!
2:09 - when it started revving up it sounded like the silent hill alarm
Steven getting down in there with his hair blowing. He became the PCMR meme. It was majestic AF
This is best collab I've ever seen!
16:12 The ghost of Linus dropped something again.
Wait, wait, wait... So you're saying if I sign up for Linode, I get Level1Techs to install the Epyc server in my place?!
love that you used a inland sata ssd in that server they are great price to performance. very cool video over all. also the fans in that server use more power than my whole ryzen 1700x build. god and also am having the atom issues in my tablet :(. so much good info in one episode.
I am loving that jet engine sound.
4TB Ram great finaly a platform that can handle a modded Cities Skylines.
You I'll need a beefy cpu too skylines is cpu intensive
4TB of RAM isn't as much as it sounds like in the server space. I've run into more situations of memory limitations than compute limitations.
Can finally have more that 4 tabs open on Chrome
@@hihaveaniceday9386 guess 128 cores/256 threads at 3.3ghz aint beefy
@@muaries12 not beefy enough.......lol
"I settled for 2080ti's 'cause I'm cheap."
I wish I had kind of problem.
When you're purchasing 5-10 of these server units, and you want to do some cheap GPU compute, the 2080ti's are pretty cheap.
Awesome content!!!!............Would love to see you and Wendell do more collaboration.
Damn this is a great video, thanks for making it guys.
AMD had this running when they tested Navi. That’s why they said it is a quiet GPU.
When everyone has tinnitus and hearing loss...?
You guys are awesome together. I really Love the collaboration vids ✌️🇨🇦
Love those hot swap fans....I replaced some of mine with 0.36 amp fans vs the 0.86 ones in there. It was just fast enough not to trip the airflow alarm but reduced the volume immensely. But 7 amps....wow.
Everyone needs someone that looks at them the way that Wendell makes eyes at that server.
For 'square footage': in Europe at least and then especially in the somewhat older datacenters, in general the limitations per square foot are more on heat (BTU) and/or power supply (Amps). So if you either produce more heat, or require more amps, you will pay a premium, if at all possible. So in both ways this server make a lot of sense if you have a high density requirement (compute heavy).
is it wrong that i wanna edit and game on those?
Not it’s intended purpose but sure, why not
I can feel you
yes
Perfect for M$ Flight Simulator 2020. Now how to link fan speed to engine RPM.
@@Neojhun 😂 ultra realistic sim
The one time I enjoy the fan noise in the back ground and you shut it off.
dear god the speed of those fans. pull one out and you can hear as it spins down. and the jet engines, OH MY GOD! I CAN'T STOP WATCHING THE RAMP UPS AND DOWNS!
"Holy shit"
"Jesus christ!"
Never thought those two phrases uttered right after another by Tech Jesus himself
I knew as soon as it started it had to be Delta fans
Ahh the memories or should I say nightmares of the old days where we had those in our pc's, From that point on I made sure my future builds were based around being quieter!
I recently tried a Nidec V12E12BS1B5-07 on my workstation... was dissapointed because it's only 6.5k rpm and only draws 1.85a.
BUT it's good to have a choice range of 1k-6.5k rpm unlike consumer stuff.
Love working on servers and storage arrays - Was a Data Center Engineer for HPE for almost 4 years until they decided to outsource us to Unisys, July of 2018. Took our company cars, changed benefits (obviously) and attempted to merge Dell products to our portfolio. Yeah, I changed jobs this past summer as a result of. Regardless of such, for those that work in IT - If you have an opportunity to work in the data centers and/or work on server hardware, applications or OS, dont pass it up. Love that you guys are exploring the enterprise side of the industry.
I would love to see more of this as professional, just for learning and see the latest tech and different kind of servers
Watch level1, or serve the home.
"If you want to see more..." I mean, Steve and Wendell. WHAT IS NOT TO LIKE?
Yeah, those are Delta fans. We used to buy them for our overclocked Athlon Thunderbird and XP CPUs back in the day.
I also work with these enterprise servers. By the end of configuring ESXi and the VMs... my ears bleed :p
I love and have a respect for delta fans!!!!
Lost the itty bitty tip of a finger by accident to one. They are amazing fans.
Richard La Rose Wear cans, I’ve had neighbor cages run way louder than my gear with HP moonshot clusters 24/7. Other neighbors had Opteron DB clusters you could tell they liked 6pm daily backups.
I really enjoy these videos with Wendell. Checking out enterprise gear is actually kinda awesome. ;)
Enterprise hardware, yeah ! Interesting series GN, TY for the this.
"I wanna say it's 3000 or 3500". It's 16,300rpm, lol
That would explain the noise and the power requirement ;-)
The PFM0812HE-01BFY "only" draws 4.3A, those mofos in the vid 7A. Guess it must be the PFM0812HE-01, A01 just can'5 find the datasheet on that one.
We can confirm it is OVER 9000
Having boxes of server fans I was going to guess around 10-12k rpm based on the noise. 16-17k rpm...I can believe it.
@@DrakkarCalethiel In the spec sheet it says "Input current 5.16 A max , safety current on label 7 A" . Also "current at back pressure MAX 7.0 A" . I'm pretty sure it's the same fan .
"especially the fans" with reference to high failure parts... I have rarely had fans within a rack mount server fail. I have, however, had quite a large number of power supplies die on me. Now, when. You're dealing with blade servers where you have extreme static pressure fan modules, those do fail pretty easily.
I worked on the flight deck of aircraft carriers for a number of years and that sound reminded me of that. lol
I could sleep to the concrete of GN's lab right next to this server. Now THAT is a proper piece of hardware! Naturally it sounds like a jet engine with afterburners on ... and it purrs like a kitten ! (actually my R710 poweredge is louder)
What a beautiful machine you have in your lab gents !
3800 linear feet per minute is the same as poking one's hand out the window of a car going at 70 km/h or 44 mph.
Not super fast, but a fair bit more then most PC fans typically reach.
THE GHOST
Also got surprised when I got our first 1U blade server. They really make fans that loud in the server space. Also got a couple of HP tower servers that get loud but the blade server is -extremely- just loud.
Ahhh...that sound reminds me of my old workplace
"Don't say it... Don't.. Don't say it...
*BUT CAN IT RUN CRYSIS?* "
- Michael Scott
It's no longer can it run crysis, it's can it run RDR2?
RDR2 on AMD? Nah.
It can run multiple RDR2 instances.
I wanna see that video of HP engineers laughing at Intel.
Very interesting. Love this content. Good Work!
Please, more videos like this one!