Full Noctua Fans & New Motherboard - Server Re-Build
Vložit
- čas přidán 22. 04. 2021
- I was very graciously sent a replacement motherboard for my server to fix the problems with my remote management. But I also needed to replace all my aging, failing, fans in my server. So I decided it was time for a complete tear down and rebuild to put in the new motherboard. And I figured I may as well go full Noctua to try and get the server as quiet as possible.
If you want to try and help me figure out the fan speed settings from Linux, let me give you some info and warnings. This motherboard is a Supermicro X9DRD-iF(Manual: www.supermicro.com/wdl/ISO_Ex... ) and it does not behave like any other Supermicro X8,X9, or X10 board I've read about. I've tried everything in the following pages:
forums.servethehome.com/index...
support.siliconmechanics.com/...
blog.pcfe.net/hugo/posts/2018...
www.haphazard.io/blog/setting...
www.fengler.ca/articles/ipmi.html
I am pretty sure this board is different and needs either a completely different set of bytes to control the fans, or it isn't possible. I'm open to advice, but I've spent a lot of time on this already and really feel like I've exhausted all known options.
If you want to get some of these and support the channel you can use these links:
80mm Fans: amzn.to/3vfOgMY
120mm Industrial high pressure fans: amzn.to/32F42Vt
Playlists of more stuff like this:
Computers: • Computers
Linux: • linux
2010s: • 2010s
Other Links
CZcams: / akbkuku
Github: github.com/AkBKukU
Thingiverse: www.thingiverse.com/AkBKukU
Patreon: / akbkuku
Discord: / discord - Věda a technologie
There was a little bit of double audio at one point, this was a complex project to render and my internet has been terrible recently so it would have take 8hrs to render and re-upload to fix that. It wasn't too bad so I decided to leave it.
If you have suggestions about controlling the server fans, I HIGHLY recommend you read the description first.
/Looks at vape pen
/Nods
Oh SMC board fans are awful on optimal! We had a bunch of machines that shipped with 3 pin instead of 4 pin splitters for the mid case fans, and sounded like a rocket taking off. Turns out if you don't have the 4th pin it runs em at full tilt. The X9s we have are X9-DRL-iFs so not the same
Could just export audio only from your timeline and then use youtube's built in tools to swap the audio?
I'm beginning to suspect that's part of the reason it had that fan controller initially.
Noctua fans often come with little plug in cables that are basically a resistor for slowing the speed down, how about testing adding those in and playing with the fan speed settings of the main board? Maybe you can a better fan speed to noise level that way.
It's a nice server, Druaga2.
Literally, they even have a similar voice
I’m still convinced they are the same person. Druaga1 is the stoned version of Tech Tangents.
@@badboybruno547 i just thought they were siblings.
I don't have any useful input to add. I just enjoy your videos and appreciate your work on them
Use the resistive extensions that come with the noctua fans, and set them to 100% and the resistors should cut down the fan speed to what you're looking for which is about 50%.
“...but it should be possible...” my favorite start for a deep dive into problem solving for a little more convenience
This is absolutely fantastic Shelby, thank you for sharing each step during the process, even when things don't go as planned its extremely helpful and entertaining!
Thank you for an excellent demonstrative video. A good way to start a server project with modding, is to know that every problem you face is part of the good experience, as you brainstorm for solutions. So not to get frustrated and depressed during the process. Thank you again.
You're running BIOS version 2.14 if I'm correct (see 20:46). The latest is 3.3. I think it could be a firmware issue. It could(?) also be an outdated BMC firmware
Something that was my first thought too. Just update both, @Kirby. :)
I've never commented on any of your videos before, but first I want to say that I really enjoy your videos, they're both entertaining and informative. Thank you! Second - it's so worth spending those $200 on Noctua fans! They're THE BEST. I have nothing but Noctua fans in my computer (and a Noctua CPU cooler) and it's just whisper quiet. Also replaced the noisy fan in my JBOD harddrive enclosures with Noctua ditto's and - SO quiet.... :-) Money well spent! :-)
They have great quality control. Arctic fans can be great, but they have (at least in my opinion) only a 70% rate of working quiet, the other 30% have grinding noises etc.
These are my favorite type of videos on CZcams currently. Not sure why.
I've been working with Supermicro servers for many years now. they can be temperamental at times HOWEVER, when they work they work well and are rock solid. We've used them in production at our office for about 6 years now and they are still running strong. I run a pair at home as well and still years later am very happy with their performance.
awesome video, as always. I couldn't help notice the Atari Tempest 2000 box in the back 😎
Dude thank you I have the X9dri-LNF4+ and it has an IPMI, Sweeeeet!! Thank you uplloader for showing me this--Now I have another project to mess around with...wooot! Thanks man, great video man. -Dave
That's one heck of a teardown and rebuild.
You also can run "full speed" profile and use fan regulators to slow down the fans to the best balance of noise and temps .
Yes, IPMI is your friend...
Good job buddy
Loved the video! Hope you find a way to 50% some day!
I went through a similar process. I started with a Norco case and fairly quiet fans. I fairly regularly got HD temp warnings when doing parity checks or other operations that spun up all the drives (17 of them) for long periods of time. I also had an unrelated (but fairly common) issue with the Norco backplane, that would occasionally drop a drive out of my drive array. Reseating the drive would work, but I would then have to rebuild the drive. I finally gave up and bought a Supermicro case with stock fans and redundant power supplies. This computer is LOUD - I think there are 7 case fans, 2 CPU fans, and 2 power supply fans. The Supermicro motherboard I have is fairly old, so the fan control is very limited. The mode I'm using isn't quite full (which is truly unbearable - can't be in the same room), but I can still hear the server everywhere in the house. The nice thing is the internal temps never go above ambient, no matter how many drives are in use. It is also rock-solid, whereas the Norco case and backplane had some sort of problem every 2-3 months.
Quite a fun adventure! Would love to know more about how Resolve setup.
Since all the fans are on the same zone I would use the resistive connectors that came with the noctua fans to regulate the fans on the CPU's and the exhaust fans at the rear of the case to run slower and therefore quieter and then let the fans at the front that are cooling the hard drives to run full speed and that way they would be somewhat muffled as they are buried further inside of the case than the ones near the rear where you were hearing the most noise.
How goes the work with the Data General mini computer by the way? Been a while since we last heard about it.
I asked him last week and he just ignored it.
He responded to another comment asking about it on this video: "Not really, the space needed to actually work on the parts was larger than I expected and having somewhere I can leave obstructed for long periods while I service it is my current bottleneck."
@@tarstarkusz He probably can't be bothered to respond to every individual person who asks the same questions over and over again, so maybe lighten up.
@@orangeActiondotcom If he's getting the question a lot, he should address it in one of the videos. Just a 2 minute explanation as to what is going on. He has all the time to ask about patreon.
Dude, HDDs are totally fine at 40c, heck my Synology NAS aims for 35c to 40c on the HDDs. HDDs are safe up to around 50c.
Even with forced cooling in a secure DC (which I've just spent an evening in, doing the resistor fix for the C2000 bug... Yes I've been soldering) the drives are at 30-34deg. Forty ish? That's genuinely fine, really.
Also, screw Intel for cheaping out on the componentry for the c2000 chipset. I had plans tonight that absolutely did not involve a train from Leeds to London with a bloody soldering iron in my backpack!
Lol some of my drives in my CrapNAS(tm) go up to 50°C on a regular basis. Maybe i should fix my cooling a bit
50C is too much for HDDs. In the google studies, they found that with 45C, but also with very low temps (below 20C) the drives had higher failure rates. I would see 40C as OK, but on the high side.
@@TheRailroad99 You got a link to that? Everything I can find on Google (including Googles pulled result at the top of the page) says up to 50c is fine.
I use a simple hardware solution for my noisy server fans. You could get some Noctua low noise adapters, or what I do is use some 50ohm metal oxide fire proof resistors on the +12v input for each of the fans. The Noctua adapters are essentially the same thing but pre-made and plug and play. I just clip the +12v wire and solder resistors inline. You can use multiple resistors in parallel, more resistors in parallel give less resistance so faster fan speed and less heat dissipating through the resistor. I have used this on many loud fans, including main drive fans in my 24 bay case , even a loud PSU. I got a 100 pack of uxcell 100Pcs 51 Ohm Resistors on amazon for like 7 bucks shipped.
Honestly, keeping the drives at 40C is plenty fine. I would just keep the fans at 30% and you should be good, temps wise.
40 C is indeed on the high side.
However 35C is perfectly fine.
I agree. I have run drives that warm for years and have not had any excessive failures. I live in South Florida where it's difficult to keep the ambient temperature down in my own small server closet. Typically I see an ambient temp in the closet of 30C and the drives under load can be much warmer.
@@TheRailroad99 depends really, more modern HDDs (basically since 2010) are rated for up to 60°C, so 40 range should be fine
I have to compliment the hat in the end of the video, because the Compaq logo
What herculean journey you've had with this server project. It's as if the universe it's self didn't want you to have a server. lol
I believe that there are quiet server fans that would be quieter than the noctuas in your use case. Mark Furneaux made some videos about quieting down your server the way you should aim to do it.
He also showed that it's possible to use foam knee boards to replace the metal bracket and completely decouple the fans from the chassis, more than the rubber nipples
Just do like those of us who run SuperMicro server boards as our desktop. Toss a Corsair Commander (or two) and run your fans off that. Running an X11SPA-TF as my daily driver, with 6 Noctua PF12's for case intake / exhaust, and a pair of AF14's on the massive cooler for the FCLGA3647
Xeon I'm running. At full blast the PF12's are loud, they are industrial class fans after all. The downside of the Corsair Commander route is if your USB host crashes (don't get me started, misbehaving USB devices shouldn't cause the other ports on the same host to drop out), your fans all go to 100% speed. At least it won't overheat.
Nice work. You're one of the few people out there with a build video using the correct amount of thermal compound. I also have some PC build videos if you're interested.
Nice, more server content
Today I swapped out a couple of 2nd and 3rd gen intel CPUs and I absolutely freak out handling those sockets. I have been this way ever since I dropped my i7 4790K on the pins at a dark moment in time. :P
The simple solution to the multi zone thing is to run the attenuators on the fans you want to run slower. I every Noctua fan box you'll get a regular 6" extension, a plstter and what looks like a 3" extension. But using that 3" will slow that fans speed down. So run the HHD fans a full speed and the others have the attenuator attached to run slower.
Awesome. :D
How are you measuring the HDD temps - via SMART or via the onboard backplane probe?
This sounds like it would be a good use case for the ESP32-based multi-channel PWM fan controller that I'm building. It has 2 thermal probe inputs and I intend to have it be able to talk to lm-sensors in some way. I will post the schematics and software on GitHub when I'm done.
Most SuperMicro server motherboards expect to be controlling the fans, and will alarm if they aren't running at the speed it sets them to. The only way you can use an external fan controller is to not connect it to the motherboard, and you'll still need some minimal set of fans (including the CPU fans) that the motherboard controls to keep it happy.
I've done the same: built a pwm fan and RGB controller for my water-cooled server boards including temp readout of the loop. It was quite an effort, but both machines run cool and sufficiently quiet now.
@@dnwheeler to fix this you can fake the tachometer signal with a simple NE555 circuit.
@@TheRailroad99 you can also just set the fan speed alarm to 0 using ipmi tool. My Supermicro doesn't need all fans running that's for sure. I had to set the CPU speed alarm lower because my CPU fan was a naturally slower one.
How open is the front of the case? That seems like the biggest issue for cooling the hard drives. You may also want to try turning the fans around. Air in the front and out the back versus air in the back and out the front may make a difference.
Also does having the second fan on the CPU heatsinks make a difference? I don't know if airflow is additive.
You could run the fans that are supposed to be a bit quieter, on 7V (connecting the GND pin to +5V, iirc, and there are ready-made adapters for this) or using a resistor (for which ready-made adapters also exist) or using fans that run slower on 12V natively.
When you flipped the CPU cooler around to make more room for the cards, did you also flip the two attached fan so the direction of air flow was correct? If not, you would be pulling warm air back in and circulating it.
The Beard is new. Suits you man
haha nice touch at the end using the Dot Matrix Printer for your Patreon Subs thats neat
I know it's not the preferred method, but can't you ran the hard drive fans using something like a molex/sata adater and that Noctua low noise thing?
Did your noctua fans come with low-noise adapters? Could you hook up just the cpu fans to the adapters to slow doen the cpu fans and leave the other running faster?
The big industrial fans are the ones causing the vast majority of the noise. But they cannot be used on the low noise adapters. They draw too much power and the low noise adapters are just resistors and will overheat.
@@TechTangents If having them run at a lower maximum speed in in principle an option for you, you can cut open the low noise adapters and replace the resistors with ones which have a suitable resistance and power rating. (P = U_drop^2 / R is your friend of course :) )
To be on the safe side you can mount the resistors in front of the fans, so they will actually receive air cooling.
That said, I'd probably be not thrilled either to lower the maximum speed of the fans in case the extra air flow is actually needed.
If you're interested in discussing creative (and sensible) add-on solutions, let me know! ;)
@@TechTangents I'm not sure the industrial line is set up to run quiet like the normal ones.
@@Breakfast_of_Champions they are not. They are optimized to get the best airflow with acceptable (but not great) noise.
oh yes!
Is there a cheap way to get lots of discs working in a machine? I have checked several sata cards and they aren't expensive. I have 6*1TB 6*2TB just laying around.
Looked like it was a bit of a project compared to a OEM system. But would have been good to get it wrapped up. I would have just ditched the rear CPU fans haha. Most Servers I've worked on don't typically have a dual CPU fan setup anyway, butttt more fans is more cooling power. Do you use any SSD Caching on your RAID setup?
How about using a speed reducer resistor cable?
kinda would be interested in a ducted 200mm fan where those 80mms are
theres definitely space, ducting would be tight, theres no real mounting for it, but the airflow and volume would be the definite improvements over literally anything else you could put in there
you could use fan adapter cables which have resistors in them to reduce the speed with the motherboard set at 100%
I have the exact same NorCo Server Case. LOL! I have my own technical issues with my setup, but, nothing i can’t solve with: a different 10gig NIC, an OS Driver update with ESXI, and a new fan...
Why don’t use LSI 92xx-16i to replace 2 raid cards setup, at least you can have a slot to spare?
Couldn't you use the adapters that came with your Noctua fans to silent the fans? I know I put one of them on each of my fans, set them to 100% fan speed, and it definitely helps.
Those temps are pretty good IMO, and I'd not worry about it. But is there a reason why you don't want to just get a fan controller and set the speeds manually outside of IPMI? You could also just do that for the HDD fans, effectively created a second, manual zone.
Do you have to install the latest version of ESXI before installing CentOS Latest version or reach out to Morton of My Playhouse in Honns Denmark.
Hey! Wich case is that on right? And wich IcyDock Model is in that 5,25 bay? :)
I think the core of your heat vs. noise issue is that commercial server rooms are, in my limited experience, kept at a temp that I would call "rather cold" (although someone like Big Clive would call "rather warm" since he likes his home at 5 C in the winter). But, in a private home, there's no real way to set up that kind of HVAC facility.
And the noise level after the upgrades...yeah, that's WAY better, but, still too loud to have right next to you for video editing.
Now, did you get a lift table to you're not trying to lift this sort of thing by hand and risking dropping in when the equipment to prevent that exists and isn't all that expensive?
Did Silverstone buy or copy Norco? All their rackmount NAS stuff is the same, but newer.
Honest question: Does it feel better knowing that problems convert to more content?
We really need to be speaking in temps above ambient because if you like it hot in your home, say above 25°C, or you like it frigid like I do, say under 15°C, those 10° along with changes in air density will change your temps.
I run everything off my Synology’s Xeon but I’m not encoding video on it... I use my M1 Airbook which is a big step up from my old 4th gen i5 laptop.
Have you looked at possibly a licencing issue with the IPMI? I vaguely remember something about licensing "advanced" licensing features?
Your PCH Temp is near 50 degrees, have you changed the Thermal Paste of it?
What model of heatsink is that
So in my circles we call this Type 2 Fun: Not fun when you're doing it, but fond memories afterwards (ok, maybe a while afterwards)
You did the same misstake as me to buy those extreme 120/140mm noctua fans(the black ones) that have insane airflow but also quite alot of noise.
Pwm normal version would be alot quieter, but ofcoarse not as much cooling, but might have been enough still since you could just run them at 100% at all time🤔
Does that Supermicro have the spy chip? /s
My server is out of the way and I have it running Proxmox with GPU pass-through to one of the VMs. Then I can use Moonlight to remote into the GPU VM machine, that's slick enough for me.
For the fans, is there perhaps a USB fan controller that works with Linux? Otherwise Github Putnam Superfans?
Oh, so you removed the LTO drive?
What were your experiences?
Would have loved to see the video about tape backups, but I'm sure you have enough projects anyway. At least until the 24h/day limit gets removed ;)
What happened to the HP Workstation?
So, my eyes were stuck on the Lego boxes on your shelf. Just kinda sorta wondering some details there! :)
What's the model name for the server chassis? Any link?
Why is 40 degrees Celsius too much for a harddrive?
So the fans spinning all the way down and then surging up is a common known problem with noctua fans with supermicro boards. The voltage dips so low on the lowest RPM that the board thinks they're off. There's a hack with a manual IPMI command to tell it what the minimum voltage level is.
The IPMI tool to send fan speed commands works great on my board (I think I have an X10). I use a python script I wrote to probe my disk temps and CPU temps on freebsd
Water cool that bad boy
Cool
OpenBMC ? (ipmi replacement soft/firmware?)
What are the model names of the keyboard and mouse ?
SuperMicro hardware not doing what it should be? No way. I've got a Dell 720xd, so I'm in my own hell. Cheers!
You mention your HDDs are slightly overheated at 35 - 40 °C? To me that’s normal? I pretty much sleep next to my 2U server (~150 W) due to cable length limits so I’ve set the fans to target 40 - 45 °C alarming at 50 °C which gets me around 45 dBA @ 30 cm @ 25 °C. My drives are rated for 60 °C (other brands are higher) with a 5-year warranty so it hasn’t worried me too much until you said that. Is there some major correlation that I don’t know about and I should be lowering the target temperature?
I tried to read up on datacenter HDD tempture/lifespan correlations and every single one of them I saw was talking about temps below 40C. So I figure it's probably bad to be above that.
@@TechTangents The server i have setup with a few old western digital drives have pretty decent airflow and are around 30-35c with a fan directly next to them.
@@TechTangents Not only that, HDDs in general last the longest if they are being kept at the same temperature for their entire lifetime. I.E. if you've always kept them at 35/40C, you'll want to keep them at temperature while running to get the most out of them.
@@thomasvnl Exactly, thermal cycling is what you want to avoid. At very low ambient temps it's reasonable even to turn the fans off.
I have a 1U 4-slot rack chassis running Linux RAID with 4TB Seagate IronWolf NAS drives that have been spinning for about 2.5 years. Currently 41C. That’s around 106F, which is a warm day in Phoenix. I don’t really see that as a problem for a HDD. :-)
What is the correct name of that device where you put four 2.5 inch drives in one 5 inch bay, featured in the desktop PC in this video?
HDD cage. But they are listed under various names. It's a pain to find cheap ones (but they exist!)
Maybe using an Arduino Nano to do that job would be an awesome solution for a video project about the fans' speed. Rocket science to open a soda can? Of course, but it'll be fun.
Kde Plasma spotted :D
Hi Shelby. What case is that on the right of the screen at the end, it's really nice?
That is a Phanteks Enthoo Pro which is a really nice case and a bargin for the price if you ask me. I'm really happy with it and I don't think I have any complaints!
Have you updated the BIOS, I had issues with a poweredge server where the remote management wouldn't work with the software, after the update it all played nice
I have intentionally not updated the BIOS because they have a product key tied to it that I don't have access to.
I didn't realise they had a license tied to the bios update, I know with the Dell poweredge servers the key is a physical dongle plugged onto the motherboard.
Genuine question here. Not a troll. As Noctua fans have a good throughput of air, why do you need two for CPU fans?
What is editing rig specs?
The reason that those profiles are so "stupid" is because those boards are supposed to be in a temp controlled environment server room. It's weird, but that's why. I used to have a board that wanted to know at what sea level/altitude it was being used at... Yeah. Who knows why the customer needed that, but they did and SuperMicro was more than happy to make it for them.
If it were me, I'd forget trying to get the board to control the fans. I'd concentrate on writing a script to pull the CPU and drive temps out, and send them to an arduino, which would be wired to control each fan individually, and program it to speed up the fans on each CPU or the hard drives as necessary.
cent os? didn't it get discontinued by Red Hat?
server board has mini computer controlling fans, no PWM curve? nice. :P
Just use bitfenix fans instead! they are just as good as noctua but cost fraction of the price and deserve more recognition!
About the sound level recognition on android... well you are actually wrong xD As long as the device manufacturer calibrated it correctly it should be just as good as iPhone, the level is not recognized in the app but on the drivers level.
I'm not nocking noctua fans but have you ever tried arctic fans? ive always used them and there quiet and decent.
Awesome! You probably get this question a lot, but any updates on the Data Generals?
Not really, the space needed to actually work on the parts was larger than I expected and having somewhere I can leave obstructed for long periods while I service it is my current bottleneck.
I've been wondering about it for awhile too, good to know he hasn't forgot about it though. Good luck and keep up the great work, love your videos!!
Could you get a fan controller and hook that to all the fans? Then hook the controller up to the fan header. Set the internal fan speeds to 100% and then just control each fan individually. Might work.
I have an HP Server PSU; I replaced the fans on it with Noctua. And they don't appear to run full speed. It ends up overheating, so I'm not sure what to do about that either... :\ Sorry for your struggle. I've been there...
The fan you replaced probably had a higher CFM, server fans push a lot of air, that's much of the reason they are noisy, not just their design. You should really, really avoid having your PSU overheat, give it a fan with more CFM
I used to have the infamous Intel Skulltrail board. There was a 10 degrees C difference between CPU 1 and 2 no matter what I did and it drove me nuts! I saw 37 and 42 degrees here and had needles in my spine for a moment :D Anyway, aren't those old industrial fans well serviceable? You've probably tried, but I'm just asking. Noctua is cool.
You should make a video about it. There are not too many good ones on youtube, and it's a legendary peace of hardware!
@@TheRailroad99 I was so glad to get rid of it for a reasonable price. It was almost impossible to sell the damn thing. Nobody wanted it. It went at the moment I added a used 650Ti to it. That was 5 years ago. So unfortunately no can do.
show output of this command: ipmitool sensor list. That will show the settings for each individual fan. You can change the fan speed with ipmitool one by one. ipmitool sensor thresh upper value value value
There's a utility called ipmitool, which makes it possible to control fan-parameters from the OS. The specific magic bytes are different for all ipmi vendors afaik, but I got a script running to control the fans based on temp-sensors on my Dell PowerEdge this way:
"--> enable dynamic fan control" /usr/local/bin/ipmitool -I lanplus -H $IDRACIP -U $IDRACUSER -P $IDRACPASSWORD raw 0x30 0x30 0x01 0x01
"--> disable dynamic fan control" /usr/local/bin/ipmitool -I lanplus -H $IDRACIP -U $IDRACUSER -P $IDRACPASSWORD raw 0x30 0x30 0x01 0x00
"--> set static fan speed" /usr/local/bin/ipmitool -I lanplus -H $IDRACIP -U $IDRACUSER -P $IDRACPASSWORD raw 0x30 0x30 0x02 0xff $STATICSPEEDBASE16
Is this why Everquest server maintenance always ran long?
actually, I'm going to be faced with a similar situation for fan control when I build my system around an X10DRi-F. what about hooking a maxim MAX31790 to the i2c bus on the board, with a little breakout? the linux kernel has an i2c driver for controlling fans hooked up to a MAX31790.
another more off the shelf option is something like the Corsair Commander Pro - which, according to phoronix, has support natively in linux as of kernel 5.9
honestly for the fans i would have assumed you would have 3d printed a mounting system?
I use Molykote 111 as a lubricant for parts such as gears and fans and I do get outstanding results with much less noise, try that out as well!
I'm using a SM X11SSL with unraid as host OS. You're probably having the same issue now that I have where the IPMI expects your fans to be at a higher RPM and therefore considers them "failed" so it keeps them close to 100%. This can be fixed with IPMITools as well but I got that working only once for about 3 months and then never again so I just gave up and started to use the fan controlller that is built into my case and set up emails for temperature warnings.
You could use a NE555 chip to fake the tachometer signal.