File Systems | Which One is the Best? ZFS, BTRFS, or EXT4
Vložit
- čas přidán 19. 01. 2020
- Let's go over File Systems in this video. We will determine which one is the best ZFS, BTRFS, and EXT4. Each one might work for you based on YOUR needs! .
►► Digital Downloads ➜ www.cttstore.com
►► Reddit ➜ / christitustech
►► Titus Tech Talk ➜ / titustechtalk
►► Twitch ➜ / christitustech - Věda a technologie
I remember when I started using GNU/Linux over 20 years ago and I couldn't find a defragmenting program in the whole distro. I actually thought that was a bad thing. Hilarious.
It isn't? I'm also new to linux...
@@SunnyGabe There are tools to do it, but I've never actually used them, except once or twice out of curiosity. Only to find out that there was no fragmentation to speak of, even though the filesystem was 95% full most of the time, for months and years. The filesystems are just that well designed. They will happily run under circumstances that would make Windows and FAT break down and cry.
@@SunnyGabe modern operating systems will do it routinely for you.
me too :)
@@SnowyRVulpix nitpicking here ...but it is not the operating system doing it it is the filesystem. e.g.
BTRFS which is very advanced can be mounted with compression algorithms and autodefrag. Or ZFS for instance can be mounted by Linux, BSD, Solaris which are different OSes and it works the same way on all of them.
Tldr:
ext4: basic functionality, little configuration, low overhead
btrfs: advanced features (checksums, compression, RAID), mostly for personal use (not as well tested as zfs)
zfs: lots of features, great reliability but complicated and resource intensive
ty
That was actually better than watching all these videos, thanks man. I already know what to use after reading short comment, lol.
How do you compress btrfs?
What about xfs?
@@AnotherSkyTV similar in principle to ext4 (traditional FS with journaling but without many of the btrfs/zfs features). Different performance characteristics (better in many scenarios, especially when scaling to larger filesystems on servers, but a few operations are more CPU intensive). Also has online expansion, defragmentation and metadata verification/repair, but the big disadvantage is that you can not shrink an xfs filesystem (not even when it's unmounted).
In (4 drive) RAID 10, if one drive is failed, there’s a 1-in-3 chance that a second drive failure will take out the whole array. RAID 6 doesn’t have this risk.
Also, when an array has more than 4 drives RAID 6 starts to make even more sense, as you can get more useable space out of the drives while still having two-point-failure redundancy.
This.
Many times in smaller servers or in home office environments (4-6 disks) RAID 10 makes sense, not to be confused with RAID 01 where if you end up losing a disk in a 6 disk array, you're essentially running RAID 0.
All the more to be careful with certain "Cheap RAID" cards that actually implement RAID 01 instead of RAID 10.
Extremely relevant comment about RAID 10!
I wish I could give you more than one like!
(I cannot comment about RAID 6 as I have never used it)
I was looking to set up a redundant system recently and was like...I don't know what raid10 is...went to look it up and that cynic in me was like...hell nah! If I lose two drives...I will probably loose two paired drives and lose everything. Thinking of lvm on top of raid1...when I have money again.
This is not fully true, the statistics are A LOT more complex than what you just wrote.
You have to take into account failure rates, load patterns and smaller data losses as well (not every data loss is due to a while disk failing).
3 Mirrors with 2 drives (6 disks total) have a much lower chance of another total drive failure when one drive failes compared to a 6 disk raid 6. Mostly due to load patterns of parity reconstruction.
However: 3 mirrors with 2 drives, does NOT have the ability to correct smaller errors after a disk failure.
If you want to correct small errors on reconstruct you NEED to go byond 1 parity or dual mirror.
That means: Tripple mirror or raid 6.
However:
2 Tripple mirrors has significant lower disk usability (33% usable) than Raid6 (66% usable)
2 Tripple mirrors also has significant higher power requirements (due to more redundancy disks)
I personally go for Raidz2(ZFS raid6), because I want to be able to correct errors when a disk is failed, disks ain't free and I want to stay somewhat power efficient.
But if I had a higher budget for power I would've gone for tripple mirrors, due to the lower chance of another disk failing on rebuild.
What I tried to explain:
There are multiple kinds of failures, all with their own chances and statistics of failures on rebuild have more variables.
ZFS was designed for this folks !
Thank Sun Microsystems for their contributions to computers and technology.
Can you make the 10 min rant on why ntfs is bad?
* Closed source
* Newest versions cannot be resized, except from when within windows
* less performance
* Need a defragment tool
@@Ju13n1s2e9 Ext XFS Btrfs also have a defrag tool. Every mechanical disk suffers from fragmentation if there is no free space to work healthy.
@@namelesske Yes, but in my experience NTFS is by far the one most prone to fragmentation and the most in need of defragmentation on a more frequent basis.
@@praetorxyn
I'm using with NTFS 3G driver and it's behaves just as any ext filesystem In terms of fragmentation. I choosed to use on Linux because of the cross platform compatibility, but the performance is really poor, but still better option to use instead of FAT32 or exfat. I'm not gambling with non journaling fs.
Krisztián Kovács What is this I don't even. To make the same partition accessible across both OS in a dual boot or something? I think there are at least experimental Ext drivers for Windows for that purpose I looked into a decade ago.
I was talking about NTFS on Windows itself dual booted with one of the ext on Linux. Windows was noticeably slower, and the more time that passed the slower Windows got compared to Linux, which means fragmentation. That was probably a decade ago though, it has to have been that long since I had anything but solid state drives in a workstation.
Thank you for the info about filesystems. However that rant on RAID 5 and 6 was entirely unwarranted, unsubstantiated, and I would say flat out wrong
I agree somewhat, especially because he only meantioned the case of 3 or 4 disks. What if I have 6 or 8 disks that I want as one array? What would he recommend then? I would actually like to here him explain that in proper detail
@@TheGodCraftMaster Totally agree. RAID 10/60 are more complicated and not more reliable in my opinion maybe better performance.
@@TheGodCraftMaster The answer to that depends on your workload/goals. TLDR: Raid 10 for 2-4 disks, Raid 6/60 for 6+ disks for cheap storage, Raid 10 for performance. Each technology has it's benefits and tradeoffs. Parity raid, "raid 5/z1,6/2,z3" has a large performance hit, but it's cheap for bulk storage. For large slow disks (>1tb) raid 5 should never be used, it's not safe. For hardware raid only use good controllers with a battery cache. In either case ECC memory is important. Calculating actual real world failure rates is complicated and more dangerous than many online calculators would have you believe. However raid 6 volumes are at least good to 10 disks or so, you could make a striped pool "raid 60" of multiple 10 disk sets to make some really large arrays. Raid 10 is the better general option, high performance and safe at scale. Raid 5 *can* be safe on ssd's because the rebuild time is so much lower, but that kind of irks me because if you are spending the money on ssds.... maybe just spend more and get raid 10 on them? But the reality is that it can make sense, a raid 5 on ssd is still going to blow a spinning set out of the water. Want some insane and totally unsafe storage for video editing or gaming? Raid 0 is the answer! You'll definitely loose the array at some point, but for fast working space it can be amazing.
Caveats to online reliability calculators: The manufacturers do tests on disks to estimate failure rates, however they can't *really* know what the real world outcomes will be years down the road and there can be large variance and sometimes firmware bugs. Data centers that buy thousands of disks have sometimes released statistics by model and brand and the results can be surprising, where you have much higher failure rate than expected with certain makes of certain drives, or even manufacturing runs of the same model. Add to that potential for raid controller bugs, other hardware bugs, etc and that should cause some significant skepticism. I've had clients loose raid 5 arrays due, not to defective disks, but buggy hardware raid that just decided to kick two disks off the array for no reason. Using onboard or low end controllers, especially ones without battery backed caches should be considered suicide. The bottom line is I don't *really* trust hardware raid, even good vendors tend to just release and forget about them, it's rare I have any issue with good ones, but with ZFS, or even just Linux MD raid, the implementation is known and used the world over, it's very known, stable, and very recoverable by reassembly tools and 3rd party data recovery services. Except for super high end NAS solutions, the reason hardware raid still exists is one reason only: Windows. Windows is actually working on their answer to ZFS, which is ReFS / StorageSpaces but nobody is really using that yet.
@@entelinI have 3 3tb disks and no space for a 4th. I wanted to put them in raid 5. What would you recommend instead of raid 5. It was recommended everywhere I looked for this kind of setup.
@@redcrafterlppa303 If these disks are SSD's then raid5, if not, then either get a 4th and do raid 10, or mirror just 2 of them. If you don't care about performance or a high degree of safety then you could do raid 5.
Safety may not always be the primary concern. Like for example if this array is specifically for holding backups, it's not really the end of the world if it dies, it's not the live data anyway. Some gamers will even make stripes of SSD's to get crazy performance with very little safety because what they care about is the performance any they don't contain data that can't be redownloaded.
Every raid level has it's place, but I can say for a client, I would never recommend raid 5 on spinning disks for use on their live data.
i use ext 4 and i am happy with it 🙂
I use btrfs........It's amazing. It's nice to see that you are happy with Ext4 😀
Internet Personal oy
UPDATE: Now I use ext4 because of some problems :)
yeah it is mature
@@bruno-zc1jo ?
When it comes to desktop, ext4 is good enough for pretty much everything. It's super performant, fairly robust, and really simple to setup.
For you with a reliable electricity net, but many people live in developing countries with many powerfails/week.
XFS is more performant than EXT4 especially with large files
@@bertnijhof5413 what do you reccomend then
@@bruhmode3041 If you have some IT experience, I recommend Ubuntu 19.10/20.04 + ZFS, I use it myself. It will protect you against corrupted files caused by crashes.
Else stick to ext4.
@@user-no3tu9kh3p I do not believe any American anymore :)
1:53 That monitor in the back synced up perfectly to his words XD
bro you got some exceptional observation skills.
EXT4 for me.. It's everywhere, it's been around for a long time, it's rock-solid stable.. What's not to like??
@@user-no3tu9kh3p No?
@@user-no3tu9kh3p UPS for this cowboy. Save my work and shut down in an orderly fashion. No more rolls of the dice when flashing BIOS either.
Forget the three letter acronyms,
name a filesystem BACONfs and everyone will want to run it
i don't think EVERYONE would use it. only those who generally buy vanity plates, and pick a computer based primarily on what color it is would do that...
I've used ZFS, but the fact that Btrfs is a native Linux filesystem made me switch to it. Spanshots are awesome but I regularly sync them to an external backup drive using btrbk utility.
the so called licensing "problem" should never be a reason to chose or not chose a filesystem. It's it's reliability, performance, caveats, features etc that drive a decision
If I were the US government, I’d make a one-time cash payment to Oracle to release ZFS to open source. Kinda like the interstate freeway, putting all our business data on a better file system could save us all from disaster. Then get our power grid and other critical systems on ZFS.
If I had Bezos money, I'd do it in a heartbeat.
Why? There is OpenZFS where you do not have to pay. No real advantage of ZFS comparing with OpenZFS.
@@catchnkill oh! So, would you recommend openzfs for use on laptop SSD by a newbie? How do I configure my disk into openzfs?
@@ekeretteekpo3004 No, I do not. It is a overkill. Btrfs is good enough. Btrfs's problems come from its advance functions like RAID5, 6, online expansion etc. Basic functions have been rock solid already. Facebook uses Btrfs extensively.
@@catchnkill oh! That helps. I'll try it out then. Thank you very much.
I hope for more detailed video comparing files systems in features and performance.
This is the first video I've seen of yours. Your home-grown background is awesome. It's clear you put plenty of effort into setting it up. Well done!
First time watching your videos but there's some misinformation here. RAID 6 doesn't use a hot spare. It's uses striping across the disk with dual parity instead of single parity that RAID 5 uses. Next in a 4 disk RAID 6 you will still gain 2X read speed but no gains in write speed. However, with most home users utilizing Gbit still, the performance benefits of RAID 10 will probably go unnoticed but with an increase in fault tolerance. Since you brought up the enterprise I'll address that as well. Before my current job (no longer dealing with storage), I used to mostly use RAID 50 or 60. This allows for a heavy increase in read speed (depending upon number of drives and RAID parity) which is what most enterprise environments require. If RAID 10 was ever used, it was used for environments that needed a large amount write speed (Big Data as an example). However, most large enterprises are moving to flash storage which would remove any reason to still use a RAID with such little fault tolerance.
Sweet. I asked you recently about ZFS and now there's a video talking about it. Thanks.
I like the dark subtle joke about ReiserFS at the end. You get my approval now.
I think raid 6 makes more sense to use with more drives. Like 6 drives or 8 drives.
@@Traumatree I use RAID6 with 6 x 6 TB drives and for me this is safe enough. _Half_ of the drives need to fail for me to lose data. For 7-12 drives ZFS offers Raidz3 (4+ drives need to fail for you to lose data). Any data that's really important to you (or your business) needs to be backed up anyway, so this is more about avoiding downtime. If you want performance and reliability and you're OK with buying twice as many drives, yes striped mirrors can be an excellent choice. If you're unlucky though, you can lose everything with only 2 failed drives (a drive and its mirror). RaidZ3 can be quite resilient especially with ~8 drives, but performance is quite bad compared to striped mirrors. There are tradeoffs here involving cost, reliability, and performance, and there still isn't one kind of setup that is best for every situation. You have to understand these tradeoffs & make the call for yourself.
@@Traumatree I've never lost data on a RAID6 array in ~15 years. Not one bit, and I've had several older drives fail on 3 RAID6 arrays with 6, 8, and 8 drives respectively. But this isn't a matter of "my chosen RAID level is correct and yours is wrong" ; my point is that we each have to make a choice based on budget & what's important to us. I could just as easily tell you that if you "really cared about your data", you'd put it on ZFS zpool whose VDEVs are RAIDZ3 of only 5 drives each (which would perform terribly), but I don't presume to tell you how to make performance/reliability/cost tradeoffs for your own data. To your concern about additional drives failing during reconstruction, for ZFS it's good precaution to "scrub" periodically (cron job) to guard against trouble during re-silvering, and for Linux md arrays my system checks arrays periodically as well. And of course, if you "truly value your data" you're not counting on _any_ single storage system to preserve it. You _must_ back it up.
P.J, so do you really think rebuilding a mirror incurs less risk than that of two additional spindles failing in a dual parity configuration?
Do you also believe this perceived reduction in risk is worth doubling the number of storage racks?
RAID 1+0 gives an effective capacity of 191 spindles per rack compared to 304 or 342 for RAID 6+0. (assuming 380 drives per 42U rack)
@@Traumatree you'd better tell NetApp. And EMC. They seem to disagree with you. Whilst I concur that the lower write speed on RAID6 increases the time exposed to risk (and therefore more likely to see a second disk fail) it's the read activity thar is doing the killing....and RAID6 is only writing the new disk, and reading all the others. On RAID10, the only disk being hammered with read is the one you are trying to mirror......guess which disk is most likely to fail? RAID6 is still in use for a good reason......they may be slow, but they are generally more highly available than RAID10 with disks upto 3TB. After that you need RAID100 if you need write performance, or RAID60 if you just need the availability. Everything has a use-case.
@@vonkruel
i use 4hdds in raid5, haven't lost data. however RAID is not a backup. You still need a backup and highly recommend it.
So raid5 has better usable space, and you got a backup to recover from, so what the heck is the problem mentioned by the youtuber? why would i use raid10 for less usable space? www.raid-calculator.com/
4x4tb
raid5 = 12tb usable space, and 1 hdd fault tolerance.
Speed gain: 3x read speed, no write speed gain
raid6 = 8 TB
usable space, and 2 hdd fault tolerance.
Speed gain: 2x read speed, no write speed gain
raid10 = 8tb usable space, and 1 hdd fault tolerance.
Speed gain: 4x read and 2x write speed gain
i rather have more usable space >_>:
I wouldn't use raid6 for 4hdds, because it's too costly in terms of getting less usable space. using raid6 for 8 hdds sounds more reasonable perhaps.
also using zfs is probably the best for a filesystem if you care about checksum auto correction and data integrity.
one problem with this video, he scares people away from raid5/6. he is right about raid 0, and i wouldn't recommend that. but raid5/6 isn't prime for btrfs since it's buggy as hell and isn't ready yet. but on ext4 and zfs raid5/6 works and is stable. you still need a backup though. haven't lost data using raid5 over many years while having backups. and yes a hdd had droped out of the raid or gone out of sync for one reason or another, but it supports 1 hdd fault tolerance. i replace the hdd and it rebuilds. and in worse case scenario, i just recover from my backup.
i recommend watching this video instead who's youtuber has a much better understanding on this topic
czcams.com/video/yAuEgepZG_8/video.html
czcams.com/video/pv9smNQ5fG0/video.html
this is why i am excited about the qnap qts hero, which is adding a zfs linux :D
5:48 *OMG* he's clueless about raid
He gives some really bad advice people. Please look this topic up yourself.
dan8t6 what is the thing he says which is incorrect?
@@neilbedwell7763 5:48 he says: "you nedd to not use *striping* like raid0, raid5, raid6"
but at 6:44 He's contradicting himself, by promoting raid 10. Which is also using *striping*
5:54 he says: "these are just really bad raids (...) just in general"
Notice the pause in the sentence. He couldn't come up with any arguments for his claim.
5:57 "like I never use them in business (...) and I never use them at here at the house, as I don't like them, at all"
Translation: I don't understand them, therefore they must be bad.
6:08 He's throwing insults at people/businesses using raid5/6 without giving any reasons, arguments, or proof why they're so bad.
6:21 Struggling to come up with something and giving up mid way through.
6:40 What he's saying only makes sense, if you only ever deal with very small raids of 3/4 disks.
In the business world, you're going to encounter raids arrays of 6 to 12 disks.
7:00 he says: "raid6, you have 4 disks. you know, 3 to establish raid5 and then that extra one is called a hot spare"
Again with the 4 disks lol. Anyways, his explanation of raid6 is 100% wrong. That is not how it works and the 4th disk would not called hot spare.
A hot spare is something that *can* be used in raid1, 3, 5, 6, 10, z1, z2, z3... but it is *not* needed in *any* raid.
7:12 More insults, no arguments.
7:32 (4 disks again) he says: "you can literally lose 2 disks and more than likely it's still gonna work"
Wrong. The chance of losing *all* data is exactly 50%. (2 drives failing in 4 disk raid 10 array)
50% is not the same as "more than likely"
TL;DR He proved he doesn't understand: raid6 at all, probability calculation and what to do with more than 4 disks.
dan8t6 thank you for the completeness
@@neilbedwell7763 No problem, glad you appreciate it.
@@dan8t669 incredibly comprehensive. I have an 8-bay RAID5 NAS, and yeah, RAID5 offers some serious benefits over RAID10
I'd never thought much about filesystems until I started noticing very big compile time differences in Windows and Linux but with the same hardware. Compiling the same source code in Windows would take minutes while doing it in Linux would take seconds.
Maybe the gcc or g++ compiler is running multiple threads on Linux and maybe on Windows, you are using the VC++ compiler or something and it is single threaded?
There could be multiple factors in this.
defender?
@@RickMyBalls It's been a while since I looked into the potential reasons, but my research at the time lead me to some sources which indicated that it was somehow related to NTFS. I've never had the need to use Windows since, so I've not really bothered looking into it again.
@@Maritims i only say that because another commenter on chris' windows defender video claimed that it was scanning everything as it was compiling and bogging it down. just windows being windows i guess. the only thing keeping me on it is gaming.
@@RickMyBalls I'll check that out just in case that's the real reason. This occurred when using Gradle and Maven. I bet Defender continuously scans the insides of JAR files. Speaking of gaming, Steam works really well on Linux now.
I use XFS, for moving files around on a partition it's faster than EXT4, and taking into consideration SSD storage because of the higher IOPS it's a no brainer
I use exFAT on my data HDD because I want it to be compatible with the different OSes I have installed. Do you know any other good file system that is compatible with both Windows and macOS?
I currently use ext4. I'm paying attention to bcachefs which is **supposed** be a next-gen fs when it's done. From what I've read it will be as fast as ext4 with the features of zfs
Do you format the whole drive to your chosen file system?
Example I am doing a manual partition but instead letting the installer do it.
I wanted to use BTRFS once, but then I got a bit scared and backed out of it and stayed with EXT4, should I give it a try? It's not too hard to set up, is it?
i use ubuntu 18.04 on BTRFS on root partition. Just select btrfs while installation.
Tip: Use TimeShift App to manage snapshot. You don't have to remember complex command. So, you never have to touch terminal for this FS.
Overall im pretty happy with this.
Great video, very informative. Thank you!
Hey, I have a question.
I started using Linux a few times now. First Linux Mint, then Zorin (but I only installed it), then Fedora Silverblue. Sadly all of them broke somehow. Linux Mint because of an update I think. I also tweaked certain files for gaming. I used a lot of PPA's. Later on I heard that PPA's could break the system when updating. I think using Flatpaks would be safer to use, right? Silverblue broke really fast, as I tweaked a file to stop the sound to go into suspend mode, as my speakers would crackle everytime a sound would play after 5 sec. of no sound. This broke Silverblue after an update.
I'm thinking of reinstalling ZorinOS as it seems to be beginner friendly and has really good support on its website.
Is there a way to use Linux without having to dig into a rabbit hole of filesystems, when using ext4? I would want to game a lot on it, so that I may replace it with Windows completely at a point.
question:
lets say ZFS is used for a git server and its UI aka Bitbucket. What recommendations do you give?
if it has licensing does it mean somehow the user has to pay Oracle?
I know this is an old video. But I wondering how well does running a Hypervisor running ZFS. Then installing windows in there with NTFS.
Does it make the underline storage any better?
Ext4 with Rsync snapshots has never let me down
DAMN, that was fast. Thanks Chris
Eh....
"Raid 6 is for idiots...."
"Raid 6 should be not exists...."
Chris: you should sleep one more night around these two sentences
In a 4 disk environment you might be true (but it's not right at all)....
..... but in an enterprise environment where we use much more than 4 disk in a raid group.
Another thing about Raid 10 vs Raid 6..... If you have 4 disks....
- using Raid 6: you can lose 2 disks of 4 ... NO matter which two
- using Raid 10: you can lose 2 disk of 4 but it DOES matter which two
Due to raid 10 (A-A | B-B) block organise...
And what about Raid Z and raid Z2..... alias Raid 5 and Raid 6 by ZFS....
I use raid z2 on an 8 drive array for over a year, lost 1 drive once, good to have because you can really stress on a HD rebuild on a 20TB array, may actually cause another failure.
@@wildmanjeff42 Yes! That's why Raid6/RaidZ2 better than Raid5/RaidZ... :D
That’s what I was thinking. If I lose a disk on RAID 6, I’m hurting but not scared yet. On RAID 10 I’d be scared, because sure I can survive a disk loss, _if it’s the right one._ At that point I might as well have no redundancy to speak of; I’ll be praying just as hard that the rebuild works before I have any other failures.
Hi Chris, great video's keep up the good work, I built a 20TB FreeNas Server and it Rocks
A friend of mine gifted me a Drobo which subsequently got bricked during a blackout. Do you have any advice for how I could migrate the data on the drives to a ZFS based system?
TThanks a lot for all your vids, they're very helpful. After switching to Fedora 31, my system(R5 2600X/32Gb) became way overkill for doing the stuff it did kind of OK before., even when running production software in VMs. I guess that means my main PC will last me at least a couple years longer than I thought. I'm grinning from ear to ear.
Hey, as you are a Freenas nut, do you happen to know where to find the early arm-compiled version of the original Freenas - I can only find later versions that work with arm7 and later i.e. rpi4 etc. Hoping to use Freenas with a rpi zero w hence the need for arm 6 compatability. thanks,
when you have 10 or 20 disks raid 6 makes a little more sense. or Raid Z3.
the equivalent to 6 is raid z2 or am i wrong?
Anything that is not NTFS.
FAT32 is good for USB-sticks and external hard drives (for cross-platform compatibility).
I've never tinkered too deep with Linux, so I've been happy with Ext4
What are the biggest problems with NTFS? Asking for a friend ;P
@@jpHasABadHandle 7777 :v
agreed. I actually have grown to appreciate that thumb drives come fat32. It's shite for anything else, but yeah. Simple. Gets the job done.
@@jpHasABadHandle NTFS has very bad compatibility with other devices.
For example, it doesn't work on Android.
On Mac computers you can read NTFS , but in order to write you need 3rd party software.
Some DVD players and digital cameras do not offer support for NTFS storage devices.
Also, NTFS is more prone to fragmentation than other file systems (at least it used to be).
I see.
whats best filesystem for gaming or will it matter?
I just bought a new RockPro64 and, after reviewing your recent videos, want to install Lubuntu with FreeNAS (ZFS) and Mandaro for my desktop. What installation steps should I follow to end up this? Or am I mixing together apples and oranges? Thank you sir.
Should I simply install each in the usual APT install manner?
I need to multiboot 3 operating systems : ubuntu20, fedora33 and centsOS8, I need snapshotting feature, out of ZFS and btrfs which one should i use ?
I use btrfs on my laptop, absolutely love the snapshots. I back up data that I actually care about, and use snapshots to restore my system if something breaks.
I use btrfs on my workstation due snapshot feature. I have JBOD configuration on three disks, it surely breaks if any of the disk has a fatal failure, but that's why there are backups.
There is openzfs, its not the same as zfs. Im waiting on the fence for now tho.
Hi! I use RAID 6 with a hardware RAID card. I've got 8 500GB drives and I need the extra space afforded over RAID 10, and I'm too broke to buy bigger drives. Is Raid 50 a better option?
Yes, RAID 10 is nice but its also costs $$$. If you're just using an array for backup data where performance is not really an issue then RAID 5 or 6 may make more sense cost wise. RAID 5 and 6 has it's place, depends on what you're using it for.
Hi, just planing a new install of KDE Neon, I would prefer using btrfs.. is there some guide ? Cant find anywhere...
what is the programming running on the screen behind you and below the matrix saver?
Hi!
Did I get you right, you use raid10 and every time you need to increase the pool you just add another raid10, raid10 on top of raid1?
I used ext4 and xfs, which are fast and rock solid. Tried btrfs but it had issues. zfs loses, performance wise.
Chris
Apple uses APFS, which one does it part of ?
ext4 or part of ZFS ?
Apple dropped ZFS.
would you use ext3 or ext4 for boot partition ??
You can also take snapshots under ext4, using LVM.
does btrfs raid10 work well with a bunch of different-sized disks?
What would be the ideal filesystem for a portable arch installation on a USB drive?
probably ext4, systems running off a usb are typically incredibly slow
Which file system would you recommend for formatting a usb stick?
Fat32
Is that star trek screen a de theme or just a screensaver?
What does "stable" mean tho? What happens if a file system is not stable, can it corrupt files from time to time or something?
"If you want to use ZFS, get yourself a lawyer"
one question, can btrfs be used on windows? and if it is, is the performance the same as in linux??
Running BTRFS RAID1 on a cluster at work,
The seamless compression works great by getting more out of our drives when storing CFD simulation data :)
I prefer XFS over Ext4 because if feels more smooth under heavy loads.
XFS had special hardware acceleration in the Sgi machines, in Linux they are not near as efficient but nowadays CPUs can handle the overhead.
Is xfs still supported? I had issues with xfs on ppc, about 15 yrs ago, though.
@@boxerfencer Yes, XFS is still in active development for Linux. It's also quite popular.
Today is the main file system promoted in Red Hat Linux systems.
I just have a Laptop. What FS is best for Daily Driver?
Fedora is going to use btrfs by default... I'm trying to figure out if I should keep using ext4
Great information!
I am noob and I am looking for a GUI database file system hybrid. Does this exists? So instead of making copies I can just make the same file exist in multiple folders. Please help.
What do u use for encrypted cloud storage?
would you recommend btrfs for gaming between windows and linux?
RAID was initially about protection against data loss through device failure. Performance enhancements should be considered a potential positive side effect, the level of which depends on the RAID level chosen, number of disks, type of disks and their performance individually.
RAID6 (roughly equivalent to ZFS RAIDZ2) protects your data availability in the case of dual drive failure. For smaller drives (say, under 1TB) this is indeed probably overkill, as long as you are able to resource a replacement drive and reintegrate into the array quickly enough. For drives at 1TB, it comes down to your data value as to whether you are comfortable with a single disk failure before data loss at the next failure. At 2TB, you run the risk of a second failure before you have managed to complete rebuild of the first failure.
Rebuilding an array puts huge strain on the remaining disks, that more than likely have all been working together in the array for the same amount of time.
So, RAID6 has a place in protecting large, bulk storage where performance is not as important as keeping the data available.
For desktops, this RAID level is overkill.
Also, performance for writing is low, due to having to calculate, write and then verify two lots of parity. Read performance should still be pretty good.
@@StephenBrown-gr3ix Thanks for the clarification! I don't know a lot about raid and this is very interesting
Can one user FreeNAS as a HA solution to clients? i.e having 2 FreeNAS hosts replicating and having clients connect to a cluster/virtual IP?
Can I save files from within a BTRFS to EXT4 storage space?
ext4 works fine for me. MicroSD from cameras (fat32): deleting files just doesn't work. I usually have to re-format the SD (gpart).
RAID 6 is needed for HDDs with capacities over 2 TB. Due to Unrecoverable Bit Errors, you’re more likely to go from a degraded array to total data loss as a result. If you’re going with large data pools, you need RAID 6 to add a layer of fault tolerance.
Also, you knock FAT32, but UEFI requires the boot filesystem to be that format regardless of OS.
ReiserFS != Reiser4. The latter trumps everything that would be on spinning rust while dealing with small files. I’ve never had any FS best R4 when applied to Portage in Gentoo.
No wonder UEFI requires FAT32. Guess what: Microsoft is part of the Unified EFI Forum. Same as with SD-Cards. Why do we have exFAT as a default FS on SDXC cards? Microsoft again. They are not part of the SD card association, but they *kindly offered* (more like: pushed them gently to use) exFAT for large flash memory devices.
Don't misinterpret it. UEFI requires *implementations* of it to support at least FAT32.
And it's perfectly fine FS for the purpose because ESP is effectively like read-only in daily operation.
@@MistorDi There is no misinterpretation, it's just fact. I did not say that FAT32 is a big problem here.
UEFI is just as bad as FAT32.
Linus Torvalds is entitled to his opinion about ZFS, but it's not like we all have to fall in line behind him. In fact ZFS on Linux is in a very good state & has been for years. Check out Proxmox VE if you're looking to build a NAS that also runs VMs and containers. ZFS + LXC + KVM is a powerful combo, and if you're already familiar with Linux (especially Debian) you'll feel right at home. It requires a little setup but you can also run Docker containers with a nice web UI to manage them.
Anyone know when it switched from butter to better?
Speaking of file systems, exFAT is clearly the way to go for an external drive you plan to share between Windows, Mac, and Linux am I right? exFAT fuse and exFAT utils being the necessary upgrades right?
ExFAT is considered a "tainted" package (MS patents) for many distributions, and it's on user land (not kernel land). The "universal" format is still FAT32. Some distributions (like Fedora) do not officially provide a fuse-exfat package (it's in RPM Fusion). Personally I have a FAT32 USB key (for sharing files with other people) and a BTRFS USB key (for my network).
If you have an external hard drive and don't care about tainted software, you can evaluate NTFS-3G (for both OSX and linux). But I strongly suggest you a BTRFS hard drive, a FAT32 USB key, and a shared folder with samba or sftp.
How about a tutorial transforming an arch system from ext4 to btrfs?
btrfs-convert.
What would be the option if I want to share data between my windows 7 or windows 10 install, with a linux dual boot? Since windows does not support EXT4 or BtrFS for as far as I know, those won't be any good. Also, the 4GB limit of FAT32 seems like a big downside. Which file systems remain that i could use for this shared partitions that have at least decent support on both Windows and Linux (I'm planning on installing Manjaro KDE).
If it's not a server or big data storage - NTFS will do the job. Reason: Windows and Linux can use it.
@@igorthelight It's not a server nor a data storage. It's just my home PC which i want to get to run linux, but keep windows for programs which I might need for school. Thanks for your advise.
I dual boot both Debian and Mint with Win 7. In Linux you can access files on the windows partitions, so if I need to move files back and forth I just do from Linux. (Unfortunately you can't access Linux partitions from Windows) Not sure if that helps but it saves making an additional partition.
Install btrfs or ext4 driver on windows and choose whatever for your hardrrive, mine is with btrfs
I mostly use btrfs for root on desktop systems. Mostly because they are developement/testing systems and easy rollback saves time. For servers with big database files I would use xfs.
Btw, snapshots do not remove the need for backups. They have different use case.
it has got extremely cool features. But it is the only Linux file system where I experience ultimately irreparable corruption on undamaged hardware. With ext2/3/4 I never saw anything like that.
No, but CoW filesystems do simplify backups.
Linux newb here. If I go to dual boot on a secondary empty drive , how do I decide a file system ? It does not give me an option to do so.
Stick withh ext4 cause it is easy to use
And what are all of those cloud server providers using? Most likely RAID 5/6 ???
It is cheaper and they have backups as well since disks ain't the only thing that can fail in a computing system. RAID is not a Backup
@@ArthursHD Do they actually make backups though. Most seem to not make backups and rely entirely on having redundant copies on a separate server / datacenter in case of failure.
@@gyroninjamodder In public cloud IaaS, backup is the customers responsibility. If you take a managed service that includes backup, it is clearly declared as a feature. As you state, cloud providers generally replicate your dataset to multiple nodes (in multiple geo-local datacenters less than 10ms apart) so in theory a crash-consistent copy of your data is always available in the event of hardware failure. If you pay extra, they will async replicate it to another region too! There is definitely some disk redundancy within hosts, however...I suspect each host carries several different arrays for different storage performance tiers.
I use ext4 for mi file system, but have a ZFS internal RAID that I love because I can use it in Linux, Mac and windows with amazing performance. Open ZFS has come a long way in the last months.
Regardless of how fast it is... Which file system is the safest, most reliable?
NTFS
ZFS far and away.
@@ChrisTitusTech Thank you. I use ext4 for my ultra fast NVMe SSD main drive, but I needed a reliable system for my secondary drives in each of my 2U rack mounts.
After about thirty minutes of research I have found that ZFS is not an option for me. For my particular situation I am left with EXT4 or XFS. For now I will stay with ext4 until I can do a little more research into data loss after a power outage on XFS. Otherwise, for large file transfers like video recording, XFS is faster and ideal.
@@ChrisTitusTech IF you follow the implementation instructions: ZFS really should have ECC memory.
Hi Chris, do you recommend a raid controller, or is software raid sufficient for a home server?
Definitely software RAID. If you buy a h/w RAID controller and it fails, you'll need to buy another of the same controller to access your data again. With software RAID you always know you can hook up your drives to a PC and access your data. Another point here is that _all_ RAID is actually software RAID. In a "hardware RAID" card that software is firmware on the card.
vonkruel thanks, I appreciate the comprehensive answer!
finally.. an explanation that should've been on the previous video..
watched the whole video.. but.. what about EXT4?
The real question for me is... which filesystem should i use for read-write support across all three : Win, Mac & Linux ?!
exfat
.... i think a raid video is in order
yes please. That went over my head completely.
Didn't Chris do something with LVM? That's Linux's built in software RAID, or at least one method to get such. But agreed, this is quite common - people thinking RAID is an answer to everything, and then sticking something like a 20TB RAID 5 in as if it will protect their data.
Raid is like a box of chocolates
Real Raid
Kinda Raid Motherboards
Software Raid
@@rwbimbie5854 And then there's the liquorice: RAIDZ and BTRFS RAID. Not to mention the Turkish delight - UnionFS.
@@PyCoder82 Similar, both LVM and MDADM are management interfaces to MD which is what's actually creating the RAID:
www.systutorials.com/docs/linux/man/7-lvmraid/
linux.die.net/man/8/mdadm
linux.die.net/man/4/md
The RAID rant is a bit silly.
I would not use a RAID 10 in a 20 disk setup for example.
That's where you use RAID 5 with one or two hot spares.
Actually, the silly person would be the one that uses a RAID 10 in this constellation.
Max F0sching I was thinking the same thing. I’m thinking he’s never run a 100 disk array and had to report to a CFO on infrastructure spend.
what is the star trek themed monitor screen behind you?
No one mentioned the "Hammer2" file system by "DragonFlyBSD". I wonder if it's more reliable than "Ext4" and"BTRFS"?
I would argue that the best file system comes down to use case. For a system drive, Windows should be NTFS, macOS needs to be HFS+/APFS, Linux works perfect with ext4. For USB drives I would go with FAT32 for capacities under 32GB because it's more compatible with multiple operating systems out of the box but, if it's larger than 32GB or any files need to be larger than 4GB, you should go with exFAT. ZFS is a great file system, but it's only real advantage is the ability to talk directly with the hardware making it much safer against data corruption. Something only a data center really needs, kinda pointless in a home environment.
I use two RAID 6's, which come into one giant volume. This has me extra fault tolerance for two drives on each RAID.
RAID60
I use BTRFS as my data partition for Windows (WinBTRFS) and Linux, I'm still waiting on a driver or FUSE module for MacOS. ZFS might also be possible, but it literally CANNOT be resized using gparted and I feel uncomfortable resizing partitions from the command line. Also, there's no documentation on using ZFS on a partition instead of a whole disk.
Edit: Decided to go with ext4, everything's good.
Can you upgrade from ext4 to btrfs?
what do you think about F2FS?
Is BTRFS SSD optimized?
I ended up loosing data on btrfs. Thankfully it was almost nothing. I installed an SSD on my family's pc so I don't need to use windows when im visiting my family, i installed fedora xfce on it , and it worked fine for like 2 months. Yesterday I noticed the system was really slow, I also tried to install a game from gog that I had on a flash drive, it was full of errors, i downloaded the game from gog and I tried tp compare the sha256 hash of the twp files but i was gettimg i/o errors, after updrading some packages and restarting, I could 't log in, i noticed input/output errors doing ls on /var/, I ended up running btrfs check --repair --force /dev/sdb3 and now everything is gonne. I've just installed xubuntu on ext4, I'm unsure if it will last, it seems there's some SSD issue. Since I will use it only for gaming and coding(pushing changes to remote repository), I think I can keep using this SSD.
"basically remove the need for backup"? Apparently you never experience HDD or or worst: RAID card failure... No file system will EVER remove the need for backup.
The scenario I work on is turning up at work to find the office and everything inside completely absent. If you can recover from that you have a backup.
Never lost data or work on a NTFS system. I had xfs for 2 weeks. An unplant poweroutage and an failed disk as a result was all I needed to loose 3TB of data.
Luckely for backups.
Between NTFS and EXT4, which is most likely to suffer data loss/corruption after a bad shutdown?
NTFS i bet
I force shutdown my systems when they hang a lot and i have never had ext4 corrupt something
I'm using FreeNAS with 3 disks in RaidZ1. The writes aren't that good, but reads are fine and it's a backup/media server anyway. That RaidZ1 is snapshot backed up to a single disk too. Backup disk is older (but larger) so I didn't include it with the other 3; also I like it being a separate managed disk backup.