ZFS Finally Has CoW Reflink Support on Linux!
Vložit
- čas přidán 21. 08. 2024
- For a long time, ZFS has been missing support for reflinks found in other filesystems like Btrfs and XFS.
That all changes now! OpenZFS 2.2.0 is introducing CoW reflinks to ZFS.
In this video, I talk about what this means, as well as some of the limitations when it comes to links across datasets.
One of the original issues on GitHub for reflink support:
github.com/ope...
Pull request bringing in block cloning:
github.com/ope...
Pull request bringing the changes to Linux:
github.com/ope...
ZFS releases:
github.com/ope...
Let me know if you're interested in this being a recurring type of video with a bi-weekly/monthly open-source news show or something along those lines!
Produced by Tony Tascioglu
tonytascioglu.com
If you enjoyed this video or learned something new, please consider subscribing! - Věda a technologie
Hey! zfs 2.2.0 is now out of rc and is released!
If you're upgrading from an older version of zfs, make sure you enable the new block cloning feature to take advantage of reflinks!
You can use `zpool get all` to check the features, then `zpool set` to enable it, or, you can just use `zpool upgrade -a` to enable all new features!
(There's other cool stuff this release like blake3 hashing, so I would just use zpool upgrade!
Thanks for your in-depth review! Great info!
NOTE:
THERE IS A BUG IN ZFS 2.2.0 AND 2.2.1 THAT CAN CAUSE IT TO EAT YOUR DATA!
UPGRADE TO 2.2.2 AS SOON AS POSSIBLE!
Very interesting. I wonder if Veeam will support ZFS ref links for vast clone (instant synthetic full creation by using ref links and linking to the blocks rather than having to write a new full)
While I'm not sure if Veeam supports reflinks, isn't that providing a very similar functionality to what could be achieved using snapshots on ZFS? ZFS snapshots already use CoW so can be created instantly and without extra usage until files start ot be written to, and are easy to restore from if needed.
@@TonyTascioglu this is in regards to veeam backups. Using a forward incremental policy what happens is - the first backup grabs the source VM disk and backs it up in its entirety - then every backup throughout the week after that is simply incremental - whatever changed blocks since the last one. However in the event that there was a loss or corruption on the production storage and the vm needed to be restored from back up you’d have to compile a new “full” backup from the original full backup + all the incrementals taken after which is time consuming and uses extra storage because it has to rewrite a new full out of all that. Reflinks and veeam fast clone allows an instant new “synthetic full” back up by just pointing to all of the already written blocks from the original plus the incrementals and can restore much faster
What's up with the artefacts in this video? It doesn't seem to be a green screen.
Hm, that is strange. It is indeed not a greenscreen. It's likely what happened to the noise/grain when it got heavily compressed.
It could also be a number of factors from my questionable job at tone mapping and colour correction, or an issue with my kdenlive workflow. I'll look through my original upload and see if it this noticable.
I think it’s the pure color LEDs reflecting off the wall and overdriving individual color pixels in the camera sensor
Now if only encrypted ZFS protected metadata... and wasn't slow.
The last time I looked at what was and wasn't encrypted was a few years ago, but I was under the impression most of the important metadata (file names, directories, structure) was all encrypted. It's just some ZFS pool and dataset values that weren't, but those also didn't seem like a major problem to me at the time.
Not sure if it is still accurate, but I had this listed on my old notes: blog.heckel.io/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/
Oh nice, that's one of the things that ties me to BTRFS. I could never use a filesystem without it again (though if it had online deduplication and a way to specify for a file that it shouldn't share space with others, I could use that instead) Unfortunately I probably won't swap to OpenZFS, as it has an awful CoC whereas BTRFS sensibly has none. And really, the only things ZFS has over BTRFS for me is higher compression support (I mount my BTRFS partitions with compress-force=zstd:15, but ZFS supports up to either level 19 or 22), and purportedly being more stable, as well as having support for other operating systems /kernels.
Interesting how at 0:40 you say "zed eff ess" but then less then 10 seconds later call it "zee eff ess" :|
Oh I haven't heard of that non-guaranteed CoW with BTRFS raid5&6. You're still not supposed to use those anyways. Does it manifest too for BTRFS RAID 10?? That's the far more important question, I think.
Wow I didn't realize a different ioctl was used for cp's different --reflink values. I alias cp to `cp --reflink=auto` anyways, only having `cp --reflink=always` for some specific snapshotting functions where I specifically want it to fail if it can't reflink. Wonder if someone's gonna patch the kernel (and mainline the patches) to make FICLONE work across ZFS datasets. ioctls are somewhat inherently unstable anyways, right? So changing that behavior shouldn't be that big of a deal, and shouldn't be "breaking userspace" or whatever.
BTRFS is so flexible and has so many features. It just kinda beats everything.
Good note! Some initial thoughts:
- CoC? You're referring to the code of conduct? I didn't realize that was an issue (though I may be mistaken), I'll have to look into it
- For higher compression, probably wouldn't want to go above 15 anyway, simply due to the performance impact. I usually benchmark my device and tune it so it's at the line between being CPU and IO bottlenecked. (and you can use the negative levels like -2 on an NVMe SSD for example)
- zed vs zee - this way, both sides can get mad at me! JK, I didn't even notice that when filming, I guess I use either.
- for btrfs raid 5/6, you probably want to avoid it, given that even the developers warn against it (though I'm not sure if there's issues other than the write-hole one that is frequently discussed)
- I didn't realize ioctl was different between auto and always either until I read the discussion on GitHub! I always thought always was "reflink or fail" whereas auto was "reflink if you can, normal if you can't", but turns out there's more to it than that. From the discussion it looked like they weren't too enthusiastic to go upstream as it's a lot of extra work.
- btrfs is indeed pretty flexible. I end up using it more frequently technically, my general rule (as bad as it may be) is that for individual (or as most 2) devices, I use btrfs, beyond that, if I want to do a raidz1 equivalent with 3 or more drives, I use ZFS. Especially given that there's extra hurdles if you want your boot partition to be ZFS...