Just for an exercise In perspective. In the late '80s, TRW moved three terabytes of data sto rage from South OC out of California. It required two full-size tractor-trailers!
Hugh, I've been doing cloud for 15 years and ran the largest AWS project at Microsoft in the past. I now run the cloud for a global consulting firm. I have multiple NAS at home for redundant storage of my RAW images and use Backblaze as my break the glass DR. It took forever to upload and it will take a while to download, but it is affordable, trustworthy, and very likely to outlive me.
If your dryer isn't performing well and it has been a long time since you have had your dryer vents cleaned out, please do it. It is maybe the biggest easily preventable fire hazard in the home now that we have LED christmas lights.
Hey Hugh - I agree with many of the principles you have in the video and will not wait for renders. About parallelizing storage... a couple things - if you keep a storage arrays on separate TB controllers, they are effectively independent. Not that you are going to saturate a TB link with an 8-bay HD (200-250MB per second x 8 disks peaks at less than a third of a TB channel), so I think it isn't a problem to be concerned with. Some Macs allow for this explicitly with four externally accessible TB controllers - certainly the MacStudios do, and the older iMac Pros did. I got a refurbed Mac Studio from Apple's site partially for the TB ports with their own controllers. (and after I got a bottom-of-the-line M1 Mac mini to compare it to my Top-of-the-line MacBook Pro...i9, Loaded to the gills, and the M1 MacMini rendered the same projects on the same external storage 75% faster. ugh. bye bye (intel) MacBook Pro. My slightly different approach: for editing current projects I am using 4x2TB NVMe blades in an OWC 4m2 (RAID 0) (peaks around 2.5 GB/s), because my next level of storage is 8- disk, RAID 4 (a bit faster at reads than RAID 5; better for editing video) (peaks around 2GB/s) array spread across two OWC Thunderbays for recent projects (we chatted about that a couple years back) which contains 84TB of usable storage, and then long term storage is in dual bare disk sets (RAID 0, in duplicate, one set offsite) (500MB/s) that I can drop into a toaster-like dual disk dock over USB if I need to bring up projects from 2016, for example. I find cloud storage waaaaaaaayyyyy too slow. Luv ya!
I have formatted two external USB4 4TB SSD with Raid 0 on my Macbook Pro m1 via two Thunderbolt4 ports and then had over 5000MBs read and write speed with an 8TB drive
The other advantage of a NAS is backing up to a second off-site NAS, to adhere to the rule of three copies of your data - two different media with one copy off-site. I keep a NAS at home and a NAS at the office. These mirror themselves in real time. That data is also backup to a paid cloud every night. But by being open to the internet, constant security vigilance from bad actors is big disadvantage.
I went the Synology route with a 5-bay DS1522+ currently filled with 4x20TB drives and a 10GbE network expansion card. 700MB/s read/write speeds and 54.5TB of storage space with single drive fault tolerance for $2500. Getting it all setup wasn't too hard, but also not a super simple plug-and-play solution and lacks TB3 connectivity. Would require a 10GbE home network to take advantage of those speeds too.
Are you planning to get a 10GbE home network, and plug in the 10gbit/s Ethernet adapter to your laptop/desktop to connect to it at full 10gbitps speed?
Hey Hugh, I like what you've laid out here. It's though provoking and well thought out as usual. I am a huge fan and a supporter. However..... I am a bit disappointed that you've published a story about storage and taken paid promotion from Sandisk without mentioning the "issues" with Sandisk SSDs that have affected many users and been widely reported. It feels like it's not consistent with your personal brand. Again, this isn't a gotcha or an attack. I'm just interested in your thought process here. peter annear
Hey, Peter. Thanks for the heads-up. I know absolutely nothing about Sandisk SSDs - I only had excellent experience for many years with my G-Speed Shuttle, which is why I was happy for them to be a sponsor. I guess: more diligence in the future. Again, thanks.
Sadly Amazon AWS Deep Archive is still at $1/TB/Month with very expensive retrieval, and Google Archive Storage seems to still be $1.2/TB/Month perhaps with a bit cheaper retrieval cost.. (they haven't lowered their prices in 5 years or so it looks like, inflation I guess so) I think Microsoft Azure cold backup storage might be around the $1/TB/Month too. To backup 50TB of data on those clouds, it's still I think way too expensive though, $600/year, compared with just backing up to 2 DAS, NAS or a bunch of External HDD.
Just for an exercise In perspective. In the late '80s, TRW moved three terabytes of data sto rage from South OC out of California. It required two full-size tractor-trailers!
Love it!
Hugh, I've been doing cloud for 15 years and ran the largest AWS project at Microsoft in the past. I now run the cloud for a global consulting firm. I have multiple NAS at home for redundant storage of my RAW images and use Backblaze as my break the glass DR. It took forever to upload and it will take a while to download, but it is affordable, trustworthy, and very likely to outlive me.
I use it as well for precisely that reason, Eric! Took a week or so for the initial upload!
If your dryer isn't performing well and it has been a long time since you have had your dryer vents cleaned out, please do it. It is maybe the biggest easily preventable fire hazard in the home now that we have LED christmas lights.
-Thank you!
You’re welcome! 😊🖖🏻
Hey Hugh - I agree with many of the principles you have in the video and will not wait for renders.
About parallelizing storage... a couple things - if you keep a storage arrays on separate TB controllers, they are effectively independent. Not that you are going to saturate a TB link with an 8-bay HD (200-250MB per second x 8 disks peaks at less than a third of a TB channel), so I think it isn't a problem to be concerned with. Some Macs allow for this explicitly with four externally accessible TB controllers - certainly the MacStudios do, and the older iMac Pros did. I got a refurbed Mac Studio from Apple's site partially for the TB ports with their own controllers. (and after I got a bottom-of-the-line M1 Mac mini to compare it to my Top-of-the-line MacBook Pro...i9, Loaded to the gills, and the M1 MacMini rendered the same projects on the same external storage 75% faster. ugh. bye bye (intel) MacBook Pro.
My slightly different approach: for editing current projects I am using 4x2TB NVMe blades in an OWC 4m2 (RAID 0) (peaks around 2.5 GB/s), because my next level of storage is 8- disk, RAID 4 (a bit faster at reads than RAID 5; better for editing video) (peaks around 2GB/s) array spread across two OWC Thunderbays for recent projects (we chatted about that a couple years back) which contains 84TB of usable storage, and then long term storage is in dual bare disk sets (RAID 0, in duplicate, one set offsite) (500MB/s) that I can drop into a toaster-like dual disk dock over USB if I need to bring up projects from 2016, for example. I find cloud storage waaaaaaaayyyyy too slow. Luv ya!
Great input, thanks for sharing
Really well done Hugh.
Thanks, Kevin!
I have formatted two external USB4 4TB SSD with Raid 0 on my Macbook Pro m1 via two Thunderbolt4 ports and then had over 5000MBs read and write speed with an 8TB drive
Love it!
The other advantage of a NAS is backing up to a second off-site NAS, to adhere to the rule of three copies of your data - two different media with one copy off-site.
I keep a NAS at home and a NAS at the office. These mirror themselves in real time. That data is also backup to a paid cloud every night. But by being open to the internet, constant security vigilance from bad actors is big disadvantage.
I went the Synology route with a 5-bay DS1522+ currently filled with 4x20TB drives and a 10GbE network expansion card. 700MB/s read/write speeds and 54.5TB of storage space with single drive fault tolerance for $2500. Getting it all setup wasn't too hard, but also not a super simple plug-and-play solution and lacks TB3 connectivity. Would require a 10GbE home network to take advantage of those speeds too.
Thanks for sharing! 😊🖖🏻
Are you planning to get a 10GbE home network, and plug in the 10gbit/s Ethernet adapter to your laptop/desktop to connect to it at full 10gbitps speed?
@@charbaxnot necessary for what we do, so no - at least, not yet. 😎
@@charbax if that question was to me: already have a 10GbE home network and 10GbE TB3 adapter for my laptop :)
if that question was not to me:
How do you find the noise level of your RAID systems?
Sufficiently loud that I power down my entire system when recording videos; otherwise, a non-issue when writing or editing.
@@3BMEP Thank you.
A master at work. Wants it right - in the same way that a long case clock could be described as..right! Great channel.
😊🙏🏻🖖🏻
Hey Hugh, I like what you've laid out here. It's though provoking and well thought out as usual. I am a huge fan and a supporter. However.....
I am a bit disappointed that you've published a story about storage and taken paid promotion from Sandisk without mentioning the "issues" with Sandisk SSDs that have affected many users and been widely reported. It feels like it's not consistent with your personal brand.
Again, this isn't a gotcha or an attack. I'm just interested in your thought process here.
peter annear
Hey, Peter. Thanks for the heads-up. I know absolutely nothing about Sandisk SSDs - I only had excellent experience for many years with my G-Speed Shuttle, which is why I was happy for them to be a sponsor. I guess: more diligence in the future. Again, thanks.
Can you do one about cloud storage? 50TB in the cloud will quickly bankrupt even a Leica owner 😂 so some strategy is needed
Backblaze!
Sadly Amazon AWS Deep Archive is still at $1/TB/Month with very expensive retrieval, and Google Archive Storage seems to still be $1.2/TB/Month perhaps with a bit cheaper retrieval cost.. (they haven't lowered their prices in 5 years or so it looks like, inflation I guess so) I think Microsoft Azure cold backup storage might be around the $1/TB/Month too. To backup 50TB of data on those clouds, it's still I think way too expensive though, $600/year, compared with just backing up to 2 DAS, NAS or a bunch of External HDD.
@@3BMEPUsed Backblaze for years, terrific company.
As a dog person, I've given up on wearing black 😆🐶
Yeah, I gave up caring about it. Even though she left us last year, she’s still here.