My Complete Photography Data Workflow
Vložit
- čas přidán 13. 09. 2024
- Want to learn more about photography and business? Head over to my Patreon page to see all the different things I offer! / travisharris
** UPDATE.. here is a new video with my new system.. • A FULL RUNDOWN ON MY N... **
If you are a photographer, or anyone else that processes very important data... this educational / workshop video on my data workflow maybe inspiring.
Join me in this complete, in-depth look at the data process starting at the point of capture, all the way to archive and everything in between.
Without doubt this is the most important thing inside of my business, and I am excited to share this with all of you. It has been comprised of over 6 years of practice, processing over 1-Million RAW images to date!
If you enjoy videos like this, please subscribe to my channel as this motivates me to continue to make more content in the future.
Thanks for watching!
To see my extended thoughts on this topic, please visit my blog post on Data Workflow here: travisharris.co...
My work: www.travisharri...
Facebook: / southfloridaweddingpho...
Instagram: / travis_harris_photography
Special thanks to: Andrew Applepie for some of the music used in this video. You can checkout his work here: www.andrewapple...
Here are some links to the products / programs referenced in my video:
Photo Mechanic: www.camerabits....
Carbon Copy Cloner: bombich.com
All hard drives used: www.g-technolog...
APC Smart UPS 2200: www.bhphotovide...
Over four years old, and this is still one of the best data workflow videos I've come across.
Also, so pleased to finally find someone else that also realizes that data on the OS should not be permanent in any way.
Wow. I have watched MANY data workflow explanations...
*THIS IS HOW ITS DONE*
Haha. Thanks so much my man! A few days ago I posted a 2020 update. Be sure to check that one out as well!
CZcams searched: "Photography workflow" > Clicked your video > watched through entirety > left very satisfied! Thank you! > THUMBS UP
Thank you so much! :-)
Hands down the very best approach on data workflow
One of the most structured Workflows Thank you! Would be interesting to see how this has changed 4 years later....
Thank you! You can see that very thing.. I have made a 2020 update, and I no longer even use Apple :-) Check it out, its a recent video. Thanks so much for watching!
@@travisharrisphotography WoW no longer use Apple... Quite a few photographers are saying that... Makes me think that's what's next for me... Thank you for replying. will look for it now. Love the Channel!
Your organizing system left me speechless! Fantastic job!
Dude, I SUPER appreciate the professionalism, order, amount of direction this video contains.
It's soooooooo rare when you encounter someone who really processes a distinctly organized workflow- one that is straightforward, directed and grasps that high level of effectiveness to keep you hitting the ground running.
It was just simply a pleasure seeing smartly done, well versed, thorough presentation like this which again, is so rare.
I'm a complete novice (Sigma lens 30 mm 50 mm - T3i). I am still struggling with white balancing on my camera and just ALL The settlings (LOL).
I've been shooting (sometimes bad blogging footage) while traveling.
6 weeks ago I lost, my laptop in a foreign city. Never was located. I'd lost edited footage I had for three important videos - one of which unrecoverable.
I stumbled upon your video while researching stuff on an external drive or SSD I can TRUST. Subscribed. True pleasure.
As a matter of fact- if you ever need anyone to back up your assistants (on a basic assisting level,) - give me 2-3 weeks notice and I will head straight over to Miami to reliably help out. Would love to shadow and be of true help to a team that's organization is superior as this. Thanks again. Regards.
Hey! Thanks for that comment and sharing the story! So sorry you lost your things while on a trip! That totally sucks. We just got back from a trip (check out the latest video I posted last night). I'd love to have some extra help once in a while.. where are you located now?
Awesome! I trust you all had a great time in travels. I will definitely check your video out.
I'm located across the peninsula from you - Naples. I can take a slide on over Alligator alley to Miami in a jiffy!
Weekends are the easiest for me but I can help out during the week as well with enough notice. I took the liberty of dropping you a line through your contact page on your website. I look forward to being in touch. If you need references, just let me know as well. Cheers!
Thank you for taking the time showing how to do the Work Flow...
I've been looking for this detail of a video about storage for the entire 3 years of my photography career!!
Thanks a million man! I have a ton of restructuring to do with my data.
You have an adorable kid! He is so lucky that when he grows up he will have tons of high quality photos of himself and his life!
Thanks so much Ryan!! Austin is the best thing in my world :-)
your organization is all I want to learn people in my job. You concentrate everything in one video. Thanks and keep on in this way.
Just ran across your video looking for effective photography workflow. Just had to say thank you for taking the time to share all this information. You have inspired me to improve my workflow, eventought at the time, I'm only a hobbyist but have plans of doing this professionally. Great work!
Juan Carlos Rodriguez right on brother! Keep me posted, and if you need any help feel free to reach out to me.
"I don't think there's any place for the term 'best' to even exist in the creative spaces at all." You had my sub right there! But, aside from that, this is, by far, the most comprehensive and thoroughly thought out workflow I have come across. It's way too much for my purposes - but I learnt a lot from watching. Thank you. It's obvious that the same principles of forethought, thoroughness and attention to detail will extend to everything you do, so no surprise that you are successful.
Thanks so much Steve. That was great of you to say!
If I could like this video 1000x I would. I almost want to like, unlike and then like it again over and over. As a beginner in photography, some of this was a little over my head but I will learn until it is second nature. Thanks for sharing!
This is the best video so far on this topic on CZcams. Thank you.
Aww. Thanks man!
Hi Travis - This is just a superb video. I love that it explains the WHYs as much as the HOWs. Frankly, my professional workflow wasn't nearly as sophisticated as yours - it didn't need to be - but my experience tells me that you hit all the right issues. I especially recommend your "keep it moving" philosophy. When the image data keeps moving so does the money. And when the data bogs down anywhere short of an archive, a creative person is bleeding profit like a wounded animal.
Wow that was by far the most helpful video on the topic! Thank you!
You are very welcome! Also, be sure to check out my latest video on the new PC build, and updated workflow! Thanks for your support!
Will do :)
Great tutorial Travis and your work is fantastic. When you mention you create a RAW folder, sort and then create your final folder, i'm assuming you are just deleting unwanted RAW files and then moving the kept RAW files to your final folder?
Nicely done! This is much higher end version of my setup. I agree with using the second card slot as back up but I tend to use the same size cards though. I keep the size low, for me anyhow with 32gb being the largest. I organize my cards by sets, so I swap out both sets at the same time. I shoot mostly landscape, so I rarely need to swap them out more than once on a single shoot. Anyhow, good info for sure!
This gave me great insight on why Lightroom was slowing down on me. Even with my files stored on an SSD and smart previews created, Lightroom eventually got bogged down. I realize that this work flow helps solves the problem and also that Lightroom was built to help create a good workflow, not be a file manager!
Hi, I am late to the party but so glad I found your channel your work is fantastic. This is a truly inspirational video that really did explain things in a logical way and I now need to re evaluate my workflow and invest in some additional hardware (for a data warehouse) . As you might expect I have a few questions that I hope you will answer or point me in the direction of finding an answer. When you make the folder of the final pics of images Using Photo Mechanic, do you copy the images from the original first file or do you actually move them - thus saving space by not doubling up on images. Also when you put your images into Lightroom for their final tweaks do you change the images to DNG format or keep them in their original RAW state whilst editing? And finally I think that your data warehouse set up is a SATA drive - how have you connected this to your MAC system? Thank you and hope to bump into you on a cruise at some stage.
Awesome video - tackles an absolutely essential topic very clearly and easy to follow. I wish I saw this before setting up my lightroom as a managed catalogue. As a result I'm now trying to change my (very large) catalogue from managed to referenced - can you help point me in the right direction?
Hey thanks so much for your comment (and on my blog) I replied over on my blog for you, hopefully that will help. All the best!
WOW. That's it. Well done Sir.
First off, job well done. Great content, clearly and concisely presented. I do have a question though. With regards to you system of shooting with smaller CF cards and backing up to a larger capacity CF. I'm assuming that the backups are JPEG files, yes? Otherwise 128gb wouldn't be enough. Please forgive me if this is a stupid question. I shoot with a Nikon D810, which doesn't afford me the luxury of having two CF card slots. So if I were to use your system, I'd shoot with a smaller CF in card slot 1 (namely because CF cards write speeds are significantly faster than SD cards) and back up to a 128gb SD card in slot 2? Thank you in advance!
Absolutely fantastic! Great information. Thanks for sharing your years of work that have paid off for you so that we don't suffer some catastrophic failure.
Thanks so much! This was my idea, wanted to help others and be that person that I never had when I started! Be sure to see my updated workflow for 2020.
Hey Travis, interesting seeing your workflow. Ours is quite similar. For the last 4 years I created individual catalogues for EVERY SHOOT. The purpose to keep things speedy which it was. BUT it became extremely difficult to create custom portfolios and find imagery with so many albums. Thus I've spent 150+ hours merging everything into a single catalogue. I can't tell you how amazing it is being able to access ALL my shoots.
I plan on exporting to catalogues by genera once everything has been organized. I still create new catalogues for new clients but now they get ingested to the master after the images are delivered.
Just thought I'd share and thanks again for sharing your process in such detail!
Hey Greg! Just seeing this comment now! Thanks so much, yeah sounds like you too have a solid game plan! I appreciate your support here on my channel very much.
Greg Samborski would you be able to provide more detail? I have also found catalogs for every shoot makes finding photos very difficult. On the other hand the poor performance of one large catalogue is almost intolerable. You seem to imply you have found a solution to both issues. Would you share more details on how you accomplished this please?
Excellent explanation thank you for sharing! I’m a hobbyist looking to better improve and up my game a bit. Well spoken and well produced vid thanks!
Thanks Adam! I just made a new 2020 update to this.. be sure to also check that out!
Wow. Exactly what I needed right now. I love you, man.
Thanks! Glad this helped!
Most impressive Travis! I come from the IT world and you are SPOT ON! I have a small Qnap NAS at home, and don't rely on any third party cloud based solutions. What's your thought on using a bank deposit for the off-site disk rotation?P.S. I plan to get into photography and pursue it further when I retire. I will definitely follow you when that time comes!
Wow how come I never heard of you till today. Awesome videos. Real pro.
lol, thanks man!
So much information to take in all at once but i do want to ask you what you would recommend for beginners? As far as workflow AND storage?
An important note: even if it's smaller APC 500s for power CONDITIONING, it's important to have a UPS on each of your PCs and/or external drive arrays!
The one I have, is huge and can handle everything. My system has now of course changed now anyway. Good ideas!
Awesome video! Do you have a tutorial video on Photo Mechanic, how to set it up and use it?
I dont. But this would be a good idea for me to do!
Thank you for sharing your - refined - workflow.
The 128gb in slot 2 is an eye-opener for me. I just ordered 2 for both of my camera's and the 32gb cards will be from now on reserved for slot 1.
Perfect! Yeah, you will like this very much! Makes things very easy.
This video was absolutely phenomenal.
try backblaze.com . You dont need to manually backup to co location. Just need to be pc/imac connected storage. It wont work in nas.
Its can be encrypted using private key. Offsite backup can be requested over pendrive or hdd upto 4tb
So great Trav! Great insight and truly inspiring to fine tune my workflow as well! Thanks for sharing!
Hey Gabby! Thanks so much! I know there is a lot of info.. but, hearing people being inspired to at least look at their current way of managing data, is really great!
+Travis Harris Photography absolutely! Keep these coming!
I think you should be running the NSA. BTW, I didn't go to Casey when I saw the glasses...I saw a deep CIA operative giving us add'l information. Great video.
The best work flow video,,,thanks
This is the best workflow I've ever seen!! I'm really "paranoid" about losing my files, and, indeed, with a workflow like yours, you will not get that chance!
Thank you so much Benny!
Hey Travis. I really appreciate you taking the time to explain your work flow so thoroughly. I am one of those who unfortunately has been ingesting all raw images straight from card to Lightroom. I was wondering if there's any way that you could explain how to bring everything back to a state of organization like you have?
Josiah Frugé hey thanks so much! Be sure to watch the updated video (link in description). I go into more detail on this process with my new setup. I think the best way would be to start a fresh new Catalog moving forward. This would be the easy way to do it. Otherwise you will need to go to each and every shoot and manually delete all the “non keep” photos from each project, remove them totally from the system and then once that is done right click on the project and “archive as new catalog” on an archive drive. When done then go back and remove everything.
Travis Harris Photography. Thank you for the quick response. I figured it would be fairly difficult to do so that's no surprise. I think I will use the rest of the year to learn and implement the new method in my current work flow and catalog and change over to the new system with a new catalog for 2018.
If there’s a hurricane and both your house and business are compromised you could have a 4th system in the cloud automatically
Hi Travis, just saw your video and it is amazing to see your level of organization and attention to detail. You have really thought out every aspect of your photography business. Do you have videos on the other 3 workflows (creative, business, and marketing) or have you thought about making some youtube videos on those subjects? Just curious, do you shoot mostly JPEG, RAW? Thank you for sharing your photo shoots and all the insights as to how you manage data flow. You sir, are a rock star.
Very good and more importantly helpful to me. While I was looking more along the photo tracking (?) aspect.
The info you you've provided will help me to get and stay organized.
The questions I have are:
Is your workflow software dependent?
What type of computer are you using and the spec? I'm looking to get a more photo centic computer.
Are the software features you mentioned unique to Lightroom?
Thanks and keep up the good work.
Hey Bill thanks so much for checking out the video, and my channel. Im running out the door now for a session, but yes the workflow is to a point dependent on Photo Mechanic, and LR. However, the "flow" of data is not. In other words, the "front end" (programs) make it easy, and thats the process outlined. However, the "backend" (data flow) is based on the drives, archive, and etc.. so, that can be done even if you use other programs. I use an old 2010 iMac. I may upgrade this year when the new ones are announced.. but, so far.. there has been no need. Lightroom is only one part. Photo Mechanic is the other. I use them together because they both excel in different areas.
Can we get a step by step of how you set everything up to operate the way it does? Like how do you program commands for drive a to transfer and copy files to drive b, how you get LR to save files outside of the software, etc.
Thanks so much for the comment. I just (this month) wrapped up a new computer build.. and I have made some adjustments to make all of this even easier.. I am going to release a video in the coming weeks as an update to this, and I think will be helpful for all.
Great stuff. Very informative . I loved the full rundown. Thank you👍
Hey Travis, thx for sharing your experiences and very valuable workflow! Thought i had a workflow, oh how i was wrong. Very helpful Video to me the next year i`ll focus more on that and already organized my data like you do. BTW if you have a workshop going in 2017 please let me know, photography and Miami might be an excellent fit!
this video is like a breath of fresh air in modern days of "1 memory card slot is okay" bullshit by CZcams clowns
thank you
My backup regimen is "ok" given my budget...the problem is how do you do a validity check on the data files over a long period of time? I have discovered a gap (all photos of a whole branch of my family) missing the images, but the folders are still there, nested in their proper locations. Since I only found it recently, and am already at 1T of data (aka 3T storage), I had already reused the space and copied nothing. Just wondering if you have run into that, and can speculate why just the files would have gone...would love to know what happened, so I am not wondering if I have been sabotaged. Thanks.
Hi Harris thanks so much for this very useful video im definitely gonna start on this workflow. i just have a few questions i hope u can answer. After editing the photos in LR do u save a JPG file before uploading it to pixieset?what drive do u save it in? also what settings do u use in exporting your photos? Also where do you keep the softcopy of the Layouted photo Albums?
Hi Travis, this is exactly what I needed to watch, especially the tips on Lightroom versus Photomechanic. Thank you for a very enlightening video.
I have a question: When a client loses pictures for whatever reason as you mentioned , do you charge them if they ask for copies (assuming they are still within the 3 month/90day limit)? What's your policy on this.
Again, thanks for the great video!
I wouldn't be mad if you did an entire video on your LR workflow. I've got the PM angle down, but my LR catalog(s) could use some work.
Will do. I agree, I could talk more about that in a short video.
I feel really educated now! Thank you!!
Thank You! ALso, be sure to check out the updated video on my channel with the new PC setup!
Love that you are in Miami
Why the thumbs down for this vid??? Am I missing something?
great info and nice video... but dude! why the glasses indoors?
Great video! I have a few questions for you. You dump all your raw photos into the folder DATE_LastName>RAW. next you move the ones worth importing into lightroom to the DATE_LastName_Final folder. Question is, do you move or copy from the original Raw location? I would imagine just move the files but I could be missing a reason for copying them there instead. One more question I have is if I'm shooting with more then one camera, would it be worth having DATE_LastName>Raw>CameraA,CameraB and the same for the _Final folder?
Thanks for the awesome tutorial!
Yes, answer 1 is move, not copy. 2. no, keep all cameras in the same folders. Then, just sort via metadata if needed. New 2020 update on this workflow too.. check it out if needed.
Genius As Fudge!!
Travis, have you thought of solid state for your drives? I know they are pricey, but may be cheap insurance, since they can withstand accidental drops or bangs compared to spindles. Thought I'd get your opinion...
Yes, I use this now. I just built a new PC.. going to make a new video.. BUT.. I still use the rotational drives.. its pretty slick.. I need to make a new video. This past month all I have been doing is working on this system upgrade LOL.
Would love to see your new system upgrade. Hopefully you will post it soon.
BRAVO is all i can say. Thank you for your help
great advice! greatly appreciated
Glad it was helpful!
This was awesome, thanks! Moving collections into their own managed catalog and archiving them makes perfect sense for professional projects that are completed. But how do you handle all of the "personal" pictures that you take that, over the course of many years, will grow quite large and could compromise the speed of lightroom and take up space on your hard drive? Moving older pictures into their own catalog for archiving means you can no longer search across all pictures for "best shots of my son playing soccer" for example. You could certainly move older pictures to an external drive to free up space, but that still leaves the master catalog growing. Would love your thoughts...thanks!
Hi Scott! Yes, this is a excellent point. I have a 5yr old son and can relate with what you are saying. Remember this.. there is always a give and take with any strategy. With that said, I rarely ever reach for my “pro camera” or gear anymore when taking images of my family. To be honest, there are 6000 images in my phone, as this is the camera that is always with me. I say that being funny, and also serious. Now, granted there are times when I will take my camera for an event (rare). His birthday, his friends birthday, school event where he is on stage, fun at the park to mark this time in his life etc.. When I get home they are ingested just like any other shoot on my system. I have a folder named “Austin” (my son) in the root folder of the master drive. This never gets deleted. Your also correct that I can’t search across the data base for something etc. BUT, it’s also very easy for me to open a folder on the drive, and look at the date and event name to find what I am looking for. So, again, give and take. Now, if I wanted.. I suppose I could make a new “personal” library for only family stuff, and then carve out a portion of the drive for that etc. That would also work. It’s just, that so far I have not had that need, as the shoots done with “family” are few. I think part of it.. is.. that.. anytime I don’t HAVE to hold my camera, I won’t. Once this is all you do, the last thing you want to be doing in taking more damn pictures on a day off. LOL. I’m serious, and it’s sad. I have FAR less images of my family BECAUSE I am a photographer, then what everyone THINKS I have because I am a photographer. Once you get to a level beyond hobby, and are dealing with paid shoots, clients, and massive downloads from weddings.. this is where my workflow shines. If you sprinkle in some personal stuff that’s fine, but you will need a separate library / setup if you plan on doing this in volume.
Which CRM are you using? Any videos on that?
This information is priceless. Thanks.
Thank you SO much for this!
I have my OS on an SSD, but cats and CR2 files are on a HDD. Can I move the catalogs to the SSD, without LR freaking out? Worst that can happen is LR needs re-pointed to where the folder is, right?
on a related note: LOOK at Black Friday & Ciber Monday specials for 1 or even 2 TB SSDs in a couple weeks! :)
Thanks Jason. Did you see my updated video, and PC build? Everything has changed since this video. Cheers to you! Hope you are well.
I have a veeery small scale version of what you have, with not nearly as many redundancies, but for now it'll do.. Managing Lightroom, the catalogs and all that has always been a little beyond me. I'd like to maybe see a more detailed LR workflow video, even after you explained it, I'm still a little confused. Oh, and definetly getting photomechanic!
Hi Oil Aponte, Thanks so much for watching the video. Good to hear you have a scaled version of this. Anything is better then nothing :-) I can def. take a look at doing a LR specific video.. there are in fact some sub flows that I use inside of LR to help the project move along the post process.. nothing fancy, but for sure helps keep me organized for volume. Thanks again so much.
How do you do the offsite backup though if you're away for more than a week? :)
After photo mechanic how do you copy over the thumbnails to lightroom like you said?
Hi Travis. What a great video. Would you be able to help out? I am using an LR for few years now and all I do it's drag original RAW in,edit,export.Whic is prob the worst you can do. My mac air has 80+GB of photos on even I deleted all photos. Is there anything I should do as I have iMac and don't wanna do the same thing. Would you be able to post step by step to avoid this.. thank you...
The main takeaway.. and only real point I can offer is to have a backup of everything. So, step 1.. is to get the LR catalog file / database on some external drives! RAID 1 would be a good idea. This way, you can remove the images from the local machine, freeing up that space. THEN.. next time yo import.. consider NOT using LR.. and copy the files to the external drives. Leave them in place, and then point LR at this folder...
worked of one memory card and one Drive for years ...nerver lost a file
Alexander Zettl good for you! I have gone 36 years and have not died in a car accident yet either. But you know what.. it could happen tomorrow. The truth is you just never know. It depends on your business and how much you want to gamble. IF something happened and resulted in data loss, it could cost your entire business and reputation and this would cost far more than what i have invested in a workflow. So, in the end it's all relative.. if you make $10K a year from photography then this is overkill. If you make over 6 figures as a full time wedding pro.. then this is an example of what it means to take what you do seriously and protect yourself.
All I Can say is wow!!! Thanks for The info!!
Hi Amos! You are very welcome sir!
Capacitized is not a word. The word you're looking for is "filled" or "full".
LOL.. Thanks Mike.. I actually had no idea. Turns out.. it is a slang word, and was hoping people could figure it out :-) www.slangvocabulary.com/wp/c/capacitize/
great information and i wish i had seen this before i invested in my full frame world..
Thank you very much for sharing this workflow! Subscribed!
Javier Bacon Ventura you are very welcome 😊. Be sure to see the updated video and new system. Link in the description.
Great post! I've learned lots from it. Would you recommend any other auto backup software at all?
Thank you for sharing your workflow. I'm spanish and I needed to see the video twice to fully understand your complex and secure worflow. I was really helpful!
Is there any software similar to Carbon Copy Cloner for Windows?
Pls. do a step by step video on how to manage lightroom. Thanks!
Loving the videos, but boy was the exiting the car shot drawn out.
Your right. It was. But.. I had FUN making it happen... and as much as I know people think I make videos for them.. I actually make them for me, and just share them here. Until I get paid to make them LOL. :-)
Dang. It would be awesome to see a visual representation of this just drawn out, where everything goes.
That is a really good idea! I may do that soon, and update my blog post on my site with it! Good call!
Awesome! Also, if you'd scale this down, how would you do it?
I'm a new grad and all my photos are on my HP laptop (fed them through LR) and my 1TB of space is completely filled so my computer is extremely slow. So I have to reorganize and your system is to expensive for me at the moment.
This past year, I shot an Indian Wedding up in NY and it was several days (4) of shooting. We were in a hotel, and I setup a compressed version of this to process all the data while away from my main system at home. It was pretty simple..
1. MacBook Pro with internal SSD. This was where the LR Catalog file was, and OS.
2. I had two other 7200rpm External LaCie HD’s chained together via. Firewire where the data was copied to after each day of shooting.
3. After the import finished (while I was sleeping) CCC (Carbon Copy Cloner) would then mirror that drive to the second LaCie HD, AND to one of the G-Tech “G-Safe’s” that I brought, which was connected with e-SATA via a card. So, in total there were three external drive connections to the MBP, with x4 actual copies of the data being made at the end of each day.
4. When I was ready to leave the hotel room, one of the small LaCie drives was hidden in my room, and the other was with me in my camera bag, and the G-Safe remained connected to the computer on the desk, with one drive with me in a sleeve that lived at the front desk.
5. When we flew home, I used Fedex to ship one of the smaller LaCie drives home, and my assistant carried the other, and I carried on the G-Safe with my Camera bag.
Hey Lauryn I'm working on a schematic diagram of what Travis explained as his data flow/backup process, once I'm done I will upload here, hopefully in the next week or so.
Hey guys! thanks for all the great info. Tony, did you end up doing a diagram of this? If so, where can I find it? Thanks
Nice Workflow, congrats on your organization.
I'm only confuse about the each drive on a top secret location. Those driver were RAID 5 wasn't it? That way you don't have a copy offsite, you only have one stripe that can't rebuild no full enclosure. You are only putting some overwork on your other drivers each 5 days to rebuild. Or am I missing something?
Regards,
Hey Luke! So, the RAID 5 is not being used in the way that you think. I only look at this as "one drive" as it should. The RAID 5 is only for a speed bump. This drive is mirrored every night to another RAID 1 drive, and one of those is what goes off site. A month ago, I just upgraded and replaced my Mac to a new Powerful PC.. and I am going to make an update video that is simple..
Thanks Travis, sorry for the insistence So you have a RAID 5 and a backup as a RAID 1 and each 5 days you get one of the RAID 1s and put on a offsite and what is rebuilding it is the RAID 1? Is that it? Just trying to wrap my heads around it :D Thanks one more time and congrats!
I think you got it. Raid 5 (pretend that is not even RAID 5.. because that part makes no difference, and now I use an SSD LOL). SO, "main drive" has all the images. Every night that drive backs up to another duplicate drive that is a RAID 1. SO.. now, the data is on THREE different drives. Every week, that third drive is in rotation offsite, and gets rebuilt with last weeks copy, so that means the data is now in 4 locations. Three locally, and one offsite.
I see, perfect, thanks and once more very nice workflow and design. Not everyone does 1/10th of that (including me). Cheers,
Thanks for all of the specifics. (pretty underrated IMO)
oh no, I can't watch more than 7min 30 cuz I have no self-control...........
Ok. First of all thank you, thank you! The most helpful and precise video I have found on this topic yet. I have a couple questions though... So the raw images after photo mechanic, are on the "warehouse drive", and backed up. Then what exactly are you saying about that you have the "master referenced catalog and lightroom on the same internal drive"? I take that to mean, you have Lightroom on the internal drive, and I believe you mean the instead of using the copy setting upon import, that you are using the add setting? So the actual image is not ever on you internal drive? I just want to be clear on what yo are referring about this.
Secondly, if the above sounds about right, I just want to check and see if you thing that using my external Seagate 1TB SSD usb 3.0 drive as a warehouse drive would work? Assuming I have most of the other steps you have outlined in place. I have a drive that would be the daily backup/clone of the warehouse as well as an archive drive.
Thanks again for the great vid. and any advise you can give to a guy who is new-ish, but craves getting it right early on.
Hi Matt! Thanks for watching, and your support on my channel! Yes, to your first question.. the RAW image data never leaves the main "warehouse". Once the directory structure has been created on that drive the files live there until archive happens. The catalog file lives on the internal SSD for speed, and correct.. on import files are not copied to this drive, but left in place.. and this is called "referenced" (at last back in the old days with Apples Aperture). Meaning the files are not being moved or copied.
Yeah, technically any decent drive will work with this workflow.. to be honest, the drive that houses all the RAW files in fact does not need to be blazing fast, as the access to the RAW files when editing in LR is not a huge deal. But, like with anything its nice to buy the best you can now, and hold on to it for a longer time. In my case in going on year 7 LOL. I know there are better drives, and faster connections out there, and someone below linked a nice NAS setup (drive array) and that would be nice too. For me, all of this is just a big box to get data from point A. to B. and works well. You will be fine :-)
The point of the video is more about the "flow" of the data vs. anything too subjective like what drive is best, or debating faster, more power systems etc. The nice thing is.. that once this flow is understood it can be applied to any setup, and tweaked for different drive setups.
Travis Harris Photography thanks Travis. You have been a great feat of help
Thanks again Travis. Again this video has been hugely helpful. I liked it so much I am in the process of revamping my workflow/and system. To that end I think, if you'd be willing, I'd like to ask for clarification on a couple points.
When you are finished culling in photo mechanic, how do you get the chosen images into your "_finals" folder? Are they exported to a folder you already created? Does photo mechanic create this folder for you?
Hi Matt! Thanks so much for watching, and your support! So, what I do is this.. (as an example) the directory structure for weddings would be this.. [Photo warehouse drive] / WEDDINGS / 20161212_Client Name/RAW. ALL the images are ingested into this RAW folder from the CF cards. Then, I cull all the date. When done, I right click on the reverse date client folder 20161212 (in this example), and "create new folder". I then create this.. 20161212_Client Name_FINAL. Then, select ALL the FINAL CULLED images, and drag & drop into this new "final" folder. This takes about 10seconds as PhotoMechanic can do this task insanely fast. Then, close Photo Mechanic, and open LR. Inside of LR, you will now only import that final folder into the catalog.
Travis Harris Photography and import folder only using "add", not copy, or move, or copy as DNG? Right? I have tried that. And it seems to be taking ages for LR to do anything so am I missing something?
Out of curiosity if I choose to copy that folder during import instead of add, how would that effect the creating new catalog during achieving? Would it still being all appropriate negatives and source files from the warehouse?
Travis, great video!! Can you tell me the export specs of the pics that you output to PixieSet. size, format etc..
Thanks for the information Travis, very informative. I am currently improving my system. How do you create the reference Catalog you mention, that will speed up lightroom processing without copying all the files?
The referenced catalog just means that when you ingest images to the catalog that you are making sure that they are NOT being copied.. the import screen will allow you to choose this. You want to LEAVE them in the place where they currently are (where you copied from the men. card to the HD).
Great. Thanks so much for replying!
Of course. Sorry for the delay.. We were on a Cruise.. (check out new video just uploaded). Good luck with everything!
Hey Travis,
Thanks for the video. It has a lot of good hints for everybody I think. Working with LR myself I just wonder about one thing: you talk about managed and referred catalogues. Maybe because my mother language is not English I never ran across these two kinds of catalogues. Could you explain the difference? I checked the adobe website, but it only speaks about catalogues in general. Maybe you happen to have a link that explains both features?
Thanks again. Any help is appreciated!
Thomas
Yes, I can explain.. and it was a mistake for me to describe them that way. Years ago (2008 - 2010) I used Apples Aperture program to manage my photography. In this system there was in fact a referenced, and managed catalog. The difference was as I described in the video, and I got used to this mindset. Now.. while LR is all referenced, all the points in my video are still valid, and when you export from LR to archives, this is effectively the same idea as the "managed" library, where all the files are pulled together on the same drive as the catalog file. Also, if you import into LR directly, and the files are "copied" over to your HD, and thats the same as your operating drive, then this is also basically a managed system.. so, yes I misspoke.. however the idea is the same, and still accurate. The bottom line... is you don't want to use LR to import all your data, and you want to keep the data flow separated for performance, and this also means only brining in the culled files into LR's catalog when your ready to process for the client, all the while they live on the drive structure as described in the video which is in fact "referenced".
Travis Harris Photography Thanks for clearing that up. I was really wondering if I overlooked a major part in LR. ;) I made good experiences when only working with smart previews instead of the original files. Worked fine when storing only on external USB drives but since I now go for a more steady office environment, I'll have to adapt.
Thanks again fort he insight! Definitely some stuff I will adapt!
Thomas Damm Right on Thomas! Sorry for the confusion.. let me know if you get stuck along the line anywhere.
What are you trying to do, put the IT folks out of business? Only kidding. As a former IT only consultant I wish my clients put this amount of thought into workflow, redundancy, and disaster recovery. Because it's not "what if it fails, it's when it fails." A 1 in a billion chance of failure is trivial in today's world of personal TB storage. The only thing I would add is something like AWS Glacier. Oh, also you are one hell of a good photographer.
Hi Travis, Thank you for probably the best explanation of a workflow that I have seen in some time. Very helpful. I have a few questions and I apologize if you have already answered them in the threads below. I have a question regarding your "Archive Drive" At a certain point this is going to get full as well. What do you do when this happens? Back it up to the cloud and reformat the drives? Also, I have been also planning on getting a SSD drive for Lightroom. Can you also store Photoshop on the same drive without affecting the performance or do you need a separate SSD drive solely for PS. Also, any recommendation on a SSD Drive? Will an external SSD drive work? Thanks Travis!!!
HI James! Thanks for watching, and your support to my channel! Yes, once the archive drives are full, I just buy more HD's (they are removable) and start over. That big, Pelican case that you see in the video shows them. However, a 2TB drive.. lasts me about 2.5 / 3years.. because the archived data is SO much smaller then the RAW data :-) Yes, an SSD is great for your applications, and really anything now. The same SSD is totally find for all your applications, and OS. I only use a secondary one because of the volume of work that I do, and also now with video projects, I like to have the seperated. I use all the OWC drives, and so far (knock on wood) have been great for 7 years!
Thank you Travis for the explanation. You have decoded the daunting task of an efficient back up and recovery system!
Thanks James! its not perfect.. but, works for my little world.. and figured others can also benefit from it..
Love your work flow Travis, very safe against Hardrive mechanical failures. But how does your Raid 5 protect against data corruption? wouldn't a corrupt data just be copied 5x over and corrupt the entire g-drive? how do you prevent against data corruption?
Hi Kenny, and thanks for checking out my channel :-) Data corruption is perhaps another entire topic, one in which I have never come across after images are ingested to the system. (going on 7 years, and over 1-Million files) Now.. I have seen files that were written to the memory card at the point of capture that had some garbage in the data, (weird blue streaks, and other strange artifacts etc.). This was the camera writing bad info, as it was present on both cards. This can happen from time to time. On my 1DX, it has over 400,000 clicks on it as I type this, and I have seen 3 or 4 “bad written files” to the cards. But, this was the source (camera and not the media). The most effective way I feel you can prevent any type of data issues, as a result of a bad CF card.. is using duel cards in the camera at the point of capture. Thats as safe of bet as you can make with yourself. Once the memory cards are downloaded to the system, there is no corruption that is going to take place from copying from point A to B. Once the data is on my system.. thats as safe as its going to get. The drives, the different RAID systems, the applications, etc, all have no bearing on data corruption. If there is some other outside reason for data issues (corruption, or anything else you want to call it) then, this would benefit from this workflow, as that data is spread across many different drives as you saw. So, in summery.. if your going to have corrupt data.. its going to happen at the point of capture (camera).. or.. a bad sector on your memory card. Its not going to happen sitting on a hard drive after the fact. The rule “garbage in, garbage out” applies here. The act of moving the said data from card to drive is not going to change, or hurt anything.
+Travis Harris Photography wow thanks for the detailed response ! your real world experience is invaluable.
Hi Travis, wow this is awesome video. Unfortunately I'm one of those photographers who didn't think about this back in 2010 and now has 8TB (2x4tb) drives of images which are not backed up. I shoot a lot more and business is growing pretty well. I have a little bit of mess here and came across your system video as I need to make an action plan on what to do now.
Question.
- I still use pretty old iMac. I think its 2009 model so internal HDD is pretty slow. I don't want to invest in another shiny thing as for me this one works fine. On Fstoppers I saw someone recommending to have LR with catalogue on external SSD for now to make it work faster. Don't store imported photos on it, but just catalogue and do editing from it. I might put PS there too. Not sure if thats possible but would this help with speed?
- I would set up external docking system with 2 internal 2TB drives to copy each other (like Raid1) this would be workstation and then across on other side of table identical set up which would clone work station HDD. Photos from Mechanics would be copied to workstation 2TB drive. I know this sounds on cheap end, but not sure if I have budget to dash out over 3K for two back up systems. I'm hoping that docking with fast 3.5" internal drives would be ok for a year and then switch to something more serious.
- My main confusion about this system is actual LR set up.
Now I workflow which you don't recommend. I have always imported photos through LR :( Card reader → LR → export as JPGS for clients finals folder on internal HDD. I know this is bad. My catalog holds around 200000 photos and is around 7GB size. How do I change now to your advised system to speed up things? Do I do anything or just leave it as it is and starting from 2017 go with your advised way.
See when you say after edit you export to archive HDD as a catalogue. Is this catalogue showing up in your LR or its just on HDD. After you delete job from main hard drive, I guess its gone from LR too. What if you want to grab few shots for portfolio from these shoots? Do you have some sort of portfolio folder set up as well. Also if clients want old photos, how do you open that catalogue. Does that is not messing up with your existing catalogue?
Sorry for all these questions. Want to get this right and have worked in LR since day 1 in business and have never heard of referencing idea.
Thanks for the share.
Regards,
Armands
Hi Armands,
Thanks so much for watching the video, and your support to my channel!
I can attempt to answer your questions..
First, my iMac is old too. 2010, and works just fine. I did however, have OWC tear out the old HDD, and use SSD drives (x2) inside to replace, and this makes a world of difference. So, however you can.. yes, get the LR catalog file (and the application) on an SSD for faster performance. I speak about that in the video, and this make a huge difference.
DO NOT copy your RAW files to the SSD. Thats a waste. Keep them on a fast rotation external drive.
What you want to start doing.. is to avoid ingesting directly into LR. This is where Photo Mechanic is gold. When you get home from your events, import directly on your external drive(s) via Photo Mechanic, and then do all your sorting / culling in there. Once done, move the “selected” photos to a new directory, and then open LR and import ONLY that folder. This will save you time, speed, and LR resources in rendering previews that it does not need.
Once archived, and exported as new catalog.. this only shows on the HD, and not in your master catalog. The idea, is to keep your master LR catalog “transit” so it never becomes large. You want it small, snappy, and efficient. When you deliver photos to clients, explain they have 90-days to download or whatever.. and if FOR ANY reason they need YOU to provide them master files again, there is a $100 charge for that, and this then requires you to pull your archive drive and re-export that data. Before you archive, you need to think about what you need for portfolio, etc.. I only archive after the 90-day mark as spoken about in the video, and this means the project is totally done. IF I need to access it for any reason down the road.. I am being paid for that time and effort, and thats a reasonable thing to ask of any client.
Hope that helps!
TH
Travis this is great info.
Spent a day researching more. Turns out my iMac even doesn't have Usb3 or thunderbolt so i have only usb2. Never realised that :) so there is no point to use SSD externally i guess. I did however found a video on adding internal SSD to my iMac and that is what i will do. Thinking if i should use 500gb or 1TB one. 1TB is almost £400 so might be a bit costly.
Someone in my town is offering G-Drive Q 12TB unit (same like yours) for £350. it is USB 2 connection but for back up might just fine. You think its worth to pay that. looks very good and has never been used in full capacity.
Need to make a decision on what to use as a first stop HD when i import cards. Some photographers have suggested to import first to computer and then after edit move it out on archival unit. Saves buying two more drives but i guess slows down my machine.
In terms of LR. this is very intersting for me. Does it mean you have only recent jobs and your private family photos only in LR catalogue? I guess maybe some favourites folder too.
Definitely want to set this up. Maybe you could do video on LR system itself. I know you are super busy but just saying :)
Will have a look on other videos you have here too.
Thanks Travis.
A
I'd avoid USB2-only drives; they are extremely slow. Firewire at a minimum, unless it is an overnight incremental backup (my TimeMachine backup is a simple USB drive). For my old iMac (2011 USB2/Firewire/Thunderbolt), I eventually got the Thunderbolt2 dock from OWC, which connects to the Thunderbolt port and provides several USB3, Firewire and Thunderbolt ports. Firewire is getting old, and hard to find drives still using it. While I never got to upgrade my 2011 iMac with SSD, it supports up to 3 SSDs if you also replace the CD drive. SSDs have gone down a lot in price, to the point where 1Tb is doable for me. I'd not want to live with anything less for my main drive.
Thank you, great video.
Does it require wearing sunglasses at night? 😆
I would never the photos to different cards. I use card 1 for stills and card 2 for B-roll.
I usually have a 2nd shooter and dedicated video person.
Very good.
I am entering into photo and video space. Are you using this workflow for video also? Using a camera that takes photo and video can you use photo mechanic to rename and import videos also?
For video I have a slightly different workflow (actually simpler).. and in fact.. I should do a simple video on just that! The short answer is no.. I don't use the same program for video.. but its very straight forward.. I will make a video soon on this!
You can rename video with photomechanic
Hi Travis, I recently tried using the technique of using a larger SD card 64gb in combination with the smaller CF cards 32gb and 32gb to break up the risk of a 64gb CF card. And all seemed to go well during the wedding, but when I went to back up my files later that evening the 64gb SD card which had all the files was corrupt. Do you think that I did something wrong when I added the second 32gb CF card? I am pretty sure I turned off the camera when I went to switch the cards. Also, can you trust the SD card after it becomes corrupt by just formatting it? Its still a relatively new card (2yrs old?). Any advice would be greatly appreciated!
Hey James! Oh no! That sounds like a nightmare. And is. Any card that has had issues, it's time for the trash. Never trust it again. What brand was it? Also, I would run smaller 16gb cards to further buffer things out. I'm so sorry that happened. As long as your turning things off you should be good. But this can happen to anyone.
@@travisharrisphotography Hey Travis, thanks for the quick reply! It's a Sandisk whom I use and trust for the most part. I'm wondering if I may have caused the issue by not turning off my camera while removing the card, so possible user error. Should I still burn the card?
@@jameskorin6963 yes, I would toss the card. Too risky. Chalk it up to either the card or user. But replace the card for sure.
James what camera were you using?
I'm currently using a Canon 5D Mark IV. Thankfully it has two card slots so my data was protected on the other card slot. BTW..saw your video on the EOS R. Would love to make a switch, but need 2 card slots as a pro for this particular reason. The lens line up looks awesome though and I am curious of how good this EVF performs. 28-70 f/2 looks like a beast and it will be very interesting what Canon can do with this new lens mount in terms of new lenses. Was thinking of switching to Sony, but like you I am invested in L glass and was waiting for Canon to step up. Hopefully, it will only be less than a year, but I am happy with the 5D Mark IV so far.
Phrase to live by: the cloud is just someone else’s computer.
Chaseme exactly! For me, I want total control and that’s what this is all about. Be sure to take a look at the new updated video I made (link in description) shows the new PC and data flow.
Still good Travis 👍
as a it tech i donot recoment swaping the raid dist and rebilding it at all it wers the disk tomuch and if the good drive fail befor finishing then you just lost the data on 2 drives raid is not ment to be used that way just copi the data to a external drive wit a script and do a rotaton betwen 2 drive ofsite no need to reimport just overite ther just in case the raid fails which ave way les chance of hapening if you dont swap the drive all the time
Thanks for the feedback. Yea, this has all changed now anyway. New 2020 video is out.
Great video, lets get more into details in your workflow :) Culling and editing :) Following