In this video, I make my case for why, when one is using Lightroom, they should create sidecar, XMP files. ** I am an affiliate for all of the companies listed EXCEPT Affinity Photo. The Best Sky Images I've Seen Available -- Ocudrone - bit.ly/3vtDpjR *Save 10% with Discount Code: Anthony10 Please read my Code of Ethics Statement: onlinephotographytraining.com/code-of-ethics/ Tap into Adobe's Creative Cloud: bit.ly/2zwQ0nW Anthony Morganti’s MUST HAVE applications: At least one Non-Destructive RAW Editor Lightroom - bit.ly/2zwQ0nW Capture One - captureone.38d4qb.net/y1Djy (Save 10% with Promo Code: MORGANTI10) Luminar - bit.ly/2JUJxKw (Save with the Promo Code AM16) On1 Photo RAW - bit.ly/2zJmA9I (Save 20% with Promo Code: AM17) Exposure X6 - bit.ly/2U8UxrK (Save 10% with Promo Code: AnthonyMorganti) PhotoLab 4: tidd.ly/2HhiN9X At least one FULL Editing App: Photoshop - bit.ly/2zwQ0nW Affinity Photo - affinity.serif.com/en-us/photo/ My MUST-HAVE Plugins: Topaz Gigapixel AI - bit.ly/3cDqa5J Topaz Sharpen AI - bit.ly/3cDqa5J Topaz Denoise AI - bit.ly/3cDqa5J *Save 15% on all Topaz Labs apps - use the Promo Code: AMDISC15 or instead of Topaz Denoise AI: On1 NoNoise AI - bit.ly/2zJmA9I (Save 20% with Promo Code: AM17) Nik Silver Efex Pro 2 - tidd.ly/3dc4gYm The Best Sky Images I've Seen Available -- Ocudrone - bit.ly/3vtDpjR *Save 10% with Discount Code: Anthony10 My MUST-HAVE Mac Utility: CleanMyMac X - Info: bit.ly/3jtVpU3 To save 10% on your purchase of CleanMyMac X: 1. Visit the CleanMyMac X store: bit.ly/3bEMuMx 2. Scroll down to the very bottom of the page and click on "Enter it now" under "Already have a coupon code?" 3. Paste the promo code TONY10 and hit "Verify" 4. The prices will be reduced, and you’ll be able to select the license and proceed with the payment ** Note that all of the promo codes listed above may not work on sale products. *** I am an affiliate for all of the companies listed EXCEPT Affinity Photo. Please read my Code of Ethics Statement: onlinephotographytraining.com/code-of-ethics/ Please follow me on Instagram: instagram.com/anthonymorganti/ View my recommended gear list: www.amazon.com/shop/anthonymorganti As an Amazon Associate, I earn from qualifying purchases. I use this software to record my screen and make parts of my screen enlarge and zoom out so you can see it: Screenflow: telestream.pxf.io/DMrW2 Unsure of how to price your photography? Check the 2019 Guide to Pricing Your Photography: amzn.to/2S1CiU7 I am an affiliate for all of the companies listed EXCEPT Affinity Photo. Please read my Code of Ethics Statement here: onlinephotographytraining.com/code-of-ethics/ Thank you!
I wonder if Scott Kelby will be advising photographers not to buy cameras with sensors over 25gb as the images will take up too much space. It seems ludicrous that he advises saving 5KB by unticking the XMP box.
Hey Anthony, One important FYI - The actual number of bytes that a file occupies is almost always larger than the size reported by the OS. That is, a file reported by the OS as 2K in size can actually take up to 4K, 8K and even 1M on disk. This has to do with how drives work in blocks (sectors, pages, clusters etc) and the interfaces that the OS uses to interact with these devices. For example, if you format a hard disk with an allocation unit of 4K then files will occupy space in multiples of 4K. - files smaller than 4K will always occupy 4K on disk, - files larger than 4K and up to 8K will occupy 8K, - files with size 8K+1 byte will occupy 12K, etc... If you choose to use an allocation unit of 1000K then a file 1 byte in size will occupy a megabyte on disk. So this can significantly affect the disk space occupied by XMP files. 10,000 XMP files can occupy 40MB, 160MB, and even 10GB of disk space.
Good video. One thing not mentioned was that files are tied to 'Allocation Units' which are created at the time the hard drive is formatted. An allocation unit is the amount of drive space taken when a file is created. On my PC, the allocation is 4096 bytes, so if I create a file of only one byte, 4096 bytes will be allocated. If I continue writing data to that file, the moment 4096+1 byte is added, a new allocation unit will be appended and the file will now take 8192 bytes and so on... In the case of the 5K file shown on the example screen, it is actually taking 8K of space (assuming your disk is also using 4k allocation units). The amount of this allocated space can vary. For example: You could format a hard drive with 32k or 64k allocation units, and if you create hundreds of 1 byte files, you could actually fill up a hard drive - even though the actual amount of data stored is very small (I actually did that once many years ago :) ) For the vast majority of users, having sidecar files will have little impact given the size of current hard drives, but it's good to be aware that there is some hidden 'overhead' whenever a file is created.
I had a client ask for some files while I was traveling. I didn't have access to the main LR catalog or images, but I could download the main image and the sidecar from my cloud backup. When I dropped them into my laptop Lightroom, I got the image and the edits I already made. It saved me a ton of work.
“Most people don’t practice good backup…” Probably the understatement of the day! I was “in the business” (that is, computer business) for years until I retired, and I will guarantee that ALL computers and ALL disks will fail at some time. Usually the worst possible time. Not only did most of my clients fail in this area, I did myself until I realized how precious these photos are to me. When you first pointed out this feature, I immediately adopted it and am grateful for the advice. And you are right: a 25 kb file is measly .1% of a 25 MB file! Anyone who values their photos should backup, have a periodic backup of the backup, and occasionally make sure you test to make sure you can restore from that backup. AND, keep your very best photos somewhere really safe, if not the cloud, on an SSD in a bank vault! Seriously.
There are two, related reasons to have time/date changes written to raw files: when you are traveling through time zones or DST/ST changes and you have forgotten to make the time/date change in camera. I’ve been in both situations.
As a PC user I definitely have more than one HDs in my computer. So my catalog is on the system SSD and the RAW files and catalog backups are on one of the other physical drives (of which again I have backups on two different external drives...).
I have to say that Scott Kelby was right back in 2012 when you could really see the performance hit from having the switch turned on. Now days there is no reason to not have it selected. For those who have had the switch off or if you only want to selectively save xmp files. You can hit Cmd (Ctrl) - S on a file or a selected group of images and the xmp's will be written to disk. Likewise you can select an entire folder by using Cmd (Ctrl) A to shorten the process or to apply a update. As I saw another poster stated the xmp's come in handy with other apps as well, Bridge being one that will show you the most recent version of the raw file with the xmp updates. Thanks for pointing this out Anthony, good stuff.
I have to say, I agree with Scott Kelby. First of all, I convert to DNG and I know that is another debate. I am religious about backups and I use an external for this purpose, this includes not only my photos, but also all settings like watermarks, export settings, catalogs, etc. I then duplicate that hard drive to another hard drive for offsite storage. When I had that box checked, it made my backups excruciatingly long because if you go into a folder and even look at the photos in the folder, it means the file has technically changed and therefore my sync app picked it up and backed it up with the rest. I unchecked the box and voila, my backups are back to quick and simple, my edits are written to catalog and since I backup my catalog every time I do my photos, all is well and I am perfectly protected. In the case of xmp, it wouldn't bog me down the same but it is a wasted step.
ON1 support also highly recommends sidecar files be turned on just in case the catalog file gets corrupted. Also, I have my backup also backup the directory the catalog is in when it backs up my images. It’s not hard.
Hi Tony. I'm a great fan of yours and frequently recommend your videos to my friends. However, I really disagree with your use of "destructive" in this video. As I understand "nondestructive editing", it means that the original IMAGE DATA is never changed within the file containing it. In this sense, DNG files edited in Adobe products are edited nondestructively, since the original raw (or whatever format) image data is never changed even though additional data about post processing is added to the file. On the contrary, bitmap image editors such as the original version of Photo Shop can irreversibly change the image file itself, and thus are "destructive" editors.
definitively ! if for any reason you loose your catalog and the backup, you can create a new catalog from scratch i backup each time i exit lightroom, and backup automatically every day on an internal drive and on an external drive stored somewhere else you're never too cautious...
Note that If you primarily shoot JPGs you may want to be aware that Lightroom will write all metadata changes directly to your files. This is technically only writing the exif data, so not truly destructive. But I had some corrupt JPG files a while back and noticed the date differences when I restored them. According to Adobe XMP is not supported on JPG/TIFF/PNG or PSD but I would have preferred the app not update my original files.
I work on 2 computers, main workstation and a laptop, I always use the XMP files so I can quickly transfer the images from one machine to the other and have all the edits ready. Also I have an old HD that contains my first 2 years of photographs and I didn't do this and I lost the catalog so now I have 2 years worth of developed images gone, only the RAW/tiff/jpegs are there but not the edits, I wish I used XMP files from the beginning.
I just had a laptop screen fail toward the end of a 3 week photography trip. The image files were stored on an external SSD, but the catalogue file was inaccessible on the laptop hard drive. Fortunately, I have been using the XMP option, so when I got back home, I just needed to import the last 3 weeks of edits. As an extra layer of safety, the XMP files are definitely worth any of the downsides, even if you only have the need once a decade!
Thanks for the video, Anthony. It raise again the question if it's better to convert the file to DNG or keep using the raw file from the camera. Hope you can make another video regarding this quesion.
Have a look at what happens when you use DNG and activate this option. Whenever you make the smallest of changes to the edit or keywords or rating - the DNG gets overwritten by a new version because for DNG Adobe doesn‘t create separate XMP files but embeds this information into the DNG itself. This will blow up your backup and also pose data integrity issues, especially for cameras which use DNG as native RAW files. So as useful I find this XMP storage, it is sabotaged by Adobe by overwriting the original DNG (even if I only use this whenever DNG is the only way, such as when doing panorama or HDR merges)…
I think you are saying something that is misleading. You said that if you write edit metadata to the file (JPEG, TIFF, DNG etc.) it will be a destructive edit. That's only true in the sense that the original file is changed to add the metadata, but the original image in that file is NOT changed.
Totally agree with you regarding the XMP files issue. Another reason to write the edits on XMP files is that it enables you to use PS with ACR to further edit the raw image without loosing the LR edits (eg, on a computer that has only PS installed). I also know that ON1 Photo Raw uses XMP files to record the edits but I'm not sure it's the same type of file.
Myself and quite a few users have the problem that Lightroom crashes after the function is activated. It is not uncommon for the Lightroom catalog to be unusable afterwards. In the Lightroom forum, the problem has now been recognized by the moderators when reporting a bug.
Catalog on local hard drive, images are on the NAS drive(s). When exiting LR back Cataloge file to NAS. Overnight LR image/Catalog duplacated to another external HD. I do not have an off sight storage yet.
Another informative upload Anthony, if i haven't already i will be checking the 'xmp' box. As you say, if my laptop has a catastrophic crash, my edits will be gone & that's that, thanks for taking the time.
Totally agree but I don't do it. The reason is that I'm working a lot with LRTimelapse (LRT). When you check this it will generate problems in the LRT-workflow. I stay sometimes several months in a second residence and make +thousands images which I like to correct in LR on my laptop before coming home (same subject with changing light conditions, HDR and panorama). But I don't like to work with the export function from LR and so for my general photography I save these XMP to my images with "Save Metadata to File". So when I copy my pictures (and XMP) after coming home I've these metadata directly with my images. So I don't fall into problems with LRT. So yes, if I wouldn't do time-lapses I would definitely follow your way !!!
What's more important to your workflow? In my view the advantages of keeping edits and metadata changes in sidecar files far outweighs any extra total file size/performance issues, which in the grand scheme of things these days (and in the future...) are minor by comparison. And if a few thousand xmp files are slowing your machine or eating your hard drive.... I wish I had that number of keepers.
Actually, there is a cluster size settings for hard drives that determines the minimum file size, not the actual size of the data in the file. So while the data in those side car files is small, the cluster sizes determines the size of the file on disk, and that is why the are wasteful. Having many of these tiny files around will eat up disk space quickly. Having said that, i’d rather eat up disk space and be able to recover my edits should the LT catalog get destroyed than to worry about disk space.
My workflow is that all of my raw images are stored as .DNG , converted on import so I have no need for sidecar files as the edits are stored in the DNG, The original Raw files (sony) are stored in a separate location until I no longer have a possible need for them and I backup both the catalogue and the images regularly. the images catalogue and even the lightroom settings are all on separate drive locations, The backups are offline and physically separate . I have a high speed SSD storage for the various locations used within my PC and I use NVME drives externally for transfer between laptop and Desktop if I do not have the option of Gigabit ethernet (or TB3 ) transfer, speed is not an issue ;) benchmarks are 3-4 times that of an M1Mac This method is not based on Photography needs but on 40+ years as an IT pro I have a high speed SSD storage for the various locations used within my PC and I use NVME drives externally for transfer between laptop and Desktop if I do not have the option of Gigabit ethernet (or TB3 ) transfer, speed is not an issue ;) [benchmarks are 3-4 times that of an M1Mac]
When I was working as a software engineer years back, we have 5 backups (we use 31/2” diskette) back than.. eg Monday’s work is save on Tuesday’s diskette and on Tuesday we will load the files from the Tuesday diskette but will save it on Wednesday’s diskette.. so at most we will only lose 1 day’s work if anything happens
I agree with you 100%. I had a HD crash once. That's all it took lost all my edits. Now I back up to 2 external drives. One I keep in my bag. I leave it goes with me. ✌🤟🖖
Great video Anthony - very clear and I agree with your reasoning. Oddball Alert: I'm guessing very few people follow my regimen. I really dislike sidecar files, not for the disk space they occupy but just the visual clutter. However, I'm a big believer in retaining my edits and in backing up. So I use LR primarily as a RAW converter. I don't feel I need it's cataloging features as I can search for metadata using other methods and have a well-organized method of naming folders/files. I also don't rate my images, which I know is a non-starter for many. So I import RAW images into an empty LR catalog, perform all my adjustments in the Develop Module, then write just the RAWs I've altered out to DNG. If I want to edit in Photoshop I export a TIF as well. After exporting I may edit the DNG in Photoshop as a smart object (if I don't need those edits preserved in the DNG. Which is ok, because in these instances, they are preserved in the PSD or layered TIF). I do not make image adjustments in LR which would be stored only in the catalog or xmp. btw, I believe I've noticed the new masking edits are also written to DNG (need to test further on that). I then delete all the imported RAWs from LR, keeping my LR catalog empty. If I need to further edit a RAW, I open its DNG in LR and repeat the above. This gives me a few advantages: I never have to worry about a corrupt LR catalog, there's no additional file clutter (other than the DNGs I keep in a separate folder), and my normal backups will protect all my edits via the DNG backups. I know I'm foregoing some of LR's advantages in doing the foregoing, but it works for me and I prefer this approach. If you've read this far, thanks for following along!
Once again Anthony there is distortion noise coming from your mike, any idea what is it. I use headphones and thought at first that was the issue. but I have not heard it on other channels? Love your work and content.
Xmp. is the only way u can work on 2 pc (when the photos are on the nas) whit the same catalog whiteout moving the catalog betwin the 2 ps . U just update the xmp ...
I do catalogue back up and XMP files, but manually. I'm not a computer expert but for my understanding in automatic mode the XMP file is written every time I push a slider. So it is a very small file for sure but written many times. No idea if this hurts a disk or SSD so... The other thing is that for bigger projects I may work with SmartPreviews, without the originals attached. After editing I attach the disk with the originals to generate the XMP files. During the editing process I'm reliant on the database, but I never had a problem with LR.
Thanks Anthony for explaining this issue in fine detail.My files are automatically backed up to OneDrive and periodically I do a back up to an external drive for extra security.
What happens if one uses both lightroom and ON1 on the same photos. Both editors work with a .xmp sidecar file which then actually is the same file which is read and written by both programs.
You said that if you ask LR to write edits to PSD, TIFF files etc. it becomes a 'destructive' editor, but doesn't it just write these changes to metadata, not the actual image data? That keeps it non-destructive no? p.s. just changed my backup location to Dropbox (thanks) ;)
I decided to take your advice and turned on the sidecar function ... my backup started three days ago and is still running ... of course, my catalog was already huge but still ... my question now is what happens if I just cancel the backup in mid stream ... does it revert to the original catalog or just stop adding new sidecar files? This was a really bad idea for me.
THanks for the informative tutorial! A question...my .xmp option is NOT checked. If I decide to do so, will xmp files be automatically created for all files that have edits?? (I suspect not!) Or only to each file when I make a change? So for the rest, there is no way to batch create them and I'm...as you put it...at the mercy of the hard drive gods, and my Catalog backup??
Hi Anthony, love your videos but I'm pretty sure that Lightroom doesn't have a destructive bone in it's body 😱 especially where DNG files are concerned. Sure, the edits are written to the DNG files metadata but not in a destructive way, you can always restore your DNG file back to it's original state at any time, hence not destructive 👍 This is one of the reasons why converting your RAW files to DNG is useful, you don't need the overhead of the sidecar .xmp files 😎
Agreed. And, at least on my system, if I check the xmp sidecar file checkbox, If I have converted my cr2 files to dng at import, I never get xmp files.
If I am using "Time Machine" on a Mac that backs up my entire hard drive to my external drive, is there any reason to backup in another location. (All files are on the external drive as well as the drive on the Mac.)
Tiny XML files (I am with Anthony on having them, see my other comment) - well "tiny" has nuances. If you have predominantly larger files on a hard drive, in formatting it, you can set a larger block size. Block size is the unit of drive space consumption. An empty XML file may start at some 10.7 KB, but on my image drive I set the block size to 64 KB and that is the minimum size of a file. In one folder I have 66 new images from my Z 7ii with lossless compression and this takes 3,548,053,504 bytes in disk when however the real data size of these files adds up to 3,545,730,950 bytes. This means that I have 99.93% utilization of drive space. The 66 tiny sidecar files add up to 729,630 bytes of data size, but these consume 4,325,376 bytes of space on the drive because of the block size and each individual file will have at least one. Or, I have 16.86% utilization. This is to also say, however, that I can do a lot of non-destructive edits without seeing growth in "on drive" storage utilization. As this utilization looks bad, we have to however see it in the context of the large photo files. If we combine the data size of 66 images and 66 XML files and compare this sum with the drive space used, then we get 99.83% utilization. Nothing to worry about.
For me, this is a big disaster. Since I activated "Automatically Write Changes into XMP" Lightroom crashes directly after starting. I don't know what else to do..
I find sidecar files most useful for maximizing compatibility across different non-Adobe applications. Without them, you're always starting with a blank slate in terms of metadata. I would disagree with your characterization that writing data to DNG files is "destructive" although it really depends on how you define the term. As originally used, "destructive" refers to making irreversible changes to the image itself, not the metadata. The ability to write metadata to the file and eliminate a sidecar file is considered a feature of the DNG format, though one that increases backup times exponentially since one small change to metadata means you have to backup the entire file again rather than just a small sidecar file.
I found that the reason i can't find my XMP files because use DNG files !!! The DNG files created i have no way to change it !!! i use Nikon Z 6 all my RAW files changed to DNG any suggestion how to stop it or keeping the NEF ???
Man, checked my check box .. like NOW! Any chance of a conflict between catalog and xmp files? Also, is there a way that I can create XMPs for files already edited from the catalog? About the checkmark to embed develop settings in JPG, etc files, does this affect the image file or just the metadata and what's the implications?
Does your comment about keeping your catalog on your computer apply even if you use an SSD external drive. I keep my catalog on an external drive in order to go from my laptop to my desktop.
As far as I know, you just should put your catalog on a fast drive. So an SSD is perfectly fine. At least, that is what I use on my desktop and the only reason for that was, that my C-drive was getting full, which was a small SSD as well. After that move I had no issues at all.
Anthony, thank you for another very informative video. You note that when using .dng files and when checking the box to automatically write changes into XMP, Lightroom will add the develop settings to the dng file. In this case are only the most recent Develop settings added, or is the complete edit history saved? If the history is saved would one be able to return to the original Image state in the history panel or be able to use the reset button to return to the original image?
Thanks for all of your wonderful and helpful Lightroom videos. In retirement I have been teaching myself photography and Lightroom but have realized that what I am least able to understand and to teach myself about is my computer. Can you suggest any books or other educational resources I can use to better understand computer specs as well as data storage, management and backups? It all feels like an overwhelming foreign language to me.
Great suggestion Anthony! I also moved my backups to my dropbox account. If my internal hard drive is maxing out, where do you suggest I move my Lightroom catalogs? To an external hard drive or to Dropbox. Thank you again for valuable advise.
Thank you ! Logical and well reasoned method. I was happy I've always done this when I upgraded to a new OS! Isn't .xmp a good idea if changing to new photo software that can't read Adobe's proprietary file types as well ? And I would note - if saving 5-20kb sidecar files would slow down a PC , then anyone would notice a slowdown in rendering/processing photos before they were even saved - meaning the system isn't hardware sufficient for LR anyway.
LRcc makes a zipped copy of the catalog. So why not backup the LR zipped backup rather than the whole catalog? I backup the zipped catalog every time I exit LR. Even losing one day’s work would be a huge hassle and it does not take much time and even including optimizing the catalog.
I create a catalog backup each time I close LR.... it seems that LR keeps every backup, even one's from years ago. Is there a setting (which I have not found) which I can set how many backups of the catalog to keep and to delete ones older that a prescribed interval?
You said in a recent video that we need to use XMP files if we use third-party software for processing images. I use Topaz Labs plugins, which seem to work without XMP files. My question is: do XMP files make Topaz run faster or more effectively?
I view my pics in camera raw… I put the whole shoot in…. I zoom to the eyes… if I like it I click auto… that makes a .xmp Then I go back and copy the pics with the .xmp and put them in a separate folder named my picks
In total agreement with you. And I will add that the xmp files are useful in other apps that can use them. Kelby's suggestions are years old if they were ever "valid". Thanks for the sensible suggestions. Note too that xmp files are simple text files like XML and need little processing. BTW I backup my main SSD every day, so my catalog being on it is fine; note the SSD in your computer is the fastest to be written to as it is on the motherboard of the computer. So backing up your main SSD (or HDD) daily is the same as what you practice IF it is backed up frequently (detail for me at EOD).
I, too, back up my entire catalog as well as the compressed version. The only caveat is that if the catalog is corrupted, backing it up copies the corruption whereas the compressed version (which I back up to external drives) passes through an integer it’s check.
Scott poleca zapisywać pliki w formie DNG i to dla tego nie poleca zaznaczenia tego pola. Ja z uwagi na korzystanie z innych wtyczek zacząłem zapisywanie plików w formie RAW z aparatu aby mieć możliwość lepszego działania tych wtyczek a szczególnie co do odszumiania zdjęć. Tak więc pomiędzy Panem a Scottem Celby nie ma żadnej rozbieżności zdań różnica polega na formacie zapisywanych zdjęć. Pozdrawiam z Polski.
This seems bizarre. Just back up your catalogue on either another drive and/or cloud storage. You say that in the end anyway and make an overwhelming case for that. The rest of the video before that is covering the loss of the catalog file. Just back that up. Two or three ways if you have to. Problem solved.
Why not adding another layer of security? If backing up your database (catalogue) is sufficient for you, that's fine. I could edit or fix a database file, if I wanted to (it's SQLite after all), but still feel better to have my edits also in normal text files.
That's what XMP files are for. To store information of your edits. Most raw editors will be able to read the XMP file and (re) apply these edits. But: I doubt, that every information, that's in the XMP file can always be interpreted by every editor. Complex masks could be an issue I think. Or generally speaking, every information, that's specific to capabilities that not all raw editors do provide.
Well, these small files do have impact on the performance of the pc. Reading all these small files has way more impact on the performance of a normal harddrive, then reading large files does. It causes lots of Input/output (IO) operations. But you have to store lot's and lot's of the XMP in a folder to even really notice a performance bottleneck. On a SD drive you will hardly even notice it, even if you have a couple of hundred XMP files in a folder. So, you are right, if you have a older computer with a low RPM harddisk it can have impact on the performance if you enable that Lightroom setting. I work in IT as a performance/problem analist. I backup my SD disks every saturday to a Synology NAS server, and the NAS server copies it al to a second Synology NAS server. So my data is save. 😉
Don't Panic, DNG editing is totally non-destructive it simply doesn't require the use of .xmp files as LR can write the changes directly to the files metadata. After all, Adobe created the DNG format so it's always going to be a good option with LR 👍
I totally agree with you Anthony., I have always checked the box for XMP data. The amount of data is tiny and takes up very little hard drive space. Who knows if some day that data might become corrupted, at least you have the sidecar alongside your original Raw file 😉
Sorry Anthony I also disagree with you. Size is not an issue but speed is really a big issue. You will notice the difference when writing an XMP. In my case I use a Loupedeck and when rotating a dial I make a lot of changes which means Lightroom has to constantly write the changes to the XMP. Now a days we all have SSDs and constantlty re-writing decreases the lifetime of that drive. If you are a user who has all of his iimages on the internal drive and doesn;t make a backup is just someone who just photographs for fun so then it isn’t a big deal if they lose some photos. You say most people don’t backup but everyone I know backups his files. Every decend macOS user also uses TimeMachine so there is always a good backup of the catalog. If a user doesn’t have a need for a decent backup then he probably keeps all of the photos on the internal drive so then when the laptop gets stolen he loses his catalog as well of all the photos. Even the argument of an external drive is a bit strange, because if someone steals your laptop and there are external drives next to it, they probably also get stolen. Therefor the whole argument of having LR constantly write the XMP’s is not the real issue or discussion. The real discussion should be having a decent allround backup strategy which just ensures that the catalog and all of the photos are kept secure. I keep my file on several Synlogy’s which replicate to an off-site location but for hobby users they could just subscribe to a online backup/storage platform like Amazon, Dropbox, Microsoft Live, iCloud, iDrive, BackBlaze, Carbonite etc. There are so many good options and they don’t need to cost that much.
Yes, I have and you can do it too. Remove a folder with some edited photos and their sidecar files from Lightroom, but leave them on your disk. Then import them. You'll see, that works flawlessly, all your edits etc. will be back, but it can take a very long time, depending on the number of photos to import. In a perfect world, you will never need this, but in the real world it's good to have - beside your db backups - another option to recover your photos from desaster, don't you agree?
U got problems whit you mic 🎤 , there is a buzzing sound , and this is your 5 or 7 clip i view whit the same problem :) Redo your settings for the new update ... Mabi ?
Totally agree with your opinion. I have benefited from the XMP files in recovering all my edits after a recent crash of my LR. I'm so glad that I have taken your suggestion before my LR crash. Thank you Anthony!
I totally agree with you, XMP files are so small in size it's daft NOT to want to have them in case of failure. I backup my catalog each time I exit LR nd it automatically syncs with the cloud. In addition I have backup which runs every night to an external drive. You can never have enough backups! Thanks for another great video. I love your calm teaching style and no nonsense attitude to giving us great tips. Happy to send you the price of a coffee as a thank you.
Another reason I write date changes to the raw files; I am digitizing family slides going back to the 1950s. I update the date of the digitized slide photo to the date the slide was taken on. That way the digitized slide has the actual date and not the date I actually digitized the slide.
Great advice Anthony! Even if you back up your catalogs to a hard drive or cloud, it's nice to have belt and suspenders. Do you know if the new version of LR with the additional catalog file stores all the mask-related edits in the XMP files?
I'm a "digital" photographer for too many years to be able to store all my images on a single drive. And .xmp files ( or whatever they're called in differents RAW editors ) are the best way not only to backup but mostly to archive images wherever i want. Mainly on external drives ( by subjects, years, jobs ... ) Whenever i need to review any edits i just plug in the drive and boom i can reedit a 15 years old image with all the improvements softwares offer you today. And if your hardrive have no space for those sidecar files what about new images ? Ad if sidecar files slow down your computer how can you run any software updates ?
In this video, I make my case for why, when one is using Lightroom, they should create sidecar, XMP files.
** I am an affiliate for all of the companies listed EXCEPT Affinity Photo.
The Best Sky Images I've Seen Available -- Ocudrone - bit.ly/3vtDpjR
*Save 10% with Discount Code: Anthony10
Please read my Code of Ethics Statement:
onlinephotographytraining.com/code-of-ethics/
Tap into Adobe's Creative Cloud: bit.ly/2zwQ0nW
Anthony Morganti’s MUST HAVE applications:
At least one Non-Destructive RAW Editor
Lightroom - bit.ly/2zwQ0nW
Capture One - captureone.38d4qb.net/y1Djy (Save 10% with Promo Code: MORGANTI10)
Luminar - bit.ly/2JUJxKw (Save with the Promo Code AM16)
On1 Photo RAW - bit.ly/2zJmA9I (Save 20% with Promo Code: AM17)
Exposure X6 - bit.ly/2U8UxrK (Save 10% with Promo Code: AnthonyMorganti)
PhotoLab 4: tidd.ly/2HhiN9X
At least one FULL Editing App:
Photoshop - bit.ly/2zwQ0nW
Affinity Photo - affinity.serif.com/en-us/photo/
My MUST-HAVE Plugins:
Topaz Gigapixel AI - bit.ly/3cDqa5J
Topaz Sharpen AI - bit.ly/3cDqa5J
Topaz Denoise AI - bit.ly/3cDqa5J
*Save 15% on all Topaz Labs apps - use the Promo Code: AMDISC15
or instead of Topaz Denoise AI:
On1 NoNoise AI - bit.ly/2zJmA9I (Save 20% with Promo Code: AM17)
Nik Silver Efex Pro 2 - tidd.ly/3dc4gYm
The Best Sky Images I've Seen Available -- Ocudrone - bit.ly/3vtDpjR
*Save 10% with Discount Code: Anthony10
My MUST-HAVE Mac Utility:
CleanMyMac X - Info: bit.ly/3jtVpU3
To save 10% on your purchase of CleanMyMac X:
1. Visit the CleanMyMac X store: bit.ly/3bEMuMx
2. Scroll down to the very bottom of the page and click on "Enter it now" under "Already have a coupon code?"
3. Paste the promo code TONY10 and hit "Verify"
4. The prices will be reduced, and you’ll be able to select the license and proceed with the payment
** Note that all of the promo codes listed above may not work on sale products.
*** I am an affiliate for all of the companies listed EXCEPT Affinity Photo. Please read my Code of Ethics Statement:
onlinephotographytraining.com/code-of-ethics/
Please follow me on Instagram: instagram.com/anthonymorganti/
View my recommended gear list: www.amazon.com/shop/anthonymorganti
As an Amazon Associate, I earn from qualifying purchases.
I use this software to record my screen and make parts of my screen enlarge and zoom out so you can see it:
Screenflow: telestream.pxf.io/DMrW2
Unsure of how to price your photography? Check the 2019 Guide to Pricing Your Photography:
amzn.to/2S1CiU7
I am an affiliate for all of the companies listed EXCEPT Affinity Photo. Please read my Code of Ethics Statement here:
onlinephotographytraining.com/code-of-ethics/
Thank you!
I wonder if Scott Kelby will be advising photographers not to buy cameras with sensors over 25gb as the images will take up too much space. It seems ludicrous that he advises saving 5KB by unticking the XMP box.
Hey Anthony,
One important FYI -
The actual number of bytes that a file occupies is almost always larger than the size reported by the OS.
That is, a file reported by the OS as 2K in size can actually take up to 4K, 8K and even 1M on disk.
This has to do with how drives work in blocks (sectors, pages, clusters etc) and the interfaces that the OS uses to interact with these devices.
For example, if you format a hard disk with an allocation unit of 4K then files will occupy space in multiples of 4K.
- files smaller than 4K will always occupy 4K on disk,
- files larger than 4K and up to 8K will occupy 8K,
- files with size 8K+1 byte will occupy 12K,
etc...
If you choose to use an allocation unit of 1000K then a file 1 byte in size will occupy a megabyte on disk.
So this can significantly affect the disk space occupied by XMP files.
10,000 XMP files can occupy 40MB, 160MB, and even 10GB of disk space.
So, you’re implying that On1 is not a FULL editing software?
I’m using a pentax K1 n the raw file is DNG. So if I have the automatic write XMP file means is destructive to the raw files?
@@stephenchung Not really. Please see my public comment of 2 days ago below for why those whose cameras create native DNG raw files need NOT worry.
Good video. One thing not mentioned was that files are tied to 'Allocation Units' which are created at the time the hard drive is formatted. An allocation unit is the amount of drive space taken when a file is created. On my PC, the allocation is 4096 bytes, so if I create a file of only one byte, 4096 bytes will be allocated. If I continue writing data to that file, the moment 4096+1 byte is added, a new allocation unit will be appended and the file will now take 8192 bytes and so on... In the case of the 5K file shown on the example screen, it is actually taking 8K of space (assuming your disk is also using 4k allocation units). The amount of this allocated space can vary. For example: You could format a hard drive with 32k or 64k allocation units, and if you create hundreds of 1 byte files, you could actually fill up a hard drive - even though the actual amount of data stored is very small (I actually did that once many years ago :) ) For the vast majority of users, having sidecar files will have little impact given the size of current hard drives, but it's good to be aware that there is some hidden 'overhead' whenever a file is created.
Good point. Used to be called “cluster size” for us old guys. ;)
I had a client ask for some files while I was traveling. I didn't have access to the main LR catalog or images, but I could download the main image and the sidecar from my cloud backup. When I dropped them into my laptop Lightroom, I got the image and the edits I already made. It saved me a ton of work.
“Most people don’t practice good backup…” Probably the understatement of the day! I was “in the business” (that is, computer business) for years until I retired, and I will guarantee that ALL computers and ALL disks will fail at some time. Usually the worst possible time. Not only did most of my clients fail in this area, I did myself until I realized how precious these photos are to me. When you first pointed out this feature, I immediately adopted it and am grateful for the advice. And you are right: a 25 kb file is measly .1% of a 25 MB file! Anyone who values their photos should backup, have a periodic backup of the backup, and occasionally make sure you test to make sure you can restore from that backup. AND, keep your very best photos somewhere really safe, if not the cloud, on an SSD in a bank vault! Seriously.
Totally Agree Anthony, the changes you make are also shown in bridge and photoshop as they read the xmp file when they open the image.
There are two, related reasons to have time/date changes written to raw files: when you are traveling through time zones or DST/ST changes and you have forgotten to make the time/date change in camera. I’ve been in both situations.
I'd been in Vietnam for 3 or 4 days, in 2019, before I remembered to change my camera's date/time from US Pacific.
As a PC user I definitely have more than one HDs in my computer. So my catalog is on the system SSD and the RAW files and catalog backups are on one of the other physical drives (of which again I have backups on two different external drives...).
I have to say that Scott Kelby was right back in 2012 when you could really see the performance hit from having the switch turned on. Now days there is no reason to not have it selected. For those who have had the switch off or if you only want to selectively save xmp files. You can hit Cmd (Ctrl) - S on a file or a selected group of images and the xmp's will be written to disk. Likewise you can select an entire folder by using Cmd (Ctrl) A to shorten the process or to apply a update. As I saw another poster stated the xmp's come in handy with other apps as well, Bridge being one that will show you the most recent version of the raw file with the xmp updates. Thanks for pointing this out Anthony, good stuff.
From now on I’m going to chuck the chuckbox.
I have to say, I agree with Scott Kelby. First of all, I convert to DNG and I know that is another debate. I am religious about backups and I use an external for this purpose, this includes not only my photos, but also all settings like watermarks, export settings, catalogs, etc. I then duplicate that hard drive to another hard drive for offsite storage. When I had that box checked, it made my backups excruciatingly long because if you go into a folder and even look at the photos in the folder, it means the file has technically changed and therefore my sync app picked it up and backed it up with the rest. I unchecked the box and voila, my backups are back to quick and simple, my edits are written to catalog and since I backup my catalog every time I do my photos, all is well and I am perfectly protected. In the case of xmp, it wouldn't bog me down the same but it is a wasted step.
ON1 support also highly recommends sidecar files be turned on just in case the catalog file gets corrupted. Also, I have my backup also backup the directory the catalog is in when it backs up my images. It’s not hard.
All good advice. Another reason is to preseve metadata across apps, especially between Adobe and others.
My computer crashed a month ago, Very thankful for backups. Just checked settings and need to change as new install was default settings.
Hi Tony. I'm a great fan of yours and frequently recommend your videos to my friends. However, I really disagree with your use of "destructive" in this video. As I understand "nondestructive editing", it means that the original IMAGE DATA is never changed within the file containing it. In this sense, DNG files edited in Adobe products are edited nondestructively, since the original raw (or whatever format) image data is never changed even though additional data about post processing is added to the file. On the contrary, bitmap image editors such as the original version of Photo Shop can irreversibly change the image file itself, and thus are "destructive" editors.
definitively !
if for any reason you loose your catalog and the backup, you can create a new catalog from scratch
i backup each time i exit lightroom, and backup automatically every day on an internal drive and on an external drive stored somewhere else
you're never too cautious...
Note that If you primarily shoot JPGs you may want to be aware that Lightroom will write all metadata changes directly to your files. This is technically only writing the exif data, so not truly destructive. But I had some corrupt JPG files a while back and noticed the date differences when I restored them. According to Adobe XMP is not supported on JPG/TIFF/PNG or PSD but I would have preferred the app not update my original files.
I work on 2 computers, main workstation and a laptop, I always use the XMP files so I can quickly transfer the images from one machine to the other and have all the edits ready. Also I have an old HD that contains my first 2 years of photographs and I didn't do this and I lost the catalog so now I have 2 years worth of developed images gone, only the RAW/tiff/jpegs are there but not the edits, I wish I used XMP files from the beginning.
I just had a laptop screen fail toward the end of a 3 week photography trip. The image files were stored on an external SSD, but the catalogue file was inaccessible on the laptop hard drive. Fortunately, I have been using the XMP option, so when I got back home, I just needed to import the last 3 weeks of edits. As an extra layer of safety, the XMP files are definitely worth any of the downsides, even if you only have the need once a decade!
Thanks for the video, Anthony. It raise again the question if it's better to convert the file to DNG or keep using the raw file from the camera. Hope you can make another video regarding this quesion.
Have a look at what happens when you use DNG and activate this option. Whenever you make the smallest of changes to the edit or keywords or rating - the DNG gets overwritten by a new version because for DNG Adobe doesn‘t create separate XMP files but embeds this information into the DNG itself. This will blow up your backup and also pose data integrity issues, especially for cameras which use DNG as native RAW files. So as useful I find this XMP storage, it is sabotaged by Adobe by overwriting the original DNG (even if I only use this whenever DNG is the only way, such as when doing panorama or HDR merges)…
I think you are saying something that is misleading. You said that if you write edit metadata to the file (JPEG, TIFF, DNG etc.) it will be a destructive edit. That's only true in the sense that the original file is changed to add the metadata, but the original image in that file is NOT changed.
I agree entirely with you about Tony's use of "destructive." I' plan to post a comment below on this unless someone else beats me to it.
Totally agree with you regarding the XMP files issue. Another reason to write the edits on XMP files is that it enables you to use PS with ACR to further edit the raw image without loosing the LR edits (eg, on a computer that has only PS installed). I also know that ON1 Photo Raw uses XMP files to record the edits but I'm not sure it's the same type of file.
I've always been doing this since day 1. Corrupt Databases can happen, but loss of ALL XPM files at once don't. Speed or space? Have enough of both...
Myself and quite a few users have the problem that Lightroom crashes after the function is activated. It is not uncommon for the Lightroom catalog to be unusable afterwards. In the Lightroom forum, the problem has now been recognized by the moderators when reporting a bug.
Catalog on local hard drive, images are on the NAS drive(s). When exiting LR back Cataloge file to NAS. Overnight LR image/Catalog duplacated to another external HD. I do not have an off sight storage yet.
I'm with you on this Anthony. Great explanation. Thanks again.
Another informative upload Anthony, if i haven't already i will be checking the 'xmp' box. As you say, if my laptop has a catastrophic crash, my edits will be gone & that's that, thanks for taking the time.
Totally agree but I don't do it. The reason is that I'm working a lot with LRTimelapse (LRT). When you check this it will generate problems in the LRT-workflow.
I stay sometimes several months in a second residence and make +thousands images which I like to correct in LR on my laptop before coming home (same subject with changing light conditions, HDR and panorama).
But I don't like to work with the export function from LR and so for my general photography I save these XMP to my images with "Save Metadata to File". So when I copy my pictures (and XMP) after coming home I've these metadata directly with my images. So I don't fall into problems with LRT.
So yes, if I wouldn't do time-lapses I would definitely follow your way !!!
What's more important to your workflow? In my view the advantages of keeping edits and metadata changes in sidecar files far outweighs any extra total file size/performance issues, which in the grand scheme of things these days (and in the future...) are minor by comparison. And if a few thousand xmp files are slowing your machine or eating your hard drive.... I wish I had that number of keepers.
Actually, there is a cluster size settings for hard drives that determines the minimum file size, not the actual size of the data in the file. So while the data in those side car files is small, the cluster sizes determines the size of the file on disk, and that is why the are wasteful. Having many of these tiny files around will eat up disk space quickly. Having said that, i’d rather eat up disk space and be able to recover my edits should the LT catalog get destroyed than to worry about disk space.
My workflow is that all of my raw images are stored as .DNG , converted on import so I have no need for sidecar files as the edits are stored in the DNG, The original Raw files (sony) are stored in a separate location until I no longer have a possible need for them and I backup both the catalogue and the images regularly. the images catalogue and even the lightroom settings are all on separate drive locations, The backups are offline and physically separate . I have a high speed SSD storage for the various locations used within my PC and I use NVME drives externally for transfer between laptop and Desktop if I do not have the option of Gigabit ethernet (or TB3 ) transfer, speed is not an issue ;) benchmarks are 3-4 times that of an M1Mac
This method is not based on Photography needs but on 40+ years as an IT pro
I have a high speed SSD storage for the various locations used within my PC and I use NVME drives externally for transfer between laptop and Desktop if I do not have the option of Gigabit ethernet (or TB3 ) transfer, speed is not an issue ;) [benchmarks are 3-4 times that of an M1Mac]
When I was working as a software engineer years back, we have 5 backups (we use 31/2” diskette) back than.. eg Monday’s work is save on Tuesday’s diskette and on Tuesday we will load the files from the Tuesday diskette but will save it on Wednesday’s diskette.. so at most we will only lose 1 day’s work if anything happens
I agree with you 100%. I had a HD crash once. That's all it took lost all my edits. Now I back up to 2 external drives. One I keep in my bag. I leave it goes with me. ✌🤟🖖
I make 4 copies.. 1 being “offline” meaning I only back up to it when I’m sure that my files are not infected by ransomware
Not so long ago I lost a week worth of editing after synchronising Lightroom since then XMP is checked and nothing will make me change my mind
Great video Anthony - very clear and I agree with your reasoning. Oddball Alert: I'm guessing very few people follow my regimen. I really dislike sidecar files, not for the disk space they occupy but just the visual clutter. However, I'm a big believer in retaining my edits and in backing up. So I use LR primarily as a RAW converter. I don't feel I need it's cataloging features as I can search for metadata using other methods and have a well-organized method of naming folders/files. I also don't rate my images, which I know is a non-starter for many.
So I import RAW images into an empty LR catalog, perform all my adjustments in the Develop Module, then write just the RAWs I've altered out to DNG. If I want to edit in Photoshop I export a TIF as well. After exporting I may edit the DNG in Photoshop as a smart object (if I don't need those edits preserved in the DNG. Which is ok, because in these instances, they are preserved in the PSD or layered TIF). I do not make image adjustments in LR which would be stored only in the catalog or xmp. btw, I believe I've noticed the new masking edits are also written to DNG (need to test further on that).
I then delete all the imported RAWs from LR, keeping my LR catalog empty. If I need to further edit a RAW, I open its DNG in LR and repeat the above. This gives me a few advantages: I never have to worry about a corrupt LR catalog, there's no additional file clutter (other than the DNGs I keep in a separate folder), and my normal backups will protect all my edits via the DNG backups. I know I'm foregoing some of LR's advantages in doing the foregoing, but it works for me and I prefer this approach. If you've read this far, thanks for following along!
On the backup, what about when your catalog is huge? Then getting it to a cloud is an issue too.
Once again Anthony there is distortion noise coming from your mike, any idea what is it. I use headphones and thought at first that was the issue. but I have not heard it on other channels? Love your work and content.
11:43 a DNG file is just a container holding the xmp and raw file. So how is that destructive?
Xmp. is the only way u can work on 2 pc (when the photos are on the nas) whit the same catalog whiteout moving the catalog betwin the 2 ps .
U just update the xmp ...
I do catalogue back up and XMP files, but manually. I'm not a computer expert but for my understanding in automatic mode the XMP file is written every time I push a slider. So it is a very small file for sure but written many times. No idea if this hurts a disk or SSD so... The other thing is that for bigger projects I may work with SmartPreviews, without the originals attached. After editing I attach the disk with the originals to generate the XMP files. During the editing process I'm reliant on the database, but I never had a problem with LR.
Thanks Anthony for explaining this issue in fine detail.My files are automatically backed up to OneDrive and periodically I do a back up to an external drive for extra security.
What the heck is a chuck box???
What happens if one uses both lightroom and ON1 on the same photos. Both editors work with a .xmp sidecar file which then actually is the same file which is read and written by both programs.
You said that if you ask LR to write edits to PSD, TIFF files etc. it becomes a 'destructive' editor, but doesn't it just write these changes to metadata, not the actual image data? That keeps it non-destructive no? p.s. just changed my backup location to Dropbox (thanks) ;)
I agree with you, Scott who?
A fast modern CPU, preferably a desktop (longevity, ability to upgrade) and multiple SSD/HDD's are essential to any pro-photog.
I decided to take your advice and turned on the sidecar function ... my backup started three days ago and is still running ... of course, my catalog was already huge but still ... my question now is what happens if I just cancel the backup in mid stream ... does it revert to the original catalog or just stop adding new sidecar files? This was a really bad idea for me.
I always have checked this.
I'm not a fan of catalogs, so they are disposable to me.
thank you for detail explanation
THanks for the informative tutorial! A question...my .xmp option is NOT checked. If I decide to do so, will xmp files be automatically created for all files that have edits?? (I suspect not!) Or only to each file when I make a change? So for the rest, there is no way to batch create them and I'm...as you put it...at the mercy of the hard drive gods, and my Catalog backup??
I just checked the XMP box in settings and LR created XMP files for ALL my pics in the catalog. Nice!
Hi Anthony, love your videos but I'm pretty sure that Lightroom doesn't have a destructive bone in it's body 😱 especially where DNG files are concerned.
Sure, the edits are written to the DNG files metadata but not in a destructive way, you can always restore your DNG file back to it's original state at any time, hence not destructive 👍
This is one of the reasons why converting your RAW files to DNG is useful, you don't need the overhead of the sidecar .xmp files 😎
Agreed. And, at least on my system, if I check the xmp sidecar file checkbox, If I have converted my cr2 files to dng at import, I never get xmp files.
I have received an error that said something like Xmp file does not match image. I forgot the exact error. Is this problem?
If I am using "Time Machine" on a Mac that backs up my entire hard drive to my external drive, is there any reason to backup in another location. (All files are on the external drive as well as the drive on the Mac.)
Tiny XML files (I am with Anthony on having them, see my other comment) - well "tiny" has nuances. If you have predominantly larger files on a hard drive, in formatting it, you can set a larger block size. Block size is the unit of drive space consumption. An empty XML file may start at some 10.7 KB, but on my image drive I set the block size to 64 KB and that is the minimum size of a file.
In one folder I have 66 new images from my Z 7ii with lossless compression and this takes 3,548,053,504 bytes in disk when however the real data size of these files adds up to 3,545,730,950 bytes. This means that I have 99.93% utilization of drive space.
The 66 tiny sidecar files add up to 729,630 bytes of data size, but these consume 4,325,376 bytes of space on the drive because of the block size and each individual file will have at least one. Or, I have 16.86% utilization. This is to also say, however, that I can do a lot of non-destructive edits without seeing growth in "on drive" storage utilization. As this utilization looks bad, we have to however see it in the context of the large photo files. If we combine the data size of 66 images and 66 XML files and compare this sum with the drive space used, then we get 99.83% utilization. Nothing to worry about.
For me, this is a big disaster. Since I activated "Automatically Write Changes into XMP" Lightroom crashes directly after starting. I don't know what else to do..
Same problem for me...
I find sidecar files most useful for maximizing compatibility across different non-Adobe applications. Without them, you're always starting with a blank slate in terms of metadata.
I would disagree with your characterization that writing data to DNG files is "destructive" although it really depends on how you define the term. As originally used, "destructive" refers to making irreversible changes to the image itself, not the metadata. The ability to write metadata to the file and eliminate a sidecar file is considered a feature of the DNG format, though one that increases backup times exponentially since one small change to metadata means you have to backup the entire file again rather than just a small sidecar file.
I found that the reason i can't find my XMP files because use DNG files !!! The DNG files created i have no way to change it !!! i use Nikon Z 6 all my RAW files changed to DNG any suggestion how to stop it or keeping the NEF ???
Just one question. What's Lightroom?
Man, checked my check box .. like NOW!
Any chance of a conflict between catalog and xmp files? Also, is there a way that I can create XMPs for files already edited from the catalog?
About the checkmark to embed develop settings in JPG, etc files, does this affect the image file or just the metadata and what's the implications?
Does your comment about keeping your catalog on your computer apply even if you use an SSD external drive. I keep my catalog on an external drive in order to go from my laptop to my desktop.
As far as I know, you just should put your catalog on a fast drive. So an SSD is perfectly fine. At least, that is what I use on my desktop and the only reason for that was, that my C-drive was getting full, which was a small SSD as well. After that move I had no issues at all.
Anthony, thank you for another very informative video. You note that when using .dng files and when checking the box to automatically write changes into XMP, Lightroom will add the develop settings to the dng file. In this case are only the most recent Develop settings added, or is the complete edit history saved? If the history is saved would one be able to return to the original Image state in the history panel or be able to use the reset button to return to the original image?
Thanks for all of your wonderful and helpful Lightroom videos. In retirement I have been teaching myself photography and Lightroom but have realized that what I am least able to understand and to teach myself about is my computer. Can you suggest any books or other educational resources I can use to better understand computer specs as well as data storage, management and backups? It all feels like an overwhelming foreign language to me.
Great suggestion Anthony!
I also moved my backups to my dropbox account. If my internal hard drive is maxing out, where do you suggest I move my Lightroom catalogs? To an external hard drive or to Dropbox. Thank you again for valuable advise.
Thank you !
Logical and well reasoned method. I was happy I've always done this when I upgraded to a new OS!
Isn't .xmp a good idea if changing to new photo software that can't read Adobe's proprietary file types as well ?
And I would note - if saving 5-20kb sidecar files would slow down a PC , then anyone would notice a slowdown in rendering/processing photos before they were even saved - meaning the system isn't hardware sufficient for LR anyway.
LRcc makes a zipped copy of the catalog. So why not backup the LR zipped backup rather than the whole catalog? I backup the zipped catalog every time I exit LR. Even losing one day’s work would be a huge hassle and it does not take much time and even including optimizing the catalog.
I agree with your position in this video!
I create a catalog backup each time I close LR.... it seems that LR keeps every backup, even one's from years ago. Is there a setting (which I have not found) which I can set how many backups of the catalog to keep and to delete ones older that a prescribed interval?
You said in a recent video that we need to use XMP files if we use third-party software for processing images. I use Topaz Labs plugins, which seem to work without XMP files. My question is: do XMP files make Topaz run faster or more effectively?
I agree with you Anthony
Also with the latest LR update there is a more efficient catalog and xmp handling so I’m pleased with that
I view my pics in camera raw… I put the whole shoot in…. I zoom to the eyes… if I like it I click auto… that makes a .xmp Then I go back and copy the pics with the .xmp and put them in a separate folder named my picks
You are correct and Scott Kelby is not correct. This video did not need to be made. It’s pretty much a waste of time
Thank you for these informative tips, Anthony.
In total agreement with you. And I will add that the xmp files are useful in other apps that can use them. Kelby's suggestions are years old if they were ever "valid". Thanks for the sensible suggestions. Note too that xmp files are simple text files like XML and need little processing. BTW I backup my main SSD every day, so my catalog being on it is fine; note the SSD in your computer is the fastest to be written to as it is on the motherboard of the computer. So backing up your main SSD (or HDD) daily is the same as what you practice IF it is backed up frequently (detail for me at EOD).
I, too, back up my entire catalog as well as the compressed version. The only caveat is that if the catalog is corrupted, backing it up copies the corruption whereas the compressed version (which I back up to external drives) passes through an integer it’s check.
Scott poleca zapisywać pliki w formie DNG i to dla tego nie poleca zaznaczenia tego pola. Ja z uwagi na korzystanie z innych wtyczek zacząłem zapisywanie plików w formie RAW z aparatu aby mieć możliwość lepszego działania tych wtyczek a szczególnie co do odszumiania zdjęć. Tak więc pomiędzy Panem a Scottem Celby nie ma żadnej rozbieżności zdań różnica polega na formacie zapisywanych zdjęć. Pozdrawiam z Polski.
This seems bizarre. Just back up your catalogue on either another drive and/or cloud storage. You say that in the end anyway and make an overwhelming case for that. The rest of the video before that is covering the loss of the catalog file. Just back that up. Two or three ways if you have to. Problem solved.
Why not adding another layer of security? If backing up your database (catalogue) is sufficient for you, that's fine. I could edit or fix a database file, if I wanted to (it's SQLite after all), but still feel better to have my edits also in normal text files.
is there a way, with those xmp files, to read your edits into another editor, so you can recover your edits outside of LR?
That's what XMP files are for. To store information of your edits. Most raw editors will be able to read the XMP file and (re) apply these edits. But: I doubt, that every information, that's in the XMP file can always be interpreted by every editor. Complex masks could be an issue I think. Or generally speaking, every information, that's specific to capabilities that not all raw editors do provide.
@@SwissNetHawk Thanks. Where can I find more info on this topic? One day I may need to leave Adobe.
Well, these small files do have impact on the performance of the pc. Reading all these small files has way more impact on the performance of a normal harddrive, then reading large files does. It causes lots of Input/output (IO) operations. But you have to store lot's and lot's of the XMP in a folder to even really notice a performance bottleneck. On a SD drive you will hardly even notice it, even if you have a couple of hundred XMP files in a folder. So, you are right, if you have a older computer with a low RPM harddisk it can have impact on the performance if you enable that Lightroom setting. I work in IT as a performance/problem analist. I backup my SD disks every saturday to a Synology NAS server, and the NAS server copies it al to a second Synology NAS server. So my data is save. 😉
I agree with you Anthony.
XMP files are uselfull in case you want to use your RAW files in another editor, Capture One or any other.
He using less clickbait like thumbnail pic?
I didn't know DNG editing wasn't non-destructive. My main camera, a Leica M10, shoots native DNG, so I'll leave the box unchecked.
Don't Panic, DNG editing is totally non-destructive it simply doesn't require the use of .xmp files as LR can write the changes directly to the files metadata. After all, Adobe created the DNG format so it's always going to be a good option with LR 👍
I totally agree with you Anthony., I have always checked the box for XMP data. The amount of data is tiny and takes up very little hard drive space. Who knows if some day that data might become corrupted, at least you have the sidecar alongside your original Raw file 😉
Sorry Anthony I also disagree with you. Size is not an issue but speed is really a big issue. You will notice the difference when writing an XMP. In my case I use a Loupedeck and when rotating a dial I make a lot of changes which means Lightroom has to constantly write the changes to the XMP. Now a days we all have SSDs and constantlty re-writing decreases the lifetime of that drive. If you are a user who has all of his iimages on the internal drive and doesn;t make a backup is just someone who just photographs for fun so then it isn’t a big deal if they lose some photos. You say most people don’t backup but everyone I know backups his files. Every decend macOS user also uses TimeMachine so there is always a good backup of the catalog. If a user doesn’t have a need for a decent backup then he probably keeps all of the photos on the internal drive so then when the laptop gets stolen he loses his catalog as well of all the photos. Even the argument of an external drive is a bit strange, because if someone steals your laptop and there are external drives next to it, they probably also get stolen. Therefor the whole argument of having LR constantly write the XMP’s is not the real issue or discussion. The real discussion should be having a decent allround backup strategy which just ensures that the catalog and all of the photos are kept secure. I keep my file on several Synlogy’s which replicate to an off-site location but for hobby users they could just subscribe to a online backup/storage platform like Amazon, Dropbox, Microsoft Live, iCloud, iDrive, BackBlaze, Carbonite etc. There are so many good options and they don’t need to cost that much.
Have you or any viewer ever tested the reconstruction of a catalog from xmp files? It really works? Any downside? Do you really get everything back?
Yes, I have and you can do it too. Remove a folder with some edited photos and their sidecar files from Lightroom, but leave them on your disk. Then import them. You'll see, that works flawlessly, all your edits etc. will be back, but it can take a very long time, depending on the number of photos to import. In a perfect world, you will never need this, but in the real world it's good to have - beside your db backups - another option to recover your photos from desaster, don't you agree?
I ALWAYS recommend to our students that they check that box. I agree 100%!
I wish I’d known this years ago.
U got problems whit you mic 🎤 , there is a buzzing sound , and this is your 5 or 7 clip i view whit the same problem :)
Redo your settings for the new update ... Mabi ?
A couple of days ago, Tim Gray wrote on this subject and agreed with you. However, you go into a better explanation.
You sold me!
Totally agree with your opinion. I have benefited from the XMP files in recovering all my edits after a recent crash of my LR. I'm so glad that I have taken your suggestion before my LR crash. Thank you Anthony!
I totally agree with you, XMP files are so small in size it's daft NOT to want to have them in case of failure. I backup my catalog each time I exit LR nd it automatically syncs with the cloud. In addition I have backup which runs every night to an external drive. You can never have enough backups! Thanks for another great video. I love your calm teaching style and no nonsense attitude to giving us great tips. Happy to send you the price of a coffee as a thank you.
Its common sense in my book i totally agree
Another reason I write date changes to the raw files; I am digitizing family slides going back to the 1950s. I update the date of the digitized slide photo to the date the slide was taken on. That way the digitized slide has the actual date and not the date I actually digitized the slide.
Great advice Anthony! Even if you back up your catalogs to a hard drive or cloud, it's nice to have belt and suspenders. Do you know if the new version of LR with the additional catalog file stores all the mask-related edits in the XMP files?
XMP all the way. Small price to pay for an added backup source
I'm a "digital" photographer for too many years to be able to store all my images on a single drive. And .xmp files ( or whatever they're called in differents RAW editors ) are the best way not only to backup but mostly to archive images wherever i want. Mainly on external drives ( by subjects, years, jobs ... ) Whenever i need to review any edits i just plug in the drive and boom i can reedit a 15 years old image with all the improvements softwares offer you today.
And if your hardrive have no space for those sidecar files what about new images ?
Ad if sidecar files slow down your computer how can you run any software updates ?