Like a lot of people I was mad when Google Announced that they were ending free photo storage for Google Photos a couple of years ago. I’d become pretty used to storing all of my photos there, and I didn’t really feel like paying Google more money than I already do, so I figured I’d find a way to manage all my photos myself. I’ve got a nice giant NAS and have got into Home Server stuff lately (more blog posts on that in the near future) so why not?
Well I had one major problem… my photos were a mess. I had kept some of them backed up on my NAS, my local PC, and some where only on Google Photos… and a few maybe just on Facebook? There was definitely a lot of overlap, so I needed some way to find the duplicates and remove them. It’s easy to find and remove IDENTICAL files, but what about a copy of the same photo that was re-compressed by Google or Facebook? The problem seemed too large to tackle, so I left it for a couple of years, trying to keep all new photos going forward organized and vowing to come back to the problem when I had a proper solution for it.
That solution is here, and it’s not even new! I came across a windows program called AllDup that does EXACTLY what I need (and it’s been around for years). It makes a tiny thumbnail style version of each photo you throw at it, and then logs any that meet a certain threshold of similarity (I found the default setting of 92% to work very well). You then go through the list and delete the ones you don’t want. Simple as that! You can even throw it two separate folders to compare against each other which worked great for comparing my Google Photos library against the mess of photos on my desktop. That way you can tell it “delete the ones from this folder where a match exists” if you know, for example, that the photos from Google will be more compressed than the ones already on your desktop.
I noticed very few false positives. It only happened when there were many shots in quick succession (with very little change between them) from a photo shoot, which were easy to identify and ignore. Overall a VERY impressive program!