HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Aren't there hooks on the filesystem layer that downloads them when you access them? E.g I can browse via terminal to my iCloud Drive somehow and cat etc works on files which aren't local (after locking to download them first).
I don't know about space-optimized storage on-phone. I know one setting for transfer to mac or pc - I have it set to "keep originals" instead of "automatic". There might be other settings I'm not aware of.
EDIT: actually, there are other directories (under /mnt but outside DCIM in my example) that seem to have other photo stuff, maybe edits? ymmv
> My current process for offloading photos off the iPhone
I'm not sure about Linux, but my workflow on Windows and MacOS is to frequently back up my iPhone locally (which you should do anyway because few incorrect PINs can security lock your phone [1]) and use utility like backup extractor (e.g. [2] but there are many others) to extract all photos from the backup. This effectively removes the need to use iCloud.
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
With Photosync I have our photos export to my NAS and have it update the file names with the timestamp + original file name, which makes it so much more sane to sort through.
I use and like PhotoSync but I thought it doesn’t export unmodified originals but your edited versions. Personally I like this behavior better but that might not be what you want?
I’m not sure Apple allows any third party app to access the unmodified originals. Imagine you crop a photo to remove some embarrassing part. A third-party app can just recover that? What a privacy risk.
Of course this won’t matter if you don’t do any photo editing on iOS.
It's right to question this. I had taken https://www.photosync-app.com/support/basics/answers/does-ph... ("PhotoSync transfers the original, unaltered images including all EXIF and GPS data") on face value without thinking of all of the iCloud behavior. Even limited copies would suffice in my case, but that's insufficient for others.
If you manually edit the date on a photo, is that also stored separately from the image file itself? Wondering because I've noticed photos I've backed up to Immich from iOS photos don't respect that edited date and reflect whatever the original date was.
I've been thinking about looking into a fix for this since it's bugged me a bit.
Yes — in Apple Photos the manually edited date/time is not written back into the image or video file. It’s stored separately in the Photos library database, which is why tools like Immich usually fall back to the original capture date in EXIF / QuickTime metadata.
Photos Backup Anywhere does handle this case: it reads the adjusted date from the Photos library and stores that modified timestamp in its own SQLite database, linked to the backed-up file, so the edited date isn’t lost even though the file itself isn’t rewritten.
Thanks for the reply, I appreciate it. Does Apple provide any APIs for interacting with this metadata or was this something you had to implement yourself with lower level DB lookups?
> As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My free open source tool osxphotos [1] does this. In addition to an "osxphotos export" command to create archival exports with all metadata, it also has an "osxphotos import" command that can restore all metadata including albums (with exception of named persons/face regions, though persons can be converted into keywords on import). It can also read XMP sidecar files to restore metadata from those. It's a CLI and definitely a power user tool.
It also includes an "osxphotos sync" command that can sync metadata between two libraries. Some people use this to sync metadata like favorites from iPhone library to the Mac library. (can't go the other way unfortunately)
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
Yeah but for some phones the storage is full ;) I had to figure out how to remove just enough to root it and then just adb in for one phone then I learned that if I extract too quickly it gives up the ghost so I have to add a delay
Awesome! Thanks for posting, I've been looking for exactly this kind of project that takes end to end pristine restoration seriously.
The FAQ states you back up the originals as plain images files in YYYY/MM/DD format, which is great for integration with other tools. What about metadata? Is it in a format that lends to integration with other tools, say if you stop developing/supporting your app for some reason?
Metadata is stored in an SQLite database alongside the backed up files. The format is not documented, but should be self explanatory if you examine the schema.
This is a great app! Thanks, I bought it. I'm backing up to a network share over SMB. Just wanted to let you know that the error message in the event that the share becomes unmounted during a backup is a bit obscure: it just says "pdb.sqlite" doesn't exist. Stopping and restarting the backup fixes it. Might be helpful to provide some clearer error handling / messaging and guidance for disappearing network shares?
You’re right about that error message. What’s happening is that when the SMB share gets unmounted, the app can no longer access its SQLite database (pba.sqlite), and the resulting error isn’t very user-friendly. I agree it would be much clearer to explicitly detect a missing or unmounted network share and provide guidance.
I’ve added this to my list of things to improve. Thanks again for taking the time to report it — feedback like this is super helpful.
My pleasure - much appreciated and thanks for being so responsive.
I’ll just add in here a sneaky humble request for graceful automatic pausing and restarting as network shares (or other volumes?) (dis-)appear - although I imagine that one’s a bit trickier to implement.
All the best! Very satisfied user. Great replacement for the script + cron job I was using before.
Pretty severe -- ADP allows users to store data with e2e encryption.
Disabling ADP is a pretty serious* thing to do -- and pretty disappointing since I was interested in the product. Since it's OSS though this might be something that can be worked around in the future.
That’s a good question. To be completely honest, I don’t know much about Advanced Data Protection myself, and I didn’t do anything specific in the app to detect or interact with it.
I’m actually curious: how did you discover that it doesn’t work when Advanced Data Protection is enabled? Was it through an error message, incomplete backups, or something else you noticed?
The app that the person you're replying to ('gerdemb) created (Photos Backup Anywhere) is different than the open-source CLI tool that is the HN submission (icloud_photos_downloader).
Yes — Photos Backup Anywhere supports Apple Family Sharing
If the app is shared via Family Sharing, each family member can install and use it on their own Mac with their own iCloud Photos library. The app works with whichever Photos library is signed into macOS on that machine, so you can absolutely use it to back up your partner’s images as well.
Yes — that’s exactly what Photos Backup Anywhere is designed to do.
The app creates file-system-level backups of each individual photo and video from your Photos library. Every asset is exported as a real file on disk (HEIC, JPEG, MOV, etc.), not a database blob or package, so you can browse, copy, verify, and back them up with standard tools.
In addition, the app keeps a small SQLite database alongside the files to track metadata (including edits, albums, and relationships like Live Photos), which allows accurate restores and verification while still giving you transparent, plain-files access to your media.
Yes — that’s exactly what it’s designed for. You can back up photos from one Photos library and later restore them into a different Photos library. The restore process recreates the items in the target library using the data stored in the backup.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
Nice to see my project on Hacker News! I started this almost 10 years ago and haven't been involved with maintenance for a long time, but I'm glad that people are still finding it useful.
Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.
That’s not true. On any Mac or iPhone you can choose the iCloud Photo Library storage option to download all instead of letting the system optimize the storage. And if you turn off iCloud Photo Library, it will also try to download it all. I know this because I stopped using iCloud Photo Library and that was how I got all my photos downloaded.
+1 to this method. After optimise storage is disabled on the Mac, wait for all photos to download. Then, open the photos library bundle and you'll see every photo there, full res. Copy them wherever you like.
Also, if you leave optimise storage disabled and continue to use Photos, every photo will be cloned in any local or cloud backups of your machine. This strategy creates additional photo redundancy separate from iCloud while still benefiting from library syncing.
I once exported my photos out of the iwhatever library. They weren't in the cloud, Apple hadn't managed to trick me into turning that on.
What I remember is that I opened the library in finder and in mc, got scared by the readable-only-by-machine directory structure and used a 3rd party tool to export them to date labeled directories.
This was my strategy too, but with a disgusting script which quit photos.app, rsync the photo library to a network share, then reopened photos.app so that it kept downloading from iCloud.
Not sure if the open/close is required, but I didn’t want to find out.
I don’t fully trust iCloud Drive / Photos therefore I use FSViewer to download all photos from my iOS device du jour (making sure to keep the HEIF formats), this way I get the Edited (slo-mo, live, portrait, usw) and pristine versions as Jobs intended. All kidding aside, after the gray area gate of 2017-2021 I had to find a more reliable backup workflow. As of today I only use iCloud Drive / Photos to extract some RAW photos that for some reason some picky apps don’t save to the photo album (looking at you ProCam 8.0). I made several tests including hash comparisons and imagemagick diffs and I am quite pleased.
Someone gave me a new iPhone (120GB) and a new MacBook Pro and asked me to download all their photos from iCloud. Long story short, after 120GB of photos were synchronised to the iPhone, the MacBook Pro refused to copy them, and now there's no storage left on the iPhone.
Also, Photos on Mac doesn't have an option to download photos directly, so the only valid option Apple offers is to download them through the web interface (max 1,000 at a time).
There is no official way to download iCloud library that is over phone capacity. Period.
You keep asserting to the contrary, but I've been syncing my entire photos library to my Mac for years, since it was iPhoto even.
Obviously if you have a larger photos library than storage space on a particular device, you cannot synchronize the entire library to that specific device. e.g. my photos library vastly exceeds my iPhone 13 mini storage, so on my iPhone, I don't sync everything. But my Mac has 2 TB of storage, and Photos is setup to sync all my photos, and does so, reliably, and has been, again, for years now.
Additionally, unlike with this open source tool, I can keep advanced data protection enabled.
I have tried 3 different Macs with different versions of macOS prior to looking for a workaround, and everywhere the result is the same: old photos are not downloaded automatically from iCloud, and there is no button to start this process - for this exact reason.
Want to prove me wrong? Create a new macOS user and open Photos with your iCloud. It will be empty until you start copying photos from your phone. It will take much less time than arguing here.
You're arguing with a lot of people who have personally seen this work. You can listen to other people. You can also go to an Apple Store and let them show you what's going wrong here.
Perhaps no one here has tried to download an entire iCloud library at once, or perhaps size is an issue, but that doesn't change the fact that there is no download button for iCloud Photos and iCloud Photos Downloader simply solves this. That's what this post is about.
I can personally confirm I've downloaded an entire iCloud library at once, to a brand new Mac, using the 'Download Originals to this Mac' option. As have many others here, I would think.
That's literally what that option is for.
If it's not working for you, you might be dealing with a bug, or perhaps you haven't given it enough time to sync. If you go to Photos > Library and scroll down, it should show you the sync status.
Thanks, that was a relief because I realised I didn't see the sync status at the bottom. It appears that Monterey hides the status message at the bottom by default, and I had to pull the page down twice to see it.
Long story short, iCloud wasn't syncing photos "due to performance" and this message was hidden.
No worries! I don't understand why Apple is so averse to surfacing the status of things, especially highly sensitive and finicky things like online sync. It would dramatically improve the feel of the software if it didn't seem like it just inexplicably wasn't working half the time.
iCloud Photos Downloader is an option, yes, but it is incorrect to say that Apple does not provide an official way to do this on Mac. Again, I direct you to the Apple Store so someone can show you in person, since you won't listen to anyone on here.
Photos on MacOS indeed synchronise photos with iCloud.
After our conversation I had tried to understand why I indeed don't see any status and I found out, that to get one in Monterey iOS I must need to scroll down of the collection and after, at the bottom pul whole page for the second time. Status message appears and it was saying that syncing was disabled due to Mac performance (I didn't asked for this).
Apologies, for misleading, code543 and thank you for consistence.
However, I must admit that I'm happy that found iCloud Photos Downloader as a result, also I liked that it's downloading all photos in date/folder structure.
Let me be one more voice telling you that you are wrong. I just did this morning.
In settings, "download originals to this mac", select all photos, file -> "export unmodified originals" will trigger the Photos app to download every file from iCloud into your local library (as well as exporting them to wherever you want)
I guess "there is no download button" but dude...I don't need iCloud Photos Downloader.
Thanks for letting me know. May I ask what macOS version you use?
Unfortunately, I'm unable to locate any button, status bar, or option to refresh or pull everything from iCloud in macOS Photos. There aren't even any details showing what percentage of iCloud is currently synchronized with macOS Photos. With nothing to debug, I can only conclude that for some reason the sync isn't working in my case.
It's great if this works for you and you don't need iCloud Photos Downloader, but for some reason I don't have that luxury.
That doesn’t sound right. My photo library is larger than my iPhone’s storage yet downloads fine on my Mac. Just need to make sure “optimise storage” is enabled on the iPhone and disabled on the Mac.
Once everything’s downloaded on the Mac, you can either export through the Apple Photos menu or just copy the “originals” directly from the Photos bundle.
This works because you had synchronised your iPhone with your Mac previously. If you start with an empty Photos library and phone, it is impossible to put all the photos on the phone and thus transfer them to your Mac.
Thanks to Apple's exceptional software quality the app has plenty of bugs and good luck exporting a lot of files out of said library - you're in for an endless game of spinners (it does some network IO on the main thread), "not responding" and memory leaks.
> Thanks to Apple's exceptional software quality the app has plenty of bugs…
I use Photos for macOS daily and I've never run into a bug with my 50K+ photos library. (To be fair, Photos doesn't do that much, and I use it more as a master catalog with Aperture's spiritual successor Nitro.)
> …and good luck exporting a lot of files out of said library…
Not sure why you would need luck to copy the "Originals" folder from the library package.
> I've never run into a bug with my 50K+ photos library.
Have you tried dragging & dropping a photo from the Photos app to your desktop to export it as a file?
I just tried it and while the Photos app UI didn't freeze (I guess either my memory is bad and the spinner behavior was on imports, or they fixed it since 2 years ago), but it takes ~3 seconds for a single photo to appear as a file on my desktop (with no UI or any indicator that something is happening), and dragging & dropping 45 photos took over a minute (again with no progress indicator).
Granted, it turns out I didn't have "download originals" on (not sure if it was always like that or got reset during an update) so hopefully for Apple it's that - but I still think at the very least a progress indicator on what seems like a network IO operation is in order (and I wonder what happens if I interrupted my network connection during its operation - does it just silently never complete and my files never appear, actually show an UI, or just crash?).
Edit: well nevermind, despite all the files apparently exporting just fine, ~5 minutes later after initiating the operation I just got a hideous popup that some files failed to export: https://imgur.com/a/SFXZB5N
It's progress, at least it's the first indication in the UI that something was actually happening. You will notice that the error text is truncated, the only way to read it all is to resize the window (no hover text) and for some reason the horizontal scroll in this UI element does not follow your touchpad - you have to scroll, lift your fingers, and ~1 second later the UI suddenly applies your scroll operation.
Also, it turns out that resizing the window finally readjusted the table control and now the table actually matches the size of its container and isn't scrollable anymore. Which might be a good thing as it at least alleviates the bug mentioned previously, but then why did the control initialize oversized to begin with?
Again this is the kind of jank I'd expect from Linux, and nowadays maybe Windows. But not Mac.
> copy the "Originals" folder from the library package
My bad, I wasn't aware of said folder; I treated the "package" as an internal implementation detail equivalent to a proprietary format and wasn't going through it. I don't think it's a fair expectation to have (potentially non-technical) people to right click "Show package contents" while the usual double-click on the file just opens the library in the Photos app.
Technically, there is: users of the European Union can get a full export of all data that Apple has about them, including all the stored photos. It can be requested from here: https://privacy.apple.com/
How does the archive they provide look like? Many zip files?
I would like to retrieve them and offload to another storage service but I don’t have local storage enough to hold all of it at the same time, unpack and then reupload. I would need to do it in stages.
Yes, many ZIP files. You can select the ZIP file sizes, from 1 to 25 GB, iirc. Although a few end up larger than 25 for some reason. And took 1-2 days for Apple to "prepare".
You can request a chunk size and then it prepares them. I specified max chunk size and it took almost a week to give me a list of file downloads from 45-60GB each. 31 zip files to download.
It sounds really weird that instead of making a separate utility, or allowing you to download iCloud Photos in the native Photos application on Mac, Apple requires you to go through a legal procedure.
I'm OK with clicking a button to download all photos to Mac, but there is no such button. Or maybe there was one previously, but it has now disappeared.
If your Mac doesn’t have enough space, export them to a USB hard drive or if you’re using the download originals option, first move your library location to the USB drive as also described on the link above.
That's exactly what I expected to work, but for some reason this approach failed for me on a new Mac with an empty Photos library. I enabled "Download Originals," but 10+ years of iCloud photos never appeared. There's no manual "fetch all from iCloud" button, no progress indicator, no way to diagnose what's wrong - the sync just silently fails. Luckily, iCloud Photos Downloader bypasses Photos entirely and pulls directly from iCloud.
You were absolutely right. Someone in the thread mentioned a status indicator, and it triggered me that I hadn't seen it at all.
It appears that on Monterey I need to pull the page down twice to show the sync status, and when I found it, the message said that photos weren't syncing due to "Performance."
> Users of Google and Apple’s photo cloud services can now transfer images between them. It was already possible to export photos and videos from iCloud to Google Photos, but now it can also be done the other way around: from Google Photos to iCloud.
As long as you are signed into the Mac with the same iCloud account used on the iPhone, this will download them all. No, you do not need to get them all downloaded to the iPhone ever for any reason for this to work. Period. You need to stop repeating that, because it is wrong. How many people have to say the same thing?
Yes, you will have to go into a hidden folder to access the Originals once they're downloaded if you want to copy them somewhere else, but it's like two clicks.
I've been using Mac since Mac OS X 10.4 (~2005) and was under the same impression.
However, in reality, when you use the same Apple account on both devices with the Photos app on macOS (yes, with the 'Download Originals' checkbox enabled), it only downloads photos that you upload from your phone.
And if you look at the iCloud tab in the Photos app, it says 'Automatically _upload_ and store all your photos and videos in iCloud', so it works from Mac to iCloud, and doesn't help to download full iCloud library.
No, you are not correct. How many people have to tell you this?
It absolutely works the way I said it does, because I have seen it work that way. Just because you accidentally turned off iCloud Photos in your Apple Account settings on that Mac (or some other similar issue) does not mean it does not work this way when properly signed in.
If you want something to try, go to System Settings -> Apple Account -> Photos and see if "Sync this mac" is turned off. It needs to be on. There could be other ways that this feature is disabled, but that is one of them.
Not seeing something work is not evidence that it does not work. You have not seen it work, but that is not proof it does not work.
Seeing it work is evidence that it works. I have seen it work.
Other people have seen it work that way, and their replies are all over this thread. Apple documents that it works this way.
Yes, it will upload photos to iCloud if enabled, but it also downloads them.
When you take a new photo, it synchronises with all your devices, and therefore you see it on your Mac, iPhone, etc. However, if you get a new Mac (I got one because my library was under capacity), Photos will not start synchronising your 10-year-old photos until you process them through the phone.
When you first turn on iCloud Photos, it can take a while to upload your photos to iCloud. You can continue to use Photos while your photos are being uploaded.
To have iCloud Photos sync photos to all your devices, sign in to the same Apple Account to turn on iCloud Photos on all your devices.
One of us is missing something.
In Photos.app I clicked download originals. The photos are there on my Mac. It’s bit gross to get at them though - right click on app > show package contents.
Are you wanting a way that doesn’t involve the photos app?
If you open the Photos app (macOS) connected to iCloud with an empty library, there will be no photos until you import them from your phone. Hope this is clear now.
iCloud via browser has a limit of 1k photos per download.
My bad — it appears that Monterey wasn't syncing with iCloud due to "Performance." Photos on Monterey hides the status message at the bottom of the page, and I had to pull the page down twice to finally see it.
This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?
Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.
I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.
It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)
I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?
While not free, and not for any other platform than macOS. The program Parachute[1] in the App Store is very nice in downloading both photos from your library as well as files from the various locations.
Another option for iOS at least is PhotoSync. It’s nice, you can pull from photos and push to basically any remote service or local server. I have it backing up to both my nas and b2.
It works well enough, but it's not without flaws either.
The desktop version works reliably, if you can get macOS to keep shares mounted for long enough, and mount them on request. The scheduler is also kinda wonky.
The iOS version has so far never finished an incremental backup overnight of our ~1TB individual libraries. It handles resume/suspend well, but for some reason, while it exports unmodified originals, it doesn't include AAE files, which the desktop version does.
PhotoSync does everything right, with the exception of trying to keep state of what has been exported, which makes little sense as it doesn't support restoring photos.
Anyone know if it works with ADP? I emailed them months ago but no one ever replied.
On a related question, is there a download solution that does work with ADP? I’m looking to mitigate any potential account lockout issues for family members (and, no, they will not switch out of the ecosystem).
It does. It uses PhotoKit to access photos, so it basically uses your Apple Photos app (iOS or Mac) to download the photos.
The only scripted solution I can think of that works with ADP is osxphotos[^1], but that also uses PhotoKit, and requires the user to be signed in.
Personally I use PhotoSync [^2] to backup our photos from phones to a NAS. It works reliably, and supports exporting unmodified originals as well as edited versions, and XMP/AAE metadata alongside it.
Automatic de-duplication of photos with the same name
I recently went through a year’s worth of photos from my wife’s phone, and found three distinct “img_0001.jpg”’s just in that single year. Apple’s naming convention is so short sighted that I’d be terrified letting a piece of software try to dedupe it “by name “
I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation
I’ve been using usbmuxd+ifuse to copy the photo files straight from the phone. No need to wait for an upload/download to some remote server, just a direct cable from the phone to my computer. I get the original files, and can even move (instead of copy) to clear up the phone.
From the project describtion: "Looking for MAINTAINER for this project"
Honestly, Apple should officially maintain tools like this. However, for obvious reasons, such as the iCloud subscription revenue model, Apple will not do it. In fact, Apple may even make life harder for such tools.
I’m in the midst of a backup-to-local project and, with this post to HN, I’m worried an Apple project manager will be on a mission this morning to get his team to cripple this software.
Thanks for this project. Our family generates about 2TB of media a year, and it’s been like that for a while, so we’re sitting at roughly 12TB total. That’s very much the long tail of personal media.
I’m not ready to pay $60/month, but I do like iCloud’s memories and other photo features. My compromise is simple:
- I use docker-icloudpd to download our iCloud Photos to local storage over time. It’s been the most practical way I’ve found to back up multiple accounts into one place, though it does require occasional re-auth every so often.
- I keep only the last ~2 years of media in iCloud and delete older ones after they’re archived locally.
- For browsing and searching the older archive, I use Immich, which has been a great self-hosted personal photo cloud experience with a modern app feel.
For storage, I’ve found fast local disk matters a lot once you’re digging up photos from 5+ years ago. Something like an OWC 4M2 with M.2 drives keeps the experience snappy; a typical HDD-based NAS can feel sluggish when you just want to quickly pull up an old memory.
How much work is it to maintain an Immich instance? I'm trying to keep my digital setup lean, but I can't find a simple way to get photos off my iPhone and onto my home server. One way synchronization is my only goal. Extra features are not desired.
If all you want is a 1-way sync from phone to server, then just go with PhotoSync. You can set up a simple SFTP connection so you don't even need anything on the server.
Immich is great if you want to be able to delete things from your phone to save space and still have access to all your content served from Immich on your server though.
Music and AI features are still lagging in Immich, and I can understand why. Immich machine learning is not flushed out yet. If Immich has plans for creating marketplace for extensibility like plugins, in the current era of Claude code, I am sure we will end up with many options or features.
Like Memories feature in Apple which uses media to make short form video but also picks and plays music with them. We love those videos. Enabling feature like that involves understanding metadata and media and music.
Encryption at rest. If someone breaks into my house and steals my server I'd rather they not be able to get data. I can do LUKS but I really want the data to only ever be decrypted client side.
My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation.
It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.
This. I don't understand why Apple don't have another checkbox beside the Download Originals to this Mac that reads 'Store Backup of Original Photos on Timemachine' This is all that's needed to solve the issue. I actually bought a Mac Studio, and a USB disk, just to be able to download originals of my photos for local backup, since a MBP is effetively a mobil limited device just like an iphone.
"I actually bought a Mac Studio"... "I don't understand why Apple don't " ... wait a minute
My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled.
I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy.
IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected?
I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.
Could the first obvious improvement please be its speed?
My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.
I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product.
I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS.
Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
A huge chunk of that 3-4GB is large files that have minor changes. Time Machine doesn't have any sort of delta support so backups the entire file again, like my local Messages or Contacts database. But I think slowness is caused by file count, not file size, so even though it's backing up 3GB+, the total number of files changes isn't that high. (I suspect).
I also use a modified version of this script[0] to identify everything that changed in the most recent Time Machine backup. This is hugely helpful and lets me find unimportant things that are the source of lots of unimportant changes which I can then exclude with `tmux addexclusion -p <PATH>`. For example I exclude 'node_modules' folders for anything that gets regularly updated. This removes ~10k files that would otherwise be wastefully backed up. Speeding it up is much more about reducing file count than total size.
I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.
In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion
> Slow motion videos are also not proprietary, they’re just a plain video file.
I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).
The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move.
For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.
It is no fun to have old iCloud photos deleted unexpectedly. Apple has provided plenty of footguns, even if they really are user errors. For examples: (1) during device restores and (2) premium subscription management fumbles.
Product idea: Apple should offer a paid service to restore the "old backups" of photos that are no longer accessible via iCloud UI/API, which were soft-removed for missing the subscription quota or whatever, if Apple happens to have that data tucked away in cold storage somewhere.
Case in point, I had some c. 2016 era photos in iMessages that I thought I handled right to not lose from iCloud, but they are apparently nowhere to be found in iCloud API based on recent checks. More than mildly irritating.
I should have used an iCloud photos backup tool like this much sooner.
Print what you want to keep onto archival paper with archival dyes. Everything else will atrophy.
Wish I’d seen this 3 days ago. Needed to backup our Shared Library and did the following (about 10K photos/videos at 300gb, had enough space so full downloaded to MacBook, not optimize. 1. Repair iPhoto library, (Wait 24 hrs to re-sync to iCloud) - initially looks like it moved all photos to personal. 2. Select small chunks, by year worked well enough - selecting All gave me the spinning ball. Then export unmodified to external hdd into folders organized by year. 3. Moved entire photo library file to another external hdd. 4. Open iPhoto and select external hdd library as primary library, let it re-sync to iCloud (Wait 24 hrs). iPhoto now running off external HDD library and I’ll backup to separate external HDD monthly. Repair function and wired Ethernet connection were biggest game changers to previous attempts.
Not in my experience, previously ran it setup externally with a shared library for at least a year or longer, https://support.apple.com/en-ca/108345. Typically I backup to external drive 2 ways, copy of library itself and unmodified export, only had to go through above after I needed to rebuild the laptop. OP app and or https://photosbackup.app/ , seems like they might enhance my setup, will have to take a closer look.
There's some technical reason (or non-reason) why Mac OS does not accept the System photolibrary on an external disk. It prevents certain things from happening which may or may not mater to you. Otherwise you can switch librarys by holding Option during photo bootup.
You can go to https://privacy.apple.com, log in with your Apple ID, select "Request to transfer a copy of your data" and then select "iCloud photos and videos to Google Photos".
I'm sure grandparent meant to modify it so they'd just have to click "Backup to cloud" on their iPhone and instead of the iPhone sending their files to Apple's servers, it sends them to a local backup server...
It's macOS-only and intentionally minimal — the goal is just to download originals from iCloud Photos to disk without syncing everything into Photos.app first.
To be clear on limitations: it doesn't preserve albums or other metadata yet, and it's not meant to replace more full-featured tools. It’s mainly for the "I just want my photos off iCloud for backup" case.
Reading the comments here, it sounds like metadata preservation is a big pain point for many people — I'd be curious whether that's the first thing folks would want added, or if simple bulk export already covers most needs.
All my vertical videos in iCloud show up cropped horizontal for some reason. If I go to edit I see the whole video. I really do not want to trust any cloud provider to maintain my years of archives of family photos and videos. Glad things like this exist. I just need properly date-foldered files, without no duplciates. Is that so hard?
Does anyone have any idea for why Apple makes it so difficult to keep photos downloaded?
For context, try tapping 'optimize photos' in iPhone storage settings and then figure out how to turn off the feature without using Google. Not only is the toggle nearly impossible to find, but it's also hidden from being searchable
> For context, try tapping 'optimize photos' in iPhone storage settings
Same place it’s always been. In Settings -> App -> Photos, toggle Download and Keep Originals. Same place it is for macOS as well. It’s not that magical. Search for “photos icloud” and you’ll be led to the setting for it.
Thanks. I cannot get iCloud sync to work at all. It consumes CPU, asks for logins repeatedly, etc but fails to actually do its job. When I think of its bugs and all the issues with the latest iOS (bugs and performance on recent hardware), I am thinking of exiting the Apple ecosystem entirely.
I’ve been using this for several years now on a little unraid box to download new photos nightly. There’s a few docker containers that wrap in support for notifying when 2FA is required etc. Always makes me nervous, the access it has, but I’d rather have my photos backed up somewhere I own.
I pay around 10 euros a month to apple just so I can sync my photos from iphone to mac and ipad. That’s the only reason I need the 2 TB for icloud service. With an app like this I could download and keep copies and get rid of iCloud subscription?
This will let you download all of your photos that already exist on iCloud Photos.
Going forward, you’d want to set up some other way to sync photos you take from your phone to your other devices. I can personally recommend Synology Photos for simplicity[1], or Immich[2] for an open-source (and in my opinion, slightly better) alternative you can run on any hardware, if you’d like to set up an always-on NAS. These are “Apple Photos” or “Google Photos” equivalents that you host yourself.
Alternatively, something like Syncthing[3] is a dead-simple way to sync your photos to various other devices as and when they are online, if you’d prefer to manage your photos in an ordinary file manager.
I’d be remiss not to mention that, for any solution where you move off the cloud to a central storage location of your own, you really must make backups to keep your photos safe. The 3-2-1 rule is a standard recommendation.
I used to use the Photos desktop app to move my photos (“select from the app and drop into a folder somewhere” worked best) to a separate folder a lot (and regularly) until I started using ente. Now ente not only backs up to their e2ee cloud but its desktop app keeps those media synced to the OS of my choice on my laptop.
But I can still not escape Apple’s gonorrhoeic naming and organisation.
Pro: FOSS of course; it works, with limitations (that’s mostly Apple) and glitches (that’s entirely ente)
Cons: really subpar non-native apps (desktop app is quite a dumb app as well) :( (and barely and useful additional features that lets a user do some batch/organisational changes or so)
I use this to sync my wife's photos to Immich and it works great, however the auth process is a bit of a pain (not the fault of icloudpd) and have to reauth every few months.
I’ve wrapped it in some short scripts which notifies on auth failure and it’s an easy process to run the auth script. But there’s no way to avoid the bi-monthly inconvenience I don’t think.
Is there no worry in giving the app your password? I would never just give my password, especially my Apple account password to a random app. Is this program reliable? However it looks like a program that I want to use.
Wow, I will definitely give this a try. I have tens of thousands of photos in iCloud and I literally can’t export them all at once. Photos app chokes and crashes and manually babysitting smaller batches is a pain. It’s pretty clear they want to make it as hard as possible
Related: does anyone know of a way to delete the original videos on trimmed ones? Apparently all edits to videos are non-destructive, so the 10-minute video that I trimmed to 2 seconds still takes up 8gb.
I have a script to scan files from my camera and add a compressed copy to a folder. This folder was supposed to work with the iCloud for windows (10) program, but one day it just stopped working.
Incredible. I have such a hacked-together system to get my iCloud Photos backed up to an external drive while not filling my main laptop drive. This would be even better.
If I'm not mistaken, the Photos app stores originals on disk and then uses a sqlite database to track metadata (and maybe edits to photos as well, given that it has the originals). Seems reasonable to me.
I mean can't you go to privacy.apple.com, ask for an archive of your data, and then they'll email you the link to a zip file in a week or two? I'm pretty sure this is what my girlfriend did when she transitioned from an iPhone to a Pixel. I think there's even a specific checkbox for photos/videos
It's fucking amazing sometimes how the Mac/iOS Photos app can't download a photo or small video for several minutes but you can easily watch YouTube on the same connection.
And it took Apple YEARS to give us a "Keep Downloaded" option for iCloud Drive documents in Finder.
And it's been years since I read any ebooks because the damn Mac/iOS Books app keeps removing my downloaded books even though I have several GBs of storage space left.
Goddamn Tim Cook and the other execs, do they even ever use their own products at all?
Fuck, my wife got a notice that she would have to increase her iCloud storage so last week began the process of ordering a backup of all her pictures so I could get them off iCloud and organized on some drives at home. We got 12 zips of the pictures along with csv's and some metadata, and I literally just finished iterating on the script to sort them into year-based folders and convert all the HEIC shit into JPG. It's running literally right now.
Became more fascinated with the history of my small hometown (Paris, Texas) TLDR: Much of it was wiped out by a 1916 fire. I spent some time recently vibe-coding this interactive map to provide some kind of historic visualization ( which enabled me to see the impact better )
https://gorch.com/parisfiremap/
HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
Project page is here if it’s useful: https://photosbackup.app/
Happy to explain details if anyone’s curious — there are a lot of sharp edges in Photos once you go beyond “export originals”.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
I use libimobiledevice on linux
plug iphone into usb. lsusb should show it.
I backup my photos with:
Actually, I backup all of /mnt not just DCIM, but that answer is for you. I also backup the entire phone with: but in this form it either does the photos as data files, or doesn't back them up. I think it is a complete backup.Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Aren't there hooks on the filesystem layer that downloads them when you access them? E.g I can browse via terminal to my iCloud Drive somehow and cat etc works on files which aren't local (after locking to download them first).
> Do you take into account the iPhone not holding the original images of every photo?
If you have enough storage space on your iPhone, you can select "Download and Keep Originals" in the photo app settings.
I think the originals and edits are both there.
I don't know about space-optimized storage on-phone. I know one setting for transfer to mac or pc - I have it set to "keep originals" instead of "automatic". There might be other settings I'm not aware of.
EDIT: actually, there are other directories (under /mnt but outside DCIM in my example) that seem to have other photo stuff, maybe edits? ymmv
> My current process for offloading photos off the iPhone
I'm not sure about Linux, but my workflow on Windows and MacOS is to frequently back up my iPhone locally (which you should do anyway because few incorrect PINs can security lock your phone [1]) and use utility like backup extractor (e.g. [2] but there are many others) to extract all photos from the backup. This effectively removes the need to use iCloud.
[1]: https://support.apple.com/en-us/105090?device-type=iphone
[2]: https://github.com/joz-k/ios_backup_extractor
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
With Photosync I have our photos export to my NAS and have it update the file names with the timestamp + original file name, which makes it so much more sane to sort through.
Example:
Original: IMG_9999.JPG
Server-side file: 2015.01.15__IMG_9999__.JPG
That looks like it might do the trick. I feel like this should be something possible only using first party apps but I'll take it! Thanks.
I use and like PhotoSync but I thought it doesn’t export unmodified originals but your edited versions. Personally I like this behavior better but that might not be what you want?
I’m not sure Apple allows any third party app to access the unmodified originals. Imagine you crop a photo to remove some embarrassing part. A third-party app can just recover that? What a privacy risk.
Of course this won’t matter if you don’t do any photo editing on iOS.
It's right to question this. I had taken https://www.photosync-app.com/support/basics/answers/does-ph... ("PhotoSync transfers the original, unaltered images including all EXIF and GPS data") on face value without thinking of all of the iCloud behavior. Even limited copies would suffice in my case, but that's insufficient for others.
I did it on Linux once I extracted them all as-is in the strange storage way that iOS stores them but I dont recall steps to make it mount the drive.
That would be perfect, I might chase down this path again. It's been a while since I've tried to directly mount the iphone as a drive on linux.
If you manually edit the date on a photo, is that also stored separately from the image file itself? Wondering because I've noticed photos I've backed up to Immich from iOS photos don't respect that edited date and reflect whatever the original date was.
I've been thinking about looking into a fix for this since it's bugged me a bit.
Yes — in Apple Photos the manually edited date/time is not written back into the image or video file. It’s stored separately in the Photos library database, which is why tools like Immich usually fall back to the original capture date in EXIF / QuickTime metadata.
Photos Backup Anywhere does handle this case: it reads the adjusted date from the Photos library and stores that modified timestamp in its own SQLite database, linked to the backed-up file, so the edited date isn’t lost even though the file itself isn’t rewritten.
Thanks for the reply, I appreciate it. Does Apple provide any APIs for interacting with this metadata or was this something you had to implement yourself with lower level DB lookups?
> As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My free open source tool osxphotos [1] does this. In addition to an "osxphotos export" command to create archival exports with all metadata, it also has an "osxphotos import" command that can restore all metadata including albums (with exception of named persons/face regions, though persons can be converted into keywords on import). It can also read XMP sidecar files to restore metadata from those. It's a CLI and definitely a power user tool.
It also includes an "osxphotos sync" command that can sync metadata between two libraries. Some people use this to sync metadata like favorites from iPhone library to the Mac library. (can't go the other way unfortunately)
[1] https://github.com/RhetTbull/osxphotos
I use your tool to backup my photo library, it's been working great! Thank you so much for this amazing piece of software.
Glad it's useful for you!
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
My current workflow for this is to install the 'simpleSSHD' app on the android phone and rsync the files off at full network speed.
The sshd running on the phone also supports key based auth , so it's pretty simple to automate.
Yeah but for some phones the storage is full ;) I had to figure out how to remove just enough to root it and then just adb in for one phone then I learned that if I extract too quickly it gives up the ghost so I have to add a delay
Awesome! Thanks for posting, I've been looking for exactly this kind of project that takes end to end pristine restoration seriously.
The FAQ states you back up the originals as plain images files in YYYY/MM/DD format, which is great for integration with other tools. What about metadata? Is it in a format that lends to integration with other tools, say if you stop developing/supporting your app for some reason?
Metadata is stored in an SQLite database alongside the backed up files. The format is not documented, but should be self explanatory if you examine the schema.
Excellent, thanks!
This is a great app! Thanks, I bought it. I'm backing up to a network share over SMB. Just wanted to let you know that the error message in the event that the share becomes unmounted during a backup is a bit obscure: it just says "pdb.sqlite" doesn't exist. Stopping and restarting the backup fixes it. Might be helpful to provide some clearer error handling / messaging and guidance for disappearing network shares?
Thanks or the detailed feedback.
You’re right about that error message. What’s happening is that when the SMB share gets unmounted, the app can no longer access its SQLite database (pba.sqlite), and the resulting error isn’t very user-friendly. I agree it would be much clearer to explicitly detect a missing or unmounted network share and provide guidance.
I’ve added this to my list of things to improve. Thanks again for taking the time to report it — feedback like this is super helpful.
My pleasure - much appreciated and thanks for being so responsive.
I’ll just add in here a sneaky humble request for graceful automatic pausing and restarting as network shares (or other volumes?) (dis-)appear - although I imagine that one’s a bit trickier to implement.
All the best! Very satisfied user. Great replacement for the script + cron job I was using before.
Thanks for creating this amazing tool. A requirement for it to work is disabling "Advanced Data Protection". What are the implications of doing this?
Pretty severe -- ADP allows users to store data with e2e encryption.
Disabling ADP is a pretty serious* thing to do -- and pretty disappointing since I was interested in the product. Since it's OSS though this might be something that can be worked around in the future.
That’s a good question. To be completely honest, I don’t know much about Advanced Data Protection myself, and I didn’t do anything specific in the app to detect or interact with it.
I’m actually curious: how did you discover that it doesn’t work when Advanced Data Protection is enabled? Was it through an error message, incomplete backups, or something else you noticed?
I did not discover it, disabling ADP is listed under "iCloud Prerequisites" on the/your GitHub page.
The app that the person you're replying to ('gerdemb) created (Photos Backup Anywhere) is different than the open-source CLI tool that is the HN submission (icloud_photos_downloader).
Is the app bound to the purchasers iCloud, or can I also use it to download my partners images?
Yes — Photos Backup Anywhere supports Apple Family Sharing
If the app is shared via Family Sharing, each family member can install and use it on their own Mac with their own iCloud Photos library. The app works with whichever Photos library is signed into macOS on that machine, so you can absolutely use it to back up your partner’s images as well.
Hey @gerdemb, just wanted to say thank you. Bought this because of your comment and this worked perfectly for backing up my 1TB photo library.
I'm extremely interested in any process that would allow me to create file-system-level backups of individual photos. Will your tool do that?
Yes — that’s exactly what Photos Backup Anywhere is designed to do.
The app creates file-system-level backups of each individual photo and video from your Photos library. Every asset is exported as a real file on disk (HEIC, JPEG, MOV, etc.), not a database blob or package, so you can browse, copy, verify, and back them up with standard tools.
In addition, the app keeps a small SQLite database alongside the files to track metadata (including edits, albums, and relationships like Live Photos), which allows accurate restores and verification while still giving you transparent, plain-files access to your media.
That's awesome. I've been low-key looking for something like this, so I just bought your tool. Thanks for creating it!
This looks interesting. Can I use it to back up photos from one (old) photo library, and restore them into another (newer) one?
Yes — that’s exactly what it’s designed for. You can back up photos from one Photos library and later restore them into a different Photos library. The restore process recreates the items in the target library using the data stored in the backup.
Thanks for commenting, do you support S3 compatible targets? Backblaze B2, for example.
Not directly, no. If you can mount the S3 target as a drive, it can be used as a backup destination.
Love this app! Kudos
have you looked at parachute backup? they also boast ability to backup the more mercurial types of iPhotos data.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
Nice to see my project on Hacker News! I started this almost 10 years ago and haven't been involved with maintenance for a long time, but I'm glad that people are still finding it useful.
Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.
That’s not true. On any Mac or iPhone you can choose the iCloud Photo Library storage option to download all instead of letting the system optimize the storage. And if you turn off iCloud Photo Library, it will also try to download it all. I know this because I stopped using iCloud Photo Library and that was how I got all my photos downloaded.
+1 to this method. After optimise storage is disabled on the Mac, wait for all photos to download. Then, open the photos library bundle and you'll see every photo there, full res. Copy them wherever you like.
Also, if you leave optimise storage disabled and continue to use Photos, every photo will be cloned in any local or cloud backups of your machine. This strategy creates additional photo redundancy separate from iCloud while still benefiting from library syncing.
Or use the great osxphotos tool that works with Apple Photo’s SQLite database to let you manage all the photos in your library.
https://github.com/RhetTbull/osxphotos
Demo .gif sold me
(Been meaning to make a software demo gif gallery, best way to understand many categories of apps)
I once exported my photos out of the iwhatever library. They weren't in the cloud, Apple hadn't managed to trick me into turning that on.
What I remember is that I opened the library in finder and in mc, got scared by the readable-only-by-machine directory structure and used a 3rd party tool to export them to date labeled directories.
This was my strategy too, but with a disgusting script which quit photos.app, rsync the photo library to a network share, then reopened photos.app so that it kept downloading from iCloud.
Not sure if the open/close is required, but I didn’t want to find out.
I don’t fully trust iCloud Drive / Photos therefore I use FSViewer to download all photos from my iOS device du jour (making sure to keep the HEIF formats), this way I get the Edited (slo-mo, live, portrait, usw) and pristine versions as Jobs intended. All kidding aside, after the gray area gate of 2017-2021 I had to find a more reliable backup workflow. As of today I only use iCloud Drive / Photos to extract some RAW photos that for some reason some picky apps don’t save to the photo album (looking at you ProCam 8.0). I made several tests including hash comparisons and imagemagick diffs and I am quite pleased.
Someone gave me a new iPhone (120GB) and a new MacBook Pro and asked me to download all their photos from iCloud. Long story short, after 120GB of photos were synchronised to the iPhone, the MacBook Pro refused to copy them, and now there's no storage left on the iPhone.
Also, Photos on Mac doesn't have an option to download photos directly, so the only valid option Apple offers is to download them through the web interface (max 1,000 at a time).
There is no official way to download iCloud library that is over phone capacity. Period.
> Photos on Mac doesn't have an option to download photos directly
Yes it does. It's called Download Originals to this Mac.
https://support.apple.com/guide/photos/use-icloud-photos-pht...
You keep asserting to the contrary, but I've been syncing my entire photos library to my Mac for years, since it was iPhoto even.
Obviously if you have a larger photos library than storage space on a particular device, you cannot synchronize the entire library to that specific device. e.g. my photos library vastly exceeds my iPhone 13 mini storage, so on my iPhone, I don't sync everything. But my Mac has 2 TB of storage, and Photos is setup to sync all my photos, and does so, reliably, and has been, again, for years now.
Additionally, unlike with this open source tool, I can keep advanced data protection enabled.
This is from the iCloud manual:
> Any new photos and videos you add to Photos appear on all your devices that have iCloud Photos turned on.
You have your photos because they are new. If they had been taken before, they would not have synchronised automatically with Photos on MacOS.
Please stop repeating your incorrect points that are contradicted by everyone else’s real experiences.
Yes, new ones will be uploaded. That doesn’t mean old ones won’t also be downloaded.
I have tried 3 different Macs with different versions of macOS prior to looking for a workaround, and everywhere the result is the same: old photos are not downloaded automatically from iCloud, and there is no button to start this process - for this exact reason.
Want to prove me wrong? Create a new macOS user and open Photos with your iCloud. It will be empty until you start copying photos from your phone. It will take much less time than arguing here.
You're arguing with a lot of people who have personally seen this work. You can listen to other people. You can also go to an Apple Store and let them show you what's going wrong here.
Perhaps no one here has tried to download an entire iCloud library at once, or perhaps size is an issue, but that doesn't change the fact that there is no download button for iCloud Photos and iCloud Photos Downloader simply solves this. That's what this post is about.
I can personally confirm I've downloaded an entire iCloud library at once, to a brand new Mac, using the 'Download Originals to this Mac' option. As have many others here, I would think.
That's literally what that option is for.
If it's not working for you, you might be dealing with a bug, or perhaps you haven't given it enough time to sync. If you go to Photos > Library and scroll down, it should show you the sync status.
Thanks, that was a relief because I realised I didn't see the sync status at the bottom. It appears that Monterey hides the status message at the bottom by default, and I had to pull the page down twice to see it.
Long story short, iCloud wasn't syncing photos "due to performance" and this message was hidden.
Thanks once again!
No worries! I don't understand why Apple is so averse to surfacing the status of things, especially highly sensitive and finicky things like online sync. It would dramatically improve the feel of the software if it didn't seem like it just inexplicably wasn't working half the time.
Happy to hear it helped. :)
iCloud Photos Downloader is an option, yes, but it is incorrect to say that Apple does not provide an official way to do this on Mac. Again, I direct you to the Apple Store so someone can show you in person, since you won't listen to anyone on here.
I confirm that you was absolutely right!
Photos on MacOS indeed synchronise photos with iCloud.
After our conversation I had tried to understand why I indeed don't see any status and I found out, that to get one in Monterey iOS I must need to scroll down of the collection and after, at the bottom pul whole page for the second time. Status message appears and it was saying that syncing was disabled due to Mac performance (I didn't asked for this).
Apologies, for misleading, code543 and thank you for consistence.
However, I must admit that I'm happy that found iCloud Photos Downloader as a result, also I liked that it's downloading all photos in date/folder structure.
Let me be one more voice telling you that you are wrong. I just did this morning.
In settings, "download originals to this mac", select all photos, file -> "export unmodified originals" will trigger the Photos app to download every file from iCloud into your local library (as well as exporting them to wherever you want)
I guess "there is no download button" but dude...I don't need iCloud Photos Downloader.
Thanks for letting me know. May I ask what macOS version you use?
Unfortunately, I'm unable to locate any button, status bar, or option to refresh or pull everything from iCloud in macOS Photos. There aren't even any details showing what percentage of iCloud is currently synchronized with macOS Photos. With nothing to debug, I can only conclude that for some reason the sync isn't working in my case.
It's great if this works for you and you don't need iCloud Photos Downloader, but for some reason I don't have that luxury.
Thank you! The status message was indeed hidden on Monterey, and syncing was blocked due to "Performance."
That doesn’t sound right. My photo library is larger than my iPhone’s storage yet downloads fine on my Mac. Just need to make sure “optimise storage” is enabled on the iPhone and disabled on the Mac.
Once everything’s downloaded on the Mac, you can either export through the Apple Photos menu or just copy the “originals” directly from the Photos bundle.
This works because you had synchronised your iPhone with your Mac previously. If you start with an empty Photos library and phone, it is impossible to put all the photos on the phone and thus transfer them to your Mac.
No, I’ve downloaded the entire library to a new Mac. It worked fine.
Thank you. Yes, indeed. I found out that Monterey is not syncing iCloud due to "Performance".
And people say Linux is hard to work with....
Okay, and if there are 300 or 500 GB of photos, how do you synchronise them with your iPhone?
You can't run random cli-tools on your iOS phone either.
that's good to know. can I then download the photos from iPhone to a backup hard-drive or transfer to a folder in my computer?
Yes over USB 2.0 until recently.
Thanks to Apple's exceptional software quality the app has plenty of bugs and good luck exporting a lot of files out of said library - you're in for an endless game of spinners (it does some network IO on the main thread), "not responding" and memory leaks.
But hey at least we've got Liquid (gl)ass now.
Worked well for me for 70k photos but took a long time
> Thanks to Apple's exceptional software quality the app has plenty of bugs…
I use Photos for macOS daily and I've never run into a bug with my 50K+ photos library. (To be fair, Photos doesn't do that much, and I use it more as a master catalog with Aperture's spiritual successor Nitro.)
> …and good luck exporting a lot of files out of said library…
Not sure why you would need luck to copy the "Originals" folder from the library package.
> I've never run into a bug with my 50K+ photos library.
Have you tried dragging & dropping a photo from the Photos app to your desktop to export it as a file?
I just tried it and while the Photos app UI didn't freeze (I guess either my memory is bad and the spinner behavior was on imports, or they fixed it since 2 years ago), but it takes ~3 seconds for a single photo to appear as a file on my desktop (with no UI or any indicator that something is happening), and dragging & dropping 45 photos took over a minute (again with no progress indicator).
Granted, it turns out I didn't have "download originals" on (not sure if it was always like that or got reset during an update) so hopefully for Apple it's that - but I still think at the very least a progress indicator on what seems like a network IO operation is in order (and I wonder what happens if I interrupted my network connection during its operation - does it just silently never complete and my files never appear, actually show an UI, or just crash?).
Edit: well nevermind, despite all the files apparently exporting just fine, ~5 minutes later after initiating the operation I just got a hideous popup that some files failed to export: https://imgur.com/a/SFXZB5N
It's progress, at least it's the first indication in the UI that something was actually happening. You will notice that the error text is truncated, the only way to read it all is to resize the window (no hover text) and for some reason the horizontal scroll in this UI element does not follow your touchpad - you have to scroll, lift your fingers, and ~1 second later the UI suddenly applies your scroll operation.
Also, it turns out that resizing the window finally readjusted the table control and now the table actually matches the size of its container and isn't scrollable anymore. Which might be a good thing as it at least alleviates the bug mentioned previously, but then why did the control initialize oversized to begin with?
Again this is the kind of jank I'd expect from Linux, and nowadays maybe Windows. But not Mac.
> copy the "Originals" folder from the library package
My bad, I wasn't aware of said folder; I treated the "package" as an internal implementation detail equivalent to a proprietary format and wasn't going through it. I don't think it's a fair expectation to have (potentially non-technical) people to right click "Show package contents" while the usual double-click on the file just opens the library in the Photos app.
Technically, there is: users of the European Union can get a full export of all data that Apple has about them, including all the stored photos. It can be requested from here: https://privacy.apple.com/
I was able to request a photo dump as a non-EU customer using this link.
How does the archive they provide look like? Many zip files? I would like to retrieve them and offload to another storage service but I don’t have local storage enough to hold all of it at the same time, unpack and then reupload. I would need to do it in stages.
Yes, many ZIP files. You can select the ZIP file sizes, from 1 to 25 GB, iirc. Although a few end up larger than 25 for some reason. And took 1-2 days for Apple to "prepare".
You can request a chunk size and then it prepares them. I specified max chunk size and it took almost a week to give me a list of file downloads from 45-60GB each. 31 zip files to download.
While that’s a pain for you, it’s also a pain if they have multiple files for those that have enough storage.
Photo management is a bit of a nightmare as it’s an awful lot of small(ish) files.
It sounds really weird that instead of making a separate utility, or allowing you to download iCloud Photos in the native Photos application on Mac, Apple requires you to go through a legal procedure.
I'm OK with clicking a button to download all photos to Mac, but there is no such button. Or maybe there was one previously, but it has now disappeared.
> or allowing you to download iCloud Photos in the native Photos application on Mac
Here’s the official documentation page for exporting directly using Photos for Mac without syncing everything locally: https://support.apple.com/guide/photos/download-photos-to-yo...
You can also choose to sync all photos locally with Photos for Mac by setting “Download Originals to this Mac” as described on this page which is what I do to keep a local copy: https://support.apple.com/guide/photos/photos-settings-pht51...
If your Mac doesn’t have enough space, export them to a USB hard drive or if you’re using the download originals option, first move your library location to the USB drive as also described on the link above.
Thank you.
That's exactly what I expected to work, but for some reason this approach failed for me on a new Mac with an empty Photos library. I enabled "Download Originals," but 10+ years of iCloud photos never appeared. There's no manual "fetch all from iCloud" button, no progress indicator, no way to diagnose what's wrong - the sync just silently fails. Luckily, iCloud Photos Downloader bypasses Photos entirely and pulls directly from iCloud.
You were absolutely right. Someone in the thread mentioned a status indicator, and it triggered me that I hadn't seen it at all.
It appears that on Monterey I need to pull the page down twice to show the sync status, and when I found it, the message said that photos weren't syncing due to "Performance."
Doesn't work if you have Advanced Data Protection (end to end encryption) enabled.
I think if you are on PC. But on Mac with the Photos App
Cmd+A > File > Export Unmodified Originals
There is no straight way to download photos from iCloud to Mac.
The comment you're replying to just gave one, I did it recently for 70k photos. select all > export. That's it. It just took 30 hours.
Photos > Settings > iCloud > Download Originals to this Mac.
I'm not sure it's surprising. Apple doesn't want you to leave and making something as important as your photos difficult to move helps with that.
From 2024:
> Users of Google and Apple’s photo cloud services can now transfer images between them. It was already possible to export photos and videos from iCloud to Google Photos, but now it can also be done the other way around: from Google Photos to iCloud.
https://www.techzine.eu/news/applications/122196/google-and-... (2023 Data Transfer Initiative (DTI))
What’s missing?
The files are there on the Mac, they are there to download on the cloud (various mentions of method mentioned here).
There is NO button on Mac to download photos from iCloud. You can only do this from your phone and then synchronise them to your Mac.
Yes, there is a button on Mac: https://support.apple.com/guide/photos/use-icloud-photos-pht...
As long as you are signed into the Mac with the same iCloud account used on the iPhone, this will download them all. No, you do not need to get them all downloaded to the iPhone ever for any reason for this to work. Period. You need to stop repeating that, because it is wrong. How many people have to say the same thing?
Yes, you will have to go into a hidden folder to access the Originals once they're downloaded if you want to copy them somewhere else, but it's like two clicks.
I've been using Mac since Mac OS X 10.4 (~2005) and was under the same impression.
However, in reality, when you use the same Apple account on both devices with the Photos app on macOS (yes, with the 'Download Originals' checkbox enabled), it only downloads photos that you upload from your phone.
And if you look at the iCloud tab in the Photos app, it says 'Automatically _upload_ and store all your photos and videos in iCloud', so it works from Mac to iCloud, and doesn't help to download full iCloud library.
No, you are not correct. How many people have to tell you this?
It absolutely works the way I said it does, because I have seen it work that way. Just because you accidentally turned off iCloud Photos in your Apple Account settings on that Mac (or some other similar issue) does not mean it does not work this way when properly signed in.
If you want something to try, go to System Settings -> Apple Account -> Photos and see if "Sync this mac" is turned off. It needs to be on. There could be other ways that this feature is disabled, but that is one of them.
Not seeing something work is not evidence that it does not work. You have not seen it work, but that is not proof it does not work.
Seeing it work is evidence that it works. I have seen it work.
Other people have seen it work that way, and their replies are all over this thread. Apple documents that it works this way.
Yes, it will upload photos to iCloud if enabled, but it also downloads them.
When you take a new photo, it synchronises with all your devices, and therefore you see it on your Mac, iPhone, etc. However, if you get a new Mac (I got one because my library was under capacity), Photos will not start synchronising your 10-year-old photos until you process them through the phone.
I hope I've made it clear now.
Your point has been clear the whole time. It is still not correct.
You can use the Photos app on your Mac and download originals
There are two options in the file menu - you can export the originals, and there is an option to export them as .jpg.
From iPhotos (Photos) app to hard drive.
Subject is to download photos from iCloud.
When you first turn on iCloud Photos, it can take a while to upload your photos to iCloud. You can continue to use Photos while your photos are being uploaded. To have iCloud Photos sync photos to all your devices, sign in to the same Apple Account to turn on iCloud Photos on all your devices.
https://support.apple.com/en-gb/guide/photos/phtf5e48489c/ma...
Select all, Files > Export. It's simple to download iCloud Photos.
One of us is missing something. In Photos.app I clicked download originals. The photos are there on my Mac. It’s bit gross to get at them though - right click on app > show package contents.
Are you wanting a way that doesn’t involve the photos app?
You can do that from iCloud over a browser.
If you open the Photos app (macOS) connected to iCloud with an empty library, there will be no photos until you import them from your phone. Hope this is clear now.
iCloud via browser has a limit of 1k photos per download.
This is not correct.
You don’t need an iPhone for Photos.app to work.
There are plenty of problems with Photos.app but the issue you describe isn’t by design.
My bad — it appears that Monterey wasn't syncing with iCloud due to "Performance." Photos on Monterey hides the status message at the bottom of the page, and I had to pull the page down twice to finally see it.
Thank you for proving me wrong.
Oh god. I’d forgotten about that tiny dialogue box.
98,281 items to sync.
I hope it comes right quickly.
This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?
Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.
I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.
It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)
I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?
immich (self hosted photos app, https://immich.app/) "stacks" live photos with their videos
I run this periodically from a little shell script; I "should" automate it, but time is scarce.
> icloudpd/icloudpd:latest
Passing your raw iCloud creds into the unverified latest tag is fine until it’s not. Better to pin to a specific tag or hash.
You're not wrong. I know I need to put more work into it. Just haven't had time.
I'm "protected" by the fact Podman doesn't automatically update the latest image even when using the latest tag.
I was more showing how simple icloudpd is to use.
While not free, and not for any other platform than macOS. The program Parachute[1] in the App Store is very nice in downloading both photos from your library as well as files from the various locations.
[1]: https://apps.apple.com/us/app/parachute-backup/id6748614170?...
Another option for iOS at least is PhotoSync. It’s nice, you can pull from photos and push to basically any remote service or local server. I have it backing up to both my nas and b2.
It works well enough, but it's not without flaws either.
The desktop version works reliably, if you can get macOS to keep shares mounted for long enough, and mount them on request. The scheduler is also kinda wonky.
The iOS version has so far never finished an incremental backup overnight of our ~1TB individual libraries. It handles resume/suspend well, but for some reason, while it exports unmodified originals, it doesn't include AAE files, which the desktop version does.
PhotoSync does everything right, with the exception of trying to keep state of what has been exported, which makes little sense as it doesn't support restoring photos.
Is there a way to verify this all is safe to use? Like it won’t do something weird privacy wise? Any equivalent for windows?
I have the same paranoia, so I was happy to learn that someone made an open-source downloader for iCloud.
Anyone know if it works with ADP? I emailed them months ago but no one ever replied.
On a related question, is there a download solution that does work with ADP? I’m looking to mitigate any potential account lockout issues for family members (and, no, they will not switch out of the ecosystem).
It does. It uses PhotoKit to access photos, so it basically uses your Apple Photos app (iOS or Mac) to download the photos.
The only scripted solution I can think of that works with ADP is osxphotos[^1], but that also uses PhotoKit, and requires the user to be signed in.
Personally I use PhotoSync [^2] to backup our photos from phones to a NAS. It works reliably, and supports exporting unmodified originals as well as edited versions, and XMP/AAE metadata alongside it.
^1: https://github.com/RhetTbull/osxphotos
^2: https://www.photosync-app.com/home
Advanced Data Protection
Thanks for the link
I was there until I saw this feature:
Automatic de-duplication of photos with the same name
I recently went through a year’s worth of photos from my wife’s phone, and found three distinct “img_0001.jpg”’s just in that single year. Apple’s naming convention is so short sighted that I’d be terrified letting a piece of software try to dedupe it “by name “
> ... Apple’s naming convention.
No, it is not. It came from DCF standard that predates smart phones.
https://en.wikipedia.org/wiki/Design_rule_for_Camera_File_sy...
The 4 digits numbers are, of cause, came from the 8.3 file name length constraint. Gonna love CP/M.
Replying to myself to tell me that I'm wrong.
I'm running this on my icloud photos now, and it seems to be dropping them nicely into folders by year and month:
D:\Pictures\icpd\2026\01\01
... so my wife will need to step up her game and start taking 10,000 photos a month before I run afoul of this.
I was just thinking about this today. Apples lack of any 3rd party integration for things like this and iMessage is really annoying sometimes. In addition to a secondary backup, I’d love to automatically sync some photos from a certain album to my parents photo frame. Or if I take a nice nature shot have it sync to a Samsung frame tv. I get the benefits of the walled garden but esp w photos and messaging it seems like opening up a little would allow for some innovation
I use PhotoSync on my phone for this - select which albums to sync where with a rule and away you go.
I’ve been using usbmuxd+ifuse to copy the photo files straight from the phone. No need to wait for an upload/download to some remote server, just a direct cable from the phone to my computer. I get the original files, and can even move (instead of copy) to clear up the phone.
This works on any iPhone? It mounts the non-privileged DCIM folder or whatever over USB to somewhere on your filesystem? With write access?
Yes, yes, and yes.
https://whynothugo.nl/journal/2024/12/22/exporting-photos-fr...
Yes, yes, no.
https://wiki.archlinux.org/title/IOS#Transferring_data
From the project describtion: "Looking for MAINTAINER for this project"
Honestly, Apple should officially maintain tools like this. However, for obvious reasons, such as the iCloud subscription revenue model, Apple will not do it. In fact, Apple may even make life harder for such tools.
I’m in the midst of a backup-to-local project and, with this post to HN, I’m worried an Apple project manager will be on a mission this morning to get his team to cripple this software.
Thanks for this project. Our family generates about 2TB of media a year, and it’s been like that for a while, so we’re sitting at roughly 12TB total. That’s very much the long tail of personal media.
I’m not ready to pay $60/month, but I do like iCloud’s memories and other photo features. My compromise is simple:
- I use docker-icloudpd to download our iCloud Photos to local storage over time. It’s been the most practical way I’ve found to back up multiple accounts into one place, though it does require occasional re-auth every so often. - I keep only the last ~2 years of media in iCloud and delete older ones after they’re archived locally. - For browsing and searching the older archive, I use Immich, which has been a great self-hosted personal photo cloud experience with a modern app feel.
For storage, I’ve found fast local disk matters a lot once you’re digging up photos from 5+ years ago. Something like an OWC 4M2 with M.2 drives keeps the experience snappy; a typical HDD-based NAS can feel sluggish when you just want to quickly pull up an old memory.
https://github.com/boredazfcuk/docker-icloudpd
Does Photos have features you use that Immich doesn't? I've switched to the latter fully and love it (though I have an Android).
How much work is it to maintain an Immich instance? I'm trying to keep my digital setup lean, but I can't find a simple way to get photos off my iPhone and onto my home server. One way synchronization is my only goal. Extra features are not desired.
If all you want is a 1-way sync from phone to server, then just go with PhotoSync. You can set up a simple SFTP connection so you don't even need anything on the server.
Immich is great if you want to be able to delete things from your phone to save space and still have access to all your content served from Immich on your server though.
This seems a lot better, thanks for the suggestion.
I change a line in the Dockerfile to upgrade it every few months, that's it. Otherwise it's zero.
Music and AI features are still lagging in Immich, and I can understand why. Immich machine learning is not flushed out yet. If Immich has plans for creating marketplace for extensibility like plugins, in the current era of Claude code, I am sure we will end up with many options or features.
Music? Like a music library? Wouldn't Subsonic be miles better for that?
Like Memories feature in Apple which uses media to make short form video but also picks and plays music with them. We love those videos. Enabling feature like that involves understanding metadata and media and music.
Ahh! They're really cool; kinda miss them from Google Photos. I'm hoping Immich will implement something similar at some point.
Encryption at rest. If someone breaks into my house and steals my server I'd rather they not be able to get data. I can do LUKS but I really want the data to only ever be decrypted client side.
My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation.
It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.
This. I don't understand why Apple don't have another checkbox beside the Download Originals to this Mac that reads 'Store Backup of Original Photos on Timemachine' This is all that's needed to solve the issue. I actually bought a Mac Studio, and a USB disk, just to be able to download originals of my photos for local backup, since a MBP is effetively a mobil limited device just like an iphone.
"I actually bought a Mac Studio"... "I don't understand why Apple don't " ... wait a minute
How about just turning off optimized storage and letting Time Machine do its job?
I have a 1.2T photo library. Carrying that around on my MacBook requires an $800 upgrade to the SSD.
Time Machine's job is to back up my data, it's not strictly to make a 1:1 copy of local storage. It should back up my cloud data too.
My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled.
I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy.
IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected?
I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.
Could the first obvious improvement please be its speed? My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.
I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
> I have had it randomly decide the backup is corrupt and it has to start over.
I have hit this too many times.
Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product.
I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS.
Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
I’ve done the sudo sysctl debug.lowpri_throttle_enabled=0. But it’s still slow.
10 minutes is great, and my changes wouldn’t seem as extensive as yours. I need to dig deeper.
A huge chunk of that 3-4GB is large files that have minor changes. Time Machine doesn't have any sort of delta support so backups the entire file again, like my local Messages or Contacts database. But I think slowness is caused by file count, not file size, so even though it's backing up 3GB+, the total number of files changes isn't that high. (I suspect).
I also use a modified version of this script[0] to identify everything that changed in the most recent Time Machine backup. This is hugely helpful and lets me find unimportant things that are the source of lots of unimportant changes which I can then exclude with `tmux addexclusion -p <PATH>`. For example I exclude 'node_modules' folders for anything that gets regularly updated. This removes ~10k files that would otherwise be wastefully backed up. Speeding it up is much more about reducing file count than total size.
[0] https://github.com/nlfiedler/timedog (though its woefully out of date, and I had to make some hacky patches to it to get it working)
I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.
In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion
> Slow motion videos are also not proprietary, they’re just a plain video file.
I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).
The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move.
For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.
It is no fun to have old iCloud photos deleted unexpectedly. Apple has provided plenty of footguns, even if they really are user errors. For examples: (1) during device restores and (2) premium subscription management fumbles.
Product idea: Apple should offer a paid service to restore the "old backups" of photos that are no longer accessible via iCloud UI/API, which were soft-removed for missing the subscription quota or whatever, if Apple happens to have that data tucked away in cold storage somewhere.
Case in point, I had some c. 2016 era photos in iMessages that I thought I handled right to not lose from iCloud, but they are apparently nowhere to be found in iCloud API based on recent checks. More than mildly irritating.
I should have used an iCloud photos backup tool like this much sooner.
Print what you want to keep onto archival paper with archival dyes. Everything else will atrophy.
Wish I’d seen this 3 days ago. Needed to backup our Shared Library and did the following (about 10K photos/videos at 300gb, had enough space so full downloaded to MacBook, not optimize. 1. Repair iPhoto library, (Wait 24 hrs to re-sync to iCloud) - initially looks like it moved all photos to personal. 2. Select small chunks, by year worked well enough - selecting All gave me the spinning ball. Then export unmodified to external hdd into folders organized by year. 3. Moved entire photo library file to another external hdd. 4. Open iPhoto and select external hdd library as primary library, let it re-sync to iCloud (Wait 24 hrs). iPhoto now running off external HDD library and I’ll backup to separate external HDD monthly. Repair function and wired Ethernet connection were biggest game changers to previous attempts.
Is there any difference in moving Photos Library.photoslibrary to an external HDD and then pointing Photos.app at that?
Not in my experience, previously ran it setup externally with a shared library for at least a year or longer, https://support.apple.com/en-ca/108345. Typically I backup to external drive 2 ways, copy of library itself and unmodified export, only had to go through above after I needed to rebuild the laptop. OP app and or https://photosbackup.app/ , seems like they might enhance my setup, will have to take a closer look.
There's some technical reason (or non-reason) why Mac OS does not accept the System photolibrary on an external disk. It prevents certain things from happening which may or may not mater to you. Otherwise you can switch librarys by holding Option during photo bootup.
Parachute Backup is a tool I use: https://parachuteapps.com/parachute
This great. I want to switch to android but the only thing stopping me is eight years of photos. No more.
You can go to https://privacy.apple.com, log in with your Apple ID, select "Request to transfer a copy of your data" and then select "iCloud photos and videos to Google Photos".
If your end goal is to move the photos to Google photos have a look at https://support.google.com/photos/answer/10502587?sjid=13067...
Does anyone know if there’s a way to self host/impersonate iCloud? I’d like to back my iPhone up locally.
All the comment here fail to address the “view” part of iCloud.
I wrote a webapp to try to solve that for myself (https://github.com/yhling/go-web-image-gallery)
You can do manual backups in macOS using Finder, or iTunes on Windows.
There's also a Program called "Apple devices" I believe from Apple for Windows that will let you backup your iPhone/iPad.
If you configure a password for your backup it will backup more (confidential) data than if you don't encrypt your local backup.
https://github.com/libimobiledevice/ifuse might help.
For pictures you can use https://immich.app/
Immich is probably the best option
I'm sure grandparent meant to modify it so they'd just have to click "Backup to cloud" on their iPhone and instead of the iPhone sending their files to Apple's servers, it sends them to a local backup server...
A lot of iPhone apps can backup the iOS photo library these days, including the abovementioned Immich, Google Photos, and Dropbox.
Immich can get pretty close to that experience, with the iPhone app and a local immich instance.
Or ente photos: https://ente.io/
I'm a big fan of Parachute Backup for this purpose. https://parachuteapps.com/parachute
Do you know if this supports multiple users? Can’t find mention of it in the site.
This looks useful. I ran into a very similar problem recently and ended up building a small tool for my own use: https://github.com/cleanexit0/darwin-photos
It's macOS-only and intentionally minimal — the goal is just to download originals from iCloud Photos to disk without syncing everything into Photos.app first.
To be clear on limitations: it doesn't preserve albums or other metadata yet, and it's not meant to replace more full-featured tools. It’s mainly for the "I just want my photos off iCloud for backup" case.
Reading the comments here, it sounds like metadata preservation is a big pain point for many people — I'd be curious whether that's the first thing folks would want added, or if simple bulk export already covers most needs.
All my vertical videos in iCloud show up cropped horizontal for some reason. If I go to edit I see the whole video. I really do not want to trust any cloud provider to maintain my years of archives of family photos and videos. Glad things like this exist. I just need properly date-foldered files, without no duplciates. Is that so hard?
Sounds like the contact sheet view is just using square preview thumbails?
All my vertical videos in iCloud show up cropped horizontal for some reason.
Turn your phone? /ducks
Does anyone have any idea for why Apple makes it so difficult to keep photos downloaded?
For context, try tapping 'optimize photos' in iPhone storage settings and then figure out how to turn off the feature without using Google. Not only is the toggle nearly impossible to find, but it's also hidden from being searchable
> For context, try tapping 'optimize photos' in iPhone storage settings
Same place it’s always been. In Settings -> App -> Photos, toggle Download and Keep Originals. Same place it is for macOS as well. It’s not that magical. Search for “photos icloud” and you’ll be led to the setting for it.
Thanks. I cannot get iCloud sync to work at all. It consumes CPU, asks for logins repeatedly, etc but fails to actually do its job. When I think of its bugs and all the issues with the latest iOS (bugs and performance on recent hardware), I am thinking of exiting the Apple ecosystem entirely.
I’ve been using this for several years now on a little unraid box to download new photos nightly. There’s a few docker containers that wrap in support for notifying when 2FA is required etc. Always makes me nervous, the access it has, but I’d rather have my photos backed up somewhere I own.
I pay around 10 euros a month to apple just so I can sync my photos from iphone to mac and ipad. That’s the only reason I need the 2 TB for icloud service. With an app like this I could download and keep copies and get rid of iCloud subscription?
This will let you download all of your photos that already exist on iCloud Photos.
Going forward, you’d want to set up some other way to sync photos you take from your phone to your other devices. I can personally recommend Synology Photos for simplicity[1], or Immich[2] for an open-source (and in my opinion, slightly better) alternative you can run on any hardware, if you’d like to set up an always-on NAS. These are “Apple Photos” or “Google Photos” equivalents that you host yourself.
Alternatively, something like Syncthing[3] is a dead-simple way to sync your photos to various other devices as and when they are online, if you’d prefer to manage your photos in an ordinary file manager.
I’d be remiss not to mention that, for any solution where you move off the cloud to a central storage location of your own, you really must make backups to keep your photos safe. The 3-2-1 rule is a standard recommendation.
[1] https://www.synology.com/en-global/dsm/feature/photos
[2] https://immich.app/
[3] https://syncthing.net/
Thank you very much for the detailed reply. I have a Synology NAS, will look into that solution.
I used to use the Photos desktop app to move my photos (“select from the app and drop into a folder somewhere” worked best) to a separate folder a lot (and regularly) until I started using ente. Now ente not only backs up to their e2ee cloud but its desktop app keeps those media synced to the OS of my choice on my laptop.
But I can still not escape Apple’s gonorrhoeic naming and organisation.
Pro: FOSS of course; it works, with limitations (that’s mostly Apple) and glitches (that’s entirely ente)
Cons: really subpar non-native apps (desktop app is quite a dumb app as well) :( (and barely and useful additional features that lets a user do some batch/organisational changes or so)
I use this to sync my wife's photos to Immich and it works great, however the auth process is a bit of a pain (not the fault of icloudpd) and have to reauth every few months.
It is a pain.
I’ve wrapped it in some short scripts which notifies on auth failure and it’s an easy process to run the auth script. But there’s no way to avoid the bi-monthly inconvenience I don’t think.
Is there no worry in giving the app your password? I would never just give my password, especially my Apple account password to a random app. Is this program reliable? However it looks like a program that I want to use.
Seems to be looking for a new maintainer. If anyone has the bandwidth, it’s a great piece of software
Wow, I will definitely give this a try. I have tens of thousands of photos in iCloud and I literally can’t export them all at once. Photos app chokes and crashes and manually babysitting smaller batches is a pain. It’s pretty clear they want to make it as hard as possible
Related: does anyone know of a way to delete the original videos on trimmed ones? Apparently all edits to videos are non-destructive, so the 10-minute video that I trimmed to 2 seconds still takes up 8gb.
Yeah, choose "Save Video as New Clip" and then delete the original.
Is there an iCloud Photos uploader?
I have a script to scan files from my camera and add a compressed copy to a folder. This folder was supposed to work with the iCloud for windows (10) program, but one day it just stopped working.
Incredible. I have such a hacked-together system to get my iCloud Photos backed up to an external drive while not filling my main laptop drive. This would be even better.
For those of you on Android / PC-Mac - Syncthing is an excellent tool for P2P syncing between multiple devices (it works over NAT and IPV6 too!).
Ok https://aionlywebsite.pythonanywhere.com/
If you are already in the progress of downloading all of your photos, this is a great time to consider completely switching over to Immich.
It's great!
Why did Apple invent this abomination of a system for storing photos?
We have had file systems for decades. They work well. They sync trivially.
If I'm not mistaken, the Photos app stores originals on disk and then uses a sqlite database to track metadata (and maybe edits to photos as well, given that it has the originals). Seems reasonable to me.
There is something similar to Google Takeout for Apple Photos. Recently used it to download thousands of photos.
Anyone got any tips for programmatically downloading photos in a shared album?
iCloud Photos.app seems to not be able to show some photos
But Photometor.app (owned by Apple) can...
So that's a little annoying... I wish I had more visibility on photos not showing up in Photos.app, and what it is that stops it showing them
You know you only need to do a GDPR request to Apple (dedicated page), select images and you get a download link after a few hours
For those in the EU only ?
Believe it work for everyone? I am located in Oslo/Norway, so can’t check if it does not apply everywhere.
Only need to go to this page to do the request https://privacy.apple.com/
You can't really file a GDPR request, but the (self-service, web based) download tools works for everybody.
I mean can't you go to privacy.apple.com, ask for an archive of your data, and then they'll email you the link to a zip file in a week or two? I'm pretty sure this is what my girlfriend did when she transitioned from an iPhone to a Pixel. I think there's even a specific checkbox for photos/videos
Thanks! I’ve been looking at options for this and didn’t realize Apple had an official method. Just submitted a request, we’ll see how it goes.
Yes... Let me ask my corporate overload and wait 2 weeks for permission to download my family photos.
It's fucking amazing sometimes how the Mac/iOS Photos app can't download a photo or small video for several minutes but you can easily watch YouTube on the same connection.
And it took Apple YEARS to give us a "Keep Downloaded" option for iCloud Drive documents in Finder.
And it's been years since I read any ebooks because the damn Mac/iOS Books app keeps removing my downloaded books even though I have several GBs of storage space left.
Goddamn Tim Cook and the other execs, do they even ever use their own products at all?
Fuck, my wife got a notice that she would have to increase her iCloud storage so last week began the process of ordering a backup of all her pictures so I could get them off iCloud and organized on some drives at home. We got 12 zips of the pictures along with csv's and some metadata, and I literally just finished iterating on the script to sort them into year-based folders and convert all the HEIC shit into JPG. It's running literally right now.
Guess I should've searched harder!
why not use rclone
Because icloudpd supports downloading photos from icloud and rclone does not (yet): https://github.com/rclone/rclone/pull/8734
"why not use rclone (?)"
This is the correct - and obvious - response to something like this.
Unfortunately, I believe that rclone has no support for iCloud photos at this time.
I created this over the holiday break to do mostly the same:
https://github.com/rcarmo/PhotosExport
...when you try to export files using the (restricted) APIs we get, it automatically triggers a download.
Vulnerability hijack incoming with the headline calling for another maintainer
Became more fascinated with the history of my small hometown (Paris, Texas) TLDR: Much of it was wiped out by a 1916 fire. I spent some time recently vibe-coding this interactive map to provide some kind of historic visualization ( which enabled me to see the impact better ) https://gorch.com/parisfiremap/
> Disable Advanced Data Protection
Nope, bzzzzt, wrong!
That's E2E encryption, right? Wouldn't you just end up with a lot of unreadable files if you didn't turn it off before running this downloader?
You need an extra tool to download your own photos? That's... not a basic feature?
I'm always surprised what kind of antifeatures people in Apple land are willing to accept and still use those things...
As is pointed out elsewhere in the thread, there are at three official ways to download your own photos. This complements those.
You absolutely can download your photos, just not via command line like this project enables.
You do not need an extra tool to download your photos. This one runs from the command line, though.