Repository navigation
Replies: 2 comments 6 replies
|
Our goal is for it to scale to millions of images. Most aspects of immich do scale to this level (tested with a library of 2 million assets), but others like the initial full sync currently don’t. We’ve made a lot of scalability improvements and others are on the way, so rest assured that we want immich to work for your library. In the meantime, I agree that elaborating more on the requirements and adding some tips for huge libraries would be helpful. |
|
Many thanks for the quick answer on what certainly isn't a critical time-sensitive issue. I instinctively didn't even try to scan the whole library at first, just added one year after another to the main directory (and finish a scan of the new files in between), and it worked well (until I run into the memory issue that got fixed eventually with the max-old-space-size=8192 after I searched for a solution for weeks, maybe months). |
|
@vb0 Can you elaborate on what your issue was and how you solved it. I currently have an immich server cataloging over 2,000,000 photos and 75,000 videos. If this is going to break because of something you encountered I'd love to fix it before that happens! |
I have 600-ish k objects, most pictures and probably half (or anyway a decent proportion) with GPS tags^1 . The change I had to do otherwise the server was running out of memory was #4530 (comment) . That was a while back. Now the server works, including the map module over the web, including over mobile browser (the layout isn't best for a small screen but the performance is good). The Android mobile app started working some months back (it was getting stuck before at "building the timeline" even on local network, flagship phone, etc.) but its "Your Map" module doesn't load any picture and it's getting stuck. ^1 that's actually the single most important feature for me, and I'm looking for an app to handle everything I've got since almost 20 years. The only ones I've found are Lightroom Classic for which the module stopped working after Adobe started with the subscription shenanigans, then you could unbreak it by using your own Google Maps API key (and paying for the hits, although you get $200 monthly from Google, so not a problem usually) but this got broken too , probably for good this time. The only other alternative is photoprism, that has no mobile app (although the web app works rather well on mobiles too), but the map module is still pretty unresponsive with lots of objects and almost unusable on lower powered browsers. |
|
Does the actual resolution or file size of the images make a difference or only the total number of objects. I am waiting to move everything I have to Immich until after the development pace stabilises and there are fewer breaking changes, but the largest portion of my library is now images between 84-200 MP and very large file sizes. I have a capable machine, but I'd prefer resources on it to be allocated to other processes. |
|
The total number is the main factor, especially the number in a single month. There'll be some big performance improvements in the timeline view for large libraries in the next release. It makes scrolling and scrubbing smooth even with 2 million assets (averaging 3300 per month). The caveats are that the initial page load for the timeline is still slow (2-3s with 2m assets), as is scrolling to a month with a huge number of assets. But these are backend bottlenecks rather than from the web app. Higher resolution images will take more time during thumbnail generation and possibly metadata extraction and use more RAM. But it doesn't really matter in normal usage since the generated thumbnails and previews are used instead. An exception to that is when opening an image: if the image is in a web-compatible format like JPEG and you either zoom in or have the "prefer original images" preference enabled, it will take longer for the original to be downloaded and decoded. |
|
This is likely still top of mind, I just wanted to link something from the big fruit company on the off chance that the optimization hasn't been considered yet: "Unload when the user has scrolled past" Even though I still see many projects fail at this, it was a genius idea of the first glass phone as the RAM and VRAM were minuscule. It allowed butter-smooth scrolling of huge lists. On the front end that looks like unloading the off-screen images while keeping the indexes (bringing them in to VRAM when they are almost on-screen) |
|
Yes, we do virtual scrolling among other optimizations; the timeline is a pretty sophisticated implementation at this point. |
Uh oh!
There was an error while loading. Please reload this page.
To preface my conundrum on a meta level I run many times into trouble with my Immich instance but I'm too embarrassed to open an issue because I do have quite a few objects and probably it isn't expected to scale with quite a few hundreds of thousands of pictures. But the problem is that I don't even know how far it should go, how much can be fixed by throwing more hardware on it, how much is a simple tweak (found only in some discussion within a github issue) or how much is a limitation of the initial naive design without much data to go on from the beginning (which will never work right until a full rewrite).
This wasn't that bad as the software matures but now Immich it's THE self-hosting pictures solution, plus the FUTO involvement, this needs to change. Somewhere very early in the documentation there should be some more specific numbers about what you need (and probably higher too, I mean 4-6GBs is barely what you need on a mobile phone nowadays), and paired with some number of objects and/or other features that might be influencing the scaling/performance. Then there should be a list of known (be it absolutely recommended or last-ditch) workarounds/config changes and lastly what can't be done for now because of known limitations.
All reactions