Repository navigation
fix(server): more robust person thumbnail generation - #17974
Merged
Merged
Conversation
Member
|
Do you have RAW assets to reproduce these edge cases? |
10 tasks done
1 of 3 tasks
mertalev
force-pushed
the
fix/server-person-thumbnail-crop
branch
from
May 5, 2025 18:57
cbc3ea8 to
d5b2143
Compare
Member
Author
The way I tested it is by setting the min detection threshold to 0.02 (after adjusting the min value for this in the dto and web client) and set the min recognized faces to 1. The result will be that just about any image you upload will have hundreds of detected faces and a person + thumbnail will be generated for each. I tested by uploading this image that has especially different dimensions between the reported |
mertalev
marked this pull request as ready for review
May 5, 2025 19:11
github-actions
Bot
added
the
documentation
alextran1502
approved these changes
May 6, 2025
alextran1502
reviewed
May 6, 2025
mertalev
force-pushed
the
fix/server-person-thumbnail-crop
branch
from
May 6, 2025 18:09
d9f12e9 to
8cf77d3
Compare
savely-krasovsky
pushed a commit
to savely-krasovsky/immich
that referenced
this pull request
Jun 8, 2025
* more robust person thumbnail generation * clamp bounding boxes * update sql * no need to process invalid images after decoding * cursed knowledge * new line
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
There are several edge cases that can lead person thumbnail generation to fail:
The image dimensions stored in the DB are not always accurate to what the decoded dimensions will be. This can happen for RAW assets in particular, which can report the cropped or uncropped dimensions, etc.
Metadata extraction may not have finished. When importing face metadata, it is possible for person thumbnail generation to be queued and run first, which can lead to failed jobs.
The metadata stored in the DB can be outdated. There is no guarantee that the data read from the DB is fully up-to-date.
If the use of embedded previews is enabled, the embedded preview would only be used for detecting faces and not for generating the thumbnail. This discrepancy can lead to failure because of different dimensions and metadata.
Faces can spill outside the image dimensions as the model predicts "where the face would be" when the face is cut off.
This PR tightens the logic here by relying more on the metadata that sharp itself reports at the time of the job, as well as considering embedded previews. It moves this job to the media service as it relates more to other thumbnail jobs and helper methods than it does to person-related jobs. Lastly, it clamps the bounding box to be within the image dimensions.
Fixes #12440
Fixes #12530
Fixes #14993