Skip to content

[BUG] force re-scan all library files doesnt update the asset metadata/thumbnails #5624

Closed as duplicate

Description

@amit-handa

The bug

the asset metadata/thumbnail is not updated on force re-scan all library files

Thank you for maintaining the product. I am really excited to use it and improve/contribute to it.

The OS that Immich Server is running on

ubuntu 23.10

Version of Immich Server

v1.90.2

Version of Immich Mobile App

v1.90.2

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /mnt/media/pics:/mnt/media/pics
      - /mnt/media/immich-albums:/mnt/media/immich-albums
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.yml
      service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /mnt/media/pics:/mnt/media/pics
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always


  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:3995fe6ea6a619313e31046bd3c8643f9e70f8f2b294ff82659d409b47d06abb
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:50d9be76e9a90da4c781554955e0ffc79d9d5c4226838e64b36aacc97cbc35ad
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always


volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/media/immich-app/immich-data
LOG_LEVEL=debug

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=abc
DB_PASSWORD=abc

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

create a folder (pics) containing a single image file (abc.jpg)

1. import [external library](https://immich.app/docs/features/libraries#external-libraries), say pics
2. create external library via user->account settings->libraries->create external library
3. view the album (pics), verify the image is present (abc.jpg)
4. update `abc.jpg` with a new file `def.jpg` (`cp def.jpg abc.jpg`)
5. refresh the external library. user->account settings->libraries->`pics` external library->`force re-scan all library files`
6. observe the server logs to ensure the process has completed (shouldnt take more than a few seconds)
7. view the album again.
Issue: 
1/ instead of seeing `def.jpg` metadata, I still see `abc.jpg` metadata
2/ instead of seeing `def.jpg`, I still see `abc.jpg`
deeper dive: on downloading the image, I see the correct image `def.jpg`.
possible bug: the thumbnail is not updated on step 5. :(

Additional information

I see following AssetService errors

immich_server              | [Nest] 9  - 12/11/2023, 2:05:46 AM   ERROR [AssetService] Cannot create read stream for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4
immich_server              | getAssetThumbnail
immich_server              | [Nest] 9  - 12/11/2023, 2:05:46 AM    WARN [AssetService] WebP thumbnail requested but not found for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4, falling back to JPEG
immich_server              | [Nest] 9  - 12/11/2023, 2:05:46 AM   ERROR [AssetService] Cannot create read stream for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4
immich_server              | getAssetThumbnail
immich_server              | [Nest] 9  - 12/11/2023, 2:07:17 AM   ERROR [AssetService] Cannot create read stream for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4
immich_server              | getAssetThumbnail
immich_server              | [Nest] 9  - 12/11/2023, 2:07:17 AM    WARN [AssetService] WebP thumbnail requested but not found for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4, falling back to JPEG
immich_server              | [Nest] 9  - 12/11/2023, 2:07:17 AM   ERROR [AssetService] Cannot create read stream for asset 3120ee43-e923-4f88-ad53-6604d4fd32d4

Activity

  1. changed the title [-][BUG] `force re-scan all library files` doesnt update the asset thumbnails[/-] [+][BUG] `force re-scan all library files` doesnt update the asset metadata/thumbnails[/+] on Dec 11, 2023
  2. nicolasshu commented on Dec 14, 2023

    @nicolasshu

    Commenting to follow. I too am not being able to see the thumbnails
    image

    Nor am I being able to see the actual image/faces
    image

    When looking at my files, I see:

    drwxrw-r--  1 username users  104 Dec 14 03:15 .
    drwxrwxrwx+ 1 root     root   632 Dec 10 23:02 ..
    -rwxrwxrwx  1 username users 2.5K Dec 14 03:15 docker-compose.yml
    -rwxrwxrwx  1 username users  764 Dec 12 22:19 .env
    drwxrw-rw-  1 username users    0 Dec 14 03:04 library
    drwx------  1 username users  162 Dec 14 02:29 model-cache
    drwx------  1       70 users  512 Dec 14 02:23 pgdata
    drwx------  1 username users   22 Dec 14 02:23 tsdata
    

    I don't think it's a permission issue, as my files look like this:

    drwxrwxrwx+ 1 admin         users 1.1K Dec 14 02:58 .
    drwxrwxrwx+ 1 root          root   71K Dec 14 02:58 ..
    drwxrwxrwx+ 1 username      users 3.3K Dec 14 02:59 @eaDir
    -rwxrwxrwx+ 1 username      users 5.0M Dec 14 02:58 IMG_6669.JPG
    -rwxrwxrwx+ 1 username      users 4.6M Dec 14 02:58 IMG_6670.JPG
    -rwxrwxrwx+ 1 username      users 4.7M Dec 14 02:58 IMG_6671.JPG
    -rwxrwxrwx+ 1 username      users 4.8M Dec 14 02:58 IMG_6672.JPG
    -rwxrwxrwx+ 1 username      users 4.0M Dec 14 02:58 IMG_6673.JPG
    -rwxrwxrwx+ 1 username      users 4.6M Dec 14 02:58 IMG_6674.JPG
    -rwxrwxrwx+ 1 username      users 4.7M Dec 14 02:58 IMG_6675.JPG
    

    I've also tried the solution in #4246 to regenerate the missing thumbnails job, but nothing changed.

  3. KenzoB73 commented on Mar 24, 2024

    @KenzoB73

    Seeing the same issue as OP. I rotated a bunch of photos, did a rescan, thumbnails stay the same, but if I download the file, it is accurate. Running v1.99.0.

  4. figadore commented on Jun 5, 2024

    @figadore

    Possibly related, I restored my database, which caused all assets to be listed as "Offline" until I force-rescanned. Now everything works except the faces at /people. It works when I manually go into a person and "Select featured photo", but I can't do this for all 400 detected people/faces. How do I get it to regenerate the person thumbnails?
    image

    v1.105.1

  5. jrasm91 commented on Sep 5, 2024

    @jrasm91
    Member

    This is a current limitation of the external libraries feature. Basically, checksum is emulated and the browser cache bust requires a checksum change. We may move this to be nonce-based in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions