feat: vectorchord by mertalev · Pull Request #18042 · immich-app/immich

mertalev · 2025-05-02T21:56:23Z

Description

This PR introduces support for VectorChord, the successor to pgvecto.rs. It's capable of higher QPS at the same recall, much faster indexing, lower RAM usage as well as better handling of inserts, updates and deletes. In addition, it is more integrated into Postgres using native indexing workers and memory management, and can coexist better with pgvector (which it comes bundled with).

The PR does not remove support for pgvecto.rs, meaning that the existing 0.2.0 image will continue to work, but will be deprecated in a future release. The correct extension to use (vchord, pgvecto.rs or pgvector) is autodetected at startup, and can be overridden with the DB_VECTOR_EXTENSION environmental variable. Switching between supported extensions is possible in this way as long as both extensions exist in the DB image. The intention is to make the migration as smooth as possible while giving both admins and build maintainers time to transition.

~~However, upgrading directly from releases using TypeSense or pgvecto.rs 0.1.11 (i.e. releases before 1.95.0) is no longer supported as of this PR.~~ However, upgrading directly from releases below 1.107.0 (released 10 months ago) is no longer supported as of this PR, as it assumes the face_search relation exists. Users on these old versions should upgrade to the release before this commit before upgrading to the latest.

A quirk of VectorChord inherent to all IVF indices is that the optimal index parameters depend on the number of vectors. Since this is not something we know beforehand, there are checks in place to ensure roughly optimal settings at startup. This only takes place if the current settings are far from ideal so as to avoid frequent reindexing. The settings applied in this PR were tested for facial recognition with 15k vectors, 130k and 166k vectors to achieve very similar results to brute-force clustering. Smart search recall was tested at 1 million and 3.3 million vectors, but could benefit from similar testing with duplicate detection.

To do:

It may be beneficial to do the parameter checks outside of startup as well in the case of a large import without restart
The number of probes for a query needs to be set as well. This number depends on the number of lists, so this should be computed whenever it checks for lists or reindexes with a different number of lists
The provided migration images only target amd64 and are for VectorChord 0.2.2 while 0.3.0 is out already, so we may need to build our own images even without targeting pg17
We need instructions on how to do this migration in a non-superuser context
Test upgrading immich from certain versions with pgvecto.rs 0.2.0, 0.2.1 and 0.3.0 (tested 1.107 with 0.2.0 and 0.2.1, 1.120 with 0.3.0, and 1.132.3 with 0.2.0 and 0.3.0).

An informal facial recognition comparison with 166495 faces (min detection=0.2, min faces=8, max distance=0.4):

	People	Clustered Faces	Faces/s	Index time	RAM (startup)	RAM (active)	Search CPU (%)
vectors	1328	61931	321	28m	472 MiB	1090 MiB	65-70
vchord	1264	66939	481	29s	136 MiB	645 MiB	55-60

The selected VectorChord parameters are extremely biased in favor of quality and it still substantially outperforms pgvecto.rs in every metric.

bo0tzz · 2025-05-03T10:17:41Z

Is there a migration path without having both extensions available at the same time?

mertalev · 2025-05-03T18:07:24Z

It should be possible to remove the references to the old extension manually with some queries before starting immich with the new extension.

jrasm91 · 2025-05-09T23:51:07Z

@@ -1,13 +1,12 @@
-import { ConfigRepository } from 'src/repositories/config.repository';
+import { getVectorExtension } from 'src/repositories/database.repository';


Why are you changing this? It's not great to keep needing to modify past migrations.

Well, we're adding another extension so it's past the point of whether to modify the migration. The change is superficial, though, and won't change the actual behavior of the migration for existing extensions.

There is an option to instead remove TypeORM migrations as of 1.107 since we're setting that as a minimum, and it happens to include all the TypeORM vector migrations.

Lol that's an option I guess. It would be nice to remove all of them at the same time. I guess it helps lessen the chance of bugs if that code path won't even be executed.

jrasm91 · 2025-05-09T23:51:45Z

    config: DatabaseConnectionParams;
    skipMigrations: boolean;
-    vectorExtension: VectorExtension;
+    vectorExtension?: VectorExtension;


Why is this no longer required?

jrasm91 · 2025-05-09T23:53:51Z

 import { vectorIndexQuery } from 'src/utils/database';
 import { isValidInteger } from 'src/validation';
-import { DataSource } from 'typeorm';
+import { DataSource, QueryRunner } from 'typeorm';


Please stop adding in more typeorm imports...

danieldietzler · 2025-05-13T15:26:34Z

This repository is quite complex by now. Is that really necessary?

It can't get much simpler without loss of functionality or robustness. The only optional parts are the auto-detection and correcting missing indices and embedding columns. The former is convenient, both for admins and for development, and the latter came up a lot during the 0.2.0 migration and resulted in people running their own SQL commands and making things worse. The more robust this code is, the less support burden we will have.

I definitely wouldn't advocate for losing robustness. Wondering if we need to support this broad variety of extensions though.

auto-detect available extensions auto-recovery, fix reindexing check use original image for ml

update images for gha

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

mmomjian · 2025-05-20T13:46:54Z

@mertalev Do extensions need to be dropped before "6. Remove the vectors.so entry from the shared_preload_libraries setting" and then removing the package from the system? I'm not familiar with extension uninstallation

mertalev · 2025-05-20T15:18:05Z

I didn't need to for the migration, but it's probably still a good idea to drop the old extension. I imagine it could cause issues with restoring from backups if it still exists. Maybe Immich should try to drop the extension too.

mmomjian · 2025-05-20T15:28:32Z

I think we probably should consider doing that, if not with this release in the near future if we want to wait for safety.

* wip auto-detect available extensions auto-recovery, fix reindexing check use original image for ml * set probes * update image for sql checker update images for gha * cascade * fix new instance * accurate dummy vector * simplify dummy * preexisiting pg docs * handle different db name * maybe fix sql generation * revert refreshfaces sql change * redundant switch * outdated message * update docker compose files * Update docs/docs/administration/postgres-standalone.md Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com> * tighten range * avoid always printing "vector reindexing complete" * remove nesting * use new images * add vchord to unit tests * debug e2e image * mention 1.107.2 in startup error * support new vchord versions --------- Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

dotlambda · 2025-06-19T20:18:02Z

+      await sql`ALTER DATABASE ${dbName} SET vchordrq.prewarm_dim = '512,640,768,1024,1152,1536'`.execute(this.db);
+      await sql`SET vchordrq.prewarm_dim = '512,640,768,1024,1152,1536'`.execute(this.db);
+      await sql`ALTER DATABASE ${dbName} SET vchordrq.probes = 1`.execute(this.db);
+      await sql`SET vchordrq.probes = 1`.execute(this.db);


Isn't the version without ALTER DATABASE redundant? It also requires superuser privileges.

It isn't because the parameters in alter database only take effect for new connections. You need to run the normal set version to make it apply to the current connection as well.

It shouldn't require superuser permission though - even the alter database command only requires db ownership, and the session-level version shouldn't require even that.

You're right. I was erroneously assuming that SET means ALTER SYSTEM SET.

github-actions Bot added the 🗄️server label May 2, 2025

mertalev added the changelog:feature label May 2, 2025

diogotcorreia mentioned this pull request May 9, 2025

postgresqlPackages.vectorchord: init at 0.4.2 NixOS/nixpkgs#392710

Merged

3 of 13 tasks

xieydd mentioned this pull request May 9, 2025

Fix typo tensorchord/pgvecto.rs-docs#115

Merged

mertalev force-pushed the feat/vectorchord branch 3 times, most recently from b024fbb to 878a80e Compare May 9, 2025 22:58

jrasm91 reviewed May 10, 2025

View reviewed changes

github-actions Bot added the documentation Improvements or additions to documentation label May 12, 2025

mertalev force-pushed the feat/vectorchord branch from ddbf125 to 6126ac7 Compare May 13, 2025 00:57

mertalev marked this pull request as ready for review May 13, 2025 14:34

mertalev requested review from bo0tzz and danieldietzler as code owners May 13, 2025 14:34

danieldietzler reviewed May 13, 2025

View reviewed changes

mertalev force-pushed the feat/vectorchord branch from 334ed7d to 49beacc Compare May 15, 2025 16:48

danieldietzler mentioned this pull request May 17, 2025

docker-compose.yml uses deprecated pgvecto.rs #18349

Closed

2 of 4 tasks

mertalev added 12 commits May 19, 2025 15:21

wip

a3d90ff

auto-detect available extensions auto-recovery, fix reindexing check use original image for ml

set probes

9b397d8

update image for sql checker

5d26d0a

update images for gha

cascade

66a4a55

fix new instance

9e22d1f

accurate dummy vector

594b0e5

simplify dummy

e14f590

preexisiting pg docs

437bb21

handle different db name

85e060e

maybe fix sql generation

e6caab4

revert refreshfaces sql change

9413538

redundant switch

abb3b53

mertalev and others added 11 commits May 19, 2025 15:21

outdated message

0a8b2ea

update docker compose files

acd0fab

Update docs/docs/administration/postgres-standalone.md

654d628

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

tighten range

ba10de7

avoid always printing "vector reindexing complete"

be7cc80

remove nesting

91cccf0

use new images

0e9696e

add vchord to unit tests

7537b36

debug e2e image

dc2c0f9

mention 1.107.2 in startup error

5482be9

support new vchord versions

c1fa389

mertalev force-pushed the feat/vectorchord branch from 7163b0c to c1fa389 Compare May 19, 2025 19:21

alextran1502 added the changelog:breaking-change label May 19, 2025

mertalev removed the changelog:feature label May 19, 2025

alextran1502 approved these changes May 20, 2025

View reviewed changes

alextran1502 merged commit 0d773af into main May 20, 2025

alextran1502 deleted the feat/vectorchord branch May 20, 2025 13:36

dotlambda reviewed Jun 19, 2025

View reviewed changes

dotlambda mentioned this pull request Jun 19, 2025

Package request: VectorChord NixOS/nixpkgs#392680

Closed

3 tasks done

coderabbitai Bot mentioned this pull request Aug 19, 2026

Remove retired services and harden environment templates TechHutTV/homelab#101

Merged

		@@ -1,13 +1,12 @@
		import { ConfigRepository } from 'src/repositories/config.repository';
		import { getVectorExtension } from 'src/repositories/database.repository';

Conversation

mertalev commented May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

bo0tzz commented May 3, 2025

Uh oh!

mertalev commented May 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mertalev May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mmomjian commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mertalev commented May 20, 2025

Uh oh!

mmomjian commented May 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mertalev commented May 2, 2025 •

edited

Loading

mertalev May 13, 2025 •

edited

Loading

mmomjian commented May 20, 2025 •

edited

Loading