Skip to content

feat: microservices be gone - #9551

Merged
zackpollard merged 2 commits into
mainfrom
feat/remove-microservices-container
May 17, 2024
Merged

zackpollard merged 2 commits into
mainfrom
feat/remove-microservices-container

Conversation

@zackpollard

@zackpollard zackpollard commented May 17, 2024 •

Copy link
Copy Markdown
Member

Caution

Suggested change: immich-microservices container can be removed from docker compose
Breaking change: start-microservices.sh and start-server.sh has been removed, we haven't used these for a while now

This PR removes the need for the seperate microservices container, we tested the workers in the 1.105.0 release by moving microservices into a worker process and we've had no issues off the back of that.

  • Removes microservices container from all docker-compose files
  • Moves the API & Web into an api worker
  • Defaults the bootstrap to run both api and microservices as seperate workers
  • Adds IMMICH_WORKERS_INCLUDE and IMMICH_WORKERS_EXCLUDE as comma seperated variables to specify which workers you want to run
  • IMMICH_WORKERS_INCLUDE will override the defaults, only running what you specify
  • IMMICH_WORKERS_EXCLUDE will apply against the defaults, or IMMICH_WORKERS_INCLUDE if for whatever reason you wanted to include and then exclude some workers 😆

This will simplify the setup process for most users by reducing the amount of containers they are running, the only real downside is that you now can't apply resource constraints against the microservices individually, however you could just go back to having the API and Microservices seperated using the env vars specified above.

Motivation

This is one of many pieces of upcoming work to change the way we handle jobs, microservices was always really a patch for the problem that we couldn't run background tasks within the main server as that would introduce contention for the API itself. Using workers, we can now have these within the same container but separated out into different V8 engines. Moving forwards, we will be looking to split out microservices into more underlying segments such as transcoding, thumbnails, etc, allowing us to have actual "microservices" and people who want to distribute just transcoding for example will be able to.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 17, 2024 •

Copy link
Copy Markdown

Deploying immich with  Cloudflare Pages  Cloudflare Pages

Latest commit: cc12c61
Status: ✅  Deploy successful!
Preview URL: https://d139b555.immich.pages.dev
Branch Preview URL: https://feat-remove-microservices-co.immich.pages.dev

View logs

@alextran1502

Copy link
Copy Markdown
Member

Can you help write up the motivations behinds this change? So we can include those information in the next release note? Thank you!

Comment thread docker/docker-compose.dev.yml Outdated
Comment thread docker/docker-compose.prod.yml Outdated
Comment thread e2e/docker-compose.yml Outdated
@zackpollard
zackpollard force-pushed the feat/remove-microservices-container branch from 65248e8 to 89da5d6 Compare May 17, 2024 13:05

@jrasm91 jrasm91 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noice

@zackpollard
zackpollard force-pushed the feat/remove-microservices-container branch from 89da5d6 to 7a7ce85 Compare May 17, 2024 13:17

@bo0tzz bo0tzz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@zackpollard
zackpollard force-pushed the feat/remove-microservices-container branch from 7a7ce85 to 264fa1b Compare May 17, 2024 13:25
@zackpollard
zackpollard force-pushed the feat/remove-microservices-container branch from 264fa1b to cc12c61 Compare May 17, 2024 13:30
@zackpollard
zackpollard merged commit 85aca2b into main May 17, 2024
@zackpollard
zackpollard deleted the feat/remove-microservices-container branch May 17, 2024 13:44
@mmomjian

mmomjian commented May 17, 2024 •

Copy link
Copy Markdown
Collaborator

Edit: leaving start.sh immich, start.sh microservices works fine.

I will move this comment into the docs as an example of a simple split worker setup.

To add to release notes:

If you prefer to retain the previous structure, with a separate immich_server and immich_microservices container, this can be accomplished as follows:

services:
  immich-server:
    ...
+   environment:
+     IMMICH_WORKERS_INCLUDE: 'api'

  immich-microservices:
    ...
+   environment:
+     IMMICH_WORKERS_EXCLUDE: 'api'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants