Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): microservices be gone #9558

Merged
merged 23 commits into from
May 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/FAQ.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Yes, with an [External Library](/docs/features/libraries.md).

### What happens to existing files after I choose a new [Storage Template](/docs/administration/storage-template.mdx)?

Template changes will only apply to _new_ assets. To retroactively apply the template to previously uploaded assets, run the Storage Migration Job, available on the [Jobs](/docs/administration/jobs.md) page.
Template changes will only apply to _new_ assets. To retroactively apply the template to previously uploaded assets, run the Storage Migration Job, available on the [Jobs](/docs/administration/jobs-workers/#jobs) page.

### Why are only photos and not videos being uploaded to Immich?

Expand Down
55 changes: 55 additions & 0 deletions docs/docs/administration/jobs-workers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Jobs and Workers

## Workers

### Architecture

The `immich-server` container contains multiple workers:

- `api`: responds to API requests for data and files for the web and mobile app.
- `microservices`: handles most other work, such as thumbnail generation and video encoding, in the form of _jobs_. Simply put, a job is a request to process data in the background.

## Split workers

If you prefer to throttle or distribute the workers, you can do this using the [environment variables](/docs/install/environment-variables) to specify which container should pick up which tasks.

For example, for a simple setup with one container for the Web/API and one for all other microservices, you can do the following:

Copy the entire `immich-server` block as a new service and make the following changes to the **copy**:

```diff
- immich-server:
- container_name: immich_server
...
- ports:
- - 2283:3001
+ immich-microservices:
+ container_name: immich_microservices
```

Once you have two copies of the immich-server service, make the following chnages to each one. This will allow one container to only serve the web UI and API, and the other one to handle all other tasks.

```diff
services:
immich-server:
...
+ environment:
+ IMMICH_WORKERS_INCLUDE: 'api'

immich-microservices:
...
+ environment:
+ IMMICH_WORKERS_EXCLUDE: 'api'
```

## Jobs

When a new asset is uploaded it kicks off a series of jobs, which include metadata extraction, thumbnail generation, machine learning tasks, and storage template migration, if enabled. To view the status of a job navigate to the Administration -> Jobs page.

Additionally, some jobs run on a schedule, which is every night at midnight. This schedule, with the exception of [External Libraries](/docs/features/libraries) scanning, cannot be changed.

:::info
Storage Migration job can be run after changing the [Storage Template](/docs/administration/storage-template.mdx), in order to apply the change to the existing library.
:::

<img src={require('./img/admin-jobs.png').default} width="80%" title="Admin jobs" />
13 changes: 0 additions & 13 deletions docs/docs/administration/jobs.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/docs/developer/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ The Immich Microservices image uses the same `Dockerfile` as the Immich Server,
- Background jobs (file deletion, user deletion)

:::info
This list closely matches what is available on the [Administration > Jobs](/docs/administration/jobs.md) page, which provides some remote queue management capabilities.
This list closely matches what is available on the [Administration > Jobs](/docs/administration/jobs-workers/#jobs) page, which provides some remote queue management capabilities.
:::

### Machine Learning
Expand Down
17 changes: 4 additions & 13 deletions docs/docs/features/libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,16 +44,16 @@ If the import paths are edited in a way that an external file is no longer in an

### Troubleshooting

Sometimes, an external library will not scan correctly. This can happen if immich_server or immich_microservices can't access the files. Here are some things to check:
Sometimes, an external library will not scan correctly. This can happen if Immich can't access the files. Here are some things to check:

- In the docker-compose file, are the volumes mounted correctly?
- Are the volumes identical between the `server` and `microservices` container?
- Are the volumes also mounted to any worker containers?
- Are the import paths set correctly, and do they match the path set in docker-compose file?
- Make sure you don't use symlinks in your import libraries, and that you aren't linking across docker mounts.
- Are the permissions set correctly?
- Make sure you are using forward slashes (`/`) and not backward slashes.

To validate that Immich can reach your external library, start a shell inside the container. Run `docker exec -it immich_microservices /bin/bash` to a bash shell. If your import path is `/data/import/photos`, check it with `ls /data/import/photos`. Do the same check for the `immich_server` container. If you cannot access this directory in both the `microservices` and `server` containers, Immich won't be able to import files.
To validate that Immich can reach your external library, start a shell inside the container. Run `docker exec -it immich_server bash` to a bash shell. If your import path is `/data/import/photos`, check it with `ls /data/import/photos`. Do the same check for the same in any microservices containers.

### Exclusion Patterns

Expand Down Expand Up @@ -98,7 +98,7 @@ First, we need to plan how we want to organize the libraries. The christmas trip

### Mount Docker Volumes

`immich-server` and `immich-microservices` containers will need access to the gallery. Modify your docker compose file as follows
The `immich-server` container will need access to the gallery. Modify your docker compose file as follows

```diff title="docker-compose.yml"
immich-server:
Expand All @@ -107,15 +107,6 @@ First, we need to plan how we want to organize the libraries. The christmas trip
+ - /mnt/nas/christmas-trip:/mnt/media/christmas-trip:ro
+ - /home/user/old-pics:/mnt/media/old-pics:ro
+ - /mnt/media/videos:/mnt/media/videos:ro
+ - "C:/Users/user_name/Desktop/my media:/mnt/media/my-media:ro" # import path in Windows system.


immich-microservices:
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
+ - /mnt/nas/christmas-trip:/mnt/media/christmas-trip:ro
+ - /home/user/old-pics:/mnt/media/old-pics:ro
+ - /mnt/media/videos:/mnt/media/videos:ro
+ - "C:/Users/user_name/Desktop/my media:/mnt/media/my-media:ro" # import path in Windows system.
```

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/guides/custom-locations.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ It is important to remember to update the backup settings after following the gu
In our `.env` file, we will define variables that will help us in the future when we want to move to a more advanced server in the future

```diff title=".env"
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables
# You can find documentation for all the supported env variables [here](/docs/install/environment-variables)

# Custom location where your uploaded, thumbnails, and transcoded video files are stored
- UPLOAD_LOCATION=./library
Expand Down
6 changes: 2 additions & 4 deletions docs/docs/guides/docker-help.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,8 @@ docker ps -a # see a list of running and stopped containers

```bash
docker exec -it <id or name> <command> # attach to a container with a command
docker exec -it immich_server sh
docker exec -it immich_microservices sh
docker exec -it immich_machine_learning sh
docker exec -it immich_server bash
docker exec -it immich_machine_learning bash
```

## Logs
Expand All @@ -22,7 +21,6 @@ docker exec -it immich_machine_learning sh
docker logs <id or name> # see the logs for a specific container (by id or name)

docker logs immich_server
docker logs immich_microservices
docker logs immich_machine_learning
```

Expand Down
8 changes: 1 addition & 7 deletions docs/docs/guides/external-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,14 @@ in a directory on the same machine.

# Mount the directory into the containers.

Edit `docker-compose.yml` to add two new mount points in the sections `immich-server:` and `immich-microservices:` under `volumes:`
Edit `docker-compose.yml` to add two new mount points in the section `immich-server:` under `volumes:`

```diff
immich-server:
volumes:
+ - ${EXTERNAL_PATH}:/usr/src/app/external

immich-microservices:
volumes:
+ - ${EXTERNAL_PATH}:/usr/src/app/external
```

Be sure to add exactly the same path to both services.

Edit `.env` to define `EXTERNAL_PATH`, substituting in the correct path for your computer:

```
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/guides/remote-machine-learning.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ To alleviate [performance issues on low-memory systems](/docs/FAQ.mdx#why-is-imm
- Start the container by running `docker compose up -d`.

:::info
Starting with version v1.93.0 face detection work and face recognize were split. From now on face detection is done in the immich_machine_learning service, but facial recognition is done in the immich_microservices service.
Starting with version v1.93.0 face detection work and face recognize were split. From now on face detection is done in the immich_machine_learning container, but facial recognition is done in the `microservices` worker.
:::

:::note
Expand Down
Loading
Loading