immich has randomized temporal parsing of a series of photos with similar timestamps to an external library.

[Gloryandel]

The bug

immich has randomized temporal parsing of a series of photos with similar timestamps to an external library.Repeatedly putting the same file in and out of the external library will have a probability that some of the files will have time recognition anomalies.
Move the recognized abnormal file out of the external library and put it in again with the probability that it will be recognized correctly again.

The OS that Immich Server is running on

Linux Arc 6.10.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 12 Sep 2024 17:21:02 +0000 x86_64 GNU/Linux && podman version 5.2.2

Version of Immich Server

v1.115.0

Version of Immich Mobile App

v1.115.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

No special configuration, only add external libraries

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/srv/pool/appdata/1000/immich/library
# The location where your database files are stored
DB_DATA_LOCATION=/srv/pool/appdata/1000/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Asia/Shanghai

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=ps

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

# Custom
EXTERNAL_PATH=/srv/pool/data/angel/files/Pictures

Reproduction steps

1.Arbitrary screenshot as a template, store it locally, copy it many times (enough, 100+ probability of time recognition anomaly)
2.Modify the timestamps of all test files to find past world errors recognized as current time. (I use Directory Opus to modify to past time)
3.Place all test files into any path in the external library and observe if the test files appear in today's image area.
4.If it does not appear, the test file will be moved out of the external library, put it in again, continue to observe, in general, 100 test files, repeat 1 to 3 times must appear a small number of time recognition error.
5.If there is a time recognition error, it is moved out of the external library individually and has a probability of being recognized correctly again.

...

Relevant log output

d76b699e8a91 [Nest] 17  - 09/26/2024, 1:12:13 PM     LOG [Api:EventRepository] Websocket Connect:    GJbN-XXtbq5GivCpAABn
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (1).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (10).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (3).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (31).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM    WARN [Microservices:MetadataService] Asset b903fd97-0218-40ba-b4ad-2af86ec78142 has no valid date (undefined), falling back to asset.fileCreatedAt
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM    WARN [Microservices:MetadataService] Asset b903fd97-0218-40ba-b4ad-2af86ec78142 has no time zone information
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (35).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh-asset): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
d76b699e8a91     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
d76b699e8a91     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
d76b699e8a91     at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
d76b699e8a91     at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
d76b699e8a91     at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
d76b699e8a91     at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
d76b699e8a91     at async LibraryService.handleAssetRefresh (/usr/src/app/dist/services/library.service.js:381:32)
d76b699e8a91     at async /usr/src/app/dist/services/job.service.js:148:36
d76b699e8a91     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
d76b699e8a91     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM   ERROR [Microservices:JobService] Object:
d76b699e8a91 {
d76b699e8a91   "id": "c54d98a3-44e6-48e0-b55e-4f11b8783a4a",
d76b699e8a91   "assetPath": "/mnt/Pictures/新建文件夹/剪贴板图片 - 复制 (36).jpg",
d76b699e8a91   "ownerId": "d16fb677-b09a-4b72-99a1-e29c4dbc1441",
d76b699e8a91   "force": false
d76b699e8a91 }
d76b699e8a91
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM    WARN [Microservices:MetadataService] Asset 615e7fa6-dfe3-44c7-912e-db8fc5b29b17 has no valid date (undefined), falling back to asset.fileCreatedAt
d76b699e8a91 [Nest] 7  - 09/26/2024, 1:12:52 PM    WARN [Microservices:MetadataService] Asset 615e7fa6-dfe3-44c7-912e-db8fc5b29b17 has no time zone information
.........
.........
d76b699e8a91 [Nest] 17  - 09/26/2024, 1:13:07 PM     LOG [Api:EventRepository] Websocket Disconnect: GJbN-XXtbq5GivCpAABn
d76b699e8a91 [Nest] 17  - 09/26/2024, 1:13:08 PM     LOG [Api:EventRepository] Websocket Connect:    qKbtWYFxy_53sRe2AABp

Additional information

In practice recognition errors can occur by placing enough temporally similar images into an external library.

[Gloryandel]

Appearance is not related to additional metadata information and has been tested several times. The metadata removal on the way to the test was all carried out using the specialized software XnViewMP.