Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classify background jobs don't seem to be doing anything and queue stays filled up. Manual recognize:classify works. #1030

Closed
Fosnine opened this issue Nov 12, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@Fosnine
Copy link

Fosnine commented Nov 12, 2023

Which version of recognize are you using?

5.0.3

Enabled Modes

Object recognition, Face recognition, Video recognition

TensorFlow mode

Normal mode

Downstream App

Memories App

Which Nextcloud version do you have installed?

NextCloudPi 27.0.1

Which Operating system do you have installed?

Debian 11.8

Which database are you running Nextcloud on?

MySQL 10.5.21

Which Docker container are you using to run Nextcloud? (if applicable)

No response

How much RAM does your server have?

6GiB

What processor Architecture does your CPU have?

x86_64

Describe the Bug

When I run recognize:classify manually, everything seems to work correctly and tags show up. I don't have any people even after running cluster manually with the command but that seems to be correct since recognize says:
'Face clustering: 79 faces left to cluster, Last clustering run: 1 hour ago, Scheduled background jobs: 0,
A minimum of 120 faces per user is necessary for clustering to kick in;
Files stay queued after adding them for 2+ days but I don't get any errors. background jobs are scheduled but the amount of queued files isn't going down. When a picture is added, it immediately shows up in the queue.
GUI shows:
'Face recognition: 72 Queued files, Last classification: 1 hour ago, Scheduled background jobs: 1, Last background job execution: 3 minutes ago'
'Object recognition: 72 Queued files, Last classification: 1 hour ago, Scheduled background jobs: 1, Last background job execution: 8 minutes ago'
'Landmark recognition: 27 Queued files, Last classification: never, Scheduled background jobs: 2, Last background job execution: 8 minutes ago'
'Video recognition: 0 Queued files, Last classification: 1 hour ago, Scheduled background jobs: 0,'

The last classification time is when I ran recognize:classify again.

Cron seems to be working correctly since it doesn't show any errors when run manually and debug logs show that other jobs get finished in 0 seconds:

e.g.
[cron] Debug: Finished OCA\Notifications\BackgroundJob\SendNotificationMails job with ID 18 in 0 seconds

at 2023-11-12T16:45:01+00:00

screenshots of oc_jobs table:
Naamloos
image

Expected Behavior

I expect the background jobs to classify when cron runs.

To Reproduce

  1. run recognize:classify to initially classify all photos and videos
  2. add pictures or videos to nextcloud
  3. queue increases but never gets processed

Debug log

There are no errors that lead me to believe anything wrong is happening. The only related lines in the logs are the following:

[cron] Debug: Skipping OCA\Recognize\BackgroundJobs\ClassifyLandmarksJob job with ID 4901 because another job with the same class is already running

at 2023-11-12T16:40:02+00:00

[cron] Debug: Skipping OCA\Recognize\BackgroundJobs\ClassifyFacesJob job with ID 4684 because another job with the same class is already running

at 2023-11-12T16:40:02+00:00

[cron] Debug: Skipping OCA\Recognize\BackgroundJobs\ClassifyImagenetJob job with ID 4683 because another job with the same class is already running

at 2023-11-12T16:40:02+00:00

[cron] Debug: Skipping OCA\Recognize\BackgroundJobs\ClassifyLandmarksJob job with ID 4678 because another job with the same class is already running

at 2023-11-12T16:40:02+00:00

@Fosnine Fosnine added the bug Something isn't working label Nov 12, 2023
Copy link

Hello 👋

Thank you for taking the time to open this issue with recognize. I know it's frustrating when software
causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at
and if possible solved.
I try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it.
Until then, please be patient.
Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation
to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can
collaborate to make this software better. For everyone.
Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge
and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and
try to fix the odd bug yourself. Everyone will be thankful for extra helping hands!
One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum,
to twitter or somewhere else. But this is a technical issue tracker, so please make sure to
focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue
Cheers 💙

@marcelklehr
Copy link
Member

Hello @Fosnine

This is a known issue and should be fixed with the upcoming server release

@marianionut
Copy link

marianionut commented Feb 11, 2024

I have the same issue with Nextcloud version 27.1.6 and Recognize version 5.0.3, a lot of pictures in my case 133228.

So the issue is still there. It was not fixed in 27.1.4.

Nextcloud is installed with snap in a VM with 16 GB RAM, OS Ubuntu, kernel, Linux 5.11.0-49-generic x86_64

CPU: Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz (8 cores)

The pictures are mounted from a second internal drive NVMe 4 TB with NTFS and external drive plugin.

nextcloud@nextcloud:~$ cat /etc/fstab | grep backup
/dev/sdb2 /media/backup_poze ntfs-3g defaults 0 0
nextcloud@nextcloud:~$ df -h | grep /media
/dev/sdb2                          3.7T  1.2T  2.6T  32% /media/backup_poze

I already reset faces ones and re-run the classify job, but it takes a few days to proces all my pictures which are added in the queue. The issue is the background jobs don't pick up the new faces.
If I manually run the command occ recognize:cluster-faces =>

Clustering face detections for user x
ClusterDebug: Retrieving face detections for user x
ClusterDebug: Not enough face detections found
Clustering face detections for user y
ClusterDebug: Retrieving face detections for user y
ClusterDebug: Not enough face detections found
root@nextcloud:/home/nextcloud#

It started fine for a few hours and after that just dropped the queue.

Face recognition: 115040 Queued files, Last classification: 17 hours ago, Scheduled background jobs: 1, Last background job execution: 12 minutes ago
Face clustering: 0 faces left to cluster, Last clustering run: 16 hours ago, Scheduled background jobs: 0,
A minimum of 120 faces per user is necessary for clustering to kick in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

3 participants