Stopped checking photos #652

k0stya63 · 2023-01-23T16:51:34Z

Which version of recognize are you using?

3.3.6

Enabled Modes

Face recognition

TensorFlow mode

Normal mode

Which Nextcloud version do you have installed?

25.0.3.2

Which Operating system do you have installed?

Ubuntu server 22.04

Which Docker container are you using to run Nextcloud? (if applicable)

How much RAM does your server have?

4

What processor Architecture does your CPU have?

x86_64

Describe the Bug

Stopped processing photos

Expected Behavior

I expected the process to go on as from the beginning

To Reproduce

Didn't seem to do anything out of the ordinary, just stopped checking, waiting for some kind of line. Tried running it through occ recognize:recrawl but nothing happened

Debug log

<?php $CONFIG = array ( 'instanceid' => '*secret*', 'passwordsalt' => '*secret*', 'secret' => '*secret*', 'trusted_domains' => array ( 0 => '192.168.0.201', ), 'datadirectory' => '/nextcloud/data/', 'dbtype' => 'mysql', 'version' => '25.0.3.2', 'overwrite.cli.url' => 'http://192.168.0.201', 'dbname' => 'nextcloud', 'dbhost' => 'localhost', 'dbport' => '', 'dbtableprefix' => 'oc_', 'mysql.utf8mb4' => true, 'dbuser' => 'nextclouduser', 'dbpassword' => '*secret*', 'installed' => true, 'app_install_overwrite' => array ( 0 => 'occweb', ), 'maintenance' => false, );

The text was updated successfully, but these errors were encountered:

ER-EPR · 2023-01-26T15:11:50Z

my face recognition stopped too, but object recognition still working. It happened after a memory exceeding 512M error, probably caused by lacking APCu enable in php.ini I'm not sure. I tried disabling classification and re-enable it. Last time I disable the whole plugin and re-enable it seems to work.

jcv8-dev · 2023-01-27T16:32:22Z

I had the same problem, but running occ recognize:classify after reinstalling the app fixed it for me.

MB-Finski · 2023-01-30T19:43:52Z

I've narrowed this issue down to Classifier.php (at least for me) . Seemingly if there's a failure within the Classifier.php at certain points, the face classifier que will not progress anymore. Specifically I had issues with getImageSize() (line 93) and getImagePreview() (line 241) causing uncaught errors.

Still, I think such errors shouldn't permanently halt the que which, I think, is the main issue at hand here.

shalak · 2023-02-02T02:25:13Z

@ER-EPR / @m1n3nfux does re-enabling / reinstalling makes the process start from scratch, or does it pick-up where it left off?

ER-EPR · 2023-02-02T04:12:15Z

If you do not delete the classify cron jobs in the database, it will star where it stoped. But, if the file which caused the process to stop still exist in the jobs, probably it will stop again. You need to resolve the errors in the log first.

…

-------- 原始邮件 -------- 发件人： Mirek ***@***.***> 日期： 2023年2月2日周四 10:25 收件人： nextcloud/recognize ***@***.***> 抄送： ER-EPR ***@***.***>, Mention ***@***.***> 主题： Re: [nextcloud/recognize] Stopped checking photos (Issue #652) @ER-EPR<https://github.com/ER-EPR> / @m1n3nfux<https://github.com/m1n3nfux> does re-enabling / reinstalling makes the process start from scratch, or does it pick-up where it left off? — Reply to this email directly, view it on GitHub<#652 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJH4OEO6RSNX7GQAQQ45YB3WVMLJLANCNFSM6AAAAAAUECIP5M>. You are receiving this because you were mentioned.Message ID: ***@***.***>

shalak · 2023-02-02T10:15:57Z

You need to resolve the errors in the log first.

How can I access the recognize logs? I don't see anything from it in the global Logging menu...

marcelklehr · 2023-02-02T12:48:44Z

the face classifier que will not progress anymore

After a cron job raises an uncaught exception, Nextcloud will pause it for 12h, but it should restart itself automatically after that. I'll try to catch those errors.

jcv8-dev · 2023-02-02T13:38:04Z

@ER-EPR / @m1n3nfux does re-enabling / reinstalling makes the process start from scratch, or does it pick-up where it left off?

I remember it reprocessing every file. In the settings panel it always said "Last classification: x seconds ago" for Object recognition, but it did not add any tags for like a day. It seems like it does not check for the "Tagged by recognize v3.0.0" Tag. Though the behaviour can have other reasons too, i'm just guessing.

marcelklehr · 2023-02-02T13:39:15Z

I remember it reprocessing every file.

the classify and recrawl commands are rather dumb, indeed.

MB-Finski · 2023-02-02T15:12:42Z

I'll try to catch those errors.

FWIW, filtering out 0 byte files helped some but, still, corrupt files will cause an uncaught exception. My naïve attempt at catching with "\Exception" didn't help.

marcelklehr · 2023-02-02T17:44:55Z

@MB-Finski What's the error stack?

MB-Finski · 2023-02-02T19:16:05Z

[PHP] Error: getimagesize(): Corrupt JPEG data: 1388 extraneous bytes before marker at /var/www/nextcloud/lib/private/legacy/OC_Image.php#598 at <<closure>>
 
 0. <<closure>>
    OC\Log\ErrorHandler::onError()
 1. /var/www/nextcloud/lib/private/legacy/OC_Image.php line 598
    getimagesize()
 2. /var/www/nextcloud/lib/private/legacy/OC_Image.php line 665
    OC_Image->checkImageSize()
 3. /var/www/nextcloud/lib/private/Preview/Image.php line 52
    OC_Image->loadFromFile()
 4. /var/www/nextcloud/lib/private/Preview/GeneratorHelper.php line 65
    OC\Preview\Image->getThumbnail()
 5. /var/www/nextcloud/lib/private/Preview/Generator.php line 343
    OC\Preview\GeneratorHelper->getThumbnail()
 6. /var/www/nextcloud/lib/private/Preview/Generator.php line 162
    OC\Preview\Generator->getMaxPreview()
 7. /var/www/nextcloud/lib/private/Preview/Generator.php line 114
    OC\Preview\Generator->generatePreviews()
 8. /var/www/nextcloud/lib/private/PreviewManager.php line 185
    OC\Preview\Generator->getPreview()
 9. /var/www/nextcloud/apps/recognize/lib/Classifiers/Classifier.php line 246
    OC\PreviewManager->getPreview()
10. /var/www/nextcloud/apps/recognize/lib/Classifiers/Classifier.php line 78
    OCA\Recognize\Classifiers\Classifier->getConvertedFilePath()
11. /var/www/nextcloud/apps/recognize/lib/Classifiers/Images/ClusteringFaceClassifier.php line 83
    OCA\Recognize\Classifiers\Classifier->classifyFiles()
12. /var/www/nextcloud/apps/recognize/lib/BackgroundJobs/ClassifyFacesJob.php line 41
    OCA\Recognize\Classifiers\Images\ClusteringFaceClassifier->classify()
13. /var/www/nextcloud/apps/recognize/lib/BackgroundJobs/ClassifierJob.php line 70
    OCA\Recognize\BackgroundJobs\ClassifyFacesJob->classify()
14. /var/www/nextcloud/apps/recognize/lib/BackgroundJobs/ClassifyFacesJob.php line 33
    OCA\Recognize\BackgroundJobs\ClassifierJob->runClassifier()
15. /var/www/nextcloud/lib/public/BackgroundJob/Job.php line 78
    OCA\Recognize\BackgroundJobs\ClassifyFacesJob->run()
16. /var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php line 103
    OCP\BackgroundJob\Job->start()
17. /var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php line 93
    OCP\BackgroundJob\TimedJob->start()
18. /var/www/nextcloud/cron.php line 152
    OCP\BackgroundJob\TimedJob->execute()

at 

2023-02-02T03:30:02+00:00

This is the one for a corrupt JPEG. I'm using NextcloudPi just because they have a ready-made script for setting up an LXC. It's possible that their bundled version of preview generator plays a role in my case..?

marcelklehr · 2023-03-19T16:48:32Z

Closing this for now as the surface issue was fixed. If this comes up again, let me know.

k0stya63 added the bug Something isn't working label Jan 23, 2023

marcelklehr mentioned this issue Feb 2, 2023

Face detection threshold too high #588

Closed

marcelklehr mentioned this issue Feb 9, 2023

fix OC_Image: Prevent E_WARNING from getimagesize* nextcloud/server#36636

Merged

4 tasks

marcelklehr closed this as completed Mar 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stopped checking photos #652

Stopped checking photos #652

k0stya63 commented Jan 23, 2023

ER-EPR commented Jan 26, 2023 •

edited

Loading

jcv8-dev commented Jan 27, 2023

MB-Finski commented Jan 30, 2023 •

edited

Loading

shalak commented Feb 2, 2023

ER-EPR commented Feb 2, 2023 via email

shalak commented Feb 2, 2023 •

edited

Loading

marcelklehr commented Feb 2, 2023

jcv8-dev commented Feb 2, 2023

marcelklehr commented Feb 2, 2023 •

edited

Loading

MB-Finski commented Feb 2, 2023 •

edited

Loading

marcelklehr commented Feb 2, 2023 •

edited

Loading

MB-Finski commented Feb 2, 2023 •

edited

Loading

marcelklehr commented Mar 19, 2023

Stopped checking photos #652

Stopped checking photos #652

Comments

k0stya63 commented Jan 23, 2023

Which version of recognize are you using?

Enabled Modes

TensorFlow mode

Which Nextcloud version do you have installed?

Which Operating system do you have installed?

Which Docker container are you using to run Nextcloud? (if applicable)

How much RAM does your server have?

What processor Architecture does your CPU have?

Describe the Bug

Expected Behavior

To Reproduce

Debug log

ER-EPR commented Jan 26, 2023 • edited Loading

jcv8-dev commented Jan 27, 2023

MB-Finski commented Jan 30, 2023 • edited Loading

shalak commented Feb 2, 2023

ER-EPR commented Feb 2, 2023 via email

shalak commented Feb 2, 2023 • edited Loading

marcelklehr commented Feb 2, 2023

jcv8-dev commented Feb 2, 2023

marcelklehr commented Feb 2, 2023 • edited Loading

MB-Finski commented Feb 2, 2023 • edited Loading

marcelklehr commented Feb 2, 2023 • edited Loading

MB-Finski commented Feb 2, 2023 • edited Loading

marcelklehr commented Mar 19, 2023

ER-EPR commented Jan 26, 2023 •

edited

Loading

MB-Finski commented Jan 30, 2023 •

edited

Loading

shalak commented Feb 2, 2023 •

edited

Loading

marcelklehr commented Feb 2, 2023 •

edited

Loading

MB-Finski commented Feb 2, 2023 •

edited

Loading

marcelklehr commented Feb 2, 2023 •

edited

Loading

MB-Finski commented Feb 2, 2023 •

edited

Loading