[Bug]: unhandled exception in ExifProvider crashes scan run #35931
Comments
luxifr
added
0. Needs triage
|
The issue is here: |
szaimen
added
25-feedback
2. developing
|
We're also experiencing this. It triggers when uploading via the web ui or via webdav. |
|
The same problem here after update from 24.x to 25.0.4 and 25.0.5 and uploading images from Android device with the app “folder sync”: With version 24.x there was no problems.
Reading line 71: https://github.com/nextcloud/server/blob/873501a141c3ddfc3d8ce4291123a7a2842ece8e/lib/private/Metadata/Provider/ExifProvider.php 1 $file->getContent() is boolean (maybe false) instead of an expected string (path with filename). External storage support 1.17.0 is used and the access is realized by SFTP. In activity log, image is created then deleted. The folder sync app try it again, create and then delete. The maintainer of the Folder Sync app can't fix the problem. He tolds, that's realted to the NextCloud Server. |
|
I believe I'm hitting this (or a very related) bug as well. I've uploaded a bunch of data (few TB) and then wanted to run a scan. So when I've finished all file copying, I ran the command: That went fine across first ~30% of data, then started stalling on files with errors such as: I thought my permissions were bad, then started thinking my file copy was corrupted, and started to delete.. well.. a lot of data sadly (will need whole day tomorrow to copy it back). Then at some point I've noticed it's just too much, and it can't all be corrupted, tried shortening my paths, excluding spaces, and such, then finally tried opening files as Anyway, long story short... Note that it actually fails on the very first file it finds, and if I delete that file manually it will just do the same for next one and next and next. But (!!) if you first do a full scan without metadata, eg.: ... that scan works fine! ... ending with... I can only cry myself to sleep about not finding the workaround sooner, now I "only" have to repeat copying 6 years of video recordings... but hopefully this comment helps finding a bug. Btw, I haven't yet actually checked if metadata is indeed imported with the subsequent scan, but scanning same folder (single year of data) shows that scan without metadata ends with while scan with metadata lasts much longer (and I can visually say it scans slower on JPG files) Edit: |
|
Finally had some more time to go through old log files. I've found three slightly different occurances of this error, as mentioned in OP, all include 1 2 3 Edit: |
|
Still present in Nextcloud Hub 3 (25.0.6) |
This one should be fixed by #37943 |
|
So, I looked into it and apart from the TypeError, this behavior is on purpose. getimagesizefromstring does not throw on errors, it triggers PHP errors, it’s a design choice of the scan command to convert that into an exception. This is why there is not try/catch on the call. That said, I opened a PR to show errors but continue, let’s see what others think of this change. |
|
Problem still present in 26.0.2 and 26.0.3rc2 |
|
Can confirm the issue persists with 26.0.2 |
This is not the same problem, it is not about files:scan stopping at first error. @Ramblurr The fix for this is in #37944 which is included in 26.0.2, so I doubt that you see the same behavior described here on this version? |
|
Have applied as workaround (until v27 is available) this patch from master and it's working now. [https://github.com/nextcloud/server/commit/997efe44ffd56959f198be2e3aa71687a2f11412](url)
|
Bug description
When doing a full file scan incl metadata generation, any file that trips the
getimagesizefromstring(...)call in theExifProviderwill make it throw an exception that is handled so late that it crashes the whole scan run.getimagesizefromstring(string $string, array &$image_info = null): array|falsebehaves likegetimagesize(string $filename, array &$image_info = null): array|falsein that it will throw Errors under certain circumstances. However, the call to this function isn't wrapped in a try/catch (see here and here)Steps to reproduce
occ files:scan --generate-metadataagainst, for example, a 0 byte file (or a file that has a header that looks malformed to that function.Expected behavior
Don't crash the whole scan run - ignore the file, maybe log a warning, and move on to the next one.
Installation method
Community Docker image
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.1
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
Configuration report
{ "system": { "memcache.local": "\\OC\\Memcache\\APCu", "allow_local_remote_servers": true, "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "loglevel": 0, "trusted_domains": [ "localhost", "***REMOVED SENSITIVE VALUE***" ], "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "overwriteprotocol": "https", "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "25.0.2.3", "overwrite.cli.url": "http:\/\/localhost", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "filesystem_check_changes": 1, "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "instanceid": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpsecure": "tls", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpauthtype": "LOGIN", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "allow_user_to_change_display_name": false, "lost_password_link": "disabled", "oidc_login_provider_url": "***REMOVED SENSITIVE VALUE***", "oidc_login_client_id": "***REMOVED SENSITIVE VALUE***", "oidc_login_client_secret": "***REMOVED SENSITIVE VALUE***", "oidc_login_auto_redirect": false, "oidc_login_end_session_redirect": false, "oidc_login_button_text": "***REMOVED SENSITIVE VALUE***", "oidc_login_hide_password_form": false, "oidc_login_use_id_token": true, "oidc_login_attributes": { "id": "preferred_username", "name": "name", "mail": "email", "groups": "groups" }, "oidc_login_default_group": "oidc", "oidc_login_use_external_storage": false, "oidc_login_scope": "openid profile email groups", "oidc_login_proxy_ldap": false, "oidc_login_disable_registration": true, "oidc_login_redir_fallback": false, "oidc_login_alt_login_page": "assets\/login.php", "oidc_login_tls_verify": true, "oidc_create_groups": false, "oidc_login_webdav_enabled": false, "oidc_login_password_authentication": false, "oidc_login_public_key_caching_time": 86400, "oidc_login_min_time_between_jwks_requests": 10, "oidc_login_well_known_caching_time": 86400, "oidc_login_update_avatar": false, "app_install_overwrite": [ "breezedark" ], "memories.exiftool_no_local": true, "enable_previews": true, "enabledPreviewProviders": [ "OC\\Preview\\TXT", "OC\\Preview\\MarkDown", "OC\\Preview\\OpenDocument", "OC\\Preview\\PDF", "OC\\Preview\\MSOffice2003", "OC\\Preview\\MSOfficeDoc", "OC\\Preview\\Image", "OC\\Preview\\HEIC", "OC\\Preview\\TIFF", "OC\\Preview\\SVG", "OC\\Preview\\Font", "OC\\Preview\\MP3", "OC\\Preview\\Movie", "OC\\Preview\\MKV", "OC\\Preview\\MP4", "OC\\Preview\\AVI" ], "memories.ffmpeg_path": "\/usr\/bin\/ffmpeg", "memories.ffprobe_path": "\/usr\/bin\/ffprobe", "memories.transcoder": "\/var\/www\/html\/apps\/memories\/exiftool-bin\/go-vod-amd64", "memories.no_transcode": true, "memories.qsv": false, "maintenance": false } }List of activated Apps
Nextcloud Signing status
Nextcloud Logs
No response
Additional info
No response
The text was updated successfully, but these errors were encountered: