Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: renaming file by just changing case leads to strange results (on SMB storage and others) #40690

Open
5 of 8 tasks
alexanderharm opened this issue Sep 28, 2023 · 6 comments
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug feature: external storage feature: filesystem hotspot: filename handling Filenames - invalid, portable, blacklisting, etc.

Comments

@alexanderharm
Copy link

⚠️ This issue respects the following points: ⚠️

Bug description

Renaming an object (here folder/directory) on external storage connected via SMB leads to strange results depending on the client.

WebGUI:

  • renaming 99_test to 99_TEST produces an error message saying that the name is already in use and that I should use a different name
  • HOWEVER: in the GUI one can see now both folders 99_test AND 99_TEST
  • if I create an object in 99_test it also shows in 99_TEST and vice versa
  • if I delete either of the folders then both are either deleted or one is deleted and the other dysfunctional (cannot create any files or folders inside it)
  • on the connected Nextcloud client on Windows the user can just see the original 99_test

Nextcloud client (Windows) with virtual files:

  • renaming 99_test to 99_TEST produces an error message saying that it failed caused by a protocol error
  • client sees folder as 99_test, sync status remains active
  • creating any objects in 99_test fails in client and WebGUI, folder is dysfunctional

Steps to reproduce

  1. create folder on external storage connected via SMB
  2. rename the folder by just changing the case of a letter

Expected behavior

renaming by just changing case is detected by WebGUI and Nextcloud client as illegal on SMB storage and no file operations are executed.

Installation method

Other Community project

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.satzkiste.de"
        ],
        "dbtype": "pgsql",
        "version": "27.1.1.0",
        "overwrite.cli.url": "https:\/\/cloud.satzkiste.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 1
        },
        "default_language": "de",
        "default_locale": "de_DE",
        "skeletondirectory": "",
        "default_phone_region": "de",
        "maintenance": false,
        "loglevel": 3,
        "enable_previews": false,
        "updater.release.channel": "stable",
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "twofactor_enforced": "false",
        "twofactor_enforced_groups": [
            "admin",
            "StoryEditor"
        ],
        "twofactor_enforced_excluded_groups": [],
        "app_install_overwrite": [
            "gpxmotion",
            "googledocsredirect",
            "admin_notifications",
            "dropit",
            "file_upload_notification",
            "flowupload",
            "integration_jira",
            "cfg_share_links",
            "apporder",
            "facerecognition",
            "folderplayer"
        ],
        "memories.exiftool": "\/config\/www\/nextcloud\/apps\/memories\/exiftool-bin\/exiftool-amd64-musl",
        "memories.vod.path": "\/config\/www\/nextcloud\/apps\/memories\/exiftool-bin\/go-vod-amd64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - announcementcenter: 6.6.2
  - apporder: 0.15.0
  - bruteforcesettings: 2.7.0
  - cfg_share_links: 4.1.0
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.11.0
  - external: 5.2.1
  - externalportal: 1.2.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - groupfolders: 15.3.1
  - impersonate: 1.14.0
  - integration_google: 2.0.2
  - integration_openai: 1.0.13
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - memories: 5.4.1
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.3
  - oauth2: 1.15.1
  - onlyoffice: 8.2.0
  - password_policy: 1.17.0
  - polls: 5.3.2
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - quota_warning: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - socialsharing_email: 2.6.0
  - spreed: 17.1.1
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - unsplash: 2.2.1
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - welcome: 1.0.10
  - workflowengine: 2.9.0
Disabled:
  - appointments: 1.15.3 (installed 1.15.3)
  - calendar: 4.5.0 (installed 4.5.0)
  - calendar_resource_management: 0.5.0 (installed 0.5.0)
  - circles: 27.0.1 (installed 26.0.0)
  - contacts: 5.4.0 (installed 5.4.0)
  - contactsinteraction: 1.8.0 (installed 1.6.0)
  - encryption: 2.15.0
  - facerecognition: 0.9.31 (installed 0.9.31)
  - file_upload_notification: 0.1.2 (installed 0.1.2)
  - files_downloadlimit: 1.1.0 (installed 1.1.0)
  - firstrunwizard: 2.16.0 (installed 2.15.0)
  - flowupload: 1.1.3 (installed 1.1.3)
  - googledocsredirect: 1.0.2 (installed 1.0.2)
  - holiday_calendars: 0.3.0 (installed 0.3.0)
  - imageconverter: 1.3.5 (installed 1.3.5)
  - mailnotifier: 0.0.3 (installed 0.0.3)
  - maps: 1.1.1 (installed 1.1.1)
  - photos: 2.3.0 (installed 2.2.0)
  - ransomware_protection: 1.14.0 (installed 1.14.0)
  - recognize: 4.3.2 (installed 4.3.2)
  - richdocuments: 8.2.0 (installed 8.2.0)
  - snowflakestheme: 1.1.0 (installed 1.1.0)
  - unroundedcorners: 1.1.1 (installed 1.1.1)
  - user_ldap: 1.17.0
  - user_saml: 5.2.2 (installed 5.2.2)
  - workflow_media_converter: 1.8.4 (installed 1.8.4)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"cn2Eniltym4LJ2uF0lYM","level":3,"time":"2023-09-28T19:08:51+00:00","remoteAddr":"142.132.133.228","user":"Maxi","app":"webdav","method":"MOVE","url":"/remote.php/dav/files/Maxi/EKS/99_test","message":"Failed to rename /EKS/99_test to /EKS/99_TEST","userAgent":"Mozilla/5.0 (Windows) mirall/3.10.0stable-Win64 (build 20230915) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"27.1.1.0","exception":{"Exception":"Sabre\\DAV\\Exception","Message":"Failed to rename /EKS/99_test to /EKS/99_TEST","Code":0,"Trace":[{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Tree.php","line":153,"function":"setName","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":612,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/app/www/public/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/config/www/nextcloud/apps/dav/lib/Server.php","line":365,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/config/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/app/www/public/remote.php","line":172,"args":["/config/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/config/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","Line":160,"message":"Failed to rename /EKS/99_test to /EKS/99_TEST","exception":[],"CustomMessage":"Failed to rename /EKS/99_test to /EKS/99_TEST"},"id":"6515d19e0186b"}

{"reqId":"nC8yKmehn3h4W7i6skb6","level":3,"time":"2023-09-28T18:46:04+00:00","remoteAddr":"87.138.252.179","user":"ncadmin","app":"no app in context","method":"POST","url":"/ocs/v2.php/apps/files/api/v1/templates/create","message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/115.0","version":"27.1.1.0","exception":{"Exception":"OCP\\Files\\NotPermittedException","Message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","Code":0,"Trace":[{"file":"/app/www/public/lib/private/Files/Template/TemplateManager.php","line":162,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->"},{"file":"/config/www/nextcloud/apps/files/lib/Controller/TemplateController.php","line":57,"function":"createFromTemplate","class":"OC\\Files\\Template\\TemplateManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/app/www/public/lib/private/AppFramework/Http/Dispatcher.php","line":230,"function":"create","class":"OCA\\Files\\Controller\\TemplateController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/app/www/public/lib/private/AppFramework/Http/Dispatcher.php","line":137,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/app/www/public/lib/private/AppFramework/App.php","line":183,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/app/www/public/lib/private/Route/Router.php","line":315,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/app/www/public/ocs/v1.php","line":64,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/app/www/public/ocs/v2.php","line":23,"args":["/app/www/public/ocs/v1.php"],"function":"require_once"}],"File":"/app/www/public/lib/private/Files/Node/Folder.php","Line":193,"message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","exception":[],"CustomMessage":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\""},"id":"6515d19e020df"}

Additional info

No response

@alexanderharm alexanderharm added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Sep 28, 2023
@joshtrichards
Copy link
Member

joshtrichards commented Sep 28, 2023

This may or may not be NC issue, but you'll need to dig a bit deeper into your SMB server's configuration and SMB implementation to help get to the bottom of the matter (your SMB server's underlying OS and OS version are a factor too).

It sounds like a bit like case sensitivity and/or case preservation is disabled on your SMB server (or any number of related options that depend on what your specific SMB server implementation supports).

Please check the manual for your SMB server to determine what it supports in terms of case preservation and sensitivity. You may just need to adjust some options. If it's really old (or oddly configured) there may even be some name mangling involved.

@alexanderharm
Copy link
Author

Thanks for your feedback. The server runs Samba (Version 4.17.8-Debian) and is on default settings which according to Samba are the same for Windows Servers, meaning case insensitive but case preserving (https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html) which would explain the above behaviour (but I would still suspect Nextcloud to handle the default config of SMB servers gracefully and not to end up with dysfunctional folders).

I can set case sensitive = yes for testing (only tonight) and I can also check if the dysfunctional folders in Nextcloud are actually dysfunctional on the SMB-server as well.

@Rello
Copy link
Contributor

Rello commented Sep 29, 2023

I can reproduce this error when using NC in docker and mounting the data volume

    volumes:
      - ./config:/var/www/html/config
      - ./data:/var/www/html/data  

from nextcloud.log
The destination node already exists, and the overwrite header is set to false

@Rello Rello changed the title [Bug]: EXTERNAL STORAGE: renaming object by just changing case on SMB storage leads to strange results [Bug]: renaming file by just changing case leads to strange results (on SMB storage and others) Sep 29, 2023
@sorbaugh
Copy link
Contributor

sorbaugh commented Oct 2, 2023

cc @icewind1991 , @artonge

@alexanderharm
Copy link
Author

I finally found some time to test further:

1. Test with Samba default settings

case sensitive = Auto
  • According to Samba manpage

The default setting of auto allows clients that support case sensitive filenames (Linux CIFSVFS and smbclient 3.0.5 and above currently) to tell the Samba server on a per-packet basis that they wish to access the file system in a case-sensitive manner (to support UNIX case sensitive semantics). No Windows or DOS system supports case-sensitive filename so setting this option to auto is that same as setting it to no for them.

  • Theoretically my server since based on Linux and using libsmbclient through a PHP-wrapper COULD use case sensitivity but doesn't
  • Renaming by changing case leads to above described issues

My test steps:

WebGUI

  1. WebGUI: renamed folder from 99_test to 99_TEST
  2. WebGUI: two folders appear namely 99_test and 99_TEST
  3. WebGUI: files can be created in both and their content is always identical in the two folders
  4. Nextcloud Client: only shows 99_test
  5. SMB-mount: only shows 99_test
  6. WebGUI: deleting 99_TEST leaves only 99_test but dysfunctional
  7. Nextcloud Client: 99_test is dysfunctional
  8. SMB-mount: 99_test and 99_TEST are both deleted
  9. WebGUI: deleting 99_test generates error but deletes folder
  10. Nextcloud Client: 99_test is deleted as well

Nextcloud Client

  1. Nextcloud Client: renamed folder from 99_test to 99_TEST
  2. Nextcloud Client: error message and 99_TEST shows but is dysfunctional
  3. WebGUI: 99_TEST shows but is dysfunctional
  4. SMB-mount: no folder anymore, it was deleted

SMB-mount

  1. SMB-mount: renamed folder from 99_test to 99_TEST
  2. Nextcloud Client: shows 99_TEST
  3. WebGUI: 99_TEST shows

2. Test with case sensitivity

case sensitive = yes
  • Seems to work flawlessly

My test steps:

  1. WebGUI: renamed folder from 99_test to 99_TEST
  2. WebGUI: no error message and just 99_TEST
  3. WebGUI: files can be created in the folder
  4. Nextcloud Client: only shows 99_TEST
  5. SMB-mount: only shows 99_TEST
  6. SMB-mount: renamed folder from 99_TEST to 99_test
  7. WebGUI: shows 99_test
  8. Nextcloud Client: shows 99_test
  9. Nextcloud Client: renaming works as well

3. Preliminary conclusion

  • Renaming folders on external storage by just changing case works in combination with Samba file servers if configured with case sensitive = yes
  • Did not have a Windows Server to test but AFAIK Windows does not support case sensitivity so the above issues might show again
  • I would still consider it a bug that the WebGUI ends up with two folders and if one is deleted the folder on the SMB share is deleted as well
  • The more serious bug probably lies in the Nextcloud client. Because renaming does lead directly to the folder being deleted on the share, the rename should abort before deleting data

@e46lux
Copy link

e46lux commented Apr 19, 2024

I feel like the issue lies with how the share is mounted via libsmbclient. I still haven't figured out how nextcloud specifies options to libsmbclient when mounting the share.

The following options:

vers=3.0,_netdev,credentials=/my_path/.secrets/.smb,iocharset=utf8,gid=100033,uid=100033,file_mode=0777,dir_mode=0777,sec=ntlmv2,cache=none,noserverino

allow linux to do a 'mv t.txt T.txt' without a problem.

Using:
vers=2.0,_netdev,credentials=/my_path/.secrets/.smb,iocharset=utf8,gid=100033,uid=100033,file_mode=0777,dir_mode=0777,sec=ntlmv2

I get this file name already exists error directly in linux terminal.

Some post I crossed when searching for the issue referenced cache being a problem and setting cache=none helped resolve that. Not quite sure if switching to vers=3.0 or the noserverio options have any effect, but came with the suggestion from the post I found online.

Can anyone point me in the right direction where I can try specifying the SMB mount options values for testing?
I found references in files.js and './apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php' but can't quite determine how the libsmbclient has options set for the connection.

@AndyScherzinger AndyScherzinger moved this to 🧭 Planning evaluation (don't pick) in 📁 Files team Apr 22, 2024
@joshtrichards joshtrichards added the hotspot: filename handling Filenames - invalid, portable, blacklisting, etc. label Sep 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug feature: external storage feature: filesystem hotspot: filename handling Filenames - invalid, portable, blacklisting, etc.
Projects
Status: 🧭 Planning evaluation (don't pick)
Development

No branches or pull requests

6 participants