Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: FIX for WebDav MacOS failed uploads with php-fpm and big files (-36 error) #48878

Closed
5 of 8 tasks
gonzalo opened this issue Oct 24, 2024 · 8 comments
Closed
5 of 8 tasks
Labels
1. to develop Accepted and waiting to be taken care of 28-feedback bug feature: dav hotspot: file transfer performance upload & download performance related optimizations

Comments

@gonzalo
Copy link
Contributor

gonzalo commented Oct 24, 2024

⚠️ This issue respects the following points: ⚠️

Bug description

There are know problems with MacOS+WebDav+Apache+php-fpm https://docs.nextcloud.com/server/latest/user_manual/en/files/access_webdav.html#accessing-files-using-macos

If you use MacOS finder client to connect via WebDavto your nextcloud instance and php-fpm running, you will soon find that transfer crashes, giving you a -36 error. This comes from a problem with Content-Length sent to backend.

For dealing with this .htaccess file has this code

# Clients like xDavv5 on Android, or Cyberduck, use chunked requests.
# When FastCGI or FPM is used with apache, requests arrive to Nextcloud without any content.
# This leads to the creation of empty files.
# The following directive will force the problematic requests to be buffered before being forwarded to Nextcloud.
# This way, the "Transfer-Encoding" header is removed, the "Content-Length" header is set, and the request content is proxied to Nextcloud.
# Here are more information about the issue:
#  - https://docs.cyberduck.io/mountainduck/issues/fastcgi/
#  - https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#troubleshooting-webdav
<IfModule mod_setenvif.c>
  SetEnvIf Transfer-Encoding "chunked" proxy-sendcl=1
</IfModule>

However this doesn't work with Macos Webdav Darwin client beacuse the header has a capital letter -> "Chunked" (checked by logs)
So you can fix it making condition case insensitive

<IfModule mod_setenvif.c>
  SetEnvIf Transfer-Encoding "(?i)chunked" proxy-sendcl=1
</IfModule>

UPDATE before PR: finally, I used an alternative method using SetEnvIfNoCase that has same effect without using regexp

<IfModule mod_setenvif.c>
  SetEnvIfNoCase Transfer-Encoding "chunked" proxy-sendcl=1
</IfModule>

NOTE: apache will upload the file to their tmp folder before passing it to php. Take care of this in case you have restrictions on that folder.

Steps to reproduce

  1. Create a webdav connection with finder. The backend with apache and php-fpm
  2. Try to upload a file of a few MB (will fail)
  3. Modify the .htaccess as I mentioned
  4. Try again (will success)

Expected behavior

To be able to upload big files to nextclcoud via webdav in macos

Nextcloud Server version

28

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

not relevant

List of activated Apps

not relevant

Nextcloud Signing status

not relevant

Nextcloud Logs

not relevant

Additional info

php8.2-fpm

@gonzalo gonzalo added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Oct 24, 2024
@solracsf
Copy link
Member

Cc @szaimen

@szaimen
Copy link
Contributor

szaimen commented Oct 24, 2024

cc @artonge

@joshtrichards
Copy link
Member

joshtrichards commented Oct 24, 2024

Makes sense. Good catch! Up to submitting as a PR? I wonder if there's any real difference between adjusting the regexp versus using SetEnvIfNoCase?

@gonzalo
Copy link
Contributor Author

gonzalo commented Oct 24, 2024

Interesting never heard about it but it seems exactly for that

The SetEnvIfNoCase is semantically identical to the SetEnvIf directive, and differs only in that the regular expression matching is performed in a case-insensitive manner. For example:
    SetEnvIfNoCase Host Example\.Org site=example
This will cause the site environment variable to be set to "example" if the HTTP request header field Host: was included and contained Example.Org, example.org, or any other combination.

Will try it tomorrow morning and confirm it to you. Cleaner is better!

@gonzalo
Copy link
Contributor Author

gonzalo commented Oct 25, 2024

Just tested on my test and prd environments SetEnvIfNoCase does the same stuff, so I suggest to use in .htaccess (currently in my vhost file, to avoid been overridden by an update).

<IfModule mod_setenvif.c>
  SetEnvIfNoCase Transfer-Encoding "chunked" proxy-sendcl=1
</IfModule>

@szaimen
Copy link
Contributor

szaimen commented Nov 22, 2024

Can you submit a PR please?

@joshtrichards joshtrichards added hotspot: file transfer performance upload & download performance related optimizations 1. to develop Accepted and waiting to be taken care of and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Nov 22, 2024
gonzalo added a commit to gonzalo/nextcloud-server that referenced this issue Nov 27, 2024
Current "SetEnvIf Transfer-Encoding "chunked" proxy-sendcl=1" setting fails with Macos Webdav Darwin client beacuse the header has a capital letter -> "Chunked" So you can fix it making condition case insensitive.
Extended description and tests results on nextcloud#48878

Signed-off-by: Gonzalo Cao Cabeza de Vaca <57393+gonzalo@users.noreply.github.com>
@gonzalo
Copy link
Contributor Author

gonzalo commented Nov 27, 2024

PR already submitted 🤞

@gonzalo
Copy link
Contributor Author

gonzalo commented Nov 29, 2024

ups I didnt' submitted it properly. Just did it again #49557

backportbot bot pushed a commit that referenced this issue Nov 29, 2024
Current "SetEnvIf Transfer-Encoding "chunked" proxy-sendcl=1" setting fails with Macos Webdav Darwin client beacuse the header has a capital letter -> "Chunked" So you can fix it making condition case insensitive.
Extended description and tests results on #48878

Signed-off-by: Gonzalo Cao Cabeza de Vaca <57393+gonzalo@users.noreply.github.com>
@szaimen szaimen closed this as completed Nov 29, 2024
szaimen pushed a commit that referenced this issue Dec 5, 2024
Current "SetEnvIf Transfer-Encoding "chunked" proxy-sendcl=1" setting fails with Macos Webdav Darwin client beacuse the header has a capital letter -> "Chunked" So you can fix it making condition case insensitive.
Extended description and tests results on #48878

Signed-off-by: Gonzalo Cao Cabeza de Vaca <57393+gonzalo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. to develop Accepted and waiting to be taken care of 28-feedback bug feature: dav hotspot: file transfer performance upload & download performance related optimizations
Projects
None yet
Development

No branches or pull requests

4 participants