S3 external storage: Cannot rename >5 GB files #24540

tlinna · 2020-12-04T01:07:18Z

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

Upload a large file (e.g. 10 GB) to Amazon S3 external storage folder
Try to rename the file

Expected behaviour

File should be renamed.

Actual behaviour

An error occurs and file is not renamed. Two error messages are logged (see below for details), the more important being:

Aws\S3\Exception\S3Exception: Error executing "CopyObject" [...] resulted in a 400 Bad Request response [...] InvalidRequest (client): The specified copy source is larger than the maximum allowable size for a copy source: 5368709120 [...]

Server configuration

Operating system: Ubuntu Server 20.04.1 LTS

Web server: Nginx 1.18.0

Database: PostgreSQL 12.5

PHP version: 7.4.3

Nextcloud version: 19.0.5

Updated from an older Nextcloud/ownCloud or fresh install: Originally installed at version 14 or 15

Where did you install Nextcloud from: Downloaded from nextcloud.com

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - accessibility: 1.5.0
  - activity: 2.12.1
  - announcementcenter: 3.8.1
  - apporder: 0.11.0
  - bruteforcesettings: 2.0.1
  - calendar: 2.0.4
  - cloud_federation_api: 1.2.0
  - comments: 1.9.0
  - contacts: 3.4.2
  - contactsinteraction: 1.0.0
  - dav: 1.15.0
  - deck: 1.2.2
  - drawio: 0.9.8
  - federatedfilesharing: 1.9.0
  - federation: 1.9.0
  - files: 1.14.0
  - files_accesscontrol: 1.9.2
  - files_automatedtagging: 1.9.0
  - files_external: 1.10.0
  - files_pdfviewer: 1.8.0
  - files_readmemd: 1.2.0
  - files_rightclick: 0.16.0
  - files_sharing: 1.11.0
  - files_trashbin: 1.9.0
  - files_versions: 1.12.0
  - files_videoplayer: 1.8.0
  - firstrunwizard: 2.8.0
  - flowupload: 1.1.2
  - forms: 2.0.4
  - groupfolders: 7.1.1
  - logreader: 2.4.0
  - lookup_server_connector: 1.7.0
  - nextcloud_announcements: 1.8.0
  - notes: 3.6.4
  - notifications: 2.7.0
  - oauth2: 1.7.0
  - onlyoffice: 6.1.0
  - password_policy: 1.9.1
  - passwords: 2020.11.1
  - photos: 1.1.0
  - polls: 1.6.2
  - previewgenerator: 2.3.0
  - privacy: 1.3.0
  - provisioning_api: 1.9.0
  - serverinfo: 1.9.0
  - settings: 1.1.0
  - sharebymail: 1.9.0
  - spreed: 9.0.6
  - support: 1.2.1
  - survey_client: 1.7.0
  - systemtags: 1.9.0
  - tasks: 0.13.6
  - text: 3.0.1
  - theming: 1.10.0
  - twofactor_backupcodes: 1.8.0
  - twofactor_nextcloud_notification: 2.3.0
  - twofactor_totp: 5.0.0
  - updatenotification: 1.9.0
  - video_converter: 0.1.5
  - viewer: 1.3.0
  - workflow_script: 1.4.0
  - workflowengine: 2.1.0
Disabled:
  - admin_audit
  - analytics
  - encryption
  - files_lock
  - recommendations
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***removed domain***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "tempdirectory": "\/data\/tmp",
        "dbtype": "pgsql",
        "version": "19.0.5.2",
        "overwrite.cli.url": "https:\/\/***removed domain***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "trashbin_retention_obligation": "14, 30",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "PLAIN",
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [
            "admin"
        ],
        "twofactor_enforced_excluded_groups": [],
        "theme": "",
        "loglevel": 2,
        "updater.release.channel": "stable"
    }
}

Are you using external storage, if yes which one: AWS S3 object storage

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Microsoft Edge 87.0.664.52 (Official build) (64-bit), Firefox 83.0

Operating system: Windows 10

Logs

Web server error log

No errors in nginx error log.

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"nzo9hgexstIQFtpUzFOw","level":4,"time":"2020-12-04T00:14:40+00:00","remoteAddr":"2001:14ba:a00c:1100:e52b:89ce:a5c0:e5c4","user":"tlinna","app":"webdav","method":"MOVE","url":"/remote.php/dav/files/tlinna/Kylm%C3%A4varasto%20S3/Videoarkisto/FinRX/2020-11-23%2019-48-39.mkv","message":{"Exception":"Sabre\\DAV\\Exception","Message":"Failed to rename /Kylmävarasto S3/Videoarkisto/FinRX/2020-11-23 19-48-39.mkv to /Kylmävarasto S3/Videoarkisto/FinRX/02 jalkkari.mkv","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":157,"function":"setName","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":641,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","Line":143,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36 Edg/87.0.664.52","version":"19.0.5.2","id":"5fc97fec59a77"}

{"reqId":"nzo9hgexstIQFtpUzFOw","level":3,"time":"2020-12-04T00:14:40+00:00","remoteAddr":"2001:14ba:a00c:1100:e52b:89ce:a5c0:e5c4","user":"tlinna","app":"files_external","method":"MOVE","url":"/remote.php/dav/files/tlinna/Kylm%C3%A4varasto%20S3/Videoarkisto/FinRX/2020-11-23%2019-48-39.mkv","message":{"Exception":"Aws\\S3\\Exception\\S3Exception","Message":"Error executing \"CopyObject\" on \"https://xxxxxxxxx.s3.eu-north-1.amazonaws.com/Videoarkisto/FinRX/02%20jalkkari.mkv\"; AWS HTTP error: Client error: `PUT https://xxxxxxxxx.s3.eu-north-1.amazonaws.com/Videoarkisto/FinRX/02%20jalkkari.mkv` resulted in a `400 Bad Request` response:\n<Error><Code>InvalidRequest</Code><Message>The specified copy source is larger than the maximum allowable size for a cop (truncated...)\n InvalidRequest (client): The specified copy source is larger than the maximum allowable size for a copy source: 5368709120 - <Error><Code>InvalidRequest</Code><Message>The specified copy source is larger than the maximum allowable size for a copy source: 5368709120</Message><RequestId>57D5FE7C7F65006B</RequestId><HostId>tOKbh3xrvJBT9lnDmGRziOKlHd7gnK45B/dMqRAW72E72VNGwDd8YEDhFlEAxGIyJpbaAHtn4RI=</HostId></Error>","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/WrappedHttpHandler.php","line":97,"function":"parseError","class":"Aws\\WrappedHttpHandler","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":203,"function":"Aws\\{closure}","class":"Aws\\WrappedHttpHandler","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":174,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/RejectedPromise.php","line":40,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\RejectedPromise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":118,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":145,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/AwsClientTrait.php","line":58,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/AwsClientTrait.php","line":86,"function":"execute","class":"Aws\\AwsClient","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Lib/Storage/AmazonS3.php","line":620,"function":"__call","class":"Aws\\AwsClient","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Lib/Storage/AmazonS3.php","line":666,"function":"copy","class":"OCA\\Files_External\\Lib\\Storage\\AmazonS3","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":278,"function":"rename","class":"OCA\\Files_External\\Lib\\Storage\\AmazonS3","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php","line":295,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":278,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->"},{"file":"/var/www/nextcloud/apps/files_accesscontrol/lib/StorageWrapper.php","line":300,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":822,"function":"rename","class":"OCA\\FilesAccessControl\\StorageWrapper","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","line":142,"function":"rename","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":157,"function":"setName","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":641,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/WrappedHttpHandler.php","Line":195,"Previous":{"Exception":"GuzzleHttp\\Exception\\ClientException","Message":"Client error: `PUT https://xxxxxxxxx.s3.eu-north-1.amazonaws.com/Videoarkisto/FinRX/02%20jalkkari.mkv` resulted in a `400 Bad Request` response:\n<Error><Code>InvalidRequest</Code><Message>The specified copy source is larger than the maximum allowable size for a cop (truncated...)\n","Code":400,"Trace":[{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":65,"function":"create","class":"GuzzleHttp\\Exception\\RequestException","type":"::"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":203,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":156,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":118,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":145,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/AwsClientTrait.php","line":58,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/aws/aws-sdk-php/src/AwsClientTrait.php","line":86,"function":"execute","class":"Aws\\AwsClient","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Lib/Storage/AmazonS3.php","line":620,"function":"__call","class":"Aws\\AwsClient","type":"->"},{"file":"/var/www/nextcloud/apps/files_external/lib/Lib/Storage/AmazonS3.php","line":666,"function":"copy","class":"OCA\\Files_External\\Lib\\Storage\\AmazonS3","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":278,"function":"rename","class":"OCA\\Files_External\\Lib\\Storage\\AmazonS3","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php","line":295,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":278,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->"},{"file":"/var/www/nextcloud/apps/files_accesscontrol/lib/StorageWrapper.php","line":300,"function":"rename","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":822,"function":"rename","class":"OCA\\FilesAccessControl\\StorageWrapper","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","line":142,"function":"rename","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":157,"function":"setName","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":641,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":320,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":167,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Exception/RequestException.php","Line":113},"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36 Edg/87.0.664.52","version":"19.0.5.2","id":"5fc97fec59c11"}

Additional information

As far as I understand, a CopyObject action is limited to files under 5 GB and should not be used here, see:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html

Unfortunately I don't have the knowledge needed to try to fix this myself.

The text was updated successfully, but these errors were encountered:

solracsf · 2020-12-07T20:55:05Z

The problem is that renaming on S3 equals to COPY that object under a new name and then DELETE the previous one.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html

You can store individual objects of up to 5 TB in Amazon S3. You create a copy of your object up to 5 GB in size in a single atomic operation using this API. However, to copy an object greater than 5 GB, you must use the multipart upload Upload Part - Copy API.

juliusknorr · 2021-05-12T12:38:11Z

https://docs.aws.amazon.com/AmazonS3/latest/userguide/CopyingObjctsMPUapi.html for reference of using the multipart upload to copy objects

Only happens on external storages as for primary object store the rename would only take action in the filecache while the object urn stays unchanged.

juliusknorr · 2023-01-24T09:21:59Z

No work happened for this so this is still valid.

tlinna added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Dec 4, 2020

solracsf added the feature: object storage label Dec 4, 2020

juliusknorr added the feature: external storage label May 12, 2021

juliusknorr added 1. to develop Accepted and waiting to be taken care of and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels May 12, 2021

This comment was marked as resolved.

Sign in to view

szaimen added needs info 0. Needs triage Pending check for reproducibility or if it fits our roadmap and removed 1. to develop Accepted and waiting to be taken care of labels Jan 23, 2023

szaimen added 26-feedback and removed needs info labels Jan 24, 2023

joshtrichards added feature: object storage and removed feature: object storage labels Sep 3, 2023

icewind1991 mentioned this issue Sep 20, 2023

use multipart copy for s3 #40531

Merged

icewind1991 closed this as completed in #40531 Sep 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

S3 external storage: Cannot rename >5 GB files #24540

S3 external storage: Cannot rename >5 GB files #24540

tlinna commented Dec 4, 2020 •

edited

Loading

solracsf commented Dec 7, 2020

juliusknorr commented May 12, 2021

This comment was marked as resolved.

juliusknorr commented Jan 24, 2023

S3 external storage: Cannot rename >5 GB files #24540

S3 external storage: Cannot rename >5 GB files #24540

Comments

tlinna commented Dec 4, 2020 • edited Loading

How to use GitHub

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Additional information

solracsf commented Dec 7, 2020

juliusknorr commented May 12, 2021

This comment was marked as resolved.

juliusknorr commented Jan 24, 2023

tlinna commented Dec 4, 2020 •

edited

Loading