[NC 20+] Legacy encryption - mitigation and documentation thereof

[Kuphi]

With Nextcloud 20, the term "Legacy encryption" was introduced. The documentation does say little about what legacy encryption is: https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_migration.html
I found another document, which gives hints about what could be meant: https://eprint.iacr.org/2020/1439.pdf

I run the command
occ encryption:scan:legacy-format
with output similar to this:
sudo -u www-data php /var/www/nextcloud/occ encryption:scan:legacy-format Scanning all files for legacy encryption Scanning all files for username /username/files_trashbin/files/file1.odt.d1592989576 does not have a proper header /username/files_trashbin/versions/folder1/file2.xls.v1585723230 does not have a proper header /username/files_versions/folder2/file3.xls.v1189677150 does not have a proper header

Now I got some files with inproper headers. Does my nextcloud still use legacy encryption for all files or does it use a more secure encryption? What are the differences?

When running occ encryption I also encountered a new command:
Command "encryption" is not defined.
Did you mean one of these?
encryption:change-key-storage-root
encryption:decrypt-all
encryption:disable
encryption:disable-master-key
encryption:enable
encryption:enable-master-key
encryption:encrypt-all
encryption:list-modules
encryption:migrate-key-storage-format
encryption:recover-user
encryption:scan:legacy-format
encryption:set-default-module
encryption:show-key-storage-root
encryption:status

There is no documentation about encryption:migrate-key-storage-format but an excerpt from the file /nextcloud/core/Command/Encryption/MigrateKeyStorage.php - https://fossies.org/linux/nextcloud/core/Command/Encryption/MigrateKeyStorage.php
75 ->setName('encryption:migrate-key-storage-format') 76 ->setDescription('Migrate the format of the keystorage to a newer format')

Am I supposed to use this command? I have key-type: user keys ( https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_details.html#key-type-user-key )
Will I still be able to use user keys after running this command, and is it save to run it?

My feature request is: please answer this questions and update the documentation of nextcloud.
Thank you very much for maintaining nextcloud! :)

[Kuphi]

For the documentation record:
The command encryption:migrate-key-storage-format can not be run successfully, when your nextcloud is in maintenance mode.
user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:migrate-key-storage-format Nextcloud is in maintenance mode - no apps have been loaded Updating key storage format Start to update the keys: 4 [----->----------------------] In Manager.php line 187: Module with ID: OC_DEFAULT_MODULE does not exist. encryption:migrate-key-storage-format
You have to turn off maintenance mode first:
sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off
then it runs smoothly, but without any notice before start. Does it not bear any risks?:

user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:list-modules
OC_DEFAULT_MODULE: Default encryption module [default*]

user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:migrate-key-storage-format
Updating key storage format
Start to update the keys:
20 [============================]Key storage format successfully updated

After the migration, occ encryption:scan:legacy-format does still output the same files as "does not have a proper header". Am I supposed to delete those files? Can I simply rm them and then do occ files:scan? (All my files with inproper header are either in the trashbin or in the folder files_versions.

[mark-orion]

Please someone explain what these "...does not have a proper header" error messages mean and how they can be fixed.

[leblitzdick]

Please someone explain what these "...does not have a proper header" error messages mean and how they can be fixed.

I also have the same question, can i set "encryption.legacy_format_support" to false without any problems?

[BenedictBauerHHN]

I have the same output on occ encryption:scan:legacy-format of course to other files then the OP. Could it lead to problems if I set encryption.legacy_format_support to false?

[developerrespig]

I would also be interested in some more information about that issue from an official side. Besides of that I just had a look into the legacy format scanner where this message is originating from (you can find the source code of that checker here: https://github.com/nextcloud/server/blob/1448b7c923d079c9616f87df3ffa52a4656ac6cc/apps/encryption/lib/Command/ScanLegacyFormat.php (I guess)).

With my limited php skills I guess it looks like these messages mean, that the check failed (because the private method returns false and the expected "All scanned files are properly encrypted. You can disable the legacy compatibility mode." doesn't get rendered). Question for me would be now: What to do about them? As far as I have seen them on my end they are only related to file versions or trash bins. But for now I just leave that configurational entry in the config.php and I guess you should do the same until someone says something different...

Hope to have something official here soon fingers crossed

[Kuphi]

Can we get a comment regarding this issue?

After upgrading to NC 20.0.3, it does anew complain about old server side encryption format. I thought I already resolved this?

"Das alte serverseitige Verschlüsselungsformat ist aktiviert. Wir empfehlen, es zu deaktivieren. Für weitere Einzelheiten sehe bitte in die Dokumentation."

[eckeSolutions]

In many instances the problem are old versions of files that are a few months old or even years.
Cleaning those versions with "occ versions:cleanup" removed all does “not have a proper header” errors for me.
Proceed with care!!! Might delete necessary file versions!!!
running "occ encryption:scan:legacy-format" after that tells me it is save do set encryption.legacy_format_support to false

This is not a general solution, but should work for some people with this problem (if “/versions/” is included in the file urls).
A solution to set the proper header would be nice though.

[Kuphi]

In many instances the problem are old versions of files that are a few months old or even years.
Cleaning those versions with "occ versions:cleanup" removed all does “not have a proper header” errors for me.
Proceed with care!!! Might delete necessary file versions!!!
running "occ encryption:scan:legacy-format" after that tells me it is save do set encryption.legacy_format_support to false

This is not a general solution, but should work for some people with this problem (if “/versions/” is included in the file urls).
A solution to set the proper header would be nice though.

Proceed with care? "occ versions:cleanup" does delete ALL previous versions of all files. This is only a solution if you do not need any previous versions at all.

[eckeSolutions]

Proceed with care? "occ versions:cleanup" does delete ALL previous versions of all files. This is only a solution if you do not need any previous versions at all.

Yes, that’s why I added 3 exclamation marks. In my instance it was no problem. And to be on the save side I have daily backups of everything.

[DPTJKKVH]

It is really sad that nobody of the team even acknowledges that encryption:migrate-key-storage-format is undocumented and elaborates in a few words what it does and if we should do it after disabling legacy encryption.

Also @developerrespig seems to be correct. I removed the bad header files manually, ran the check, got the "you can disbale legacy encryption" message and did just that.

I see neither issues nor changes so far which is good. But it drives me nuts that I don't see the point and don't know if we should migrate the storage format because it seems to be security related.

@schiessle @rullzer @nickvergessen can you please explain if we should run encryption:migrate-key-storage-format or not after disabling legacy encryption, even if you don't feel to explain it any further? Just a YES or NO is sufficient for the time being.

Many thanks in advance!

[thomwiggers]

I don't think encryption:migrate-key-storage-format solves this problem. There doesn't seem to be any migration option if I look in the source code of the Encryption module.

[DPTJKKVH]

Well I can report that disabling legacy encryption breaks my installation, even though the encryption:scan:legacy-format tells me I can disable it.

To be more specific: Encrypted files cannot be downloaded anymore via web GUI and sharing files (e.g. publicly) doesn't work as well.

The GUI tells me that my private key is missing and I should log out and back in. But that doesn't fix it.

Then it tells me to set a new password. When changing paswords it tells me again that my private key is missing and I should log out and back in. Doing that again doesn't fix it.

So neither logging in nor changing passwords works.

Fortunately just enabling legacy encryption fixes the issue and I can access all files again. However this foreshadows doom for when legacy encryption won't be supported anymore.

Either the scanning tool is broken or there is an important step missing after or before disabling legacy encryption.

I find it rather odd that simply disabling legacy encryption in the config is supposed to change encryption schemes. Also the "has been migrated" boolean in config.php is suggesting there might be more to it.

Oh btw: The encryption:migrate-key-storage-format does not run through and produces and error about an expexted boolean value for decrypt.

[thomwiggers]

It's not unlikely that the files need to be migrated from the client side if you're using user key encryption. I only have a single user, and managed to migrate by touch-ing the few affected files. This triggered a re-upload in the client. This does however still leave old versions in files_versions that now show up in the scan tool.

Either the scanning tool is broken or there is an important step missing after or before disabling legacy encryption.

I don't think the tool is broken, there simply isn't any migration path yet.

[DPTJKKVH]

I don't think the tool is broken, there simply isn't any migration path yet.

That might be and it would be nice to have some official input on that.

The main issue I have at this point is that there is a scary orange warning in the admin panel that highly recommends admins to disable legacy encryption asap while at the same time it seems like there isn't a proper way to do that without breaking things.

I know everyone is de facto on leave for Christmas at this point and probably only paying customers get emergency support but one simple "Yea it's broken at the moment. Don't worry, we will fix it in 2021." would be nice or a little hotfix that simply removes the warning so nobody else falls victim to that would go a long way.

I know Nexctloud owes us nothing when we don't pay for support but it would be nice to not let people run into their doom right before Christmas.

[RedKage]

Hmmm
I have been using nextcloud for many years, migrated from owncloud 8 or something. So I must have started with nextcloud 10 or 11.
I have been using the default encryption module since then, and done web upgrades.

I have seem some posts somewhere about encryption.legacy_format_support, and I set it to false
Recently then I upgraded to 20.0.4.
Then emptied the trashbin.

Then I ran encryption:scan:legacy-format which took a while, and it told me a had a few files. they were all versions files.
I cleaned them up using versions:cleanup

Finally I have ran encryption:migrate-key-storage-format on version 20.0.4.
Maintenance mode was activated while doing so
Nothing much was showing on stdout, and after about 1 hour it was done.
Everything is working properly, @DPTJKKVH I have had no weird things about changing password.

For me all is good, though I'm not sure anything was changed. I did huge backups before doing this, so in theory I could make some diffs to see if some stuff were actually changed... but, we are talking hundreds of gigs here.

[DPTJKKVH]

@RedKage I also run Nextcloud from the day there was a migration path from ownCloud.

What's interesting is that you ran key storage migration in maintenance mode. @Kuphi said it only worked when he disabled maintenance mode.

Also just for more background info: I had some files with bad/missing headers but none of them were versions or in the trash. I just backed them up on my client, removed them and emptied trash. Then I ran the legacy check and it said all is fine.

[Amon-san]

i also had this issue. i "fixed" it by copying the respective files to another directory, deleting the original and then moving the copy back in place. note: using 'move file' in the first place did not yield the same result for me.

[Kuphi]

i also had this issue. i "fixed" it by copying the respective files to another directory, deleting the original and then moving the copy back in place. note: using 'move file' in the first place did not yield the same result for me.

Yes. By copying the files, they will lose all previous file versions and will be newly encrypted.

[Alvarord]

Hello,

I face a similar issue as @DPTJKKVH

Well I can report that disabling legacy encryption breaks my installation, even though the encryption:scan:legacy-format tells me I can disable it.

I have upgraded from 19.0.7.1 to 20.0.5 yesterday

I cleaned the result from:

occ encryption:scan:legacy-format
When I deactivate the 'encryption.legacy_format_support' => false then I can't access files and I see the #8546 Invalid private key. I tried this without success.

Then I change the value of 'encryption.legacy_format_support' => true and I can access back to the data but the message Invalid private key for encryption app. Please update your private key password in your personal settings to recover access to your encrypted files. keeps appearing.

Not sure how can I help to track it.

I migrated from owncloud to nextcloud long ago (not sure in which version).

In the meantime I will do a full backup of my files (now that I can access them) just in case I break anything.

Any ideas about how to proceed?

More information:

root@machine:/var/www/html# sudo -u www-data php occ   encryption:scan:legacy-format
Scanning all files for legacy encryption
Scanning all files for user1
......
All scanned files are properly encrypted. You can disable the legacy compatibility mode.

root@machine:/var/www/html# sudo -u www-data php occ  encryption:status
  - enabled: true
  - defaultModule: OC_DEFAULT_MODULE

root@machine:/var/www/html# sudo -u www-data php occ  config:system:get version
20.0.5.2

Edit: I tried to ran this step:
sudo -u www-data ./occ encryption:encrypt-all
But I don't have same output as reported by @yahesh

I will try execute it again right after I finish the backup and I will report my result.

[DPTJKKVH]

@Alvarord I ran encrypt-all multiple times and it doesn't fix anything. The issue as we experience it, stays the same.

I can only guess that people like us, who migrated from owncloud to nextcloud years ago, need some command to migrate everything encrypted in legacy mode to the new encryption, possibly including the new key storage format?

But alas two months go by and the developers have not even acknowledged that people have problems.

I fear for the worst when we have to update to v21 and our encryption breaks for good, forcing us to deploy from scratch, setup dozens of users and reupload gigabytes or terabytes worth of data.

[micah]

On 2021-01-19 06:51:47, DPTJKKVH wrote: I can only guess that people like us, who migrated from owncloud to nextcloud years ago, need some command to migrate everything encrypted in legacy mode to the new encryption, possibly including the new key storage format?

I did not upgrade from owncloud, and I had this problem.

[Alvarord]

Hello @DPTJKKVH ,

Is strange maybe they consider it should be solved with the encrypt-all, or they havent been able to reproduce it.

I saw in the bash history this morning that I executed the command yesterday but it was quite fast and I though it was because it was ciphering just a few old files. Today after doing the backup (luckily for me is only 100GB) I re-executed it and I noticed 2 things

First it finish always with an error:

In Storage.php line 287:

  Could not decrypt key


In Crypto.php line 122:

  Authenticated ciphertext could not be decoded.

@DPTJKKVH do you have the same issue?

And every time I execute it it cipher just a few files less than 1000 files But at every try it is trying different groups of files (normally shared ones).

I will try to gather which file/s produce the issue.

Regards.

[DPTJKKVH]

@Alvarord you can't encrypt your files with encrypt-all because when you disable legacy encryption the server, for some reason, can't decrypt your private key. Try the same with legacy encryption enabled (just switch the "false" flag back to "true" in the config.php) and it will work.

Disabling legacy encryption does seemingly require the private keys to be stored in a different format. However encryption:migrate-key-storage-format interupts with an error. Maybe try this in both legacy and non legacy mode and tell us what your output is.

I think you and me have the same problem.

@micah do you HAVE the problem or HAD the the problem and fixed it?

[micah]

On 2021-01-19 07:27:40, DPTJKKVH wrote: @micah do you HAVE the problem or HAD the the problem and fixed it?

well, honestly I dont *know* if I have the problem still. I saw in my administration panel that this was deprecated and I needed to run the encryption:scan:legacy-format command. It produced a number of weird errors and seemed like it failed. I tried moving those files out of the way, like people here said, and re-ran it. As far as I know, this just 'scanned' for files, and didn't do anything? Will I need to do something more? If I run this command and I don't get an error, do I no longer have the problem? When I run it now, it does not fail, and it says: All scanned files are properly encrypted. You can disable the legacy compatibility mode. How do I do that?

[Alvarord]

Hello @DPTJKKVH

@Alvarord you can't encrypt your files with encrypt-all because when you disable legacy encryption the server, for some reason, can't decrypt your private key. Try the same with legacy encryption enabled (just switch the "false" flag back to "true" in the config.php) and it will work.

same issue enabling or disabling it. A few files are encrypted and it finish with same error.

Trying right now the second part with legacy active (I think this took yesterday a while). I will try with both values and update you later.

Thanks for the ideas to try.

[DPTJKKVH]

@micah when it runs through without an error you are (in theory) ready to disable legacy encryption as described here:

https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_migration.html

Try that and then log into your normal user account on your nextcloud server and see if it tells you that your private key couldn't be decrypted. If that happens, you have the same issue as we have.

If you don't get any message, try downloading and opening your files via the web gui. If it works: Congratulations, you were lucky. If not, you suffer the same issue as we do.

[jknockaert]

The fun thing is, even after disabling encryption (occ encryption:disable) I still get the message:

The old server-side-encryption format is enabled. We recommend disabling this. For more details see the documentation.

That's quite a bit off. And don't expect the documentation to help you any further; it rarely ever did when it comes to the encryption app.

[kaysond]

#24240 (comment)
So the solution to this, in my case, was to run occ files:scan --all (requires maintenance mode being off). Then it ran with no issues!

[kaysond]

Followup - I'm now getting a lot of encryption-related errors like:

{"reqId":"vlJTJhWNMRpObwgx8h8i","level":3,"time":"2021-05-08T16:32:32+00:00","remoteAddr":"99.240.225.175","user":"samir","app":"PHP","method":"PUT","url":"/remote.php/dav/files/samir/Samir's%20Pictures/ControlCenter4/Scan/CHQ%20for%20Jack20210508.pdf","message":{"Exception":"Error","Message":"Undefined index: encryptedVersion at /var/www/nubcloud/lib/private/Files/Storage/Wrapper/Encryption.php#669","Code":0,"Trace":[{"file":"/var/www/nubcloud/lib/private/Files/Storage/Wrapper/Encryption.php","line":669,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/nubcloud/lib/private/Files/Storage/Wrapper/Encryption.php","line":767,"function":"updateEncryptedVersion","class":"OC\\Files\\Storage\\Wrapper\\Encryption","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nubcloud/lib/private/Files/Storage/Wrapper/Encryption.php","line":623,"function":"copyBetweenStorage","class":"OC\\Files\\Storage\\Wrapper\\Encryption","type":"->"},{"file":"/var/www/nubcloud/apps/files_trashbin/lib/Storage.php","line":240,"function":"moveFromStorage","class":"OC\\Files\\Storage\\Wrapper\\Encryption","type":"->"},{"file":"/var/www/nubcloud/apps/dav/lib/Connector/Sabre/File.php","line":292,"function":"moveFromStorage","class":"OCA\\Files_Trashbin\\Storage","type":"->"},{"file":"/var/www/nubcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":156,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1098,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/apps/dav/lib/Server.php","line":332,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nubcloud/remote.php","line":167,"args":["/var/www/nubcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nubcloud/lib/private/Log/ErrorHandler.php","Line":92,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows) mirall/2.6.2stable-Win64 (build 20191224) (Nextcloud)","version":"21.0.1.1"}

after I ran the migration and disabled legacy encryption support.

Anyone have any ideas?

[DPTJKKVH]

@kaysond I don't think it's related to the issue at hand. I suddenly have gotten such errors as well, after the server ran more or less okay for a time. I set up a second server, 100% fresh via the Snap repository, no backup restore or anything. I enabled the encryption module uploaded some files, and there were the errors already.

This is where I gave up. We are on SeaFile now.

But again: I don't think it's related to the migration since the errors happen on a fresh install as well, right after enabling the default encryption module.

[kaysond]

@kaysond I don't think it's related to the issue at hand. I suddenly have gotten such errors as well, after the server ran more or less okay for a time. I set up a second server, 100% fresh via the Snap repository, no backup restore or anything. I enabled the encryption module uploaded some files, and there were the errors already.

This is where I gave up. We are on SeaFile now.

But again: I don't think it's related to the migration since the errors happen on a fresh install as well, right after enabling the default encryption module.

Thanks for the info. I wonder if it is just the upgrade to the new version that caused this.

[DPTJKKVH]

That's very much possible. To be honest now that you mentioned it, I think the issues did start shortly after a server update.

Do keep in mind that those issue mean that your files on the server one by one become unreadable/undecryptable due to those errors. At least that is how it was on my server. You can try to verify if you suffer the same fate by logging in via the web client, downloading some files and then trying to open them. In my case the file names were okay but the files themselves were all messed up. (Either still encrypted or corrupted. It's hard to tell the difference.)

Fortunately the files on the clients stayed unharmed, so we were able to fully recover everything.

Before someone chimes in: I'm not trying to shit on Nextcloud in any way. I'm actually still helping people because I do care. I was a big proponent of Nextcloud and I somewhat still am but the issues with the encryption module and the devs not even acknowledging it, forced me to quit to avoid data loss, after literal months of dreadful work. I know the devs owe me/us nothing for offering free open source software. So no hate, no hard feelings. Just helping people to avoid data loss.

[szaimen]

Maybe this comment shelds some lights on something here?
#27501 (comment)

[ajs124]

In my case, the files are just plain unencrypted files, in the filesystem. Nextcloud however seems to think that they are encrypted. I've already run occ files:scan --all and so on, but encryption:scan:legacy-format still finds the same files.

I fixed this by deleting the folders and files relating to those files from files_encryption/keys, dropping the entries referencing them from the database and setting the encrypted field to 0 in the database. Then I ran files:scan --all again and now it seems to work. Idk if that helps anyone here.

[PVince81]

for the case of unencrypted files wrongly detected as encrypted, please run: https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#problems-when-downloading-or-decrypting-files

[Gajusbonus]

for the case of unencrypted files wrongly detected as encrypted, please run: https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#problems-when-downloading-or-decrypting-files

Returns:
Command "encryption:fix-encrypted-version" is not defined.
occ config:system:get version
20.0.10.1
So....what version do you expect me to run, to be able to fix decrypting files?

[RedKage]

Do you guys a favor and stop using encryption. This is broken since I have ever used nextcloud.
And started with version 9 or 10.
All these years I had 500 errors with bad encryption errors, had to re-up files to fix these issues.
For 4 years I coped with this, until version 21 or something.

Then decided to reinstall everything, re-upload everything without encryption and do my own encryption with gocrypt-fs. I'm never going back. This is a broken feature from the start. And also prevent many apps to work with it, like the music app which tries (tried? Maybe fixed) to save tags on encrypted files and corrupt them or many other apps not handling encryption.
Also now that I have removed encryption it's very easy for to backup and restore my data, as the data is unencrypted I can pull files directly from a backup. While still benefiting from fs encryption on top.

Anyway my 2cents guys.

[Pazu]

Hear, hear. The worst part is there is no apparent will or perhaps even ability on the part of the developers to do anything about this problem.

[asheroto]

Even on a brand new nextcloud installation the issue exists. If it's broken, it should be disabled and not offered as a working product. Related to #8546.

Going to try the solution @RedKage proposed.

[Flonky]

Hi, running ./occ encryption:fix-encrypted-version user seems to have fixed this for me, although I also had to run ./occ files:scan --all afterwards.

For reference, occ config:system:get version returns 28.0.1.1.

[asheroto]

Indeed, it will appear to fix it at first. But I found several days later that encryption was still broken for new or changed files moving forward, especially larger than a few KB even though running that command fixes the existing files.

This of course may not be the case for everyone, but when using the previously mentioned encrypted method encryption with S3, I found this to be the case.

[Flonky]

I found several days later that encryption was still broken for new or changed files moving forward, especially larger than a few KB even though running that command fixes the existing files.

Even after key migration? I haven't noticed anything like that yet, although my instance doesn't have too much traffic either.

[asheroto]

Even after key migration? I haven't noticed anything like that yet, although my instance doesn't have too much traffic either.

Yes! Even after that. I worked on this for days and made several posts. :-/ I ended up just switching to SSE with my S3 provider. That worked out fine, although it still rubs me the wrong way that my data is sent to the S3 provider without being encrypted first. I realize it is encrypted in transmit, but the S3 provider technically receives the raw data, then encrypts it at rest. 😊

[brotkastn]

Hello everyone.
Since there have been some time since the last comment here, i thought i would take my chance again at disabling the legacy encryption.

Previously, i did the encryption:migrate-key-storage-format . Today i have taken the following steps:

www-data@srv-nextcloud:~/nextcloud$ php occ status
  - installed: true
  - version: 29.0.4.1
  - versionstring: 29.0.4
  - edition: 
  - maintenance: false
  - needsDbUpgrade: false
  - productname: Nextcloud
  - extendedSupport: false
www-data@srv-nextcloud:~/nextcloud$ php occ files:scan --all
www-data@srv-nextcloud:~/nextcloud$ php occ encryption:scan:legacy-format | tee ../../nextcloud-datadir/nextcloud_encryption_legacy_2024-07-25
Scanning all files for legacy encryption
Scanning all files for brot
Scanning all files for otheruser1
Scanning all files for otheruser2
Scanning all files for otheruser3
Scanning all files for otheruser4
Scanning all files for otheruser5
All scanned files are properly encrypted. You can disable the legacy compatibility mode.

However, if i try to access a share my wife made with a collection of family photos, via the link she used to send to our extended family, it seems to miss the legacy encryption. I can view the previews of the photos, but nextcloud throws an error when i try to access stuff. Neither the "Download" button on the top of the page or the download of an individual image seems to work. The nextcloud log shows the following entries:

{"reqId":"MSajkOfIBJz2hC2BEbwH","level":3,"time":"2024-07-25T18:10:54+00:00","remoteAddr":"2001:1234::1234","user":"--","app":"no app in context","method":"GET","url":"/s/shareViaLink/download","mes
sage":"Exception thrown: OC\\ServerNotAvailableException","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","version":"29.0.4.1","exception":{"Exception":"OC\\ServerNotAvailab
leException","Message":"Legacy cipher is no longer supported!","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/encryption/lib/Crypto/Crypt.php","line":378,"function":"getLegacyCipher","class":"OCA\\Encryptio
n\\Crypto\\Crypt","type":"->"},{"file":"/var/www/nextcloud/apps/encryption/lib/KeyManager.php","line":403,"function":"decryptPrivateKey","class":"OCA\\Encryption\\Crypto\\Crypt","type":"->","args":["*** sensiti
ve parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/encryption/lib/Crypto/Encryption.php","line":155,"function":"getFileKey","class":"OCA\\Encryption\\KeyManager","type":"->"},{"file":"/var/www/nextc
loud/lib/private/Files/Stream/Encryption.php","line":284,"function":"begin","class":"OCA\\Encryption\\Crypto\\Encryption","type":"->"},{"function":"stream_open","class":"OC\\Files\\Stream\\Encryption","type":"-
>"},{"file":"/var/www/nextcloud/lib/private/Files/Stream/Encryption.php","line":212,"function":"fopen"},{"file":"/var/www/nextcloud/lib/private/Files/Stream/Encryption.php","line":187,"function":"wrapSource","c
lass":"OC\\Files\\Stream\\Encryption","type":"::"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Encryption.php","line":489,"function":"wrap","class":"OC\\Files\\Stream\\Encryption","type":"::"}
,{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":307,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Encryption","type":"->"},{"file":"/var/www/nextcloud/lib/private/F
iles/View.php","line":1169,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":997,"function":"basicOperation","class":"
OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/File.php","line":116,"function":"fopen","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Streamer.p
hp","line":132,"function":"fopen","class":"OC\\Files\\Node\\File","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Files.php","line":217,"function":"addDirRecursive","class":"OC\\Streamer","type":
"->"},{"file":"/var/www/nextcloud/apps/files_sharing/lib/Controller/ShareController.php","line":492,"function":"get","class":"OC_Files","type":"::"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dis
patcher.php","line":232,"function":"downloadShare","class":"OCA\\Files_Sharing\\Controller\\ShareController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"fu
nction":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Ht
tp\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":338,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":
1050,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/apps/encryption
/lib/Crypto/Crypt.php","Line":301,"CustomMessage":"Exception thrown: OC\\ServerNotAvailableException"}}
{"reqId":"AjIzOqiDuIKw0n44A543","level":3,"time":"2024-07-25T18:11:11+00:00","remoteAddr":"2001:1234::1234","user":"--","app":"no app in context","method":"GET","url":"/s/shareViaLink/download?path=
&files=singleImage.jpg","message":"Exception thrown: OC\\ServerNotAvailableException","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","version":"29.0.4.1","excep
tion":{"Exception":"OC\\ServerNotAvailableException","Message":"Legacy cipher is no longer supported!","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/encryption/lib/Crypto/Crypt.php","line":378,"function":"
getLegacyCipher","class":"OCA\\Encryption\\Crypto\\Crypt","type":"->"},{"file":"/var/www/nextcloud/apps/encryption/lib/KeyManager.php","line":403,"function":"decryptPrivateKey","class":"OCA\\Encryption\\Crypto\
\Crypt","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/encryption/lib/Crypto/Encryption.php","line":466,"function":"getFileKey","class":"OCA\\Encryption\\KeyManag
er","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Encryption.php","line":354,"function":"isReadable","class":"OCA\\Encryption\\Crypto\\Encryption","type":"->"},{"file":"/var/www/nex
tcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":182,"function":"isReadable","class":"OC\\Files\\Storage\\Wrapper\\Encryption","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","l
ine":1171,"function":"isReadable","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":491,"function":"basicOperation","class":"OC\\Files\\V
iew","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Filesystem.php","line":530,"function":"isReadable","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Files.
php","line":344,"function":"isReadable","class":"OC\\Files\\Filesystem","type":"::"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Files.php","line":123,"function":"getSingleFile","class":"OC_Files","type":
"::"},{"file":"/var/www/nextcloud/apps/files_sharing/lib/Controller/ShareController.php","line":487,"function":"get","class":"OC_Files","type":"::"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dis
patcher.php","line":232,"function":"downloadShare","class":"OCA\\Files_Sharing\\Controller\\ShareController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"fu
nction":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Ht
tp\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":338,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":
1050,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/apps/encryption
/lib/Crypto/Crypt.php","Line":301,"CustomMessage":"Exception thrown: OC\\ServerNotAvailableException"}}
{"reqId":"AjIzOqiDuIKw0n44A543","level":3,"time":"2024-07-25T18:11:11+00:00","remoteAddr":"2001:1234::1234","user":"--","app":"PHP","method":"GET","url":"/s/shareViaLink/download?path=&files=singleImage.jpg","message":"Undefined variable $event at /var/www/nextcloud/lib/private/legacy/OC_Files.php#242","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","versio
n":"29.0.4.1","data":{"app":"PHP"}}

I do wonder why that exact folder still relies on legacy encryption, as said pictures have been uploded around may - the "active" encryption-module was already set to the default one.

www-data@srv-nextcloud:~/nextcloud$ php occ encryption:show-key-storage-root
Current key storage root:  default storage location (data/)
www-data@srv-nextcloud:~/nextcloud$ php occ encryption:status
  - enabled: true
  - defaultModule: OC_DEFAULT_MODULE

I wonder what the way forward to disabling the legacy encryption module is. The re-download and start with a fresh nextcloud approach is a bit problematic since we are talking about 1.2TB worth of files, family spread across a few thousand kilometers and a huge number of share links which would be lost:

MariaDB [nextcloud]> select count(*) from oc_share where token IS NOT NULL;
+----------+
| count(*) |
+----------+
|      391 |
+----------+
1 row in set (0.002 sec)

Thanks for reading. I am happy that the community proved so helpful for people trying to get their data back. If there is someone with an idea how to proceed in my case, i am happy to test even more 'creative' solutions to this problem and report back.

Also, thanks to the devs for the awesome suite of software that nextcloud is :)