Can not decrypt file on mobile client

[team-alpeinsoft]

Steps to reproduce

1. Install owncloud.
2. Enable encryption.
3. Share file with user.
4. Try download file from mobile client

Actual behaviour

Cannot download file from mobile client when using server side encryption

1. Web interface - working
2. Problem actual not only for ios .
   Can not decrypt file on mobile client android#1711

Server configuration

Web server:
apache2 2.4.10-10+deb8u4 amd64

Database:
mysql-server-5.5 5.5.47-0+deb8u1 amd64

PHP version:
PHP 5.6.20-0+deb8u1 (cli) (built: Apr 27 2016 11:26:05)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
with Xdebug v2.2.5, Copyright (c) 2002-2014, by Derick Rethans

Client

iOS version: All versions

ownCloud app version: All versions

Device model: All models

Logs

Web server error log

[10/Jun/2016:14:45:33 +0200] "GET /owncloud/remote.php/webdav/community.txt HTTP/1.1" 500 1630 "-" "Jakarta Commons-HttpClient/3.1"

ownCloud log (data/owncloud.log)

{"reqId":"m01/psu9n5hlK1KbxI1/","remoteAddr":"37.17.19.59","app":"no app in context","message":"Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.","level":3,"time":"2016-06-10T12:45:34+00:00","method":"GET","url":"/owncloud/remote.php/webdav/community.txt","user":"C96C88E4-9F71-4E45-9D6D-4FF46F01AEB6"}
{"reqId":"m01/psu9n5hlK1KbxI1/","remoteAddr":"37.17.19.59","app":"webdav","message":"Exception: {"Message":"Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.","Exception":"OC\Encryption\Exceptions\DecryptionFailedException","Code":0,"Trace":"#0 /var/www/owncloud/lib/private/files/stream/encryption.php(459): OCA\Encryption\Crypto\Encryption->decrypt('NB2yToafXnn8ROy...', '0end')\n#1 /var/www/owncloud/lib/private/files/stream/encryption.php(290): OC\Files\Stream\Encryption->readCache()\n#2 [internal function]: OC\Files\Stream\Encryption->stream_read(8192)\n#3 /var/www/owncloud/3rdparty/icewind/streams/src/Wrapper.php(83): fread(Resource id #83, 8192)\n#4 /var/www/owncloud/3rdparty/icewind/streams/src/CallbackWrapper.php(91): Icewind\Streams\Wrapper->stream_read(8192)\n#5 [internal function]: Icewind\Streams\CallbackWrapper->stream_read(8192)\n#6 /var/www/owncloud/3rdparty/sabre/http/lib/Sapi.php(78): stream_copy_to_stream(Resource id #86, Resource id #88, '861')\n#7 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(470): Sabre\HTTP\Sapi::sendResponse(Object(Sabre\HTTP\Response))\n#8 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#9 /var/www/owncloud/apps/dav/appinfo/v1/webdav.php(55): Sabre\DAV\Server->exec()\n#10 /var/www/owncloud/remote.php(138): require_once('/var/www/ownclo...')\n#11 {main}","File":"/var/www/owncloud/apps/encryption/lib/crypto/encryption.php","Line":360,"User":"C96C88E4-9F71-4E45-9D6D-4FF46F01AEB6"}","level":4,"time":"2016-06-10T12:45:34+00:00","method":"GET","url":"/owncloud/remote.php/webdav/community.txt","user":"C96C88E4-9F71-4E45-9D6D-4FF46F01AEB6"}

[rperezb]

@PVince81 does this make you ring any bell?
@SergioBertolinSG FYI

[PVince81]

Question is whether this user ever logged in before at the time the share was created ?
Or was this user added to a group recently and the file was shared with that group ?

Best is to try unsharing the file again, ask that user to log in (if not already) and then share the file again.

[team-alpeinsoft]

It`s not working not only for share file, but and other files( for example for my own files and folders ).

[team-alpeinsoft]

For new user(from LDAP) all works ( share & downloads ).

[PVince81]

@team-alpeinsoft are you able to log out then log in again the user who has trouble with the decryption ? Maybe something went wrong with the current session.

[team-alpeinsoft]

@PVince81 I logout - no result, remove and add account in mobile version - no result.

[PVince81]

@team-alpeinsoft can that user download files using the web UI ? Or any other Webdav client ? (goal is to find out whether there is something with the mobile client or something is wrong with this account)

[team-alpeinsoft]

1. Web interface +
2. Windows client +

"+" -> works

[PVince81]

"+" meaning it works ? Or getting the same error ?

[team-alpeinsoft]

Works :)

[PVince81]

Hmmmm... what about a mobile IOS from another device ? So far I think the IOS client has always worked with encryption so I'm not sure what would be wrong with that user apart from maybe a session that isn't cleared/reset properly.

[team-alpeinsoft]

We login in new ios device - no result.

Error on mobile:

OC\Encryption\Exceptions\DecryptionFailedException/s:exception Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you./s:message /d:error

Error on server side:

{"reqId":"z19T/GcaqR+aSQQ4ObBp","remoteAddr":"80.249.84.82","app":"webdav","message":"Exception: {"Message":"Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.","Exception":"OC\Encryption\Exceptions\DecryptionFailedException","Code":0,"Trace":"#0 \/var\/www\/owncloud\/lib\/private\/files\/stream\/encryption.php(459): OCA\Encryption\Crypto\Encryption->decrypt('NB2yToafXnn8ROy...', '0end')\n#1 \/var\/www\/owncloud\/lib\/private\/files\/stream\/encryption.php(290): OC\Files\Stream\Encryption->readCache()\n#2 [internal function]: OC\Files\Stream\Encryption->stream_read(8192)\n#3 \/var\/www\/owncloud\/3rdparty\/icewind\/streams\/src\/Wrapper.php(83): fread(Resource id #117, 8192)\n#4 \/var\/www\/owncloud\/3rdparty\/icewind\/streams\/src\/CallbackWrapper.php(91): Icewind\Streams\Wrapper->stream_read(8192)\n#5 [internal function]: Icewind\Streams\CallbackWrapper->stream_read(8192)\n#6 \/var\/www\/owncloud\/3rdparty\/sabre\/http\/lib\/Sapi.php(78): stream_copy_to_stream(Resource id #120, Resource id #122, '861')\n#7 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(470): Sabre\HTTP\Sapi::sendResponse(Object(Sabre\HTTP\Response))\n#8 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#9 \/var\/www\/owncloud\/apps\/dav\/appinfo\/v1\/webdav.php(55): Sabre\DAV\Server->exec()\n#10 \/var\/www\/owncloud\/remote.php(138): require_once('\/var\/www\/ownclo...')\n#11 {main}","File":"\/var\/www\/owncloud\/apps\/encryption\/lib\/crypto\/encryption.php","Line":360,"User":"96DB31B3-7E84-4C2D-B640-7FC2EF61A0BF"}","level":4,"time":"2016-06-13T13:56:06+00:00","method":"GET","url":"/owncloud/remote.php/webdav/community.txt","user":"96DB31B3-7E84-4C2D-B640-7FC2EF61A0BF"}

[PVince81]

Okay thanks. So this means that the IOS app is doing something differently with the session/cookies.

[team-alpeinsoft]

And thank you. p.s. Problem actual not only for ios .

[team-alpeinsoft]

Additional information. Via mobile client i can create folder and files, upload on the server. Only downloading options with error.

[muppeth]

I can confirm the same issue on my instance with Android app. I can upload, but no download. Works perfect with desktop client and web interface.

[muppeth]

@team-alpeinsoft do you think it's LDAP related?
I've created new users with default backend (no Ldap) to see whether the error persist, and it was gone. All users not in LDAP can download files with mobile app. The issue only affects LDAP users.
Did you get any further with your investigation?

[team-alpeinsoft]

@muppeth We have a 50/50 result. Some LDAP users are all well, others - does not work.

[muppeth]

@team-alpeinsoft AS far as I checked on accounts I have access to, none of them work. Same applies to newly created LDAP users. Only non-ldap users can decrypt data via android app.

It seems strange that in your case you have 50/50 result. Do you see a pattern there?

[team-alpeinsoft]

Yes. Its strange and we dont know how fix this problem.

[rperezb]

@team-alpeinsoft thanks for reporting this, currently @owncloud/qa team is checking this

[muppeth]

@team-alpeinsoft when did you realized the problem started? I've only realised after one of our users reported the problem, but it might be as long as oc8 > oc9 update.

I've checked if problem stays when upgrading to Nextcloud (both core and android app) and it is still there.

My only guesses atm are:

• something broke when updating to owncloud9.
• some recent OS update is causing the error.

I'm also looking for people running similar setup (oc+encryption+ldap) to see if this problem is affecting more ppl.

@rperezb thanks for taking time looking into this issue.

[team-alpeinsoft]

@rperezb And thank you!

@muppeth When we realized the problem? At random, when we can`t downloud files via mobile apps.

[muppeth]

@team-alpeinsoft I meant when was the first time you spotted problem exist.

[jesmrec]

Checking this issue with the following set up:

Server:

{"installed":true,"maintenance":false,"version":"9.1.0.9","versionstring":"9.1.0 beta 2","edition":"Enterprise"}

Encryption enabled.

Clients:

Android v5.0.1. App version 2.0.1 (market), and masterbranch
iOS v9.1. App version 3.4.9. App version 3.4.9 (market) and masterbranch

Test cases:

1. User1 shares some docs with User2 (LDAP user)
2. User1 & User2 download the files and view them
   __
3. User1(LDAP) shares some docs with User2 (LDAP user)
4. User1 & User2 download the files and view them

Checked also with folders

All downloads work fine.

Any input related to the doc type? Which kinds did you check? Were there in any external mount point stored?

[muppeth]

@jesmrec Did you use encryption module?

[jesmrec]

@muppeth yes, i used the default encryption module in order to encrypt server side files as you can check in my previous message.

[muppeth]

@jesmrec sorry I saw your msg before the edit.

Any input related to the doc type? Which kinds did you check? Were there in any external mount point stored?

In my case I checked jpg, pdf, png. All fail. User can create and upload files via android app but can't download (as stated before).

[team-alpeinsoft]

@muppeth : I meant when was the first time you spotted problem exist. ---> when open access( share folders) for few users.

[team-alpeinsoft]

@ muppeth and other guys from owncloud team : thank you for research.

We also use debian jessie.

We try recreate this problem ( install few new vm, ldap databases ) - all works fine. Problem with only one instance. And why it`s not work on mobile version? Web and mobile application have cardinal differences in code?

[team-alpeinsoft]

May be interesting : owncloud/core#18000

[muppeth]

Yet another update form my side, though not in anyway fixing the problem unfortunately.
First maybe some info that I haven't post previously about our setup.
We run Database, owncloud and ldap as seperate vms.

Here is what I did last night (great way of seeing sun rise btw :P)
I've found some old backup dating march (pre oc9). I deployed databse, ldap and owncloud vms. Tested the ldap users with android app and it all worked fine. Now:

1. I then copied the oc code into that older vm, and I migrated db to old db vm. Running new OC on old setup (old system packages etc), was causing the issue.
2. I reverted database back to the old one (oc8.2 i belive), and run owncloud update to oc9. That resulted in the issue appearing again.
3. I removed all users from ldap vm, leaving only one for testing. Problem persisted.
4. I moved old oc code and database to new/current vms (to see if it breaks with new packages). Everything worked fine so we can dismiss server setup at this point.
5. I installed oldest owncloud client i could find on f-droid (1.9.1) but the problem is still there when using it against oc9.

Next step is to install everything on single vm, as I'm now running out of ideas.

[team-alpeinsoft]

@muppeth:

Here is what I did last night (great way of seeing sun rise btw :P)
Yes :)

Next step is to install everything on single vm, as I'm now running out of ideas.

In our case all servrices on a single vm.

[muppeth]

Next step is to install everything on single vm, as I'm now running out of ideas.

That didnt work either.
I'm now pretty sure the problem must be related to some little silly package missing in my template. I will install Full Debian 8 and see if that changes anything, otherwise seems like I'm cursed and even when installing owncloud from scratch, I'm forbiden to use Android app :P.

[team-alpeinsoft]

We migrated to another VM:
uname -a
Linux nxt-hst 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u2 (2016-06-25) x86_64 GNU/Linux

php -v
PHP 5.6.22-0+deb8u1 (cli) (built: Jun 9 2016 07:14:06)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

apache2 -v
Server version: Apache/2.4.10 (Debian)

php -m
[PHP Modules]
bcmath
bz2
calendar
Core
ctype
curl
date
dba
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
imap
intl
json
ldap
libxml
mbstring
mcrypt
mhash
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
pgsql
Phar
posix
readline
Reflection
session
shmop
SimpleXML
soap
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

No results.

[team-alpeinsoft]

p.s. owncloud/android#1711 -> silence

[muppeth]

@team-alpeinsoft
I also did some more tests and whatnot and so far got to conclusion that something must have gone wrong with database when updating from OC8 to OC9. I will focus on this tonight.

[jvillafanez]

@team-alpeinsoft @muppeth Did you run the "encryption:migrate" occ command?

Steps should be as follow:

1. Enable files_encryption app in OC 8.2
2. Create files, user accounts, etc (normal usage of ownCloud)
3. Download and upgrade to OC 9
4. Enable encryption app: occ app:enable encryption
5. Enable default encryption module: occ encryption:enable
6. Migrate encryption keys: occ encryption:migrate (This step is what might be missing)
7. Normal usage

If you did everything through the web UI is very likely that the key migration didn't happen. I'm not sure if it's possible to execute it from the web UI.

[muppeth]

@jvillafanez I'm pretty sure I've done that, though cannot be 100% sure as I updated to oc9 in April. I am about to redo the update process on my test machine since i seem to have backup (i think.).
I only dont understand if key migration was not done, why is my encryption working in webui and desktop client, but fails with mobile clients.
Also why are new users affected by it and why it works perfectly fine with non-ldap users.

In any case @jvillafanez thanks for advice. I will play around with pre oc9 backup and let you know if I find anything.

As I was comparing both my production db with just freshly installed oc instance, I haven't seen anything related to encryption. All ldap replated entries in both instances seem to be in order. I even deleted all entries in oc_filecache to see if this is related, but did not change anything besides not being able to generate thumbnails etc. Anyone has any idea what to look for? Again it seems to be only related to ldap users.

[FlorianFranzen]

Could you check if any of you file actually do no have a key? You could use a script similar to this one:

#!/bin/bash

if [ $# -ne 1 ]; then
    echo "usage: $0 <user>"
    exit 0
fi

files=/var/www/html/owncloud/data/$1/files
keys=/var/www/html/owncloud/data/$1/files_encryption/keys/files

file_list=$(mktemp)
key_list=$(mktemp)
diff_list=$(mktemp)

find $files -type f | sed "s:^$files::" | sort > $file_list
find $keys -name fileKey | sed "s:^$keys::" | sed "s:/OC_DEFAULT_MODULE/fileKey::" | sort > $key_list
diff $file_list $key_list > $diff_list

rm $file_list $key_list

echo "Missing keys: $(egrep "^<" $diff_list | wc -l)" 1>&2
echo "Missing files: $(egrep "^>" $diff_list | wc -l)" 1>&2

rm $diff_list 

[neuhausjulian]

I also have the same problem. And it is also only related to ldap-users (Owncloud9 - Android-Client)

[muppeth]

@neuhausjulian sad at the same good to see more of us, maybe we can pin point the issue. When did you notice the issue? Was it any particular thing you did (update, installing/removing app, changing any config?).

I haven't paid too much attention to this issue as I was busy with other things on my todo list. However according to my planning I should be done with all of them by the end of the week, so the next thing on the list is solving that issue. If everything goes well I will dedicate all my free time next week to finally fix it (or at least go further with it).

Last resort would be dropping encryption I guess.

[neuhausjulian]

@muppeth

I installed and started running owncloud 2 weeks ago. At first I only used the admin account with no problems on every client.
On monday I installed a ldap-server and yesterday (2016-08-30) I added my ldap-server as login-authentication to owncloud.

With my first ldap-account I checked the encryption behaviour if I change the password. No problems on web- and desktop-client bevor and after. I didn't checked the mobile-client before.
I added also a new ldap-user uploaded a single picture and added the new user on my mobile-client. Also broken without any password changes.

The admin-account (not-ldap) is still working

Error-log:

Exception: {"Message":"HTTP/1.1 404 File with name Dokumente/ich.JPG could not be located","Exception":"Sabre\DAV\Exception\NotFound","Code":0,"Trace":"#0 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(79): OCA\DAV\Connector\Sabre\ObjectTree->getNodeForPath('Dokumente/ich.J...', 0)\n#1 [internal function]: Sabre\DAV\CorePlugin->httpGet(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#2 /var/www/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\n#3 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(459): Sabre\Event\EventEmitter->emit('method:GET', Array)\n#4 /var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#5 /var/www/owncloud/apps/dav/appinfo/v1/webdav.php(56): Sabre\DAV\Server->exec()\n#6 /var/www/owncloud/remote.php(164): require_once('/var/www/ownclo...')\n#7 {main}","File":"/var/www/owncloud/apps/dav/lib/Connector/Sabre/ObjectTree.php","Line":170,"User":"2b1bd114-0309-1036-9a86-435a68297ecb"}

[muppeth]

After long while I finally have time to take care of that issue.
I decided to check if anything happend.
Owncloud server side. I haven't update the core nor any of the apps.
Mobile client
I installed fresh nextcloud beta and owncloud beta to see if the issue is still there.
Nextcloud beta - Just works! \o/ It uploads files without problem.
Owncloud beta - at first. When uploading straight from gallery app I was getting errors, but after selecting a file via "Upload" it also did upload the file.

@neuhausjulian @team-alpeinsoft could you guys check and confirm?

[muppeth]

eeeh scratch that.. So much time has passed I forgot it was download that was the issue and not upload.

There is still problem downloading. :(

What I also noticed is that notes gives the same error.

[team-alpeinsoft]

Hi!
@muppeth We reinstall from scratch owncloud 9.1 and problems have disappeared.

[neuhausjulian]

I will try the same as solution. After reinstall I will post my result.

[muppeth]

I found the problem, though I don't know why is it happening.

Basically removing mail attribute form "Email Field" in LDAP settings solves the problem. I don't know why is that happening.

The down side is that by leaving it blank, new users email fields are empty in their personal setting, meaning they won't receive notifications.
Anyone has thoughts on this one?

[muppeth]

@team-alpeinsoft does your current setting have "Email field" filled in?

[PVince81]

Hmmmm, there was a known issue with the LDAP email field. Basically every time the user logs in in OC it would get the email value from LDAP and then set it to the local user, and that itself would trigger some change events even when the email did not change. Maybe these change events were causing additional side effects. See owncloud/core#25553

Please update to OC 9.0.5 or 9.1.1 and see if the problem is solved there.

[muppeth]

You are probably right. I'm still running owncloud 9.01. Yesterday night
I've updated my test instance to nextcloud 10.01 and the issue is gone,
and it was. I will test it with up to date owncloud as well tonight to
see if it helps.

On 10/04/2016 09:50 AM, Vincent Petry wrote:

Hmmmm, there was a known issue with the LDAP email field. Basically
every time the user logs in in OC it would get the email value from
LDAP and then set it to the local user, and that itself would trigger
some change events even when the email did not change. Maybe these
change events were causing additional side effects. See
owncloud/core#25553 owncloud/core#25553

Please update to OC 9.0.5 or 9.1.1 and see if the problem is solved there.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#701 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIFoPqRcVD_6hXYxjjLQBkQig_OEeomCks5qwgU5gaJpZM4IzfLn.

[neuhausjulian]

Hi guys, I updated to owncloud 9.1.1-1.2 and the problem is also gone.

The problem in the android-mobile client disappeared immediately after update

[team-alpeinsoft]

Hi! We reinstall owncloud and can`t diagnose problem.
@muppeth: may be you?

[muppeth]

I'm quite certain it was a bug in LDAP settings, where "Email Attribute" was the root cause. I haven't test it on latest owncloud (didnt have time), but I did on latest nextcloud, which should be quite similar in this regard

[team-alpeinsoft]

May be other can repeat error?

[muppeth]

@team-alpeinsoft I don't think it's an issue anymore when using latest version of owncloud/nextcloud.

I did not install from scratch but found the root cause and when upgrading to latest nextcloud (form owncloud) the issue was solved. I assume when updating the instance to the latest owncloud the result would be the same. They are using the same LDAP auth plugin.