Add cloud-safe configuration option

[mrjanse]

We have an encrypted folder on an OwnCloud server which is shared by 2 users. If user1 moves a file from a subfolder to another, user2 gets an I/O error when accessing this file. User1 has no problem with this file. If user1 copies this file, there is no error.Without encfs there was no problem when moving files.

Regards, Jan

[rfjakob]

So the encrypted folder is synchronised via owncloud to two computers?

[mrjanse]

The folder is synchronized via owncloud to different computers - to 2 computers of user1 and 1 computer of user2.

[johseg]

I have a similar problem with dropbox. Not two different users, but two machines that share a encfs encrypted folder. If I change the name of a directory sometimes all the files in there get corrupted. I see these messages:
MAC comparison failure in block 0
withFileNode: error caught in read: MAC comparison failure, refusing to read

[rfjakob]

Are they corrupted on both ends? If not, can you check the md5sum of the encrypted files on both sides?

[johseg]

no, they are only corrupt on one end
system1:
md5sum storage/aerzte/2016-02-mrt-hws/syngo_fv/help/lang09/fv_help.chm
7274e5f6e711188389a97b47fdaecb67 storage/aerzte/2016-02-mrt-hws/syngo_fv/help/lang09/fv_help.chm
system2:
md5sum storage/aerzte/2016-02-mrt-hws/syngo_fv/help/lang09/fv_help.chm
md5sum: storage/aerzte/2016-02-mrt-hws/syngo_fv/help/lang09/fv_help.chm: Eingabe-/Ausgabefehler

[rfjakob]

When you run "ls -li" on those files, you get the inode number in the first column, lets say it is 12345. Then, using "find . -inum 12345" in the encrypted directory, you get the encrypted file. Please run md5sum on it on both sides. If it is different, there is a bug in owncloud's sync algorithm that was caught by encfs.

[johseg]

The have different md5 hashes. In my case it's not owncloud but dropbox.

[rfjakob]

Ok, a bug in Dropbox seems less likely.

Can you post the encrypted version of fv_help.chm from both sides? I'd like to take a look at the difference.

[johseg]

encfs.zip

So I xxd'ed them, only the first eight bytes differ. Some IV problem?

[rfjakob]

$ diff -u <(hexdump -C /tmp/kINmTAvz2MsSv9N9hUx,tIiY_local) <(hexdump -C /tmp/kINmTAvz2MsSv9N9hUx,tIiY_remote)
--- /dev/fd/63  2016-05-30 22:58:48.922385221 +0200
+++ /dev/fd/62  2016-05-30 22:58:48.923385225 +0200
@@ -1,4 +1,4 @@
-00000000  03 6a 80 d7 89 df 01 fe  15 a0 49 90 f8 e0 6d 8c  |.j........I...m.|
+00000000  97 23 41 d1 d5 79 7f ff  15 a0 49 90 f8 e0 6d 8c  |.#A..y....I...m.|
 00000010  5b a3 c1 5e 1b 79 4e be  a1 e4 c0 3c 40 97 02 bc  |[..^.yN....<@...|
 00000020  de 7e d5 d9 73 37 ce b0  bb 6f 12 b7 d8 c3 18 d0  |.~..s7...o......|
 00000030  e3 26 2c 8b ff 94 0a ce  92 7c f5 d1 e7 a0 fa 93  |.&,......|......|

[rfjakob]

Ok, I think I understand what is going on. You must be running paranoia mode (or you have enabled "filename to IV header chaining" manually). I'll quote the help text first:

Enable filename to IV header chaining?
This makes file data encoding dependent on the complete file path.
If a file is renamed, it will not decode sucessfully unless it
was renamed by encfs with the proper key.

So with that enabled, a rename modifies the file header. But a rename should not change the timestamp on the file, so encfs sets it back to the original value. And I guess this is why Dropbox and OwnCloud to not pick up the change.

I bet if you run touch fv_help.chm on the working system so the file gets a new timestamp, Dropbox will copy it again and the problem will go away.

[johseg]

yes, that worked perfectly. So is the solution to this not to use "filename to IV header chaining" when using dropbox etc? Or maybe add another option to set a new timestamp once a file is renamed?

[rprieto]

Avoiding filename to IV header chaining is a great tip for cloud sync. It seems that it might be a common pitfall for new users (got me too). What do you think of a new option when creating a file system?

Please choose from one of the following options:
     enter "x" for expert configuration mode,
     enter "p" for pre-configured paranoia mode,
     enter "c" for cloud-friendly mode,
     anything else, or an empty line will select standard mode.
    ?> 

[vgough]

Thanks for tracking this down. I've changed the title of this issue - a safe setting for cloud seems like a great idea @rprieto.

[johseg]

The "cloud-friendly" options sounds like a great idea. A lot of people use encfs for securing their files before they get sent online, and that would be very helpful and could enable other optimizations later on

[xenithorb]

Sorry to resurrect this issue but for those of us less capable of understanding what's going on here:

What if anything can be done if I've run into this for a few files that I want to recover?

Both synced copies in this case seem to be "bad" in the same way - i.e. the file with the wrong IV header propagated. Likely what happened is that I did a full re-sync some time ago and never noticed the issue with these files as one can remain completely oblivious until they go to access the files in question

Is this equivalent to data loss?

[rfjakob]

Hmm. If you remember from what to what these files were renamed, it may be possible to recover.

[xenithorb]

To confirm: I was able to recover a few by remembering the original names, and using encfsctl encode to rename the outside files back to the original.

The rest I couldn't and had to restore them from backups. I immediately migrated away from External IV Chaining according to the message on the README:

Use standard mode. There have been reports of a pathological interaction of paranoia mode with Dropbox' rename detection. The problem seems to be with External IV chaining, which is not active in standard mode.

However this is what my repo's info looks like according to encfsctl:

Version 6 configuration; created by EncFS 1.9.5 (revision 20100713)
Filesystem cipher: "ssl/aes", version 3:0:0 (using 3:0:2)
Filename encoding: "nameio/stream", version 2:1:0 (using 2:1:2)
Key Size: 256 bits
Using PBKDF2, with 321427 iterations
Salt Size: 160 bits
Block Size: 1024 bytes, including 8 byte MAC header
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.                          <-----
File holes passed through to ciphertext.

Is it specifically External IV chaining or filename chaining? This ticket seems to suggest filename chaining, please advise :)

[rfjakob]

Standard mode:

$ encfsctl info foo
Version 6 configuration; created by EncFS 1.9.5 (revision 20100713)
Filesystem cipher: "ssl/aes", version 3:0:0 (using 3:0:2)
Filename encoding: "nameio/block", version 4:0:0 (using 4:0:2)
Key Size: 192 bits
Using PBKDF2, with 399930 iterations
Salt Size: 160 bits
Block Size: 1024 bytes
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File holes passed through to ciphertext.

Paranoia mode:

$ encfsctl info bar
Version 6 configuration; created by EncFS 1.9.5 (revision 20100713)
Filesystem cipher: "ssl/aes", version 3:0:0 (using 3:0:2)
Filename encoding: "nameio/block", version 4:0:0 (using 4:0:2)
Key Size: 256 bits
Using PBKDF2, with 1619163 iterations
Salt Size: 160 bits
Block Size: 1024 bytes, including 8 byte MAC header
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File data IV is chained to filename IV.
File holes passed through to ciphertext.

So "External IV chaining" is called File data IV is chained to filename IV in this listing. And you don't seem to have enabled, which is weird. Because then you should have neither experienced the problems, nor should the recovery by renaming to the old file have done anything.

[benrubson]

@rfjakob, encfsctl info @xenithorb just gave is the one from his new repo :)
So all is OK, and the old repo certainly had External IV Chaining enabled.
@xenithorb you can find Filename Initialization Vector Chaining (which you have enabled) meaning in the man page.

[xenithorb]

Sorry for the confusion, but my encfsctl snippet is indeed custom options and from my new repo. (@benrubson is correct).

Let's take a step back for a moment and I want to reiterate what I was really getting after earlier.

---

So, it seems that in this ticket there seems to be some confusion, and for my own sake I want to be completely clear that it is ok to have Filenames encoded using IV chaining mode. - "Filename Initialization Vector Chaining" enabled but not File data IV is chained to filename IV. - "External IV Chaining" - Is that correct? My settings from #141 (comment) are "cloud safe?"

[benrubson]

Correct !
This is what is said here in this ticket :)