[File Firewall] If you deleted the rules is not deleted on the Desktop

[Dianafg76]

Steps to reproduce

1. If you have two rules with File Firewall (related [File Firewall] If i have two rules on the Desktop is not working very well #4070) for example
2. Go to the Server Web
3. Deleted the rules

Expected behaviour

The rule should disappear

Actual behaviour

When sync is complete, the rules remain on the Desktop

Server configuration

Desktop v ownCloud-2.1.0.2851-nightly20151104.pkg
Server v {"installed":true,"maintenance":false,"version":"8.2.0.12","versionstring":"8.2.0","edition":"Enterprise"}

[Dianafg76]

Logs

11-04 11:18:08:211 0x7fe654b56030 csync_walker: directory: ownclouds://docker.oc.solidgear.es:53412/remote.php/webdav/new folder [file_id=00000094ocrd8w867cxx]
11-04 11:18:08:211 0x7fe654b56030 _csync_detect_update: Database entry found, compare: 1446632242 <-> 1446632242, etag: 5639db3283a28 <-> 5639db3283a28, inode: 0 <-> 4676322, size: 0 <-> 0, perms: RDNVCK <-> RDNVCK, ignore: 0
11-04 11:18:08:211 0x7fe654b56030 _csync_detect_update: Reading from database: new folder
11-04 11:18:08:211 0x7fe654b56030 _csync_detect_update: file: new folder, instruction: INSTRUCTION_NONE <<=
11-04 11:18:08:211 0x7fe654b56030 csync_statedb_get_below_path: 0 entries read below path new folder from db.
11-04 11:18:08:211 0x7fe654b56030 OCC::DiscoveryJob::remote_vio_closedir_hook: static void OCC::DiscoveryJob::remote_vio_closedir_hook(csync_vio_handle_t *, void *) OCC::DiscoveryJob(0x7fe654c986b0) ""
11-04 11:18:08:211 0x7fe654b56030 csync_ftw:  <= Closing walk for ownclouds://docker.oc.solidgear.es:53412/remote.php/webdav with read_from_db 0
11-04 11:18:08:211 0x7fe654b56030 csync_update: Update detection for remote replica took 0.40 seconds walking 8 files.
11-04 11:18:08:211 0x7fe654b56030 csync_statedb_close: sqlite3_close=0
11-04 11:18:08:215 0x7fe652722210 OCC::SyncEngine::slotDiscoveryJobFinished: <<#### Discovery end ####################################################  411
11-04 11:18:08:216 0x7fe652722210 csync_statedb_load: sqlite3 version "3.9.1"
11-04 11:18:08:216 0x7fe652722210 csync_statedb_load: Success
11-04 11:18:08:216 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client file: Documents/Example.odt
11-04 11:18:08:216 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client dir:  Photos
11-04 11:18:08:216 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client dir:  new folder
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client file: Photos/Squirrel.jpg
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client file: Photos/Paris.jpg
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NEW      client file: 8.1.txt
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client file: Photos/San Francisco.jpg
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client file: IMG_0007prename.MOV
11-04 11:18:08:217 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     client dir:  Documents
11-04 11:18:08:218 0x7fe652722210 csync_reconcile: Reconciliation for local replica took 0.00 seconds visiting 9 files.
11-04 11:18:08:218 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server file: Documents/Example.odt
11-04 11:18:08:218 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server dir:  Photos
11-04 11:18:08:218 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server dir:  new folder
11-04 11:18:08:218 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server file: Photos/Squirrel.jpg
11-04 11:18:08:219 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server file: Photos/Paris.jpg
11-04 11:18:08:219 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server file: Photos/San Francisco.jpg
11-04 11:18:08:219 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server file: IMG_0007prename.MOV
11-04 11:18:08:219 0x7fe652722210 _csync_merge_algorithm_visitor: INSTRUCTION_NONE     server dir:  Documents
11-04 11:18:08:219 0x7fe652722210 csync_reconcile: Reconciliation for remote replica took 0.00 seconds visiting 8 files.
11-04 11:18:08:219 0x7fe652722210 csync_statedb_close: sqlite3_close=0
11-04 11:18:08:220 0x7fe652722210 OCC::SyncEngine::slotDiscoveryJobFinished: <<#### Reconcile end ####################################################  415
11-04 11:18:08:220 0x7fe652722210 OCC::SyncEngine::checkErrorBlacklisting: Item is on blacklist:  "8.1.txt" retries: 1 for another 86308 s
11-04 11:18:08:220 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:221 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:221 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:222 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:223 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:223 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:SYNC:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:224 0x7fe652722210 OCC::SyncEngine::slotDiscoveryJobFinished: Permissions of the root folder:  "RDNVCK"
11-04 11:18:08:224 0x7fe652722210 OCC::SyncJournalDb::commitInternal: void OCC::SyncJournalDb::commitInternal(const QString &, bool) Transaction commit  "post treewalk" and starting new transaction
11-04 11:18:08:225 0x7fe652722210 OCC::SyncJournalDb::commitInternal: void OCC::SyncJournalDb::commitInternal(const QString &, bool) Transaction commit  "post stale entry removal" and starting new transaction
11-04 11:18:08:225 0x7fe652722210 OCC::OwncloudPropagator::start: Using QNAM/HTTP parallel code path
11-04 11:18:08:225 0x7fe652722210 OCC::SyncEngine::slotDiscoveryJobFinished: <<#### Post-Reconcile end ####################################################  421
11-04 11:18:08:226 0x7fe652722210 OCC::Folder::slotSyncStarted: #### Propagation start #################################################### >>
11-04 11:18:08:226 0x7fe652722210 OCC::SocketApi::slotUpdateFolderView: Not sending UPDATE_VIEW for "ownCloud4" because status() is 3
11-04 11:18:08:227 0x7fe652722210 OCC::ownCloudGui::slotComputeOverallSyncStatus: Folder in overallStatus Message:  OCC::Folder(0x7fe654d85540)  with name  "ownCloud"
11-04 11:18:08:227 0x7fe652722210 OCC::ownCloudGui::slotComputeOverallSyncStatus: Folder in overallStatus Message:  OCC::Folder(0x7fe6551736c0)  with name  "ownCloud1"
11-04 11:18:08:227 0x7fe652722210 OCC::ownCloudGui::slotComputeOverallSyncStatus: Folder in overallStatus Message:  OCC::Folder(0x7fe655165500)  with name  "ownCloud2"
11-04 11:18:08:227 0x7fe652722210 OCC::ownCloudGui::slotComputeOverallSyncStatus: Folder in overallStatus Message:  OCC::Folder(0x7fe654da0e10)  with name  "ownCloud3"
11-04 11:18:08:227 0x7fe652722210 OCC::ownCloudGui::slotComputeOverallSyncStatus: Folder in overallStatus Message:  OCC::Folder(0x7fe655253750)  with name  "ownCloud4"
11-04 11:18:08:228 0x7fe652722210 OCC::ownCloudGui::slotSyncStateChange: Sync state changed for folder  "ownCloud4" :  "Sync Running"
11-04 11:18:08:228 0x7fe652722210 OCC::SyncEngine::slotItemCompleted: void OCC::SyncEngine::slotItemCompleted(const OCC::SyncFileItem &, const OCC::PropagatorJob &) "8.1.txt" INSTRUCTION_ERROR 6 "The item is not synced because of previous errors: Error downloading https://docker.oc.solidgear.es:53412/remote.php/webdav/8.1.txt - server replied: Forbidden (Access to this resource has been forbidden by a file firewall rule.)"
11-04 11:18:08:229 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:230 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:230 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:231 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:231 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"
11-04 11:18:08:232 0x7fe652722210 OCC::SocketApi::sendMessage: SocketApi:  Sending message:  "STATUS:ERROR:/Users/Diana/FileFirewalluser2/8.1.txt"

[guruz]

@nickvergessen @ckamm @ogoffart @dragotin Maybe the file firewall's actions should not be blacklisted?

@Dianafg76 If it says The item is not synced because of previous errors it's our client error black list

[ogoffart]

@guruz

Maybe the file firewall's actions should not be blacklisted?

This is the whole point of the blacklist.
This is not the first time i heard this comment that stuff should not be blacklisted. If the blacklist is causing problem, should we totaly remove it?

[Dianafg76]

related #3490

[ckamm]

The blacklist is mostly a "don't annoy users" thing. Syncing would mostly work better without it, but we'd also bother users more.

Blacklisting file firewall errors means that, if the firewall rules change, it can take up to a day (max blacklist time) for a file that previously failed due to the firewall to be retried.

Maybe it'd be beneficial to just mute the error instead of not even trying files that are blacklisted? The main drawback would be retrying costly uploads more often.

[ogoffart]

The goal is to also avoid unecessary traffic.
Otherwise we would try to re-upload this file again and again every sync.

In practice I would not expect the rules to be changed that often anyway.
The user could also manually click retry.

But maybe 1 day is too much and we should limit to a couple of hours?

[nickvergessen]

Well, since the firewall has an option to block timebased, hourly sounds way better than 1 day.

[guruz]

Note that @butonic for other reasons has proposed that the client should honour the Retry-After header. Maybe it's an option for you to emit that for a time-based block and then the client would (when implemented) handle it better.
#3932

[nickvergessen]

Well firewall will use that header in 9.0, for 8.2 that's not possible due to the architecture.
So maybe status code 403 and message contains firewall then retry after 1hour/30mins?

[ckamm]

@nickvergessen Is firewall guaranteed to be part of the message for firewall related blocks?

[ckamm]

@Dianafg76 Note that the symptoms will still be the same. But the client will now retry a previous firewall-related error after 1h or less (instead of 1 day).

[nickvergessen]

Response as of 9.0 is:

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:o="http://owncloud.org/ns">
  <s:exception>OCA\DAV\Connector\Sabre\Exception\Forbidden</s:exception>
  <s:message>Access to this resource has been forbidden by a file firewall rule.</s:message>
  <o:retry xmlns:o="o:">true</o:retry>
  <o:reason xmlns:o="o:">Access to this resource has been forbidden by a file firewall rule.</o:reason>
</d:error>

In 8.2 I think it's something like:

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:o="http://owncloud.org/ns">
  <s:exception>OCP\SabrePluginException</s:exception>
  <s:message>Access to this resource has been forbidden by a file firewall rule.</s:message>
</d:error>

So yes the message should contain the f*** word

[guruz]

@nickvergessen Is firewall guaranteed to be part of the message for firewall related blocks?

Note that the server is i18n'ed

[ckamm]

@guruz @nickvergessen Then it won't work. I think it would be safe to do this for 403s generally.

[nickvergessen]

Currently 403 is only used for this case, so yes sounds save

[Dianafg76]

I tested this issue, and is working ok
Desktop v ownCloud-2.1.0.2916-nightly20151127.pkg
Server v {"installed":true,"maintenance":false,"version":"8.2.1.4","versionstring":"8.2.1","edition":"Enterprise"}
Closed