-
Notifications
You must be signed in to change notification settings - Fork 665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
After sync aborts, resume in a more graceful manner #5414
Comments
I do not think that this will work. If a discovery run breaks and later continues, how would the system be able to know how long this "later" would be, and if the filetree has not changed meanwhile? That is a problem that can not be solved. A way to approach this kind of problems is to speed up the discovery phase, as discussed in #4117 for example. And the best option probably would be to constantly keep the file tree in memory and only apply changes through the local file watcher or the remote ETag changes and process the differences from there. But that is probably far out. @ogoffart @ckamm |
As I understand it, the issue of files changing after discovery would also arise during a discovery run that just takes a long time to complete:
Would the change to Speeding up discovery through parallelization, as you mention, might mitigate this issue but probably wouldn't eliminate it. As for how long “later” would be—that is always going to be an arbitrary decision. I’d resume rather than start over if the duration of the interruption is inferior to the duration of the sync run. If you want to rework the whole sync logic, the cleanest way is probably to keep a journal of changes on the server and on each client, and remember the last journal entry that has been synced.
|
Update: apparently the “change files during sync” scenario has issues of that very same kind, which are potentially even more severe (modifications made during sync get reset), see #5437. That needs to be tackled with priority, but maybe this one can be considered in the solution. |
Would be improved by #2976 |
When sync aborts, e.g. due to a flaky network connection or an overloaded server, discovery (of changed items) seems to start all over again.
When syncing large amounts of data in a setup that is experiencing this issue, this behavior adds a huge overhead (each abort/resume causes a full discovery even though very little, if anything at all, may have changed since the last discovery).
Suggestion:
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The text was updated successfully, but these errors were encountered: