Synchronization progress no longer reported / full sync doesn't work

[twwn]

Reproduction Steps

1. Upload note edits to AnkiWeb (it gets esp. obvious when there's lots of them, like to a 1000 ones reported worth 100s of kB in Anki)
2. Pull them onto your AnkiDroid

Expected Result

The synchronization popup reports received & sent data.

Actual Result

Since a couple of weeks (can't recall if it occurred pre-2.1a), the received figure stays flat at 0, while the sent one increments only by 1 kB roughly every 5s. This for any changes, including ones that desktop Anki reported as MBs in size. But they do succeed normally.

Reporting of full sync download progress still works fine, though (since) today I only managed to complete such for one of my devices (on 6th try). On the other device AnkiDroid (both a65) still hasn't managed to start the download even after a dozen attempts, instead apparently indefinitely waiting/remaining at 0/0 (also: rather than offer to abort, the back button does nothing in this state, though the app can still be closed fine via Android's app overview).

[david-allison]

Hi @rathsky,

Thanks for the report!

Could you post your ACRA ID if you have crash reporting enabled:

1. Open the navigation drawer by tapping the button on the top left of the screen
2. Tap "settings"
3. Tap "advanced"
4. Tap "about AnkiDroid" at the bottom
5. Tap the "copy debug info" button at the bottom (which copies "debug info" to the clipboard)

Thanks,
David

[mikehardy]

Sync success or not is not up to us unfortunately, it's an external network or ankiweb server issue. The client code works though except...download reporting perhaps.

I have seen what you see and have been meaning to log it so I'm happy you did so. It started happening when I had to switch us to mandatory TLS1.2 or higher #5623 / #5658

I think what is happening is this:

https://github.com/mikehardy/Anki-Android/blob/a0957344677d3b13d7fbc9aa661db94c12cc3e6f/AnkiDroid/src/main/java/com/ichi2/libanki/sync/Tls12SocketFactory.java#L145-L154

        // Note if progress tracking needs to be more granular than default OkHTTP buffer, do this:
//        try {
//            s.setSendBufferSize(16 * 1024);
//            // We will only know if this is a problem if people complain about progress bar going to 100%
//            // on small transfers (indicating buffer ate all contents) before transfer finishes (because buffer is still flushing)
//            // Important to say that this can slow things down dramatically though so needs tuning. With 16kb throughput was 40kb/s
//            // By default throughput was maxing my 50Mbit line out (!)
//        } catch (SocketException se) {
//            Timber.e(se, "Unable to set socket send buffer size");
//        }

[mikehardy]

if it is not that exact chunk then it will be something intimately related - perhaps multiple little transfers not aggregating correctly or similar. I was under time pressure getting that code in at all so that our Android 4.x and 5.x users could sync at all so making it work was the criteria for merge vs getting reporting correct, though I'm sad I didn't get the reporting correct at the time too...

[mikehardy]

As an aside to @dae - I was working on something else that required a full sync from AnkiWeb just now and I noticed that full sync right now is behaving very strangely, either no throughput or very low throughput from AnkiWeb. I don't think that is an AnkiDroid problem - I think you might have something going on at the server side?

[dae]

The servers don't appear to be under strain - it may be an issue with an international link.

[twwn]

@david-allison-1

Sorry about forgetting that, it's

AnkiDroid Version = 2.10alpha66

Android Version = 10

ACRA UUID = ca4be91c-f334-4a16-837e-7ce318e2ca69

New attempts are still stuck at 0/0. Same network as the device that eventually synced (e: turning the device off & on again made no difference…).

[david-allison]

@rathsky It's you! Thanks so much for the great bug reports!!!!

You previously had a bug caused by the latest version of Anki Desktop master. It's fixed here: ankitects/anki@6e6776a. I'm unsure if this is what's affecting you above.

Ref: #5963

[mikehardy]

@david-allison-1

Sorry about forgetting that, it's

AnkiDroid Version = 2.10alpha66
Android Version = 10
ACRA UUID = ca4be91c-f334-4a16-837e-7ce318e2ca69

New attempts are still stuck at 0/0. Same network as the device that eventually synced (e: turning the device off & on again made no difference…).

Thanks for posting that, I can confirm something is wrong. 2.9.6 works, 2.10alpha66 full sync doesn't work. It's something in the last couple alphas I'm sure, I'm bisecting now. @david-allison-1 I don't think this is related to that issue, I think it's more fundamental - either a dependency update like #5885 where I bumped the OkHTTP library or database access synchronization/threading issue like #5947

[mikehardy]

Okay, it's #5947 - which has a separate / related follow-on fixup here #5972 indicating it may have some deep issue. I'm going to investigate the code looking for a real fix since it appeared solid going in, or will revert - whichever direction I'll issue alpha67 shortly

@rathsky - your testing on the alphas and good reports are gold I might not have checked full sync for longer and this could have sat for a while, thank you!

[mikehardy]

Reverted problematic commit with 44aea97 - alpha67 released just now