Offline download batches

[ivovandongen]

Fixes #11217
Closes #10252

Alternative to #11235 using batching of resource update/inserts.

Somewhat slower: +/- 30 seconds on Pixel XL 2, 130 seconds on Galaxy Nexus. Higher memory consumption; had to keep the batch size relatively small to avoid running out of memory on old devices.

[kkaefer]

Instead of exposing a generic function like this, can we pass a list/collection of Resource/Response pairs and handle iteration over those, and transaction management internally? E.g. have a public putRegionResources() function that internally calls putRegionResource for every item, wrapping everything in a transaction.

[jfirebaugh]

I just discovered that you can REPLACE in a way that avoids generating a new id:

REPLACE INTO resources (url, ..., id)
VALUES                 (?1,  ..., (SELECT id FROM resources WHERE url = ?1))

This takes advantage of the fact that id is the ROWID column, which will replace a NULL value with an autogenerated value on inserts.

However, there does not appear to be a way of determining whether such a statement replaced or inserted, which putResource / putTile needs for the return value. 😢If there were, we could avoid the transaction entirely.

[jfirebaugh]

@ivovandongen Can you benchmark this approach versus https://github.com/mapbox/mapbox-gl-native/compare/no-transaction? I'm not sure if small automatic deferred transactions, or batched immediate transactions will be faster.

[ivovandongen]

@ivovandongen Can you benchmark this approach versus https://github.com/mapbox/mapbox-gl-native/compare/no-transaction? I'm not sure if small automatic deferred transactions, or batched immediate transactions will be faster.

@jfirebaugh Between 95 - 100 seconds on the Pixel XL 2. The Galaxy Nexus is still charging, but I don't think it will be much better I'm afraid.

[jfirebaugh]

I'm not clear how 95-100 seconds compares. Can you summarize the results from the three approaches (master, no-transation, offline-download-batches)?

[ivovandongen]

	Pixel 2	Galaxy Nexus
master	~90 seconds	15 minutes
no transaction	95 - 100 seconds	not tested
offline-download-batches	~30 seconds	~130 seconds
longer running transactions	~20 seconds	~130 seconds

Tested with


• Berlin region: 5195 tiles, north:52.6780473464 east:13.7603759766 south:52.3305137868 west:13.0627441406 max-zoom:15

• Warmed up cache on the edge location

[jfirebaugh]

Great -- I think that offline-download-batches is the best balance between improved performance and code complexity.

[kkaefer]

\o/

[friedbunny]

iOS changelog added in #12086.