Skip to content
This repository has been archived by the owner on Aug 8, 2023. It is now read-only.

Slow offline downloads #10914

Closed
tobrun opened this issue Jan 12, 2018 · 8 comments
Closed

Slow offline downloads #10914

tobrun opened this issue Jan 12, 2018 · 8 comments
Labels
Android Mapbox Maps SDK for Android

Comments

@tobrun
Copy link
Member

tobrun commented Jan 12, 2018 •
edited
Loading

Capturing from internal analysis that the Android binding is slower in downloading offline regions as the iOS binding. While there are improvements to be made to offline downloads in general, there shouldn't be a big difference in performance between the 2 platforms. This ticket tries to identify where the bottleneck is coming from.

Platform Time Size Speed
iOS 0:32 49.2 MB 84.81 MB/s
Android 1:25 45.57 MB 118 MB/s

A couple of possible hypothesis:

  • for each tile download, core will need to cross the JNI boundary and ask HttpRequest.java to download a tile, when that http request has finished the result is passed through JNI back to core again. It's possible that this part is a bottleneck. If so we would need to rewrite the interaction to take a batched approach to offline downloads.
  • for every region we are downloading, we are posting a large amount of status update runnables to the main thread. It's possible that we are sending a too large amount that results in trashing the main thread and adding CPU overhead (which would explain the slower download).
  • move from enqueue to execute (HttpRequest itself is already ran on BG, no need to schedule the request).
  • Validate if we are maximising the amount of simultaneous requests
  • TBA

cc @mapbox/maps-android
@tobrun tobrun added the Android Mapbox Maps SDK for Android label Jan 12, 2018
@tobrun
Copy link
Member Author

tobrun commented Jan 12, 2018

Limit posting runnables to the main thread

Offline region definition:

    LatLng southWest = new LatLng(64.149385, -22.036754);
    LatLng northEast = new LatLng(64.156640, -22.020452);
    LatLngBounds latLngBounds = new LatLngBounds.Builder().include(southWest).include(northEast).build();
    OfflineTilePyramidRegionDefinition definition = new OfflineTilePyramidRegionDefinition(Style.MAPBOX_STREETS, latLngBounds, 17, 18, densityDpi);

Test results

Type test Time Size Speed
Baseline 0:41 73.30 MB 109.86 MB/s
Limited 0:44 73,30 MB 128.11 MB/s

The messaging mechanism doesn't seem to be the bottleneck. This can also be confirmed when looking at the CPU traces. We are only seeing the CPU maxing out at 25% of its capacity with the baseline test.

screenshot from 2018-01-12 11-34-05

@tobrun
Copy link
Member Author

tobrun commented Jan 12, 2018

Batch requests

Been looking into adapting the HttpRequest.java interaction with core to integrate a concept of batching requests and responses together. This approach doesn't seem feasible without changing any core code due to the HTTPFileSource abstraction that spawns an AsyncTask for each tile request. This AsyncTask is the equivalent of HTTPRequest which matches our HttpRequest.java.

@zugaldia
Copy link
Member

Related ticket: #10252.

@dylan-chong
Copy link

dylan-chong commented Jan 14, 2018 •
edited
Loading

I've been very bad mapview performance while a ofline area is downloading. It can take 15+ seconds for a mapview to load when you scroll to some unexplored area while the download is progressing.

This is related issue so i commented this here rather than making a new issue. Perhaps the offlien area download is preventing mapview tile donwloads from progressing

Replicated on mapbox android 5.2.1, android 8.1.0

@tobrun
Copy link
Member Author

tobrun commented Jan 15, 2018

Been looking at optimising how we perform web requests through HttpRequest. One attempt on making that integration faster is by changing our call#enqueue to a call#execute. This mitigates any additional time we spend waiting in the queue before being able to execute. After running benchmarks it seems that enqueue is more optimised as the synchronous execute.

@tobrun
Copy link
Member Author

tobrun commented Jan 15, 2018

Been looking at if we are maximising the amount of simultaneous requests from #10567:

I'm seeing the following output when logging out simultaneous request value:

01-15 14:56:41.241 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 1
01-15 14:56:41.405 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 1
01-15 14:56:42.081 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:42.082 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:42.094 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:42.106 20096-20236/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:42.114 20096-20231/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:42.127 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:42.255 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:42.260 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:42.263 20096-20237/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:42.265 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:42.278 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:42.279 20096-20220/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:42.286 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:42.295 20096-20232/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:42.375 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:42.420 20096-20222/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:42.425 20096-20230/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:42.469 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:42.628 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:42.655 20096-20227/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:42.657 20096-20231/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:42.733 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:42.762 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:42.983 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:43.125 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:43.153 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:43.278 20096-20227/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:43.421 20096-20226/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 1
01-15 14:56:44.068 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 1
01-15 14:56:44.314 20096-20227/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 1
01-15 14:56:44.398 20096-20226/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:44.415 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:44.441 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 16
01-15 14:56:44.446 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 15
01-15 14:56:44.460 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 14
01-15 14:56:44.473 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 13
01-15 14:56:44.498 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:44.601 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:44.616 20096-20220/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.618 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:44.657 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:44.665 20096-20232/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:44.718 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.720 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:44.759 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:44.770 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:44.845 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:44.859 20096-20220/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.867 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:44.883 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.904 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:44.905 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.927 20096-20218/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:44.933 20096-20232/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:44.938 20096-20230/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:44.950 20096-20226/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:44.955 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:44.963 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:44.994 20096-20237/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:44.994 20096-20220/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:44.997 20096-20231/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:45.004 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:45.010 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:45.074 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:45.126 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:45.167 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:45.524 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:45.527 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:45.545 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:45.556 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 13
01-15 14:56:45.557 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:45.570 20096-20231/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:45.573 20096-20237/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:45.583 20096-20222/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:45.665 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 16
01-15 14:56:45.686 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 15
01-15 14:56:45.721 20096-20230/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 14
01-15 14:56:45.753 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 14
01-15 14:56:45.772 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 13
01-15 14:56:45.785 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 14
01-15 14:56:45.810 20096-20226/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 13
01-15 14:56:45.821 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:45.854 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:45.864 20096-20236/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:45.899 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:45.919 20096-20227/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:45.920 20096-20231/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 11
01-15 14:56:45.922 20096-20230/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 12
01-15 14:56:45.929 20096-20229/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:45.972 20096-20232/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:46.052 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:46.051 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 10
01-15 14:56:46.066 20096-20222/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:46.079 20096-20220/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.088 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:46.091 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.122 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:46.139 20096-20237/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:46.178 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:46.273 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3
01-15 14:56:46.294 20096-20218/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 2
01-15 14:56:46.691 20096-20218/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.754 20096-20223/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:46.760 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.764 20096-20233/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:46.830 20096-20225/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 9
01-15 14:56:46.842 20096-20227/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 8
01-15 14:56:46.877 20096-20234/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.877 20096-20237/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.911 20096-20228/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 7
01-15 14:56:46.936 20096-20235/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 6
01-15 14:56:46.965 20096-20216/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 5
01-15 14:56:47.023 20096-20224/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 4
01-15 14:56:47.048 20096-20221/com.mapbox.mapboxsdk.testapp E/HTTPRequest: onResponse: 3

Ideally the value above would alternate between 19-20, it seems we aren't maximising the amount of parallel downloads possible.

@tobrun
Copy link
Member Author

tobrun commented Jan 18, 2018

Going to close this in favour of #10252. No immediate optimisations found for now.

@tobrun tobrun closed this as completed Jan 18, 2018
@dylan-chong
Copy link

@tobrun Did you find anything out regarding my comment above?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Android Mapbox Maps SDK for Android
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zugaldia @tobrun @dylan-chong