suggestion: resource Loader with a callback mechanism to provide locally served files.

[Yermo]

Motivation

In my use case, I need to be able to render a map on a device in a completely offline context. As such, json styles, mbtiles, etc all have to be served from localstorage on the phone.

mapbox-gl-js calls ajax methods directly whenever it needs to pull a resource making the assumption the resource will be remotely hosted.

I noticed the semi-complete "addSourceType" features and read the discussion. I don't want to create a new source type. I merely want to pull the 'vector' source from the local phone.

This may be related to issue #4740 Resource Transforms.

Design Alternatives

While adding new source types to address this is an option, abstracting the loading of resources would avoid some code duplication and be much simpler.

I just want to be able to:

1. host the json style file locally and refer to it using a "file:///" url in map constructor options.
2. reference all resources specified in the JSON files using a file:/// prefix.
3. host a small mbtiles file locally, also specified via a file:/// url.

Design

As a proof of concept, I have implemented a "src/util/resourceLoader,js" shim that mirrors the interface in src/util/ajax.js and replaced all ajax calls in the code with the corresponding calls to resourceLoader.

(eg. ajax.getJSON() becomes resourceLoader.getJSON()).

There is a map level configuration option for a user provided object that expects two methods: getJSON() and getArrayBuffer(). In my case I'm using Cordova File and Sqlite plugins.

In src/utils/resourceLoader I check the URL. If it's a remote URL I pass on the request to the ajax handler, business as usual. If the URL is not prefixed by http:// https:// or mapbox://, I forward the request to the caller supplied resourceLoader.

A complication is that Cordova plugins can only be called from the main thread so I've modified the interface to LoadVectorData to include the actor so I can send a message to the main thread. I've modified style.js to receive those calls and forward them to the user provided loader.

Doing it this way would provide a great deal of flexibility for developers to host offline maps or pull data from various sources without having to maintain separate source implementations.

Mock-Up

Concepts

Implementation

Since this is just a proof of concept and needs some cleanup work, I didn't want to send a pull request yet but I have put up a fork with this work.

https://github.com/Yermo/mapbox-gl-js/tree/resource-loader

At present, there is an ugly hack in map.js to make the reference for the user supplied loader available in src/util/resourceLoader.js.

Ideally, I'd like to get something like this merged into the code. With a little guidance, I am happy to improve upon this work and package it up for a pull request.

[anandthakker]

Thanks for this thoughtful proposal, @Yermo!

A complication is that Cordova plugins can only be called from the main thread

This is a challenge in the normal browser environment, too, and probably the biggest blocker to a complete/successful design for a feature like this. There's not a straightforward way to provide a user-supplied object to the web workers with its methods intact. It's something we're thinking about over in #4740, too (which does overlap with this proposal, as you mentioned) cc @asheemmamoowala.

[Yermo]

Thank you for taking the time to consider my suggestions!

In my implementation, the Web Worker passes the buck back to the main thread when a user supplied method needs to be invoked. This is determined by inspecting the scheme of the resource URL. It seems to be working pretty well. Obviously, one wouldn't want to do anything super involved in such a user provided callback, but given the limitations we face it seems a reasonable compromise to me.

[asheemmamoowala]

@Yermo can you confirm if #4740(comment) addresses your use case as well ?

[Yermo]

See my comment on #4740

[Yermo]

@asheemmamoowala Would it be worthwhile for me to put together a pull request? I've just re-applied my localResourceLoader changes to your 4740-per-map-transform branch and gotten it working.

[asheemmamoowala]

@Yermo - thanks for updating your branch. I took a look and think that it would be better to wait for the Custom Sources project to build in the support needed to put in an PR. We are just beginning work to design the Custom Sources API, we'll update here with tickets as we get further along.

[Yermo]

@asheemmamoowala Thank you. I'll be interested to see the design.

[itbeyond]

@Yermo Did this ever get sorted - I am looking to do something very similar to what you did with local storage in an MBTiles db.

[Yermo]

@itbeyond I did get it to work but my approach was less clean than this one:

https://github.com/JpmGuides/mapbox-gl-cordova-offline

It has a few bugs notably at high zoom levels that I haven't tracked down yet.