-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow GeoJSON to be passed as stringify JSON. #2001
Conversation
Awesome idea! Thanks @dcervelli! Can you add some unit tests and fix the lint errors showing up on CircleCI (or locally by running Thoughts @mourner? |
f59bb1c
to
a4510a5
Compare
Fixed up the lint errors and added a test. Also, FWIW, I think the test in geojson_source.test.js:146 doesn't do anything. As far as I could tell |
ref #762 |
I'm surprised it's 6 times faster but 👍 on the change. @dcervelli just wondering, are you changing all the features in the GeoJSON real-time that makes it impossible to load through an url? |
Yeah, our |
Merged as f9afff0 Thanks again @dcervelli! I really appreciate the PR! |
Hi all, sorry I didn't chime in here earlier. @dcervelli, really appreciate the contribution but I have a concern about this approach. I'm not crazy about widening the Passing stringified JSON might let you avoid one string ⇢ object ⇢ string roundtrip. But does it really, in practice? That would mean you never need the GeoJSON in object format on the main thread, which if that's the case, you're probably better off passing a URL anyway. Thoughts? |
@mourner Yes, our use case is very much around mapbox-gl-js being the view layer on a very dynamic app where feature sets and styling will be changing quite often. (The same use case is driving this PR: mapbox/feature-filter#9). A couple of years ago we hosted some Mapbox folks to show them some of our geospatial workflows and this is similar to those. @lucaswoj You're very welcome! Thanks for the quick merge. @jfirebaugh Maybe I took the easy way out here but this seemed like the safest change. I could imagine there are GeoJSONs where it's faster to go natively/doesn't matter. Basically I didn't want to have to convince myself that stringify-ing was always better/correct. In particular JSON.stringify is fundamentally different than Also, to your second point. Honestly we don't want to store GeoJSON in object format but it's the path of least resistance to getting geo data from our app into mapbox-gl-js. The surface area of the change required to add a new source that super-efficiently transfers (i.e. transferables) a very memory-efficient geo data format is pretty big. If you guys are interested in taking such a change, there's a good chance we'd be up for developing it. |
That actually led me to a thought that https://github.com/mapbox/geobuf fits perfectly for the use case, because its encoding/decoding is about the same or faster than JSON.parse/stringify, AND it can be transferred immediately as Uint8Array. This would also make hacks like @lucaswoj @jfirebaugh @ansis thoughts? This would be really awesome application of geobuf, and I can make a quick proof of concept. P.S. Thinking about this further, a stringified JSON can also be represented as a typed array to make it transferrable in theory. Should we try to do it? |
Sounds like there's more discussion to be had here. Lets revert this commit for the time being and move all discussion to #1504 |
reverted in b5c52e1 |
When are you guys planning to release this thing? My UI is blocking for 4 seconds right now with 90k lines in my GeoJSON object. 😬 |
@mourner In master |
Mitigation for #1504.
With a complex GeoJSON object (say 20k features with 100k coordinates) the
postMessage
call to the worker takes several seconds on the UI thread. By manuallyJSON.stringify
ing the GeoJSON object andJSON.parse
ing on the worker side we're seeing a 6x perf improvement.Ideally we would figure out a way to use transferables or similar (there is still a small UI freeze in our usage), but this is a step in the right direction.