Supercluster.hpp 0.3.0 update #12726

tobrun · 2018-08-23T14:00:19Z

This PR exposes the getChildren, getLeaves and getClusterExpansionZoom from v0.3.0 of supercluster.hpp (see mapbox/supercluster.hpp#16 for upstream PR).

Wasn't really sure about what the best approach is to expose these methods to the public API of geojson_source.cpp. For now, I introduced virtual methods that are no-op for GeoJSONVTData. Lmk if there is a cleaner way of handling this.

tobrun · 2018-08-24T09:50:35Z

Going to refactor this using the visitor pattern as this would be more clean as the virtual function approach.

kkaefer

This PR adds a few functions, but they don't seem to be used or tested anywhere?

mourner · 2018-10-12T11:15:17Z

See also mapbox/mapbox-gl-js#6829 for discussion on unit tests for this in the JS version.

tobrun · 2018-10-12T11:32:03Z

@kkaefer android integration is in #12727, I can work on updating that PR to current state and merging into here.

update: iOS in #12952

…perclusterData

alexshalamov · 2018-10-18T10:47:20Z

@kkaefer @mourner Added unit test for GeoJSONSource d4485c9 do you think it is sufficient?

Also, noticed that GeoJSONSource::Impl shares its private data as a raw pointer and GeoJSONSource doesn't check whether data is null. Fixed in fce4bff. Do you think it is required? If not, I will drop that commit.

alexshalamov · 2018-10-23T06:24:01Z

@kkaefer @tobrun could you please take a look at this PR when you have time.

tobrun

Thank you for fixing up the code and the tests look great! Since I added a part of the code, I don't feel that I'm the right person to sign off on this. @kkaefer can you take a look?

mourner

This looks great to me from the API standpoint. Can't comment on the C++ specifics though (pointer/type changes), maybe needs another pair of eyes.

mourner · 2018-10-26T10:03:11Z

test/style/source.test.cpp

+        ASSERT_NE(geojson_source, nullptr);
+        EXPECT_FALSE(geojson_source->getClusterChildren(1).empty());
+        EXPECT_FALSE(geojson_source->getClusterLeaves(1).empty());
+        EXPECT_NE(geojson_source->getClusterExpansionZoom(1), 0);


Can we test that the API returns specific values here instead of just making sure they're not default ones like for non-clustered sources?

@mourner Done. I noticed that default GeoJSONSource clustering options differ from supercluster's defaults. Should default options be synced or there was a specific reason for different default options?

Supercluster (the JS version) can be used in different contexts than GL Native (e.g. with Leaflet), so it's not a problem that we don't match default options.

GeoJSONSource::Impl shares its private data as a raw pointer, this may lead to unwanted side-effects. Moreover GeoJSONSource, used to access its Impl data without checking whether data is valid, this PR fixes both issues.

asheemmamoowala

There is a synchronization issue exposed by the new methods on mbgl::style::GeoJSONSource. These methods return content from the current data available on the main thread. Cluster node information from queryRenderedFeatures is retrieved from the data available on the render thread.

Using cluster_ids from results of qRF data may not match the data on the main thread when already modified with a call to GeoJSONSource::setURL or GeoJSONSource::setGeoJSON.

One way to provide the results here is to use a ClusterFeature class for cluster nodes and implement that in the supercluster module or in mbgl::style::SuperclusterData. Features from qRF can then be cast to ClusteredFeature which would provide the getLeaf, getChildren, and getExpansionZoom methods. That said, each of these instances would need a shared_ptr<> back to to the root data.

Alternately, the new APIs can be provided in Renderer with a sourceId. That would move the implementation of the methods to RenderGeoJSONSource. Mobile SDKs already forward the qRF invocations from source peers to the renderer, so their implementation would remain on the source objects.

asheemmamoowala · 2018-10-26T17:37:20Z

src/mbgl/renderer/sources/render_geojson_source.cpp

@@ -63,7 +63,7 @@ void RenderGeoJSONSource::update(Immutable<style::Source::Impl> baseImpl_,
                       util::tileSize,
                       impl().getZoomRange(),
                       optional<LatLngBounds>{},
-                       [&] (const OverscaledTileID& tileID) {
+                       [&, data = data_] (const OverscaledTileID& tileID) {


nit: use data_ or rename so as to not shadow the member variable.

asheemmamoowala · 2018-10-26T17:37:27Z

src/mbgl/renderer/sources/render_geojson_source.cpp


-    if (data_ != data) {
+    if (data.lock() != data_) {


Is it necessary to create the shared_ptr<> here for the comparison or can you compare the internal pointers directly?

asheemmamoowala · 2018-10-26T17:45:36Z

test/style/source.test.cpp

+        ASSERT_NE(geojson_source, nullptr);
+
+        auto children = geojson_source->getClusterChildren(1);
+        EXPECT_EQ(children.size(), 4U);


nit: use lower-case u to match style convention in this repo.

alexshalamov · 2018-10-29T16:47:09Z

@asheemmamoowala Thanks for review.

Features from qRF can then be cast to ClusteredFeature which would provide the getLeaf, getChildren, and getExpansionZoom methods.

Would that require heap allocated Features, right? At the moment most of the code passes vectors of POD Features.

If I would move methods to Renderer, do you think that would acceptable?

asheemmamoowala · 2018-10-29T23:58:24Z

Would that require heap allocated Features, right? At the moment most of the code passes vectors of POD Features

Ahhh, good catch @alexshalamov . Yes, they would not be POD-types anymore, and would have to hold shared_ptr<>s back to the mbgl::style::SuperclusterData they reference.
Thinking through this further, it would require the platform SDKs use wrapper types to hold on to the underlying core object (they use copy core data to POD types now), and be aware of it's lifetime. This is not how the SDKs return data from core at the moment, so it would be a change to how those APIs are written.

Using the Renderer to front the required methods seems awkward if they only apply to supercluster-based GeoJSON sources. This might be OK in core, because the platform SDKs can (as they do with qRF) expose the public facing methods from the Source peer wrapper.

cc @1ec5 @tobrun

alexshalamov · 2018-11-01T12:29:18Z

@tobrun Do you have any suggestions?
@asheemmamoowala can querySourceFeatures be used to get more data about the Feature or it was meant to be used for different use-cases?

If I understand correctly, generic problem is that there should be an interface for querying additional data about the Feature provided by the source. For example, when features are generated at runtime.

asheemmamoowala · 2018-11-03T00:03:18Z

can querySourceFeatures be used to get more data about the Feature or it was meant to be used for different use-cases?

querySourceFeatures is meant to query all available tiles of a source. It is similar to queryRenderedFeatures but not limited to visible features or the visible screen bounds. I don't think it is the appropriate way to query for additional information about specific features.

generic problem is that there should be an interface for querying additional data about the Feature provided by the source

This is a good way to think about this problem. How can different classes of features be queried for their particular unique datum. For the supercluster nodes though it does not address the need for identifying nodes using clusterId, as opposed to featureId, but maybe that can be made opaque to the interface by using a method like:

class Renderer {
...
mbgl::Value queryFeatureExtensions(
    std::string sourceId,
    optional<std::string> sourceLayer,
    const mbgl::Feature& feature,
    std::string extension,
    std::string extensionField );
}

//Usage:
auto zoomValue = queryFeatureExtensions("my_geosjon", {}, featureFromQRF, "supercluster", "expansionZoom" );

The "supercluster" extension would be registered with the Renderer::Impl and implemented on the RenderGeoJSONSource. Still need to come up with a way to statically register these, but I'll keep looking into it.

alexshalamov · 2018-11-15T15:00:50Z

@asheemmamoowala could you please take a look at #13382 and tell what do you think?

tobrun · 2018-12-20T18:37:33Z

This PR is now obsolete with #13631

tobrun added the Core The cross-platform C++ core, aka mbgl label Aug 23, 2018

tobrun self-assigned this Aug 23, 2018

tobrun mentioned this pull request Aug 23, 2018

Supercluster.hpp 0.3.0 Android #12727

Merged

tobrun mentioned this pull request Sep 13, 2018

gcc compatible by renaming shadowed decleration mapbox/supercluster.hpp#20

Merged

tobrun force-pushed the tvn-next-gen-cluster branch from 0bc2db4 to 09fd2a2 Compare September 21, 2018 14:05

tobrun requested review from kkaefer, mourner and alexshalamov October 12, 2018 09:09

kkaefer reviewed Oct 12, 2018

View reviewed changes

tobrun and others added 4 commits October 18, 2018 10:46

[core] - bump supercluster dependency to 0.3.0

a1744e5

[core] - expose getChildren, getLeaves, getClusterExpansionZoom on Su…

1db31ca

…perclusterData

Updated supercluster to 0.3.1

39e4de0

[android] - integrate supercluster api 0.3.0

a1eb283

alexshalamov force-pushed the tvn-next-gen-cluster branch from d009eda to 35b0557 Compare October 18, 2018 08:13

[all] - rename functions to include cluster

78c5a8e

alexshalamov force-pushed the tvn-next-gen-cluster branch from 35b0557 to fce4bff Compare October 18, 2018 09:12

tobrun mentioned this pull request Oct 22, 2018

Cluster items on map #13143

Closed

tmpsantos added the lift: 1 label Oct 22, 2018

tobrun commented Oct 23, 2018

View reviewed changes

mourner approved these changes Oct 26, 2018

View reviewed changes

alexshalamov added 2 commits October 26, 2018 17:19

[core] Add unit tests for GeoJSONSource with cluster data

a7c217e

[core] Share GeoJSONData pointer as weak_ptr instead of raw ptr

34c3373

GeoJSONSource::Impl shares its private data as a raw pointer, this may lead to unwanted side-effects. Moreover GeoJSONSource, used to access its Impl data without checking whether data is valid, this PR fixes both issues.

alexshalamov force-pushed the tvn-next-gen-cluster branch from fce4bff to 34c3373 Compare October 26, 2018 14:23

asheemmamoowala reviewed Oct 26, 2018

View reviewed changes

tobrun mentioned this pull request Nov 2, 2018

Implement Spiderifier for same coordinates markers #13259

Closed

tmpsantos added lift: 5 and removed lift: 1 labels Nov 5, 2018

tmpsantos assigned alexshalamov Nov 5, 2018

alexshalamov mentioned this pull request Nov 15, 2018

[core] Add queryFeatureExtension API #13382

Merged

3 tasks

tmpsantos added lift: 3 and removed lift: 5 labels Nov 16, 2018

tobrun mentioned this pull request Dec 20, 2018

Expose query feature extensions #13631

Merged

tobrun closed this Dec 20, 2018

tobrun deleted the tvn-next-gen-cluster branch December 20, 2018 18:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Supercluster.hpp 0.3.0 update #12726

Supercluster.hpp 0.3.0 update #12726

tobrun commented Aug 23, 2018

tobrun commented Aug 24, 2018 •

edited

Loading

kkaefer left a comment

mourner commented Oct 12, 2018

tobrun commented Oct 12, 2018 •

edited

Loading

alexshalamov commented Oct 18, 2018

alexshalamov commented Oct 23, 2018

tobrun left a comment

mourner left a comment

mourner Oct 26, 2018

alexshalamov Oct 26, 2018

mourner Oct 26, 2018

asheemmamoowala left a comment

asheemmamoowala Oct 26, 2018

asheemmamoowala Oct 26, 2018

asheemmamoowala Oct 26, 2018

alexshalamov commented Oct 29, 2018

asheemmamoowala commented Oct 29, 2018

alexshalamov commented Nov 1, 2018

asheemmamoowala commented Nov 3, 2018

alexshalamov commented Nov 15, 2018

tobrun commented Dec 20, 2018

Supercluster.hpp 0.3.0 update #12726

Supercluster.hpp 0.3.0 update #12726

Conversation

tobrun commented Aug 23, 2018

tobrun commented Aug 24, 2018 • edited Loading

kkaefer left a comment

Choose a reason for hiding this comment

mourner commented Oct 12, 2018

tobrun commented Oct 12, 2018 • edited Loading

alexshalamov commented Oct 18, 2018

alexshalamov commented Oct 23, 2018

tobrun left a comment

Choose a reason for hiding this comment

mourner left a comment

Choose a reason for hiding this comment

mourner Oct 26, 2018

Choose a reason for hiding this comment

alexshalamov Oct 26, 2018

Choose a reason for hiding this comment

mourner Oct 26, 2018

Choose a reason for hiding this comment

asheemmamoowala left a comment

Choose a reason for hiding this comment

asheemmamoowala Oct 26, 2018

Choose a reason for hiding this comment

asheemmamoowala Oct 26, 2018

Choose a reason for hiding this comment

asheemmamoowala Oct 26, 2018

Choose a reason for hiding this comment

alexshalamov commented Oct 29, 2018

asheemmamoowala commented Oct 29, 2018

alexshalamov commented Nov 1, 2018

asheemmamoowala commented Nov 3, 2018

alexshalamov commented Nov 15, 2018

tobrun commented Dec 20, 2018

tobrun commented Aug 24, 2018 •

edited

Loading

tobrun commented Oct 12, 2018 •

edited

Loading