Kibana GIS UI Phase 1

[kindsun]

🚧 This ticket is under construction.

---

Summarizes scope of the first phase for the upcoming GIS app plugin for Kibana.

---

Overall scope

The GIS app

• must have feature parity with the current visualizations (coordinate map and region map)
  • Any map that can be created now with the visualizations, can be created in the GIS app.
• introduces new features:
  • Add layered UI. A traditional concept in GIS. The user should not longer be restricted to just be able to configure a single base-map and single overlay. This means that a single map can hold data from multiple datasources concurrently. Related ERs: Overlay GeoShapes without filtering #11529, Support for GeoJson as alternative to WMS #10757, Display journeys on a tilemap #8173
  • Can map individual documents. The Kibana-visualizations can only show aggregation-results. Users should be able to show individual documents (similar to how Discover shows individual documents) if they have a spatial data-type (e.g. geo_point or geo_shape). Related ERs Pin for every document in tile map #4234, Zoom tile map #11227, Display journeys on a tilemap #8173, Individual Features in Tilemap #5505.
  • provides flexible styling options for vector data. Users should not have to rely on 3rd party map-server stacks for basic cartography (e.g. WMS should not be a requirement for custom styled maps). Related ERs: Show bounding box on the map #14821, Support for GeoJson as alternative to WMS #10757
  • introduces basic data upload functionality. Users should not have to rely on setting up an outside file-server to bring in custom vector layers. Kibana can be that server.
• must be future proof:
  • technology choices need to provide pathway to 2.5D or 3D visualizations
  • UI needs to anticipate analytical use-cases (e.g. spatial join)

Features

The following feature list roughly summarizes the scope of the phase 1 version of the GIS plugin.

Note: the below list will only be checked off once the features have been merged into master. Right now, you can track day-2-day progress on the development branch: #24804.

high priority

Layered UX

• add layer wizard
• layer ordering
• layer visibility
• zoom to bounds of layer

supported layer types

• raster layer
  • all TMS sources
  • WMS sources ()
• vector layer
  • EMS sources
  • ES sources
  • geohash grid source
• heatmap layer
  • geohash grid source

Supported datasources

• From EMS
  • EMS file layers (aka "regions", (geojson/topojson))
  • EMS TMS service(s)
• WMS (custom config)
• TMS (custom config) (decide on url-pattern convention {} or ${})
• From Kibana.yml
  • TMS from kibana-yml
  • file layers from kibana-yml (geojson/topojson)
• from index pattern (current-extent aware (?) only consume data that is in view)
  • index patterns with geo_point
    • consume with geohash_grid
    • as individual docs (_search)
  • index patterns with geo_shape
    • as individual docs (_search)

Each data source needs:

• a UX-form in the add data layer wizard
• display layer details in the Layer details panel

styling

vector

• styling properties (need to be data-driven from ordinal field-values or statically selected)
  • Points as circle markers
    • size
    • fill color
  • Lines
    • line thickness
    • line color
  • Polygons
    • fill color
    • outline color
    • outline thickness

raster

• stylable EMS layers (e.g. dark theme/light theme)

supported operations

"inner joining" of layers to ES term aggregation

• operation for vector layers, needs UX layer details (support joining to all simple feature geometries)
• must be filterable (Keury or Lucene-query)

balloon

• feature identifier with balloon
• configurable field list

user-based filtering based on drawings

• must apply to all filterable-layers

workspace management

• create new GIS workspace
• open existing GIS workspace
• persist map state (layer order/visibility/extent/zoom level) in workspace

filterability

• heatmap/vector layers that are tied to filterable sources, should have UX for people to define filter
  • ES index patterns
    • for linked term aggregations in enrichment UX
    • for individual docs
  • ES geohash grids

data upload

• upload geojson -> store in index
• copy EMS file -> store in index (on-prem solution)

dashboard integration

• embed workspaces in dashboard
• "global state aware" (time, query/filter)

time filter control

• dedicated time-slider for time-selection and playback

Table view

• may require using Inspector
• table view for vector layer

Proof of concept

The following image shows an example of the scope of Phase 1

• Layered UX showing data from
  • Elasticsearch index (downloads/log and elasticon2018 layers)
  • EMS file data (usa states and country layers)
  • EMS styled background data (dark theme map)
  • custom geojson
• Mapping invididual documents
  • point vector data is not aggregated. each point corresponds to an individual document
• Vector data styling
  • Choropleth mapping (the pink color ramp shows document count per country/state)
  • Icon styling (custom icons for the masonic layer and colored circles for the elasticon2018 layer)
• Applying dark theme stylesheets to EMS world map data

[trevan]

Is this x-pack specific?

[kindsun]

@trevan No hard decisions on this yet. We'll be discussing this more as we get closer to finalizing the code and targeting a release for integration.

[thomasneirynck]

For posterity:

(earlier misc notes in the description of this ticket)

## Resolve immediate friction

Multilayer UIs
- Ability to add more than one layer
- Determine challenges with 3D maps integration (shouldn't hold up integration, but a consideration)

Style layers on-the-fly
- Consider rules for multiple layers styled and stacked, especially global
  - One issue is when you have multiple global bounding boxes, they collectively block out the map
  - Consider strategies for multiple styles & persisting custom styles

Visualize individual documents from index
- Defaults to entire index(?)
- Via “save search” as datastore
  - Filter on data through Kibana

3D Capabilities
- MapBox vs OpenLayers w/ Cesium vs Kepler?
- "2.5D" compromise?

## Feature requests
Real-time analytics for choropleth mapping/nearest neighbor. Where would these go? Some ideas
- Another outside component 
- An embedded popup
- Toggle-able sidebar 

Easy way to add data
- This could be implemented with limited capabilities for Phase 1, expanded for Phase 2
- Quick upload / drag and drop preferred (could re-invigorate CSV upload with GeoJSON support)
- File selection pop-up
- Determine interplay of adding files via config vs. interface. Easiest is that everything is just additive
- Perhaps integrate automated process with Beats (?)

[alepuccetti]

I propose another enhancement to improve map navigation and usability.

A search box google maps style where you can type an address or landmark and resolve it to the lat long using openstreet map API.
http://nominatim.openstreetmap.org/search?q="london"&format=json (other formats are available, see http://wiki.openstreetmap.org/wiki/Nominatim).

This was also discussed on awesome @nreese enhanced_tilemap plugin discussed in nreese/enhanced_tilemap#75.

Automatically adjust the zoom level dpending on what you are searhed e.g. if you look for "France" or "London" the map should center on the tager on different zoom level relevant to the "interested" area.

Benefits:

• improve usebility.
• reduce number of useless intermediate queries running when zooming/moving the map when looking for the place you want ot reach.

[thomasneirynck]

Closed with 5840370 (7.0) and 7e65c47 (6.7)

Any outstanding issues are logged separately. Issue titles start with [Maps].

[foreman3]

Congratulations! Can't wait!