-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Maps] pew pew source #41504
[Maps] pew pew source #41504
Conversation
Pinging @elastic/kibana-gis |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
Sorry about that, the POC had yet to implement metric selection which is why the "layer style" select was empty. I have pushed some changes that add the ability to configure metrics for the source. These metrics are then available for dynamic styling in the layer style field select. |
There will never be a tooltip configuration for this source because its an aggregation source. The tooltip will just display the metric aggregations that have been configured for the source. |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💚 Build Succeeded |
x-pack/legacy/plugins/maps/public/layers/sources/es_pew_pew_source/create_source_editor.js
Outdated
Show resolved
Hide resolved
x-pack/legacy/plugins/maps/public/layers/sources/es_pew_pew_source/update_source_editor.js
Show resolved
Hide resolved
x-pack/legacy/plugins/maps/public/layers/sources/es_pew_pew_source/update_source_editor.js
Outdated
Show resolved
Hide resolved
x-pack/legacy/plugins/maps/public/layers/sources/es_pew_pew_source/convert_to_lines.js
Outdated
Show resolved
Hide resolved
x-pack/legacy/plugins/maps/public/layers/sources/es_pew_pew_source/es_pew_pew_source.js
Show resolved
Hide resolved
retest |
const indexPattern = await this._getIndexPattern(); | ||
const geoField = indexPattern.fields.byName[this._descriptor.destGeoField]; | ||
if (!geoField) { | ||
throw new Error(i18n.translate('xpack.maps.source.esSource.noGeoFieldErrorMessage', { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update i18n to xpack.maps.source.pewPew.noGeoFieldErrorMessage
💚 Build Succeeded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall a really nice feature. Nice work! You've finished most of the requested changes already. Some final minor i18n work requested. If you can, get that in and then with green CI, lgtm!
- code review
- tested locally in chrome
* [Maps][POC] pew pew source * refetch data on zoom level change * add metric aggs to request * fix bug where initial draw did not have styles set up * make tooltips work * fix import broken with merging master * use custom labels in tooltips * remove duplicate keys and clean up title and desc wording * update source description * update references migration to extract references for pew pew source * add percy visual test to ensure pew pew map data is fetch, processed, and visualized as expected * update title and description * use GEO_FIELD_TYPES in filterGeoField function * remove unneeded Fragment wrapper * fix typo * update inspector description id and message
* [Maps][POC] pew pew source * refetch data on zoom level change * add metric aggs to request * fix bug where initial draw did not have styles set up * make tooltips work * fix import broken with merging master * use custom labels in tooltips * remove duplicate keys and clean up title and desc wording * update source description * update references migration to extract references for pew pew source * add percy visual test to ensure pew pew map data is fetch, processed, and visualized as expected * update title and description * use GEO_FIELD_TYPES in filterGeoField function * remove unneeded Fragment wrapper * fix typo * update inspector description id and message
## Summary This PR uses the Embeddables API to embed a [Connections Map](#41504) on the Network Page of the SIEM App. A [Map Configuration](#43878) is generated on page load for configured Kibana Index Patterns that match the `siem:defaultIndex` setting in Kibana Advanced Settings, with a `Source`, `Destination`, and `Line` layer being created for each configured Kibana Index Pattern. Features includes: * Global KQL Bar Filtering * Global Timerange Filtering * Global Refresh * Click on a feature on the map to view details in tooltip * Line Selection: shows `Total Source/Destination Bytes` and `Total Documents` in tooltip * Source/Destination Selection: shows `host.name` and `host.ip` * Send filter to Global KQL Bar from Source/Destination feature tooltip (`host.name` and `host.ip`) * Informative error when index patterns aren't configured that links to specific beats setup documentation (e.g. [auditbeat setup docs](https://www.elastic.co/guide/en/beats/auditbeat/current/load-kibana-dashboards.html)) #### Dark Theme: ![image](https://user-images.githubusercontent.com/2946766/63667505-4dbe1900-c791-11e9-9b1e-9b690eafc405.png) #### Light Theme: ![image](https://user-images.githubusercontent.com/2946766/63667685-f2d8f180-c791-11e9-840e-6d3fbe0adaf2.png) ### Visual Feature Catalogue® <details><summary>Global KQL Bar Filtering</summary> ![pewpew_global_kql_filter](https://user-images.githubusercontent.com/2946766/63803146-72210f00-c8d1-11e9-951a-18571bbf9069.gif) </details> <details><summary>Global Timerange Filtering + Refreshing</summary> ![pewpew_global_time_filter](https://user-images.githubusercontent.com/2946766/63803214-9b419f80-c8d1-11e9-8349-e4f40fa39200.gif) </details> <details><summary>Filter on `host.name` or `host.ip` From Map Tooltip</summary> ![pewpew_filter_on_property](https://user-images.githubusercontent.com/2946766/63803302-d6dc6980-c8d1-11e9-990d-c364b53e247b.gif) </details> <details><summary>Using Field Formatters in Tooltip to link to Host/Network Details</summary> ![pewpew_field_formatter_links](https://user-images.githubusercontent.com/2946766/63803333-e65bb280-c8d1-11e9-9743-81165e0f78d7.gif) </details> <details><summary>Error displayed when Index Patterns aren't available</summary> ![image](https://user-images.githubusercontent.com/2946766/63804367-418ea480-c8d4-11e9-9357-73418a197809.png) </details> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) * Tested and all functionality works in Chrome/FF/Safari * IE11 fails to load map with `mapbox-gl.js` exception. Details: #44155 - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Not yet, but will work with @benskelker on this. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
## Summary This PR uses the Embeddables API to embed a [Connections Map](elastic#41504) on the Network Page of the SIEM App. A [Map Configuration](elastic#43878) is generated on page load for configured Kibana Index Patterns that match the `siem:defaultIndex` setting in Kibana Advanced Settings, with a `Source`, `Destination`, and `Line` layer being created for each configured Kibana Index Pattern. Features includes: * Global KQL Bar Filtering * Global Timerange Filtering * Global Refresh * Click on a feature on the map to view details in tooltip * Line Selection: shows `Total Source/Destination Bytes` and `Total Documents` in tooltip * Source/Destination Selection: shows `host.name` and `host.ip` * Send filter to Global KQL Bar from Source/Destination feature tooltip (`host.name` and `host.ip`) * Informative error when index patterns aren't configured that links to specific beats setup documentation (e.g. [auditbeat setup docs](https://www.elastic.co/guide/en/beats/auditbeat/current/load-kibana-dashboards.html)) #### Dark Theme: ![image](https://user-images.githubusercontent.com/2946766/63667505-4dbe1900-c791-11e9-9b1e-9b690eafc405.png) #### Light Theme: ![image](https://user-images.githubusercontent.com/2946766/63667685-f2d8f180-c791-11e9-840e-6d3fbe0adaf2.png) ### Visual Feature Catalogue® <details><summary>Global KQL Bar Filtering</summary> ![pewpew_global_kql_filter](https://user-images.githubusercontent.com/2946766/63803146-72210f00-c8d1-11e9-951a-18571bbf9069.gif) </details> <details><summary>Global Timerange Filtering + Refreshing</summary> ![pewpew_global_time_filter](https://user-images.githubusercontent.com/2946766/63803214-9b419f80-c8d1-11e9-8349-e4f40fa39200.gif) </details> <details><summary>Filter on `host.name` or `host.ip` From Map Tooltip</summary> ![pewpew_filter_on_property](https://user-images.githubusercontent.com/2946766/63803302-d6dc6980-c8d1-11e9-990d-c364b53e247b.gif) </details> <details><summary>Using Field Formatters in Tooltip to link to Host/Network Details</summary> ![pewpew_field_formatter_links](https://user-images.githubusercontent.com/2946766/63803333-e65bb280-c8d1-11e9-9743-81165e0f78d7.gif) </details> <details><summary>Error displayed when Index Patterns aren't available</summary> ![image](https://user-images.githubusercontent.com/2946766/63804367-418ea480-c8d4-11e9-9357-73418a197809.png) </details> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) * Tested and all functionality works in Chrome/FF/Safari * IE11 fails to load map with `mapbox-gl.js` exception. Details: elastic#44155 - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Not yet, but will work with @benskelker on this. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
## Summary This PR uses the Embeddables API to embed a [Connections Map](elastic#41504) on the Network Page of the SIEM App. A [Map Configuration](elastic#43878) is generated on page load for configured Kibana Index Patterns that match the `siem:defaultIndex` setting in Kibana Advanced Settings, with a `Source`, `Destination`, and `Line` layer being created for each configured Kibana Index Pattern. Features includes: * Global KQL Bar Filtering * Global Timerange Filtering * Global Refresh * Click on a feature on the map to view details in tooltip * Line Selection: shows `Total Source/Destination Bytes` and `Total Documents` in tooltip * Source/Destination Selection: shows `host.name` and `host.ip` * Send filter to Global KQL Bar from Source/Destination feature tooltip (`host.name` and `host.ip`) * Informative error when index patterns aren't configured that links to specific beats setup documentation (e.g. [auditbeat setup docs](https://www.elastic.co/guide/en/beats/auditbeat/current/load-kibana-dashboards.html)) #### Dark Theme: ![image](https://user-images.githubusercontent.com/2946766/63667505-4dbe1900-c791-11e9-9b1e-9b690eafc405.png) #### Light Theme: ![image](https://user-images.githubusercontent.com/2946766/63667685-f2d8f180-c791-11e9-840e-6d3fbe0adaf2.png) ### Visual Feature Catalogue® <details><summary>Global KQL Bar Filtering</summary> ![pewpew_global_kql_filter](https://user-images.githubusercontent.com/2946766/63803146-72210f00-c8d1-11e9-951a-18571bbf9069.gif) </details> <details><summary>Global Timerange Filtering + Refreshing</summary> ![pewpew_global_time_filter](https://user-images.githubusercontent.com/2946766/63803214-9b419f80-c8d1-11e9-8349-e4f40fa39200.gif) </details> <details><summary>Filter on `host.name` or `host.ip` From Map Tooltip</summary> ![pewpew_filter_on_property](https://user-images.githubusercontent.com/2946766/63803302-d6dc6980-c8d1-11e9-990d-c364b53e247b.gif) </details> <details><summary>Using Field Formatters in Tooltip to link to Host/Network Details</summary> ![pewpew_field_formatter_links](https://user-images.githubusercontent.com/2946766/63803333-e65bb280-c8d1-11e9-9743-81165e0f78d7.gif) </details> <details><summary>Error displayed when Index Patterns aren't available</summary> ![image](https://user-images.githubusercontent.com/2946766/63804367-418ea480-c8d4-11e9-9357-73418a197809.png) </details> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) * Tested and all functionality works in Chrome/FF/Safari * IE11 fails to load map with `mapbox-gl.js` exception. Details: elastic#44155 - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Not yet, but will work with @benskelker on this. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
## Summary This PR uses the Embeddables API to embed a [Connections Map](#41504) on the Network Page of the SIEM App. A [Map Configuration](#43878) is generated on page load for configured Kibana Index Patterns that match the `siem:defaultIndex` setting in Kibana Advanced Settings, with a `Source`, `Destination`, and `Line` layer being created for each configured Kibana Index Pattern. Features includes: * Global KQL Bar Filtering * Global Timerange Filtering * Global Refresh * Click on a feature on the map to view details in tooltip * Line Selection: shows `Total Source/Destination Bytes` and `Total Documents` in tooltip * Source/Destination Selection: shows `host.name` and `host.ip` * Send filter to Global KQL Bar from Source/Destination feature tooltip (`host.name` and `host.ip`) * Informative error when index patterns aren't configured that links to specific beats setup documentation (e.g. [auditbeat setup docs](https://www.elastic.co/guide/en/beats/auditbeat/current/load-kibana-dashboards.html)) #### Dark Theme: ![image](https://user-images.githubusercontent.com/2946766/63667505-4dbe1900-c791-11e9-9b1e-9b690eafc405.png) #### Light Theme: ![image](https://user-images.githubusercontent.com/2946766/63667685-f2d8f180-c791-11e9-840e-6d3fbe0adaf2.png) ### Visual Feature Catalogue® <details><summary>Global KQL Bar Filtering</summary> ![pewpew_global_kql_filter](https://user-images.githubusercontent.com/2946766/63803146-72210f00-c8d1-11e9-951a-18571bbf9069.gif) </details> <details><summary>Global Timerange Filtering + Refreshing</summary> ![pewpew_global_time_filter](https://user-images.githubusercontent.com/2946766/63803214-9b419f80-c8d1-11e9-8349-e4f40fa39200.gif) </details> <details><summary>Filter on `host.name` or `host.ip` From Map Tooltip</summary> ![pewpew_filter_on_property](https://user-images.githubusercontent.com/2946766/63803302-d6dc6980-c8d1-11e9-990d-c364b53e247b.gif) </details> <details><summary>Using Field Formatters in Tooltip to link to Host/Network Details</summary> ![pewpew_field_formatter_links](https://user-images.githubusercontent.com/2946766/63803333-e65bb280-c8d1-11e9-9743-81165e0f78d7.gif) </details> <details><summary>Error displayed when Index Patterns aren't available</summary> ![image](https://user-images.githubusercontent.com/2946766/63804367-418ea480-c8d4-11e9-9357-73418a197809.png) </details> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) * Tested and all functionality works in Chrome/FF/Safari * IE11 fails to load map with `mapbox-gl.js` exception. Details: #44155 - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Not yet, but will work with @benskelker on this. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
## Summary This PR uses the Embeddables API to embed a [Connections Map](#41504) on the Network Page of the SIEM App. A [Map Configuration](#43878) is generated on page load for configured Kibana Index Patterns that match the `siem:defaultIndex` setting in Kibana Advanced Settings, with a `Source`, `Destination`, and `Line` layer being created for each configured Kibana Index Pattern. Features includes: * Global KQL Bar Filtering * Global Timerange Filtering * Global Refresh * Click on a feature on the map to view details in tooltip * Line Selection: shows `Total Source/Destination Bytes` and `Total Documents` in tooltip * Source/Destination Selection: shows `host.name` and `host.ip` * Send filter to Global KQL Bar from Source/Destination feature tooltip (`host.name` and `host.ip`) * Informative error when index patterns aren't configured that links to specific beats setup documentation (e.g. [auditbeat setup docs](https://www.elastic.co/guide/en/beats/auditbeat/current/load-kibana-dashboards.html)) #### Dark Theme: ![image](https://user-images.githubusercontent.com/2946766/63667505-4dbe1900-c791-11e9-9b1e-9b690eafc405.png) #### Light Theme: ![image](https://user-images.githubusercontent.com/2946766/63667685-f2d8f180-c791-11e9-840e-6d3fbe0adaf2.png) ### Visual Feature Catalogue® <details><summary>Global KQL Bar Filtering</summary> ![pewpew_global_kql_filter](https://user-images.githubusercontent.com/2946766/63803146-72210f00-c8d1-11e9-951a-18571bbf9069.gif) </details> <details><summary>Global Timerange Filtering + Refreshing</summary> ![pewpew_global_time_filter](https://user-images.githubusercontent.com/2946766/63803214-9b419f80-c8d1-11e9-8349-e4f40fa39200.gif) </details> <details><summary>Filter on `host.name` or `host.ip` From Map Tooltip</summary> ![pewpew_filter_on_property](https://user-images.githubusercontent.com/2946766/63803302-d6dc6980-c8d1-11e9-990d-c364b53e247b.gif) </details> <details><summary>Using Field Formatters in Tooltip to link to Host/Network Details</summary> ![pewpew_field_formatter_links](https://user-images.githubusercontent.com/2946766/63803333-e65bb280-c8d1-11e9-9743-81165e0f78d7.gif) </details> <details><summary>Error displayed when Index Patterns aren't available</summary> ![image](https://user-images.githubusercontent.com/2946766/63804367-418ea480-c8d4-11e9-9357-73418a197809.png) </details> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) * Tested and all functionality works in Chrome/FF/Safari * IE11 fails to load map with `mapbox-gl.js` exception. Details: #44155 - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Not yet, but will work with @benskelker on this. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
This PR adds a new
Connection
source. The source uses a terms aggregation to identify destinations and nests a geo_grid aggregation to get rolled up source connections. Each destination/source grid cell are converted into line feature.Here is the elasticsearch request