Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support dashboard loading without Elasticsearch #5653

Merged
merged 2 commits into from
Nov 21, 2017
Merged

Support dashboard loading without Elasticsearch #5653

merged 2 commits into from
Nov 21, 2017

Conversation

kvch
Copy link
Contributor

@kvch kvch commented Nov 20, 2017

Make it possible to import dashboards without Elasticsearch using Kibana API.

Usage

$ ./metricbeat setup --dashboards --force-kibana

The flag force-kibana can be specified only when dashboards are loaded.

$ ./metricbeat --template --force-kibana
Error setting up beat: --force-kibana is only used in case of dashboard loading

The name of the flag is not the best. I am open to proposals.

Based on a discussion over IRC.

@ruflin
Copy link
Contributor

ruflin commented Nov 20, 2017

@kvch Can you share some more details on why we need this? Instead of requiring a force param couldn't we change the current load dashboard command so it works with ES in 6.x? Why is ES required at all here?

@tsg
Copy link
Contributor

tsg commented Nov 21, 2017

@ruflin The current algorithm is something like this:

  • Check the ES version
  • if it's < 6.0, load the dashboards directly in ES 5.x style
  • if it's >= 6.0, load the dashboards via the Kibana API

What this PR changes is a way to skip the ES version check for when the Beat has access to the Kibana API but not the ES API.

@kvch We generally prefer configuration file options more than CLI flags (because any config can be set from the CLI anyway), so maybe a configuration option in the setup.kibana namespace. Since this is adding a flag only to the setup command, it's not too bad, though. But it could be that the config option results in cleaner code.

@kvch kvch force-pushed the feature/libbeat/do-not-require-elasticsearch-to-import-dashboards branch from d1a83d2 to 8664fa3 Compare November 21, 2017 12:07
@kvch
Copy link
Contributor Author

kvch commented Nov 21, 2017

I added a new config option called load_dashboards under setup.kibana as @tsg requested. By default it is false.

@kvch kvch force-pushed the feature/libbeat/do-not-require-elasticsearch-to-import-dashboards branch from 8664fa3 to 85514d3 Compare November 21, 2017 12:37
@kvch
Copy link
Contributor Author

kvch commented Nov 21, 2017

The new config option is under setup.dashboards called always_kibana. By default it is false.

# Force loading of dashboards using Kibana instead of Elasticsearch
#always_kibana: false

@tsg
Copy link
Contributor

tsg commented Nov 21, 2017

@kvch LGTM. Can you also add documentation for the new option and a CHANGELOG entry, please?

tsg
tsg reviewed Nov 21, 2017
View reviewed changes
libbeat/_meta/config.reference.yml Outdated
@@ -628,6 +628,9 @@ output.elasticsearch:
# dashboards and index pattern. Example: testbeat-*
#setup.dashboards.index:

# Force loading of dashboards using Kibana instead of Elasticsearch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: "Force loading of dashboards using the Kibana API without querying Elasticsearch for the version"

@kvch kvch force-pushed the feature/libbeat/do-not-require-elasticsearch-to-import-dashboards branch from 85514d3 to 778d96b Compare November 21, 2017 12:59
@kvch
libbeat: support dashboard loading without Elasticsearch
c588860 
A new boolean option for dashboards is introduced named alwas_kibana.
If it is true, Elasticsearch version check is skipped and dashboards
are loaded using Kibana API. It should be used when no Elasticsearch
output is configured.
@kvch kvch force-pushed the feature/libbeat/do-not-require-elasticsearch-to-import-dashboards branch from 778d96b to c588860 Compare November 21, 2017 13:00
@kvch
Copy link
Contributor Author

kvch commented Nov 21, 2017

I added the entry and changed the documentation.

tsg
tsg reviewed Nov 21, 2017
View reviewed changes
CHANGELOG.asciidoc Outdated
@@ -14,6 +14,9 @@ https://github.com/elastic/beats/compare/v6.0.0-beta2...master[Check the HEAD di

*Affecting all Beats*

- Determine log level for kafka output. {pull}5397[5397]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a rebase artifact. Can you remove the line, please?

@tsg
Copy link
Contributor

tsg commented Nov 21, 2017

Please also add the config option to libbeat/docs/dashboardsconfig.asciidoc.

@kvch kvch force-pushed the feature/libbeat/do-not-require-elasticsearch-to-import-dashboards branch from 73c7e16 to a134b80 Compare November 21, 2017 14:08
@tsg tsg merged commit 0bc3186 into elastic:master Nov 21, 2017
ruflin
ruflin reviewed Nov 22, 2017
View reviewed changes
auditbeat/auditbeat.reference.yml
@@ -722,6 +722,9 @@ output.elasticsearch:
# dashboards and index pattern. Example: testbeat-*
#setup.dashboards.index:

# Force loading of dashboards using the Kibana API without querying Elasticsearch for the version
#always_kibana: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kvch Should this be prefixed by setup.dashboards.?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruflin ruflin ruflin left review comments

@tsg tsg tsg left review comments

Assignees
No one assigned
Labels
libbeat review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kvch @ruflin @tsg