[Metricbeat] Add STAN dashboard for module #15654

devon-kim · 2020-01-17T19:19:40Z

What does this PR do?

Add Kibana dashboards for the Metricbeat STAN module

Why is it important?

NATS module already has a dashboard, STAN module should have one as well for consistency in support

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works

Author's Checklist

[ ]

Related issues

Relates STAN (NATS streaming) module #14629

Screenshots

Logs

N/A

fearful-symmetry

Looks like you need to run make update :

Error: some files are not up-to-date. Run 'mage fmt update' then review and commit the changes. Modified: [metricbeat/docs/modules/stan.asciidoc metricbeat/docs/modules_list.asciidoc]

ChrsMark · 2020-01-20T08:36:35Z

metricbeat/include/list_common.go

@@ -128,6 +128,7 @@ import (
 	_ "github.com/elastic/beats/metricbeat/module/redis/info"
 	_ "github.com/elastic/beats/metricbeat/module/redis/key"
 	_ "github.com/elastic/beats/metricbeat/module/redis/keyspace"
+	_ "github.com/elastic/beats/metricbeat/module/stan"


It seems that this shouldn't be here. It will be fixed with a make update command inside /beats/metricbeat folder.

Unsure why running make check in my local repo doesn't cause this error though while Travis complains? Been driving me nuts

make update adds it back after I removed it and ran locally. Something is out of sync and even merging the PR into a local copy of master and running these updates don't reproduce the error for myself either.

ChrsMark

Thanks for resolving the problem. I just left some minor suggestions regarding the naming.

Also could you please update the https://github.com/elastic/beats/blob/10c741a8d8a16de9ee317887fcf90036fa0fd214/x-pack/metricbeat/module/stan/module.yml accordingly with the proper filename and dashobard ID?

Last but not least a changelog entry would be needed (

beats/CHANGELOG.next.asciidoc

Line 95 in 3dd38c1

- [Metricbeat][Istio] Add mixer metricset {pull}15696[15696]

) at Metricbeat Added section.

x-pack/metricbeat/module/stan/_meta/kibana/7/Metricbeat-STAN-overview.json

…module

…verview.json Co-Authored-By: Chris Mark <chrismarkou92@gmail.com>

ChrsMark · 2020-01-24T08:52:32Z

metricbeat/docs/fields.asciidoc

@@ -31244,7 +31244,6 @@ type: object
 stan Module


-


This is not part of this PR right? Could we clean this?

ChrsMark · 2020-01-24T08:53:57Z

metricbeat/module/stan/fields.go

@@ -0,0 +1,36 @@
+// Licensed to Elasticsearch B.V. under one or more contributor


I don't think this file should be here? Is there any reason for adding this? I think we should clean this up.

ChrsMark · 2020-01-24T08:54:35Z

x-pack/metricbeat/module/stan/_meta/fields.yml

@@ -1,7 +1,6 @@
 - key: stan
  title: "Stan"
-  description: >
-    stan Module
+  description: stan Module


Is this change needed?

ChrsMark · 2020-01-24T10:06:45Z

@devon-kim I tried to fix the problems with make check it seems to be ok now, but feel free to comment if I missed anything.

We will need an approval from @fearful-symmetry so as to enable merging.

fearful-symmetry · 2020-01-24T15:43:25Z

Tests are still failing:

  error loading /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard/Metricbeat-STAN-overview.json: returned 422 to import file: <nil>. Response: {"statusCode":422,"error":"Unprocessable Entity","message":"Document \"fbc095e0-37cc-11ea-a9c8-152a657da3ab\" has property \"visualization\" which belongs to a more recent version of Kibana (7.4.2)."}

Exiting: Failed to import dashboard: Failed to load directory /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard:

  error loading /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard/Metricbeat-STAN-overview.json: returned 422 to import file: <nil>. Response: {"statusCode":422,"error":"Unprocessable Entity","message":"Document \"fbc095e0-37cc-11ea-a9c8-152a657da3ab\" has property \"visualization\" which belongs to a more recent version of Kibana (7.4.2)."}

ChrsMark · 2020-01-24T15:58:02Z

Tests are still failing:

  error loading /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard/Metricbeat-STAN-overview.json: returned 422 to import file: <nil>. Response: {"statusCode":422,"error":"Unprocessable Entity","message":"Document \"fbc095e0-37cc-11ea-a9c8-152a657da3ab\" has property \"visualization\" which belongs to a more recent version of Kibana (7.4.2)."}

Exiting: Failed to import dashboard: Failed to load directory /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard:

  error loading /go/src/github.com/elastic/beats/x-pack/metricbeat/build/system-tests/run/test_xpack_base.Test.test_dashboards/kibana/7/dashboard/Metricbeat-STAN-overview.json: returned 422 to import file: <nil>. Response: {"statusCode":422,"error":"Unprocessable Entity","message":"Document \"fbc095e0-37cc-11ea-a9c8-152a657da3ab\" has property \"visualization\" which belongs to a more recent version of Kibana (7.4.2)."}

ouch, thanks for noticing! I changed the problematic objects to an older version, let's see if this helps. Maybe we need to update our CI though?

ChrsMark · 2020-01-27T08:12:15Z

@fearful-symmetry CI looks good now.

(cherry picked from commit e3588da)

…#15863)

…#15862)

ChrsMark · 2020-01-31T08:37:43Z

Some issues with the dashboard found that will be fixed by #15960.

ChrsMark · 2020-02-06T08:31:23Z

Tested this with BC5 and works as expected after changes applied on #15960

devon-kim added enhancement Metricbeat Metricbeat labels Jan 17, 2020

devon-kim requested a review from a team as a code owner January 17, 2020 19:19

devon-kim changed the title ~~[Metricbeat] Add dashboard for module~~ [Metricbeat] Add STAN dashboard for module Jan 17, 2020

fearful-symmetry requested changes Jan 17, 2020

View reviewed changes

ChrsMark reviewed Jan 20, 2020

View reviewed changes

devon-kim force-pushed the stan-kibana-dashboard branch from ca608ea to ada055c Compare January 21, 2020 17:02

ChrsMark reviewed Jan 22, 2020

View reviewed changes

devon-kim force-pushed the stan-kibana-dashboard branch from edb4db7 to 249db3f Compare January 23, 2020 16:23

devon-kim and others added 11 commits January 23, 2020 11:33

elastic#14629 initial commit of kibana dashboard for STAN metricbeat …

550e96e

…module

Fix docs

4da2980

Add fields for STAN metricbeat module

12d8dce

STAN module dashboard name fix

8616f50

Trying to fix staleness checks despite make fmt update galore

e16ccd8

change dashboard title formatting

b65cdd7

Update x-pack/metricbeat/module/stan/_meta/kibana/7/Metricbeat-STAN-o…

126989b

…verview.json Co-Authored-By: Chris Mark <chrismarkou92@gmail.com>

Update x-pack/metricbeat/module/stan/_meta/kibana/7/Metricbeat-STAN-o…

cd4fd2e

…verview.json Co-Authored-By: Chris Mark <chrismarkou92@gmail.com>

Fix STAN file path

159d76d

Update x-pack/metricbeat/module/stan/_meta/kibana/7/Metricbeat-STAN-o…

01e3d8b

…verview.json Co-Authored-By: Chris Mark <chrismarkou92@gmail.com>

Add changelog entry

35ad75b

devon-kim force-pushed the stan-kibana-dashboard branch 2 times, most recently from 945b79b to ac050f6 Compare January 23, 2020 19:13

Fix dashboard path presumption from tests

129b9e8

devon-kim force-pushed the stan-kibana-dashboard branch from ac050f6 to 129b9e8 Compare January 23, 2020 19:28

ChrsMark approved these changes Jan 24, 2020

View reviewed changes

ChrsMark added 3 commits January 24, 2020 10:56

Update fields.yml

cb1c13f

Update fields.asciidoc

9773298

Delete fields.go

c14fae1

ChrsMark requested a review from fearful-symmetry January 24, 2020 09:24

Update fields.go

5fb3b74

Update fields.go

ab6fd7d

ChrsMark added needs_backport PR is waiting to be backported to other branches. v7.6.0 v7.7.0 labels Jan 24, 2020

Update Metricbeat-STAN-overview.json

3d885ea

fearful-symmetry approved these changes Jan 27, 2020

View reviewed changes

Merge branch 'master' into stan-kibana-dashboard

6a657f6

ChrsMark merged commit e3588da into elastic:master Jan 27, 2020

ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Jan 27, 2020

[Metricbeat] Add STAN dashboard for module (elastic#15654)

c73fc0d

(cherry picked from commit e3588da)

ChrsMark mentioned this pull request Jan 27, 2020

Cherry-pick #15654 to 7.x: [Metricbeat] Add STAN dashboard for module #15862

Merged

5 tasks

ChrsMark removed the needs_backport PR is waiting to be backported to other branches. label Jan 27, 2020

ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Jan 27, 2020

[Metricbeat] Add STAN dashboard for module (elastic#15654)

9214d17

(cherry picked from commit e3588da)

ChrsMark mentioned this pull request Jan 27, 2020

Cherry-pick #15654 to 7.6: [Metricbeat] Add STAN dashboard for module #15863

Merged

5 tasks

ChrsMark added a commit that referenced this pull request Jan 28, 2020

Cherry-pick #15654 to 7.6: [Metricbeat] Add STAN dashboard for module (…

841eb71

…#15863)

ChrsMark self-assigned this Jan 28, 2020

ChrsMark added the test-plan Add this PR to be manual test plan label Jan 28, 2020

ChrsMark added a commit that referenced this pull request Jan 28, 2020

Cherry-pick #15654 to 7.x: [Metricbeat] Add STAN dashboard for module (…

d8db10d

…#15862)

ChrsMark mentioned this pull request Jan 30, 2020

Update stan dashboard #15960

Merged

ChrsMark added the test-plan-regression Manually testing this PR found a regression label Jan 31, 2020

This was referenced Jan 31, 2020

Cherry-pick #15960 to 7.x: Update stan dashboard #15976

Merged

Cherry-pick #15960 to 7.6: Update stan dashboard #15977

Merged

ChrsMark added test-plan-ok This PR passed manual testing and removed test-plan-regression Manually testing this PR found a regression labels Feb 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Metricbeat] Add STAN dashboard for module #15654

[Metricbeat] Add STAN dashboard for module #15654

devon-kim commented Jan 17, 2020

fearful-symmetry left a comment

ChrsMark Jan 20, 2020

devon-kim Jan 21, 2020

devon-kim Jan 21, 2020

ChrsMark left a comment

ChrsMark Jan 24, 2020

ChrsMark Jan 24, 2020

ChrsMark Jan 24, 2020 •

edited

Loading

ChrsMark commented Jan 24, 2020

fearful-symmetry commented Jan 24, 2020

ChrsMark commented Jan 24, 2020

ChrsMark commented Jan 27, 2020

ChrsMark commented Jan 31, 2020

ChrsMark commented Feb 6, 2020

		@@ -0,0 +1,36 @@
		// Licensed to Elasticsearch B.V. under one or more contributor

[Metricbeat] Add STAN dashboard for module #15654

[Metricbeat] Add STAN dashboard for module #15654

Conversation

devon-kim commented Jan 17, 2020

What does this PR do?

Why is it important?

Checklist

Author's Checklist

Related issues

Screenshots

Logs

fearful-symmetry left a comment

Choose a reason for hiding this comment

ChrsMark Jan 20, 2020

Choose a reason for hiding this comment

devon-kim Jan 21, 2020

Choose a reason for hiding this comment

devon-kim Jan 21, 2020

Choose a reason for hiding this comment

ChrsMark left a comment

Choose a reason for hiding this comment

ChrsMark Jan 24, 2020

Choose a reason for hiding this comment

ChrsMark Jan 24, 2020

Choose a reason for hiding this comment

ChrsMark Jan 24, 2020 • edited Loading

Choose a reason for hiding this comment

ChrsMark commented Jan 24, 2020

fearful-symmetry commented Jan 24, 2020

ChrsMark commented Jan 24, 2020

ChrsMark commented Jan 27, 2020

ChrsMark commented Jan 31, 2020

ChrsMark commented Feb 6, 2020

ChrsMark Jan 24, 2020 •

edited

Loading