Subcommands enhancements #4510

exekias · 2017-06-15T18:56:28Z

Mark configtest, setup and version flags as deprecated, this hides them from the help 76a3164
Add a new hidden completion subcommand. It dumps bash completion script to stdout, this can be handy to load it on the fly, while it would make sense to dump it into packages, will try to figure out how to do it with packer. Check https://blog.fabric8.io/enable-bash-completion-for-kubernetes-with-kubectl-506bc89fe79e for a good example of how this would be used
Add a new export subcommand, at the moment it allows to export:
- config: Current config, good to ask for debugging info
- template: Index template, this could replace index_template script

For examples check:

$ filebeat completion
$ filebeat export config
$ filebeat export template

andrewkroh · 2017-06-16T21:39:48Z

libbeat/beat/beat.go

@@ -279,13 +279,17 @@ func (b *Beat) launch(bt Creator) error {

 	// If -configtest was specified, exit now prior to run.
 	if cfgfile.IsTestConfig() {
+		logp.Deprecate("6.0", "-configtest flag has been deprectad, use configtest subcommand")


s/deprectad/deprecated/g (occurs multiple times)

exekias · 2017-06-19T14:18:49Z

Fixed the typo and rebased to master, tests should pass now

exekias · 2017-06-21T13:01:30Z

jenkins retest this

tsg · 2017-06-21T22:50:39Z

I get a crash when trying the Packetbeat export template:

$ ./packetbeat export template                                                                      1 ↵
Error getting template settings: (assert) config is nil
Trace:goroutine 1 [running]:
runtime/debug.Stack(0x4cab66c, 0xb, 0xc42127fab0)
	/usr/local/Cellar/go/1.8.3/libexec/src/runtime/debug/stack.go:24 +0x79
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.raiseCritical(0x528f600, 0xc420019250, 0xc421195a60, 0x16, 0x0, 0xc42118bf20)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/error.go:132 +0x66
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.raiseNil(0x528f600, 0xc420019250, 0x3, 0xc420452380)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/error.go:235 +0x47
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.(*Config).Unpack(0x0, 0x4abfac0, 0xc42006f140, 0x52edf20, 0x3, 0x3, 0x1, 0xc42006f140)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/reify.go:108 +0x1fb
github.com/elastic/beats/libbeat/common.(*Config).Unpack(0x0, 0x4abfac0, 0xc42006f140, 0xc, 0xc420451a00)
	/Users/tsg/src/github.com/elastic/beats/libbeat/common/config.go:193 +0x63
github.com/elastic/beats/libbeat/cmd/export.GenTemplateConfigCmd.func1(0xc421408480, 0x58377a8, 0x0, 0x0)
	/Users/tsg/src/github.com/elastic/beats/libbeat/cmd/export/template.go:38 +0x338
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).execute(0xc421408480, 0x58377a8, 0x0, 0x0, 0xc421408480, 0x58377a8)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:651 +0x23a
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4213d6500, 0xc4213d6500, 0x0, 0x0)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:726 +0x339
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4213d6500, 0xc4200001a0, 0xc4200001a0)
	/Users/tsg/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:685 +0x2b
main.main()
	/Users/tsg/src/github.com/elastic/beats/packetbeat/main.go:13 +0x2f

tsg · 2017-06-21T22:55:28Z

libbeat/cmd/configtest.go

@@ -15,6 +16,7 @@ func genConfigTestCmd(name, version string, beatCreator beat.Creator) *cobra.Com
 		Run: func(cmd *cobra.Command, args []string) {
 			b, err := beat.New(name, version)
 			if err != nil {
+				fmt.Fprintf(os.Stderr, "Error initializing beat: %s\n", err)


I don't like it too much that we get these writes directly to stdout/stderr. Perhaps it would be possible to init logging with stderr on any non-run command?

What's your exact concern? I tried to do it just the other way around, strip all log metadata as this is command line

It would be possible though

I just don't like it that logp and fmt.Println are interleaved in the code. But def. not a show stopper.

tsg · 2017-06-21T22:58:31Z

libbeat/cmd/export/template.go

+
+			// Make it compatible with the sem versioning
+			if version == "2x" {
+				version = "2.0.0"


If we have magic for 2x, we should also add 5x and 6x, I guess. But, tbh, I don't think it's worth the trouble.

Took it from here https://github.com/elastic/beats/blob/master/dev-tools/cmd/index_template/index_template.go#L40, I guess it's there for historical reasons?

I'd say we remove it, or @ruflin, do you see it otherwise?

tsg · 2017-06-21T23:19:38Z

I think the export template command covers the functionality of the setup.template.output_to_file functionality. The latter was never released, so I think we should delete it before it makes it out in a release.

tsg · 2017-06-21T23:23:11Z

I have buyer remorse for removing --setup :). The reason is that:

./filebeat -e -setup --modules=nginx,system

Is just nicer and more intuitive than:

./filebeat -e setup --modules=nginx,system && ./filebeat -e --modules=nginx,system

This can get much worse when one has to overwrite multiple config file settings (e.g. ES output).

If it's not too much effort, I'd say we don't deprecate/hide the --setup flag for now.

tsg · 2017-06-21T23:24:25Z

Nice work @exekias, and exporting the config as well as the bash completion are really nice touches.

exekias · 2017-06-26T11:12:38Z

I have added some commits:

Fix export template issue in packetbeat (and others)
Show -setup flag back
Remove 2x mangling in export template
Remove output_to_file code in favor of export template

I think this should be ready to merge

Deprecated in favor of `beat export template`

tsg

WFG

exekias added libbeat review v6.0.0-beta1 labels Jun 15, 2017

exekias force-pushed the subcommands-enhancements branch 4 times, most recently from 8eb0d54 to 905e5c4 Compare June 15, 2017 21:48

exekias added the in progress Pull request is currently in progress. label Jun 16, 2017

exekias force-pushed the subcommands-enhancements branch 3 times, most recently from dc04512 to 7402fe8 Compare June 16, 2017 13:32

andrewkroh reviewed Jun 16, 2017

View reviewed changes

exekias added review and removed review in progress Pull request is currently in progress. labels Jun 16, 2017

exekias force-pushed the subcommands-enhancements branch 2 times, most recently from 2f3cd83 to a1df699 Compare June 19, 2017 14:18

exekias force-pushed the subcommands-enhancements branch from 32758cc to f158446 Compare June 21, 2017 16:54

tsg reviewed Jun 21, 2017

View reviewed changes

exekias force-pushed the subcommands-enhancements branch 3 times, most recently from 2e38d1c to bc2c7f2 Compare June 26, 2017 10:45

exekias added 10 commits June 26, 2017 15:38

Add deprecation messages for deprecated flags

1180a92

Ensure RootCmd works when imported somewhere outside main.go

af6cbde

Add completion command

f4923ae

Add export config subcommand

b57ac77

Add export template subcommand

c2a9e13

Be more verbose on cmd errors

8e0f242

Do not fail when no template settings are given

11df391

Do not deprecate -setup flag

155f07a

Remove version mangling in export template

310bb52

Remove output_to_file template settings

d7d5e0a

Deprecated in favor of `beat export template`

exekias force-pushed the subcommands-enhancements branch from bc2c7f2 to d7d5e0a Compare June 26, 2017 13:39

tsg approved these changes Jun 26, 2017

View reviewed changes

tsg merged commit dc6e846 into elastic:master Jun 26, 2017

exekias mentioned this pull request Jun 26, 2017

Document new command line interface #4490

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Subcommands enhancements #4510

Subcommands enhancements #4510

exekias commented Jun 15, 2017 •

edited

Loading

andrewkroh Jun 16, 2017 •

edited

Loading

exekias commented Jun 19, 2017

exekias commented Jun 21, 2017

tsg commented Jun 21, 2017

tsg Jun 21, 2017

exekias Jun 22, 2017

tsg Jun 22, 2017

tsg Jun 21, 2017

exekias Jun 22, 2017

tsg Jun 22, 2017

tsg commented Jun 21, 2017

tsg commented Jun 21, 2017

tsg commented Jun 21, 2017

exekias commented Jun 26, 2017

tsg left a comment

Subcommands enhancements #4510

Subcommands enhancements #4510

Conversation

exekias commented Jun 15, 2017 • edited Loading

andrewkroh Jun 16, 2017 • edited Loading

Choose a reason for hiding this comment

exekias commented Jun 19, 2017

exekias commented Jun 21, 2017

tsg commented Jun 21, 2017

tsg Jun 21, 2017

Choose a reason for hiding this comment

exekias Jun 22, 2017

Choose a reason for hiding this comment

tsg Jun 22, 2017

Choose a reason for hiding this comment

tsg Jun 21, 2017

Choose a reason for hiding this comment

exekias Jun 22, 2017

Choose a reason for hiding this comment

tsg Jun 22, 2017

Choose a reason for hiding this comment

tsg commented Jun 21, 2017

tsg commented Jun 21, 2017

tsg commented Jun 21, 2017

exekias commented Jun 26, 2017

tsg left a comment

Choose a reason for hiding this comment

exekias commented Jun 15, 2017 •

edited

Loading

andrewkroh Jun 16, 2017 •

edited

Loading