Cherry-pick #18630 to 7.x: Validate config before init NewInput to avoid uneccessory cleanup and kubernetes watcher leak

[exekias]

Cherry-pick of PR #18630 to 7.x branch. Original message:

An alternative Fix #18629

When Filebeat using add_kubernetes_metadata processor and enabled input.reload, It will cause Kubernetes watcher goroutine leaks when the input config is wrong.

I looked through the code in filebeat/input/log/input.go, notice that there is a defer function to cleanup when executing NewInput failed.

beats/filebeat/input/log/input.go

Line 76 in d11d609

func NewInput(

// NewInput instantiates a new Log
func NewInput(
	cfg *common.Config,
	outlet channel.Connector,
	context input.Context,
) (input.Input, error) {
	cleanupNeeded := true
	cleanupIfNeeded := func(f func() error) {
		if cleanupNeeded {
			f()
		}
	}

But it seems not easy to add Kubernetes watcher stopping function here, So I move the validate section to the top, then NewInput will return directly if the config not qualified.

// NewInput instantiates a new Log
func NewInput(
	cfg *common.Config,
	outlet channel.Connector,
	context input.Context,
) (input.Input, error) {
	cleanupNeeded := true
	cleanupIfNeeded := func(f func() error) {
		if cleanupNeeded {
			f()
		}
	}

	inputConfig := defaultConfig

	if err := cfg.Unpack(&inputConfig); err != nil {
		return nil, err
	}
	if err := inputConfig.resolveRecursiveGlobs(); err != nil {
		return nil, fmt.Errorf("Failed to resolve recursive globs in config: %v", err)
	}
	if err := inputConfig.normalizeGlobPatterns(); err != nil {
		return nil, fmt.Errorf("Failed to normalize globs patterns: %v", err)
	}

	if len(inputConfig.Paths) == 0 {
		return nil, fmt.Errorf("each input must have at least one path defined")
	}

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #18810 opened]

• Start Time: 2020-05-28T07:27:06.691+0000

• Duration: 79 min 4 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6639
Skipped	1053
Total	7692

Steps errors

Expand to view the steps failures

• Name: Report to Codecov

  • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

  • Duration: 2 min 22 sec

  • Start Time: 2020-05-28T07:52:19.263+0000

  • log

• Name: Install Go 1.13.10

  • Description: .ci/scripts/install-go.sh

  • Duration: 1 min 10 sec

  • Start Time: 2020-05-28T07:49:48.871+0000

  • log

• Name: Report to Codecov

  • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

  • Duration: 2 min 22 sec

  • Start Time: 2020-05-28T08:40:21.016+0000

  • log