Make log messages in the file scanner less noisy

[rdner]

• Demote warning about empty files to the Debug level
• Demote warnings about files too small to the Debug level
• Improve wording, so it does not sound like an error

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

cd ./filebeat/input/filestream
go test

Related issues

• Relates Fix empty file edge case #36076
• Relates Add new fingerprint file identity #35734

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @rdner? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5bebd8c

cc @rdner

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 134 min 24 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 5bebd8c

Failed CI Steps

• :windows:-family/core-windows-2016 Unit Tests
• :windows:-family/core-windows-2022 Unit Tests

History

cc @rdner

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 5bebd8c

Failed CI Steps

• :windows: Unit Tests / family/core-windows-2016
• :windows: Unit Tests / family/core-windows-2022

History

cc @rdner

[kbujold]

Note that this log was missed

filestream input with ID 'my-unique-id' already exists, this will lead to data duplication, please use a different ID

See #31767

[kbujold]

Also still seeing this one flooding the system

points to an already known ingest target

{"log.level":"warn","@timestamp":"2024-09-09T17:46:41.990Z","log.logger":"scanner","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileScanner).GetFiles","file.name":"filestream/fswatch.go","file.line":404},"message":"\"/var/log/dcdbsync/dcdbsync_xxx.log\" points to an already known ingest target \"/var/log/dcdbsync/dcdbsync.log\" [80a036b73f0a2c6d42cfcdfe9d6a5f527262630c6ff7fa5f349cde60158451f8==80a036b73f0a2c6d42cfcdfe9d6a5f527262630c6ff7fa5f349cde60158451f8]. Skipping","service.name":"filebeat","ecs.version":"1.6.0"}