Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Telemetry] Add integration tests #181927

Merged
merged 4 commits into from
Apr 29, 2024

Conversation

szaffarano
Copy link
Contributor

@szaffarano szaffarano commented Apr 26, 2024

Summary

  • Improve the telemetry logging by including execution context as structured arguments to facilitate searching by those fields. The TelemetryLogger now includes always by default some basic data (like cluster uuid), so it is highly recommended to start using it when possible (instantiate it using the newTelemetryLogger function)
  • Start using the above feature in the receiver as well as in some tasks.
  • Add integration tests mainly for endpoint-meta-telemetry
  • Refactor endpoint-meta-telemetry to simplify its logic and also improve error handling. Based on the tests, I had to add or change how we managed errors for some (edge) cases. Also, the errors now log more information introducing fields for relevant data instead of using a nonstatic string (e.g., instead of Running task: ${taskId} [last: ${taskExecutionPeriod.last} - current: ${taskExecutionPeriod.current}], now taskId and the executionPeriod are searchable fields and not part of the log message, which becomes a "static string").
    Regarding error handling, the goal is to send as much information as possible and not abort the execution when we encounter weird (but still feasible) errors.
  • Enable the async sender in endpoint-meta-telemetry using the already defined feature flag, just in case we need to disable it through a configuration artifact. This change deprecates the current way we split the events to send them into batches (although if we disable the feature flag, it still uses this approach).
  • Integration tests:

image

@szaffarano szaffarano requested review from a team as code owners April 26, 2024 22:40
@szaffarano szaffarano added release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. 8.14 candidate v8.14.0 and removed 8.14 candidate labels Apr 26, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@szaffarano szaffarano requested review from donaherc and JDKurma April 26, 2024 22:51
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id before after diff
securitySolution 517 518 +1

Total ESLint disabled count

id before after diff
securitySolution 595 596 +1

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@JDKurma JDKurma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@szaffarano szaffarano merged commit 2cc5109 into elastic:main Apr 29, 2024
34 checks passed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Apr 29, 2024
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.14

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Apr 29, 2024
…#182024)

# Backport

This will backport the following commits from `main` to `8.14`:
- [ [Security Solution][Telemetry] Add integration tests
(#181927)](#181927)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Sebastián
Zaffarano","email":"sebastian.zaffarano@elastic.co"},"sourceCommit":{"committedDate":"2024-04-29T19:02:51Z","message":"
[Security Solution][Telemetry] Add integration tests
(#181927)","sha":"2cc5109484ee8336b04bb52b4e58b0aa34480b21","branchLabelMapping":{"^v8.15.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:
SecuritySolution","v8.14.0","v8.15.0"],"title":" [Security
Solution][Telemetry] Add integration
tests","number":181927,"url":"https://github.com/elastic/kibana/pull/181927","mergeCommit":{"message":"
[Security Solution][Telemetry] Add integration tests
(#181927)","sha":"2cc5109484ee8336b04bb52b4e58b0aa34480b21"}},"sourceBranch":"main","suggestedTargetBranches":["8.14"],"targetPullRequestStates":[{"branch":"8.14","label":"v8.14.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.15.0","branchLabelMappingKey":"^v8.15.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/181927","number":181927,"mergeCommit":{"message":"
[Security Solution][Telemetry] Add integration tests
(#181927)","sha":"2cc5109484ee8336b04bb52b4e58b0aa34480b21"}}]}]
BACKPORT-->

Co-authored-by: Sebastián Zaffarano <sebastian.zaffarano@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.14.0 v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants