Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Exclude Synthetics from per-policy-outputs #161949

Conversation

jillguyonnet
Copy link
Contributor

@jillguyonnet jillguyonnet commented Jul 14, 2023

Summary

Closes #155725

Context

Basic license users cannot use Synthetics private locations and a default Logstash output (cf. this comment). This is because under Basic license per policy output is not allowed.

There was a similar issue with Fleet server that was fixed by adding logic to exclude the fleet server integration from the checks (PR).

This PR follows a very similar approach for the synthetics integration.

Changes

Compare these to the corresponding changes that were made for Fleet server.

  • If the user tries to change the default output from ES to Logstash and there are agent policies using synthetics, then an error is thrown.
  • If an existing Logstash output is made to be the default output and there are agent policies using synthetics, then these policies will still be using the previous default ES output.
  • If the default output is of type Logstash with an agent policy using synthetics and another ES output, then if the user tries to delete the ES output an error is thrown.
  • Logstash type outputs are disabled in the agent policy settings if the policy uses synthetics.

Steps for testing

⚠️ Ideally the complete scenario reported here should be tested. I was having some issues running a dev setup with a proper Logstash output, so the steps below are not end-to-end (similar to #153226).

  1. Start with a single default ES output, an agent policy (1) with synthetics and another (2) without (NB: you can also test combinations with fleet server; if an error is thrown, the error for fleet server would be thrown before the one for synthetics).
  2. Edit the default ES output to be a Logstash output; this should fail and thrown the following error:
    Screenshot 2023-07-14 at 15 07 56
  3. Create a new Logstash output and make it default on creation. Go the agent policies and check the policies settings: the Output for integrations setting should still be set to the previous ES default output for policies with synthetics (and/or fleet server), and should have switched to the new Logstash default for other policies.
  4. Repeat the previous steps, but this time first create the Logstash output and then edit it to make it default. This tests the update behaviour as opposed to create. You will see the following warning:
    Screenshot 2023-07-14 at 15 06 35
  5. Still with the default Logstash output, try to delete the ES output (that is still used by policies with synthetics/fleet server(. This should fail with the following error:
    Screenshot 2023-07-14 at 15 16 03
  6. (Basic license constraint) Create another ES output (not default). Go the agent policies and check the policies settings: policies with synthetics (or fleet server) should be allowed to switch the Output for integrations setting to the new ES output:
    Screenshot 2023-07-14 at 15 18 56
    In contrast, policies without these integrations should be constrained to the default Logstash output:
    Screenshot 2023-07-14 at 15 19 19

Checklist

@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@jillguyonnet jillguyonnet self-assigned this Jul 14, 2023
@jillguyonnet jillguyonnet added v8.10.0 Team:Fleet Team label for Observability Data Collection Fleet team release_note:fix labels Jul 14, 2023
@jillguyonnet jillguyonnet marked this pull request as ready for review July 14, 2023 13:57
@jillguyonnet jillguyonnet requested a review from a team as a code owner July 14, 2023 13:57
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@jillguyonnet
Copy link
Contributor Author

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 981.2KB 981.2KB +62.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 134.3KB 134.6KB +304.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jillguyonnet

Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the tests

@jillguyonnet jillguyonnet added the QA:Needs Validation Issue needs to be validated by QA label Jul 18, 2023
Copy link
Contributor

@criamico criamico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚢

@jillguyonnet jillguyonnet merged commit e1352db into elastic:main Jul 18, 2023
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jul 18, 2023
@jillguyonnet jillguyonnet deleted the fleet/exclude-synthetics-from-per-policy-outputs branch July 18, 2023 08:53
ThomThomson pushed a commit to ThomThomson/kibana that referenced this pull request Aug 1, 2023
## Summary

Closes elastic#155725

### Context

Basic license users cannot use Synthetics private locations and a
default Logstash output (cf. [this
comment](elastic#155725 (comment))).
This is because under Basic license per policy output is not allowed.

There was a [similar
issue](elastic#152234) with Fleet
server that was fixed by adding logic to exclude the fleet server
integration from the checks
([PR](elastic#153226)).

This PR follows a very similar approach for the synthetics integration.

### Changes

Compare these to [the corresponding changes that were made for Fleet
server](elastic#153226).

- If the user tries to change the default output from ES to Logstash and
there are agent policies using synthetics, then an error is thrown.
- If an existing Logstash output is made to be the default output and
there are agent policies using synthetics, then these policies will
still be using the previous default ES output.
- If the default output is of type Logstash with an agent policy using
synthetics and another ES output, then if the user tries to delete the
ES output an error is thrown.
- Logstash type outputs are disabled in the agent policy settings if the
policy uses synthetics.

### Steps for testing

⚠️ Ideally the complete scenario reported
[here](elastic#155725 (comment))
should be tested. I was having some issues running a dev setup with a
proper Logstash output, so the steps below are not end-to-end (similar
to elastic#153226).

1. Start with a single default ES output, an agent policy (1) with
synthetics and another (2) without (NB: you can also test combinations
with fleet server; if an error is thrown, the error for fleet server
would be thrown before the one for synthetics).
2. Edit the default ES output to be a Logstash output; this should fail
and thrown the following error:
![Screenshot 2023-07-14 at 15 07
56](https://github.com/elastic/kibana/assets/23701614/bf716a7a-745b-4b89-aa49-0ced2cc825a2)
3. Create a new Logstash output and make it default on creation. Go the
agent policies and check the policies settings: the `Output for
integrations` setting should still be set to the previous ES default
output for policies with synthetics (and/or fleet server), and should
have switched to the new Logstash default for other policies.
4. Repeat the previous steps, but this time first create the Logstash
output and then edit it to make it default. This tests the update
behaviour as opposed to create. You will see the following warning:
![Screenshot 2023-07-14 at 15 06
35](https://github.com/elastic/kibana/assets/23701614/283ee6c2-6d84-4847-bd3d-3c4ff734b2d6)
5. Still with the default Logstash output, try to delete the ES output
(that is still used by policies with synthetics/fleet server(. This
should fail with the following error:
![Screenshot 2023-07-14 at 15 16
03](https://github.com/elastic/kibana/assets/23701614/62dfe4c1-aed6-4d85-83f0-92985eab9336)
6. (Basic license constraint) Create another ES output (not default). Go
the agent policies and check the policies settings: policies with
synthetics (or fleet server) should be allowed to switch the `Output for
integrations` setting to the new ES output:
![Screenshot 2023-07-14 at 15 18
56](https://github.com/elastic/kibana/assets/23701614/98deb4fc-abb1-40e2-9556-1a4ecfa96923)
In contrast, policies without these integrations should be constrained
to the default Logstash output:
![Screenshot 2023-07-14 at 15 19
19](https://github.com/elastic/kibana/assets/23701614/9856de1f-675b-47db-a471-9796239f0f85)

### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@amolnater-qasource amolnater-qasource removed the QA:Needs Validation Issue needs to be validated by QA label Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:fix Team:Fleet Team label for Observability Data Collection Fleet team v8.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Fleet] Exclude Synthetics Integration from License Limitation
8 participants