[Fleet] Updates to output logic #153226
Merged
[Fleet] Updates to output logic #153226
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
criamico
changed the title
criamico
added
Team:Fleet
|
@elasticmachine merge upstream |
|
Pinging @elastic/fleet (Team:Fleet) |
|
@elasticmachine merge upstream |
nchaulet
reviewed
Mar 20, 2023
|
@elasticmachine merge upstream |
…-ref HEAD~1..HEAD --fix'
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 23, 2023
Closes elastic#152234 ## Summary - If the user changes an output from `ES` to `logstash` when there are Fleet server policies that use that output, we throw asking the user to create a new output - If the user changes the default output to a `logstash` one, we update the Fleet server policies to use the previous default output (that should be ES) - Removed limitations to basic licenses so that fleet server policies can select a per-policy output (between the available ES outputs). This means that "Basic" users should be able to set their default output to Logstash but still have the Fleet Server policy attached to an ES output. - Added some tests to cover new functionalities ### UI changes When editing a "logstash" output the modal displays a warning: <img width="878" alt="Screenshot 2023-03-16 at 15 04 54" src="https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png"> ### Testing 1. **Edit existing logstash output to be default** - Have a default ES output and a fleet server policy (verify that has fleet server integration) - Create a new logstash output - Update it to make it default (this is valid for both integrations and monitoring output) - Check that fleet server policy keeps the ES output, it can be verified from agent policies > fleet server policy > settings. - The other policies should be switched to the logstash output **Note** if the previous output is logstash, it will be switched to the default one. However, users who updated the system and already had a logstash ouput set as a default will need to to manually set the output from the agent policy settings page. 2. **Edit existing ES output to become a logstash one** - Have a default ES output and a fleet server policy (verify that has fleet server integration) - Edit the output and change it to type 'logstash' - Check that the endpoint throws with an error <img width="366" alt="Screenshot 2023-03-21 at 18 06 25" src="https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png"> 3. **Create new logstash output as default** - Have a default ES output and a fleet server policy (verify that has fleet server integration) - Create a new logstash output as set it as default output - Check that the fleet server policy keeps the previous ES default output - basically this should behave like case 1 (PUT and POST should have the same behaviour) 5. **Delete ES Output** - Once in the previous case (fleet server on ES output, other policies on logstash), try to delete the ES output - It should be prevented with an error <img width="444" alt="Screenshot 2023-03-16 at 15 05 44" src="https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png"> 6. **Custom output per policy** - With "basic" license, go to an agent policy with fleet server > settings - Check that "output for integrations" can select any output of type `elasticsearch`, while the logstash ones are disabled <img width="981" alt="Screenshot 2023-03-21 at 18 09 29" src="https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png"> ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit ec3294f)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Mar 23, 2023
# Backport This will backport the following commits from `main` to `8.7`: - [[Fleet] Updates to output logic (#153226)](#153226) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Cristina Amico","email":"criamico@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-03-23T08:00:26Z","message":"[Fleet] Updates to output logic (#153226)\n\nCloses https://github.com/elastic/kibana/issues/152234\r\n\r\n## Summary\r\n\r\n- If the user changes an output from `ES` to `logstash` when there are\r\nFleet server policies that use that output, we throw asking the user to\r\ncreate a new output\r\n- If the user changes the default output to a `logstash` one, we update\r\nthe Fleet server policies to use the previous default output (that\r\nshould be ES)\r\n- Removed limitations to basic licenses so that fleet server policies\r\ncan select a per-policy output (between the available ES outputs). This\r\nmeans that \"Basic\" users should be able to set their default output to\r\nLogstash but still have the Fleet Server policy attached to an ES\r\noutput.\r\n- Added some tests to cover new functionalities\r\n\r\n### UI changes\r\n\r\nWhen editing a \"logstash\" output the modal displays a warning:\r\n\r\n<img width=\"878\" alt=\"Screenshot 2023-03-16 at 15 04 54\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png\">\r\n\r\n### Testing\r\n\r\n1. **Edit existing logstash output to be default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output \r\n- Update it to make it default (this is valid for both integrations and\r\nmonitoring output)\r\n- Check that fleet server policy keeps the ES output, it can be verified\r\nfrom agent policies > fleet server policy > settings.\r\n- The other policies should be switched to the logstash output\r\n\r\n**Note** if the previous output is logstash, it will be switched to the\r\ndefault one. However, users who updated the system and already had a\r\nlogstash ouput set as a default will need to to manually set the output\r\nfrom the agent policy settings page.\r\n\r\n2. **Edit existing ES output to become a logstash one**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Edit the output and change it to type 'logstash' \r\n- Check that the endpoint throws with an error\r\n\r\n<img width=\"366\" alt=\"Screenshot 2023-03-21 at 18 06 25\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png\">\r\n\r\n3. **Create new logstash output as default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output as set it as default output\r\n- Check that the fleet server policy keeps the previous ES default\r\noutput\r\n- basically this should behave like case 1 (PUT and POST should have the\r\nsame behaviour)\r\n\r\n5. **Delete ES Output**\r\n- Once in the previous case (fleet server on ES output, other policies\r\non logstash), try to delete the ES output\r\n- It should be prevented with an error\r\n\r\n<img width=\"444\" alt=\"Screenshot 2023-03-16 at 15 05 44\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png\">\r\n\r\n6. **Custom output per policy**\r\n- With \"basic\" license, go to an agent policy with fleet server >\r\nsettings\r\n- Check that \"output for integrations\" can select any output of type\r\n`elasticsearch`, while the logstash ones are disabled\r\n\r\n<img width=\"981\" alt=\"Screenshot 2023-03-21 at 18 09 29\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png\">\r\n\r\n### Checklist\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"ec3294f2b340a5f6dbfdada3a997e17047bc8f17","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Fleet","backport:prev-minor","v8.8.0"],"number":153226,"url":"https://github.com/elastic/kibana/pull/153226","mergeCommit":{"message":"[Fleet] Updates to output logic (#153226)\n\nCloses https://github.com/elastic/kibana/issues/152234\r\n\r\n## Summary\r\n\r\n- If the user changes an output from `ES` to `logstash` when there are\r\nFleet server policies that use that output, we throw asking the user to\r\ncreate a new output\r\n- If the user changes the default output to a `logstash` one, we update\r\nthe Fleet server policies to use the previous default output (that\r\nshould be ES)\r\n- Removed limitations to basic licenses so that fleet server policies\r\ncan select a per-policy output (between the available ES outputs). This\r\nmeans that \"Basic\" users should be able to set their default output to\r\nLogstash but still have the Fleet Server policy attached to an ES\r\noutput.\r\n- Added some tests to cover new functionalities\r\n\r\n### UI changes\r\n\r\nWhen editing a \"logstash\" output the modal displays a warning:\r\n\r\n<img width=\"878\" alt=\"Screenshot 2023-03-16 at 15 04 54\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png\">\r\n\r\n### Testing\r\n\r\n1. **Edit existing logstash output to be default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output \r\n- Update it to make it default (this is valid for both integrations and\r\nmonitoring output)\r\n- Check that fleet server policy keeps the ES output, it can be verified\r\nfrom agent policies > fleet server policy > settings.\r\n- The other policies should be switched to the logstash output\r\n\r\n**Note** if the previous output is logstash, it will be switched to the\r\ndefault one. However, users who updated the system and already had a\r\nlogstash ouput set as a default will need to to manually set the output\r\nfrom the agent policy settings page.\r\n\r\n2. **Edit existing ES output to become a logstash one**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Edit the output and change it to type 'logstash' \r\n- Check that the endpoint throws with an error\r\n\r\n<img width=\"366\" alt=\"Screenshot 2023-03-21 at 18 06 25\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png\">\r\n\r\n3. **Create new logstash output as default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output as set it as default output\r\n- Check that the fleet server policy keeps the previous ES default\r\noutput\r\n- basically this should behave like case 1 (PUT and POST should have the\r\nsame behaviour)\r\n\r\n5. **Delete ES Output**\r\n- Once in the previous case (fleet server on ES output, other policies\r\non logstash), try to delete the ES output\r\n- It should be prevented with an error\r\n\r\n<img width=\"444\" alt=\"Screenshot 2023-03-16 at 15 05 44\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png\">\r\n\r\n6. **Custom output per policy**\r\n- With \"basic\" license, go to an agent policy with fleet server >\r\nsettings\r\n- Check that \"output for integrations\" can select any output of type\r\n`elasticsearch`, while the logstash ones are disabled\r\n\r\n<img width=\"981\" alt=\"Screenshot 2023-03-21 at 18 09 29\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png\">\r\n\r\n### Checklist\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"ec3294f2b340a5f6dbfdada3a997e17047bc8f17"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/153226","number":153226,"mergeCommit":{"message":"[Fleet] Updates to output logic (#153226)\n\nCloses https://github.com/elastic/kibana/issues/152234\r\n\r\n## Summary\r\n\r\n- If the user changes an output from `ES` to `logstash` when there are\r\nFleet server policies that use that output, we throw asking the user to\r\ncreate a new output\r\n- If the user changes the default output to a `logstash` one, we update\r\nthe Fleet server policies to use the previous default output (that\r\nshould be ES)\r\n- Removed limitations to basic licenses so that fleet server policies\r\ncan select a per-policy output (between the available ES outputs). This\r\nmeans that \"Basic\" users should be able to set their default output to\r\nLogstash but still have the Fleet Server policy attached to an ES\r\noutput.\r\n- Added some tests to cover new functionalities\r\n\r\n### UI changes\r\n\r\nWhen editing a \"logstash\" output the modal displays a warning:\r\n\r\n<img width=\"878\" alt=\"Screenshot 2023-03-16 at 15 04 54\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png\">\r\n\r\n### Testing\r\n\r\n1. **Edit existing logstash output to be default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output \r\n- Update it to make it default (this is valid for both integrations and\r\nmonitoring output)\r\n- Check that fleet server policy keeps the ES output, it can be verified\r\nfrom agent policies > fleet server policy > settings.\r\n- The other policies should be switched to the logstash output\r\n\r\n**Note** if the previous output is logstash, it will be switched to the\r\ndefault one. However, users who updated the system and already had a\r\nlogstash ouput set as a default will need to to manually set the output\r\nfrom the agent policy settings page.\r\n\r\n2. **Edit existing ES output to become a logstash one**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Edit the output and change it to type 'logstash' \r\n- Check that the endpoint throws with an error\r\n\r\n<img width=\"366\" alt=\"Screenshot 2023-03-21 at 18 06 25\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png\">\r\n\r\n3. **Create new logstash output as default**\r\n- Have a default ES output and a fleet server policy (verify that has\r\nfleet server integration)\r\n- Create a new logstash output as set it as default output\r\n- Check that the fleet server policy keeps the previous ES default\r\noutput\r\n- basically this should behave like case 1 (PUT and POST should have the\r\nsame behaviour)\r\n\r\n5. **Delete ES Output**\r\n- Once in the previous case (fleet server on ES output, other policies\r\non logstash), try to delete the ES output\r\n- It should be prevented with an error\r\n\r\n<img width=\"444\" alt=\"Screenshot 2023-03-16 at 15 05 44\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png\">\r\n\r\n6. **Custom output per policy**\r\n- With \"basic\" license, go to an agent policy with fleet server >\r\nsettings\r\n- Check that \"output for integrations\" can select any output of type\r\n`elasticsearch`, while the logstash ones are disabled\r\n\r\n<img width=\"981\" alt=\"Screenshot 2023-03-21 at 18 09 29\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png\">\r\n\r\n### Checklist\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"ec3294f2b340a5f6dbfdada3a997e17047bc8f17"}}]}] BACKPORT--> Co-authored-by: Cristina Amico <criamico@users.noreply.github.com>
1 task
jen-huang
pushed a commit
that referenced
this pull request
Mar 28, 2023
Fixes #153622 ## Summary [Fleet] Fix an error when creating new logstash output. The bug was added with #153226 and depended on this [else branch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312) that shouldn't have been added. I also added unit and integration tests to cover for this and other cases. ### Testing - Create a new `logstash` output, it can be `default` or not - The output creation works as expected in both cases ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 28, 2023
Fixes elastic#153622 ## Summary [Fleet] Fix an error when creating new logstash output. The bug was added with elastic#153226 and depended on this [else branch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312) that shouldn't have been added. I also added unit and integration tests to cover for this and other cases. ### Testing - Create a new `logstash` output, it can be `default` or not - The output creation works as expected in both cases ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 2e08470)
kibanamachine
added a commit
that referenced
this pull request
Mar 28, 2023
…153819) # Backport This will backport the following commits from `main` to `8.7`: - [[Fleet] Fix error when creating new logstash output (#153752)](#153752) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Cristina Amico","email":"criamico@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-03-28T00:43:03Z","message":"[Fleet] Fix error when creating new logstash output (#153752)\n\nFixes https://github.com/elastic/kibana/issues/153622\r\n\r\n## Summary\r\n[Fleet] Fix an error when creating new logstash output. \r\n\r\nThe bug was added with #153226 and\r\ndepended on this [else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat shouldn't have been added.\r\n\r\nI also added unit and integration tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n- Create a new `logstash` output, it can be `default` or not\r\n- The output creation works as expected in both cases\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","backport:prev-minor","v8.8.0"],"number":153752,"url":"https://github.com/elastic/kibana/pull/153752","mergeCommit":{"message":"[Fleet] Fix error when creating new logstash output (#153752)\n\nFixes https://github.com/elastic/kibana/issues/153622\r\n\r\n## Summary\r\n[Fleet] Fix an error when creating new logstash output. \r\n\r\nThe bug was added with #153226 and\r\ndepended on this [else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat shouldn't have been added.\r\n\r\nI also added unit and integration tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n- Create a new `logstash` output, it can be `default` or not\r\n- The output creation works as expected in both cases\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/153752","number":153752,"mergeCommit":{"message":"[Fleet] Fix error when creating new logstash output (#153752)\n\nFixes https://github.com/elastic/kibana/issues/153622\r\n\r\n## Summary\r\n[Fleet] Fix an error when creating new logstash output. \r\n\r\nThe bug was added with #153226 and\r\ndepended on this [else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat shouldn't have been added.\r\n\r\nI also added unit and integration tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n- Create a new `logstash` output, it can be `default` or not\r\n- The output creation works as expected in both cases\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2"}}]}] BACKPORT--> Co-authored-by: Cristina Amico <criamico@users.noreply.github.com>
kpollich
added
v8.7.1
QA:Needs Validation
1 task
criamico
added a commit
that referenced
this pull request
Apr 11, 2023
…igured policies (#154445) Closes #154326 ## Summary After the merge of #153226, when creating/updating a Logstash output as default, the `Elastic Cloud agent policy` preconfigured on Cloud gets reassigned to the "default" ES policy instead than keeping the `Elastic Cloud internal output`. <img width="1418" alt="Screenshot 2023-04-04 at 12 51 21" src="https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png"> Tee bug is fixed by checking if any given fleet server policy is `preconfigured` or if it has already an assigned `data_output_id`, in which cases it doesn't get updated. ### Testing - Create an ES output additional to the default one - Create a preconfigured fleet server policy and make sure it has `fleet-server` integration (this is to simulate the preconfigured cloud policy) - Assign the previous output to the preconfigured policy - Now create a new `logstash` output and make it default - Check that the preconfigured policy maintains the custom output previously assigned ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
nchaulet
pushed a commit
to nchaulet/kibana
that referenced
this pull request
Apr 11, 2023
…igured policies (elastic#154445) Closes elastic#154326 ## Summary After the merge of elastic#153226, when creating/updating a Logstash output as default, the `Elastic Cloud agent policy` preconfigured on Cloud gets reassigned to the "default" ES policy instead than keeping the `Elastic Cloud internal output`. <img width="1418" alt="Screenshot 2023-04-04 at 12 51 21" src="https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png"> Tee bug is fixed by checking if any given fleet server policy is `preconfigured` or if it has already an assigned `data_output_id`, in which cases it doesn't get updated. ### Testing - Create an ES output additional to the default one - Create a preconfigured fleet server policy and make sure it has `fleet-server` integration (this is to simulate the preconfigured cloud policy) - Assign the previous output to the preconfigured policy - Now create a new `logstash` output and make it default - Check that the preconfigured policy maintains the custom output previously assigned ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co> (cherry picked from commit a5de314) # Conflicts: # x-pack/plugins/fleet/server/services/output.test.ts
nchaulet
added a commit
that referenced
this pull request
Apr 12, 2023
…reconfigured policies (#154445) (#154771) # Backport This will backport the following commits from `main` to `8.7`: - [[Fleet] Logstash Output - prevent updating data_output_id for preconfigured policies (#154445)](#154445) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Cristina Amico","email":"criamico@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-04-11T18:38:17Z","message":"[Fleet] Logstash Output - prevent updating data_output_id for preconfigured policies (#154445)\n\nCloses https://github.com/elastic/kibana/issues/154326\r\n\r\n## Summary\r\n\r\nAfter the merge of #153226, when\r\ncreating/updating a Logstash output as default, the `Elastic Cloud agent\r\npolicy` preconfigured on Cloud gets reassigned to the \"default\" ES\r\npolicy instead than keeping the `Elastic Cloud internal output`.\r\n\r\n<img width=\"1418\" alt=\"Screenshot 2023-04-04 at 12 51 21\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png\">\r\n\r\nTee bug is fixed by checking if any given fleet server policy is\r\n`preconfigured` or if it has already an assigned `data_output_id`, in\r\nwhich cases it doesn't get updated.\r\n\r\n### Testing\r\n- Create an ES output additional to the default one\r\n- Create a preconfigured fleet server policy and make sure it has\r\n`fleet-server` integration (this is to simulate the preconfigured cloud\r\npolicy)\r\n- Assign the previous output to the preconfigured policy\r\n- Now create a new `logstash` output and make it default\r\n- Check that the preconfigured policy maintains the custom output\r\npreviously assigned\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>","sha":"a5de314687a195f4b73fb4b10bf6d3f7f5e58fd9","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","backport:prev-minor","v8.8.0"],"number":154445,"url":"https://github.com/elastic/kibana/pull/154445","mergeCommit":{"message":"[Fleet] Logstash Output - prevent updating data_output_id for preconfigured policies (#154445)\n\nCloses https://github.com/elastic/kibana/issues/154326\r\n\r\n## Summary\r\n\r\nAfter the merge of #153226, when\r\ncreating/updating a Logstash output as default, the `Elastic Cloud agent\r\npolicy` preconfigured on Cloud gets reassigned to the \"default\" ES\r\npolicy instead than keeping the `Elastic Cloud internal output`.\r\n\r\n<img width=\"1418\" alt=\"Screenshot 2023-04-04 at 12 51 21\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png\">\r\n\r\nTee bug is fixed by checking if any given fleet server policy is\r\n`preconfigured` or if it has already an assigned `data_output_id`, in\r\nwhich cases it doesn't get updated.\r\n\r\n### Testing\r\n- Create an ES output additional to the default one\r\n- Create a preconfigured fleet server policy and make sure it has\r\n`fleet-server` integration (this is to simulate the preconfigured cloud\r\npolicy)\r\n- Assign the previous output to the preconfigured policy\r\n- Now create a new `logstash` output and make it default\r\n- Check that the preconfigured policy maintains the custom output\r\npreviously assigned\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>","sha":"a5de314687a195f4b73fb4b10bf6d3f7f5e58fd9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/154445","number":154445,"mergeCommit":{"message":"[Fleet] Logstash Output - prevent updating data_output_id for preconfigured policies (#154445)\n\nCloses https://github.com/elastic/kibana/issues/154326\r\n\r\n## Summary\r\n\r\nAfter the merge of #153226, when\r\ncreating/updating a Logstash output as default, the `Elastic Cloud agent\r\npolicy` preconfigured on Cloud gets reassigned to the \"default\" ES\r\npolicy instead than keeping the `Elastic Cloud internal output`.\r\n\r\n<img width=\"1418\" alt=\"Screenshot 2023-04-04 at 12 51 21\"\r\nsrc=\"https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png\">\r\n\r\nTee bug is fixed by checking if any given fleet server policy is\r\n`preconfigured` or if it has already an assigned `data_output_id`, in\r\nwhich cases it doesn't get updated.\r\n\r\n### Testing\r\n- Create an ES output additional to the default one\r\n- Create a preconfigured fleet server policy and make sure it has\r\n`fleet-server` integration (this is to simulate the preconfigured cloud\r\npolicy)\r\n- Assign the previous output to the preconfigured policy\r\n- Now create a new `logstash` output and make it default\r\n- Check that the preconfigured policy maintains the custom output\r\npreviously assigned\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>","sha":"a5de314687a195f4b73fb4b10bf6d3f7f5e58fd9"}}]}] BACKPORT--> --------- Co-authored-by: Cristina Amico <criamico@users.noreply.github.com>
|
Hi @criamico We have revalidated this PR on latest 8.7.1 BC1 kibana cloud environment We created a policy with Fleet Server integration and had below observations:
Screenshots:
Screenshot:
Screenshots:
Screenshot:
Screenshots: Build details: Could you please confirm if the above shared results are expected? Please let us know if we are missing anything here. Thanks! |
|
Hi @amolnater-qasource, I can confirm that these are all following the expected behavior. |
|
Hi @criamico Thank you for the confirmation. Further as per feedback from @kpollich, we have created 05 testcases for these changes under Fleet test suite at links:
Please let us know if anything else is required from our end. |
|
Hi Team, We have executed test run for this feature on latest 8.7.1 BC3 kibana cloud environment at link: Status: Build details: Hence we are marking this feature as QA:Validated. |
amolnater-qasource
added
QA:Validated
2 tasks
jillguyonnet
added a commit
that referenced
this pull request
Jul 18, 2023
## Summary Closes #155725 ### Context Basic license users cannot use Synthetics private locations and a default Logstash output (cf. [this comment](#155725 (comment))). This is because under Basic license per policy output is not allowed. There was a [similar issue](#152234) with Fleet server that was fixed by adding logic to exclude the fleet server integration from the checks ([PR](#153226)). This PR follows a very similar approach for the synthetics integration. ### Changes Compare these to [the corresponding changes that were made for Fleet server](#153226). - If the user tries to change the default output from ES to Logstash and there are agent policies using synthetics, then an error is thrown. - If an existing Logstash output is made to be the default output and there are agent policies using synthetics, then these policies will still be using the previous default ES output. - If the default output is of type Logstash with an agent policy using synthetics and another ES output, then if the user tries to delete the ES output an error is thrown. - Logstash type outputs are disabled in the agent policy settings if the policy uses synthetics. ### Steps for testing⚠️ Ideally the complete scenario reported [here](#155725 (comment)) should be tested. I was having some issues running a dev setup with a proper Logstash output, so the steps below are not end-to-end (similar to #153226). 1. Start with a single default ES output, an agent policy (1) with synthetics and another (2) without (NB: you can also test combinations with fleet server; if an error is thrown, the error for fleet server would be thrown before the one for synthetics). 2. Edit the default ES output to be a Logstash output; this should fail and thrown the following error:  3. Create a new Logstash output and make it default on creation. Go the agent policies and check the policies settings: the `Output for integrations` setting should still be set to the previous ES default output for policies with synthetics (and/or fleet server), and should have switched to the new Logstash default for other policies. 4. Repeat the previous steps, but this time first create the Logstash output and then edit it to make it default. This tests the update behaviour as opposed to create. You will see the following warning:  5. Still with the default Logstash output, try to delete the ES output (that is still used by policies with synthetics/fleet server(. This should fail with the following error:  6. (Basic license constraint) Create another ES output (not default). Go the agent policies and check the policies settings: policies with synthetics (or fleet server) should be allowed to switch the `Output for integrations` setting to the new ES output:  In contrast, policies without these integrations should be constrained to the default Logstash output:  ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
ThomThomson
pushed a commit
to ThomThomson/kibana
that referenced
this pull request
Aug 1, 2023
## Summary Closes elastic#155725 ### Context Basic license users cannot use Synthetics private locations and a default Logstash output (cf. [this comment](elastic#155725 (comment))). This is because under Basic license per policy output is not allowed. There was a [similar issue](elastic#152234) with Fleet server that was fixed by adding logic to exclude the fleet server integration from the checks ([PR](elastic#153226)). This PR follows a very similar approach for the synthetics integration. ### Changes Compare these to [the corresponding changes that were made for Fleet server](elastic#153226). - If the user tries to change the default output from ES to Logstash and there are agent policies using synthetics, then an error is thrown. - If an existing Logstash output is made to be the default output and there are agent policies using synthetics, then these policies will still be using the previous default ES output. - If the default output is of type Logstash with an agent policy using synthetics and another ES output, then if the user tries to delete the ES output an error is thrown. - Logstash type outputs are disabled in the agent policy settings if the policy uses synthetics. ### Steps for testing⚠️ Ideally the complete scenario reported [here](elastic#155725 (comment)) should be tested. I was having some issues running a dev setup with a proper Logstash output, so the steps below are not end-to-end (similar to elastic#153226). 1. Start with a single default ES output, an agent policy (1) with synthetics and another (2) without (NB: you can also test combinations with fleet server; if an error is thrown, the error for fleet server would be thrown before the one for synthetics). 2. Edit the default ES output to be a Logstash output; this should fail and thrown the following error:  3. Create a new Logstash output and make it default on creation. Go the agent policies and check the policies settings: the `Output for integrations` setting should still be set to the previous ES default output for policies with synthetics (and/or fleet server), and should have switched to the new Logstash default for other policies. 4. Repeat the previous steps, but this time first create the Logstash output and then edit it to make it default. This tests the update behaviour as opposed to create. You will see the following warning:  5. Still with the default Logstash output, try to delete the ES output (that is still used by policies with synthetics/fleet server(. This should fail with the following error:  6. (Basic license constraint) Create another ES output (not default). Go the agent policies and check the policies settings: policies with synthetics (or fleet server) should be allowed to switch the `Output for integrations` setting to the new ES output:  In contrast, policies without these integrations should be constrained to the default Logstash output:  ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
![https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png\">\r\n\r\n###](https://user-images.githubusercontent.com/16084106/225665515-6090b85c-b84f-4295-9b30-c480d2a84879.png\">\r\n\r\n###){kind=link}
![https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png\">\r\n\r\n3](https://user-images.githubusercontent.com/16084106/226687174-296e6e2b-3cef-4649-8173-18ee70c6a4a7.png\">\r\n\r\n3){kind=link}
![https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png\">\r\n\r\n6](https://user-images.githubusercontent.com/16084106/226687206-f22e1330-4435-4077-a7ef-f29846f25e00.png\">\r\n\r\n6){kind=link}
![https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png\">\r\n\r\n###](https://user-images.githubusercontent.com/16084106/226687854-95d239a3-4122-470a-88e0-73c456ad5f7d.png\">\r\n\r\n###){kind=link}
![https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png\">\r\n\r\nTee](https://user-images.githubusercontent.com/16084106/230112067-a2767d1a-1191-4877-8dec-546d1590e41f.png\">\r\n\r\nTee){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #152234
Summary
EStologstashwhen there are Fleet server policies that use that output, we throw asking the user to create a new outputlogstashone, we update the Fleet server policies to use the previous default output (that should be ES)UI changes
When editing a "logstash" output the modal displays a warning:
Testing
Note if the previous output is logstash, it will be switched to the default one. However, users who updated the system and already had a logstash ouput set as a default will need to to manually set the output from the agent policy settings page.
elasticsearch, while the logstash ones are disabledChecklist