Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet]: Unable to add Logstash output under Fleet>Settings tab. #153622

Closed
amolnater-qasource opened this issue Mar 24, 2023 · 16 comments · Fixed by #153752
Closed

[Fleet]: Unable to add Logstash output under Fleet>Settings tab. #153622

amolnater-qasource opened this issue Mar 24, 2023 · 16 comments · Fixed by #153752
Assignees
@criamico
Labels
bug Fixes for quality problems that affect the customer experience impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@amolnater-qasource
Copy link

Kibana version: 8.7 BC9 kibana cloud environment

Host OS and Browser version: All, All

Build details:

VERSION: 8.7 BC9
BUILD: 61093
COMMIT: 8eda067283f541c673beb406ae5480da6dab9296

Preconditions:

  1. 8.7 BC9 kibana cloud environment should be available.

Steps to reproduce:

  1. Navigate to Fleet>Settings tab.
  2. Add Output>Select Logstash and add all the required fields.
  3. Click Save and apply settings and observe below error:
    Logstash output cannot be used with Fleet Server integration in Elastic Cloud agent policy. Please create a new ElasticSearch output.

Screen Recording:

Settings.-.Fleet.-.Elastic.-.Google.Chrome.2023-03-24.10-34-59.mp4

Expected Result:
User should be able to add Logstash output under Fleet>Settings tab.
@amolnater-qasource amolnater-qasource added bug Fixes for quality problems that affect the customer experience impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:Fleet Team label for Observability Data Collection Fleet team Team:Elastic-Agent-Control-Plane labels Mar 24, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@amolnater-qasource
Copy link
Author

@manishgupta-qasource Please review.

@manishgupta-qasource
Copy link

Secondary review for this ticket is Done

@amolnater-qasource amolnater-qasource mentioned this issue Mar 24, 2023
Closed
@jen-huang
Copy link
Contributor

@criamico This seems like it might have resulted from changes for #152234. Since this is just a new output that's not being set as default for anything, we shouldn't be blocking it, right?

@criamico criamico mentioned this issue Mar 27, 2023
Merged
1 task
@criamico
Copy link
Contributor

@jen-huang yes, you're right. I have a PR to fix it with a lot more tests to cover edge cases.

jen-huang pushed a commit that referenced this issue Mar 28, 2023
@criamico @kibanamachine
[Fleet] Fix error when creating new logstash output (#153752)
2e08470 
Fixes #153622

## Summary
[Fleet] Fix an error when creating new logstash output. 

The bug was added with #153226 and
depended on this [else
branch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)
that shouldn't have been added.

I also added unit and integration tests to cover for this and other
cases.


### Testing
- Create a new `logstash` output, it can be `default` or not
- The output creation works as expected in both cases

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Mar 28, 2023
@criamico
[Fleet] Fix error when creating new logstash output (elastic#153752)
169be89 
Fixes elastic#153622

## Summary
[Fleet] Fix an error when creating new logstash output.

The bug was added with elastic#153226 and
depended on this [else
branch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)
that shouldn't have been added.

I also added unit and integration tests to cover for this and other
cases.

### Testing
- Create a new `logstash` output, it can be `default` or not
- The output creation works as expected in both cases

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 2e08470)
kibanamachine referenced this issue Mar 28, 2023
@kibanamachine @criamico
[8.7] [Fleet] Fix error when creating new logstash output (#153752) (#…
d59a68f 
…153819)

# Backport

This will backport the following commits from `main` to `8.7`:
- [[Fleet] Fix error when creating new logstash output
(#153752)](#153752)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Cristina
Amico","email":"criamico@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-03-28T00:43:03Z","message":"[Fleet]
Fix error when creating new logstash output (#153752)\n\nFixes
https://github.com/elastic/kibana/issues/153622\r\n\r\n##
Summary\r\n[Fleet] Fix an error when creating new logstash output.
\r\n\r\nThe bug was added with
#153226 and\r\ndepended on this
[else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat
shouldn't have been added.\r\n\r\nI also added unit and integration
tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n-
Create a new `logstash` output, it can be `default` or not\r\n- The
output creation works as expected in both cases\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","backport:prev-minor","v8.8.0"],"number":153752,"url":"https://github.com/elastic/kibana/pull/153752","mergeCommit":{"message":"[Fleet]
Fix error when creating new logstash output (#153752)\n\nFixes
https://github.com/elastic/kibana/issues/153622\r\n\r\n##
Summary\r\n[Fleet] Fix an error when creating new logstash output.
\r\n\r\nThe bug was added with
#153226 and\r\ndepended on this
[else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat
shouldn't have been added.\r\n\r\nI also added unit and integration
tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n-
Create a new `logstash` output, it can be `default` or not\r\n- The
output creation works as expected in both cases\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/153752","number":153752,"mergeCommit":{"message":"[Fleet]
Fix error when creating new logstash output (#153752)\n\nFixes
https://github.com/elastic/kibana/issues/153622\r\n\r\n##
Summary\r\n[Fleet] Fix an error when creating new logstash output.
\r\n\r\nThe bug was added with
#153226 and\r\ndepended on this
[else\r\nbranch](https://github.com/criamico/kibana/blob/30d3843b142142d19f1fda8f8107f088c03ff1cb/x-pack/plugins/fleet/server/services/output.ts#L309-L312)\r\nthat
shouldn't have been added.\r\n\r\nI also added unit and integration
tests to cover for this and other\r\ncases.\r\n\r\n\r\n### Testing\r\n-
Create a new `logstash` output, it can be `default` or not\r\n- The
output creation works as expected in both cases\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"2e0847007a6e870b87b9401cf2360c015afb82a2"}}]}]
BACKPORT-->

Co-authored-by: Cristina Amico <criamico@users.noreply.github.com>
@amolnater-qasource amolnater-qasource added QA:Validated Issue has been validated by QA QA:Ready for Testing Code is merged and ready for QA to validate and removed QA:Validated Issue has been validated by QA labels Mar 28, 2023
@amolnater-qasource
Copy link
Author

Hi Team,

We have revalidated this issue on latest 8.7.0 BC10 and found it still reproducible.
Looks like the merges are not available in the latest BC.

Build details:
VERSION: 8.7.0
BUILD: 61109
COMMIT: 05f1259

Screen Recording:

Settings.-.Fleet.-.Elastic.-.Google.Chrome.2023-03-29.15-14-29.mp4

Please let us know if anything else is required from our end.
cc: @joshdover

Thanks!

@hop-dev
Copy link
Contributor

hop-dev commented Mar 29, 2023

The fix missed BC10, the job started at midnight and the backport for this was merged around 2am. From the looks of it we were just trying to sneak it in as opposed to blocking the release for it @jen-huang? There is no mention of BC11 yet but you never know

@criamico
Copy link
Contributor

@hop-dev Do you think we are still in time for 8.7.0? I see that FF is tomorrow (Thu the 30th).

@hop-dev
Copy link
Contributor

hop-dev commented Mar 29, 2023

I think tomorrow is the release date not FF. The only way to make 8.7.0 is if another build candidate is done and I can't see any mention of another one being done. We could raise this as a release blocker but I think we would have to have a very strong case.

@criamico
Copy link
Contributor

criamico commented Mar 29, 2023 •
edited
Loading

@jlind23 What do you suggest? This bug prevents the creation of any new logstash policy as long as the customer has already a policy with fleet server installed. :/

@criamico
Copy link
Contributor

criamico commented Mar 29, 2023 •
edited
Loading

As discussed with @jlind23, the bug wasn't identified as a blocker for 8.7.0 so it should be fine.

There is a workaround for customers trying to add a new Logstash output; the bug doesn't affect existing outputs, but only the creation of new ones.

I'll document the workaround in a Known Issue.

@jen-huang
Copy link
Contributor

@amolnater-qasource In the meantime you can validate this with a snapshot build of 8.7.1 or 8.8.0. Thank you!

@amolnater-qasource
Copy link
Author

Hi @jen-huang

We have revalidated this issue on latest 8.8.0-SNAPSHOT and found it fixed there.

  • We are successfully able to add Logstash output under Fleet>Settings tab.

Screen Recording:

Settings.-.Fleet.-.Elastic.-.Google.Chrome.2023-03-31.10-24-13.mp4

Build details:
BUILD: 61903
COMMIT: 5437cda

Further, we are currently unable to deploy latest 8.7.1 SNAPSHOT and we will revalidate once we are successfully able to deploy the latest 8.7.1 build.
Please let us know if separate issue is required to be logged for this deployment issue.

image

Please let us know if anything else is required from our end.
Thanks!

@amolnater-qasource
Copy link
Author

Hi Team,

We are successfully able to deploy 8.7.1 SNAPSHOT today, so we have revalidated this issue and found it fixed there too.

  • We are successfully able to add Logstash output under Fleet>Settings tab.

Screen Recording:

Settings.-.Fleet.-.Elastic.-.Google.Chrome.2023-04-03.11-39-00.mp4

Build details:
VERSION: 8.7.1
BUILD: 61140
COMMIT: b26cf5e

Further we will mark this issue as QA:Validated once testing on 8.7.1 BC1 [when available].
Thanks!

@dikshachauhan-qasource
Copy link

Bug Conversion

Test-Case not required as this particular checkpoint is already covered in the following testcase:

Thanks!

@amolnater-qasource
Copy link
Author

amolnater-qasource commented Apr 19, 2023 •
edited
Loading

Hi Team,

We have revalidated this on latest 8.7.1 BC2 kibana cloud environment and found it fixed now.

Observations:

  • We are successfully able to add Logstash output under Fleet>Settings tab.

Build details:
VERSION: 8.7.1 BC2
BUILD: 61192
COMMIT: e33260d

Screenshot:
image

Hence, we are marking this issue as QA:Validated.

Please let us know if anything else is required from our end.
Thanks!

@amolnater-qasource amolnater-qasource added QA:Validated Issue has been validated by QA and removed QA:Ready for Testing Code is merged and ready for QA to validate labels Apr 19, 2023
@jen-huang jen-huang mentioned this issue May 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@criamico criamico

Labels
bug Fixes for quality problems that affect the customer experience impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fleet] Fix error when creating new logstash output criamico/kibana
7 participants
@jen-huang @hop-dev @elasticmachine @criamico @manishgupta-qasource @dikshachauhan-qasource @amolnater-qasource