Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Unskip rules bulk actions Serverless Cypress tests #168307

Conversation

maximpn
Copy link
Contributor

@maximpn maximpn commented Oct 7, 2023

Addresses: #161540

Summary

This PR unskips rules bulk actions Serverless Cypress tests (besides role based bulk_edit_rules_actions.cy.ts)

  • bulk_duplicate_rules.cy.ts
  • bulk_edit_rules.cy.ts
  • bulk_edit_rules_data_view.cy.ts

Flaky test runner

detection_response/rule_management/rule_actions/bulk_actions folder 150 runs 🟢

@maximpn maximpn added test release_note:fix Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team v8.11.0 labels Oct 7, 2023
@maximpn maximpn self-assigned this Oct 7, 2023
@maximpn maximpn added release_note:skip Skip the PR/issue when compiling release notes and removed release_note:fix labels Oct 7, 2023
@maximpn maximpn requested a review from banderror October 7, 2023 12:24
@maximpn maximpn force-pushed the unskip-rules-bulk-actions-serverless-cypress-test branch from afecfcf to 06a1e3a Compare October 7, 2023 20:38
@maximpn maximpn marked this pull request as ready for review October 7, 2023 20:38
@maximpn maximpn requested a review from a team as a code owner October 7, 2023 20:38
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@maximpn
Copy link
Contributor Author

maximpn commented Oct 8, 2023

@elasticmachine merge upstream

3 similar comments
@maximpn
Copy link
Contributor Author

maximpn commented Oct 9, 2023

@elasticmachine merge upstream

@banderror
Copy link
Contributor

@elasticmachine merge upstream

@banderror
Copy link
Contributor

@elasticmachine merge upstream

@banderror
Copy link
Contributor

@maximpn I think there's something wrong with the changed tests because the Security Solution Cypress tests, burning changed specs job has been consistently failing in this PR.

@banderror banderror marked this pull request as draft October 13, 2023 10:27
@maximpn maximpn force-pushed the unskip-rules-bulk-actions-serverless-cypress-test branch 2 times, most recently from 2b00b77 to 1ed17f3 Compare October 17, 2023 15:23
@maximpn
Copy link
Contributor Author

maximpn commented Oct 17, 2023

@banderror Security Solution Cypress tests, burning changed specs failed due to a timeout as each test repeated 5 times. I rolled it back to two repeats so it passed the last time.

@maximpn maximpn marked this pull request as ready for review October 17, 2023 19:59
@maximpn maximpn force-pushed the unskip-rules-bulk-actions-serverless-cypress-test branch 2 times, most recently from 1ed17f3 to da8e814 Compare October 18, 2023 11:04
Copy link
Contributor

@banderror banderror left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is my understanding correct that bulk_edit_rules_actions.cy.ts is skipped because all tests in this file depend on roles? It just doesn't look like from the code, maybe we could unskip a subset of them that doesn't depend on roles.

https://github.com/banderror/kibana/blob/da8e81434fa7a435b77d84b5532e2c6479dd2e99/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/rule_actions/bulk_actions/bulk_edit_rules_actions.cy.ts#L77-L81

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maximpn maximpn force-pushed the unskip-rules-bulk-actions-serverless-cypress-test branch 2 times, most recently from 8e01918 to 6f8bca8 Compare October 18, 2023 16:29
@maximpn maximpn requested a review from banderror October 18, 2023 16:32
Copy link
Contributor

@banderror banderror left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you for addressing the comments @maximpn! 🚀

@kibana-ci
Copy link
Collaborator

kibana-ci commented Oct 18, 2023

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #75 / visualize app visual builder table should display correct values on changing metrics aggregation

Metrics [docs]

✅ unchanged

History

  • 💚 Build #169122 succeeded 6f8bca84984af6fa58b0a3b640da01a3bfcea3f9
  • 💛 Build #169085 was flaky 8e01918465e617f546d8fd43b25689cddcdefc3b
  • 💛 Build #168967 was flaky da8e81434fa7a435b77d84b5532e2c6479dd2e99
  • 💔 Build #168780 failed 22f0e75ae25570db7cac326076d91f22aaa75626

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maximpn

@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.11

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2023
elastic#168307)

**Addresses:** elastic#161540

## Summary

This PR unskips rules bulk actions Serverless Cypress tests (besides role based `bulk_edit_rules_actions.cy.ts`)

- `bulk_duplicate_rules.cy.ts`
- `bulk_edit_rules.cy.ts`
- `bulk_edit_rules_data_view.cy.ts`

## Flaky test runner

`detection_response/rule_management/rule_actions/bulk_actions` folder [150 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3596) 🟢

(cherry picked from commit 4f91082)
kibanamachine added a commit that referenced this pull request Oct 18, 2023
…ss tests (#168307) (#169322)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Unskip rules bulk actions Serverless Cypress
tests (#168307)](#168307)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2023-10-18T22:26:55Z","message":"[Security
Solution] Unskip rules bulk actions Serverless Cypress tests
(#168307)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/161540\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules bulk actions Serverless Cypress
tests (besides role based `bulk_edit_rules_actions.cy.ts`)\r\n\r\n-
`bulk_duplicate_rules.cy.ts`\r\n- `bulk_edit_rules.cy.ts`\r\n-
`bulk_edit_rules_data_view.cy.ts`\r\n\r\n## Flaky test
runner\r\n\r\n\r\n`detection_response/rule_management/rule_actions/bulk_actions`
folder [150
runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3596)
🟢","sha":"4f9108273c3bbdf03af26a8a0d2b7a571311751a","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","v8.11.0","v8.12.0"],"number":168307,"url":"https://github.com/elastic/kibana/pull/168307","mergeCommit":{"message":"[Security
Solution] Unskip rules bulk actions Serverless Cypress tests
(#168307)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/161540\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules bulk actions Serverless Cypress
tests (besides role based `bulk_edit_rules_actions.cy.ts`)\r\n\r\n-
`bulk_duplicate_rules.cy.ts`\r\n- `bulk_edit_rules.cy.ts`\r\n-
`bulk_edit_rules_data_view.cy.ts`\r\n\r\n## Flaky test
runner\r\n\r\n\r\n`detection_response/rule_management/rule_actions/bulk_actions`
folder [150
runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3596)
🟢","sha":"4f9108273c3bbdf03af26a8a0d2b7a571311751a"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/168307","number":168307,"mergeCommit":{"message":"[Security
Solution] Unskip rules bulk actions Serverless Cypress tests
(#168307)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/161540\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules bulk actions Serverless Cypress
tests (besides role based `bulk_edit_rules_actions.cy.ts`)\r\n\r\n-
`bulk_duplicate_rules.cy.ts`\r\n- `bulk_edit_rules.cy.ts`\r\n-
`bulk_edit_rules_data_view.cy.ts`\r\n\r\n## Flaky test
runner\r\n\r\n\r\n`detection_response/rule_management/rule_actions/bulk_actions`
folder [150
runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3596)
🟢","sha":"4f9108273c3bbdf03af26a8a0d2b7a571311751a"}}]}] BACKPORT-->

Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
@maximpn maximpn deleted the unskip-rules-bulk-actions-serverless-cypress-test branch October 19, 2023 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test v8.11.0 v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants