Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Add comment about ftr test environment differences #169596

Merged
merged 3 commits into from
Oct 24, 2023

Conversation

TinaHeiligers
Copy link
Contributor

@TinaHeiligers TinaHeiligers commented Oct 23, 2023

Skipped tests are release blockers and fixing flaky tests isn't always easy, especially if we assume that if a test passes locally, it will also pass in CI.

We need to keep the differences between our local environments and CI in mind and make appropriate changes that take these differences into account but to do that, we first need to know what these differences are!

This PR adds a high level description of a couple of key differences.

Checklist

Delete any items that are not applicable to this PR.

@TinaHeiligers TinaHeiligers requested a review from a team October 23, 2023 22:34
@TinaHeiligers TinaHeiligers added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting v8.12.0 backport:all-open Backport to all branches that could still receive a release and removed backport:skip This commit does not require backporting labels Oct 23, 2023
Copy link
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this! 🧡

dev_docs/operations/writing_stable_functional_tests.mdx Outdated Show resolved Hide resolved
grammar correction

Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
@TinaHeiligers TinaHeiligers enabled auto-merge (squash) October 24, 2023 14:29
@TinaHeiligers TinaHeiligers merged commit 456d980 into elastic:main Oct 24, 2023
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 24, 2023
…9596)

Skipped tests are release blockers and fixing flaky tests isn't always
easy, especially if we assume that if a test passes locally, it will
also pass in CI.

We need to keep the differences between our local environments and CI in
mind and make appropriate changes that take these differences into
account but to do that, we first need to know what these differences
are!

This PR adds a high level description of a couple of key differences.

### Checklist

Delete any items that are not applicable to this PR.

- [X] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials

---------

Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
(cherry picked from commit 456d980)
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
7.17 Backport failed because of merge conflicts
8.11

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 169596

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 24, 2023
…9596) (#169665)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[docs] Add comment about ftr test environment differences
(#169596)](#169596)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Christiane (Tina)
Heiligers","email":"christiane.heiligers@elastic.co"},"sourceCommit":{"committedDate":"2023-10-24T14:46:03Z","message":"[docs]
Add comment about ftr test environment differences (#169596)\n\nSkipped
tests are release blockers and fixing flaky tests isn't always\r\neasy,
especially if we assume that if a test passes locally, it will\r\nalso
pass in CI.\r\n\r\nWe need to keep the differences between our local
environments and CI in\r\nmind and make appropriate changes that take
these differences into\r\naccount but to do that, we first need to know
what these differences\r\nare!\r\n\r\nThis PR adds a high level
description of a couple of key differences.\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or
tutorials\r\n\r\n---------\r\n\r\nCo-authored-by: Alejandro Fernández
Haro
<afharo@gmail.com>","sha":"456d980e771846ee82ae246c7fca204d4a4c9f50","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v8.12.0"],"number":169596,"url":"https://github.com/elastic/kibana/pull/169596","mergeCommit":{"message":"[docs]
Add comment about ftr test environment differences (#169596)\n\nSkipped
tests are release blockers and fixing flaky tests isn't always\r\neasy,
especially if we assume that if a test passes locally, it will\r\nalso
pass in CI.\r\n\r\nWe need to keep the differences between our local
environments and CI in\r\nmind and make appropriate changes that take
these differences into\r\naccount but to do that, we first need to know
what these differences\r\nare!\r\n\r\nThis PR adds a high level
description of a couple of key differences.\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or
tutorials\r\n\r\n---------\r\n\r\nCo-authored-by: Alejandro Fernández
Haro
<afharo@gmail.com>","sha":"456d980e771846ee82ae246c7fca204d4a4c9f50"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169596","number":169596,"mergeCommit":{"message":"[docs]
Add comment about ftr test environment differences (#169596)\n\nSkipped
tests are release blockers and fixing flaky tests isn't always\r\neasy,
especially if we assume that if a test passes locally, it will\r\nalso
pass in CI.\r\n\r\nWe need to keep the differences between our local
environments and CI in\r\nmind and make appropriate changes that take
these differences into\r\naccount but to do that, we first need to know
what these differences\r\nare!\r\n\r\nThis PR adds a high level
description of a couple of key differences.\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or
tutorials\r\n\r\n---------\r\n\r\nCo-authored-by: Alejandro Fernández
Haro
<afharo@gmail.com>","sha":"456d980e771846ee82ae246c7fca204d4a4c9f50"}}]}]
BACKPORT-->

Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
adamkasztenny added a commit to adamkasztenny/kibana that referenced this pull request Oct 24, 2023
adamkasztenny added a commit that referenced this pull request Oct 24, 2023
## Summary

Follow up to #169596

---------

Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 24, 2023
## Summary

Follow up to elastic#169596

---------

Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
(cherry picked from commit 6f30fa3)
kibanamachine referenced this pull request Oct 24, 2023
# Backport

This will backport the following commits from `main` to `8.11`:
- [[docs] Fix functional test doc formatting
(#169715)](#169715)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Adam
Kasztenny","email":"adam.kasztenny@elastic.co"},"sourceCommit":{"committedDate":"2023-10-24T20:45:52Z","message":"[docs]
Fix functional test doc formatting (#169715)\n\n## Summary\r\n\r\nFollow
up to
https://github.com/elastic/kibana/pull/169596\r\n\r\n---------\r\n\r\nCo-authored-by:
Christiane (Tina) Heiligers
<christiane.heiligers@elastic.co>","sha":"6f30fa32857fa8352c50327ecb55e60e1f879a29","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","v8.12.0"],"number":169715,"url":"https://github.com/elastic/kibana/pull/169715","mergeCommit":{"message":"[docs]
Fix functional test doc formatting (#169715)\n\n## Summary\r\n\r\nFollow
up to
https://github.com/elastic/kibana/pull/169596\r\n\r\n---------\r\n\r\nCo-authored-by:
Christiane (Tina) Heiligers
<christiane.heiligers@elastic.co>","sha":"6f30fa32857fa8352c50327ecb55e60e1f879a29"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169715","number":169715,"mergeCommit":{"message":"[docs]
Fix functional test doc formatting (#169715)\n\n## Summary\r\n\r\nFollow
up to
https://github.com/elastic/kibana/pull/169596\r\n\r\n---------\r\n\r\nCo-authored-by:
Christiane (Tina) Heiligers
<christiane.heiligers@elastic.co>","sha":"6f30fa32857fa8352c50327ecb55e60e1f879a29"}}]}]
BACKPORT-->

Co-authored-by: Adam Kasztenny <adam.kasztenny@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:all-open Backport to all branches that could still receive a release release_note:skip Skip the PR/issue when compiling release notes v8.11.0 v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants