📖 Add trigger phrases and status badge for e2e tests

README.md

@@ -4,17 +4,18 @@
 [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/metal3-io/baremetal-operator/badge)](https://securityscorecards.dev/viewer/?uri=github.com/metal3-io/baremetal-operator)
 [![Ubuntu daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_ubuntu&subject=Ubuntu%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_ubuntu/)
 [![CentOS daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_centos&subject=CentOS%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_centos/)
+[![BMO e2e periodic main](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/badge/icon?subject=BMO%20e2e%20periodic%20main)](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/)
 
 The Bare Metal Operator implements a Kubernetes API for managing bare metal
 hosts. It maintains an inventory of available hosts as instances of the
 `BareMetalHost` Custom Resource Definition. The Bare Metal Operator knows how
 to:
 
-* Inspect the host’s hardware details and report them on the corresponding
+- Inspect the host’s hardware details and report them on the corresponding
   `BareMetalHost`. This includes information about CPUs, RAM, disks, NICs, and
   more.
-* Provision hosts with a desired image.
-* Clean a host’s disk contents before or after provisioning.
+- Provision hosts with a desired image.
+- Clean a host’s disk contents before or after provisioning.
 
 More capabilities are being added regularly. See open issues and pull requests
 for more information on work in progress.
@@ -24,11 +25,11 @@ components, see the [Metal³ docs](https://github.com/metal3-io/metal3-docs).
 
 ## Resources
 
-* [API documentation](docs/api.md)
-* [Setup Development Environment](docs/dev-setup.md)
-* [Configuration](docs/configuration.md)
-* [Testing](docs/testing.md)
-* [Publishing Images](docs/publishing-images.md)
+- [API documentation](docs/api.md)
+- [Setup Development Environment](docs/dev-setup.md)
+- [Configuration](docs/configuration.md)
+- [Testing](docs/testing.md)
+- [Publishing Images](docs/publishing-images.md)
 
 ## Integration tests
 
@@ -39,32 +40,41 @@ Eventually these tests will be replaced with BMO's own
 [e2e tests](test/e2e/README.md). BMO e2e tests are currently under active
 development. Here are the tests which run on different branches in BMO:
 
+**E2E tests:**
+
+- **/test metal3-bmo-e2e-test-pull** runs required BMO e2e tests on your PR.
+  Works for main and release-0.5 branches.
+- **/test metal3-bmo-e2e-test-optional-pull** runs optional BMO e2e tests on
+  your PR. These are under active development currently.
+
+For more details on the e2e tests, see [test/e2e/README.md](test/e2e/README.md).
+
 **Main branch:**
 
-* **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with
+- **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with
   CAPM3 main branch and BMO **main** branch on Centos
-* **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3
+- **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3
   main branch and BMO **main** branch on Ubuntu
 
 **Release-0.5 branch:**
 
-* **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests
+- **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests
   with CAPM3 release-1.6 branch and BMO **release-0.5** branch on Centos
-* **/test-ubuntu-integration-release-1-6** runs ansible integration tests with
+- **/test-ubuntu-integration-release-1-6** runs ansible integration tests with
   CAPM3 release-1.6 branch and BMO **release-0.5** branch on Ubuntu
 
 **Release-0.4 branch:**
 
-* **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests
+- **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests
   with CAPM3 release-1.5 branch and BMO **release-0.4** branch on Centos
-* **/test-ubuntu-integration-release-1-5** runs ansible integration tests with
+- **/test-ubuntu-integration-release-1-5** runs ansible integration tests with
   CAPM3 release-1.5 branch and BMO **release-0.4** branch on Ubuntu
 
 **Release-0.3 branch:**
 
-* **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests
+- **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests
   with CAPM3 release-1.4 branch and BMO **release-0.3** branch on Centos
-* **/test-ubuntu-integration-release-1-4** runs ansible integration tests with
+- **/test-ubuntu-integration-release-1-4** runs ansible integration tests with
   CAPM3 release-1.4 branch and BMO **release-0.3** branch on Ubuntu
 
 ### Important Notes

test/e2e/README.md

@@ -11,14 +11,15 @@ in `config`.
 In the CI system we set up a pre-configured minikube cluster that is then used
 for the tests, as seen in [ci-e2e.sh](../../hack/ci-e2e.sh). This allows us to
 have control over the network and have a static configuration for Ironic and the
-Baremetal Operator. The script creates a VM to represent a BareMetalHost,
-and configures either VBMC or sushy-tools to be used as BMC.
+Baremetal Operator. The script creates a VM to represent a BareMetalHost, and
+configures either VBMC or sushy-tools to be used as BMC.
 
 Currently there are two sets of tests, which cannot be ran together in the same
-cluster. One is the "optional" set, currently consists of only the [upgrade tests](upgrade_test.go),
-and the "main" set, which are the ones required to run in every BMO PR. One can
-switch between these sets by manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env
-vars. In the default setting, the script sets `GINKGO_SKIP` to `upgrade`.
+cluster. One is the "optional" set, currently consists of only the
+[upgrade tests](upgrade_test.go), and the "main" set, which are the ones
+required to run in every BMO PR. One can switch between these sets by
+manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env vars. In the default
+setting, the script sets `GINKGO_SKIP` to `upgrade`.
 
 E.g. Here is how to run the E2E main tests:
 
@@ -34,13 +35,44 @@ export GINKGO_FOCUS="upgrade"
 ```
 
 It is also possible to run the tests with the fixture provider instead of
-Ironic. Without any changes, the whole suite (including optional tests) will be run.
-Please note, however, that it is quite questionable to call this configuration
-"end to end".
+Ironic. Without any changes, the whole suite (including optional tests) will be
+run. Please note, however, that it is quite questionable to call this
+configuration "end to end".
 
-Fixture provider is configured to be the default one. This is how to run
-the e2e test suite with it:
+Fixture provider is configured to be the default one. This is how to run the e2e
+test suite with it:
 
 ```bash
 make test-e2e
 ```
+
+## Tests
+
+Here is a list of the currently implemented tests. As mentioned above, we divide
+them into two sets, one required and one optional. The required tests must pass
+on every PR before merging. Optional tests may be triggered for extra
+verification but are not expected to run on all PRs. All tests are checked with
+both redfish and ipmi protocols in CI.
+
+**Required tests:**
+
+- Inspection: Check that a BareMetalHost is registered and inspected when
+  created, and that it becomes available after the inspection.
+- External inspection: Check that a BareMetalHost, with inspection disabled and
+  hardware details added through an annotation, skips inspection, accepts the
+  hardware details from the annotation and becomes available.
+- Re-inspection: Check that an available BareMetalHost is re-inspected when the
+  inspect annotation is added and that this updates the hardware details.
+- Provisioning: Check that an available BareMetalHost can be provisioned, that
+  it can be detatched, deleted and re-created, with the status annotation,
+  without affecting the host. Finally checks that the BareMetalHost can be
+  deprovisioned and becomes available again.
+- Live-ISO: Check that an available BareMetalHost can be provisioned with a
+  live-ISO image.
+
+**Optional tests:**
+
+- Bare Metal Operator upgrade: Check that an older version of the Bare Metal
+  Operator works by creating a BareMetalHost with external inspection. Then
+  upgrade the Bare Metal Operator and check the the BareMetalHost can be
+  provisioned with the upgraded version.