From 7455fbc310074da7a44d833926ca1a89e22c75a2 Mon Sep 17 00:00:00 2001 From: Sandra G Date: Mon, 30 Sep 2024 19:07:01 -0400 Subject: [PATCH] Serverless API integration tests (#192219) Tests for serverless - copies over and modifies all tests from stateful to work in serverless. ~~deployment agnostic tests do not yet support enterprise license for stateful, so are tests don't yet qualify as being deployment agnostic~~. Given how difficult it is to see differences from the stateful tests, I've added PR comments where I've changed something that might be of interest. - changes to `createObservabilityAIAssistantApiClient` to use supertest without basic auth and accept headers for serverless and use roles - removes creating persisted users when tests start and [use roles](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#roles-based-testing) within tests. its not possible to create custom users with the serverless test framework at the moment. See https://github.com/elastic/kibana/issues/192711 Skipped tests - knowledge base tests https://github.com/elastic/kibana/issues/192886 - any test suite that uses the LLM proxy has been skipped on MKI https://github.com/elastic/kibana/issues/192751 - all tests that depend on the config.modelId skipped in MKI https://github.com/elastic/kibana/issues/192757 TODO: - [x] move over remaining tests - [x] test in MKI environment before merging - [x] create issues for skipped tests - [ ] this will not run on MKI (after merging) unless we ping the appex-qa team to add it to the pipeline. this is due to creating a separate config. ask appex-qa team to add our config. Followup / related issues to be tracked in a newly created issue: - [ ] https://github.com/elastic/kibana/issues/192757 - [ ] https://github.com/elastic/kibana/issues/192886 - [ ] https://github.com/elastic/kibana/issues/192751 - [ ] https://github.com/elastic/kibana/issues/192701 - [ ] https://github.com/elastic/kibana/issues/192497 - [ ] https://github.com/elastic/kibana/issues/192711 - [ ] https://github.com/elastic/kibana/issues/192718 - [ ] serverless functional tests - [ ] inquire with ml-ui-team to have the ability to delete system indices which we do after uninstalling tiny elser with .ml indices --- .buildkite/ftr_oblt_serverless_configs.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.buildkite/ftr_oblt_serverless_configs.yml b/.buildkite/ftr_oblt_serverless_configs.yml index 422c548c28ae7..9c836bfaf376d 100644 --- a/.buildkite/ftr_oblt_serverless_configs.yml +++ b/.buildkite/ftr_oblt_serverless_configs.yml @@ -9,6 +9,7 @@ disabled: - x-pack/test_serverless/api_integration/test_suites/observability/config.feature_flags.ts - x-pack/test_serverless/api_integration/test_suites/observability/common_configs/config.group1.ts - x-pack/test_serverless/api_integration/test_suites/observability/fleet/config.ts + - x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/config.ts - x-pack/test_serverless/functional/test_suites/observability/config.ts - x-pack/test_serverless/functional/test_suites/observability/config.examples.ts - x-pack/test_serverless/functional/test_suites/observability/config.feature_flags.ts