Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] Release version 2.6.0 #3081

Closed
41 of 48 tasks
github-actions bot opened this issue Jan 11, 2023 · 24 comments
Closed
41 of 48 tasks

[RELEASE] Release version 2.6.0 #3081

github-actions bot opened this issue Jan 11, 2023 · 24 comments
Assignees

Comments

@github-actions
Copy link
Contributor

github-actions bot commented Jan 11, 2023

Release OpenSearch and OpenSearch Dashboards 2.6.0

I noticed that a manifest was automatically created in manifests/2.6.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. More details are included in the Maintainers Release owner section.

Release Steps

There are several steps to the release process, these steps are completed as the whole release and components that are behind present risk to the release. The release owner completes the tasks in this ticket, whereas component owners resolve tasks on their ticket in their repositories.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

Component List

To aid in understanding the state of the release there is a table with status indicating each component state. This is updated based on the status of the component issues.

Preparation

  • Assign this issue to a release owner.
  • Declare a pencils down date for new features to be merged.
  • Feb 21, 2023 is pencils down date for feature freeze.
  • Update the Campaigns section to include monitoring campaigns during this release.
  • Update this issue so all __REPLACE_RELEASE-__ placeholders have actual dates.
  • Document any new quality requirements or changes.
  • Finalize scope and feature set and update the Public Roadmap.
  • Create a release issue in every component repo that links back to this issue, update Components section with these links.
  • Ensure the label is created in each component repo for this new version, and the next minor release. Create a version label
  • Ensure that all release issues created above are assigned to an owner in the component team.
  • Increase the build frequency for the this release from once a day (H 1 * * *) to once every hour (H/60 * * * *) in jenkinsFile.

Campaigns

Release Branch - _Ends Feb 17, 2022

  • Component versions are auto-incremented to 2.6.0 version.
  • Plugins team to ensure working CI in component repositories and merge the version increment PRs.
  • OpenSearch / OpenSearch-Dashboards core cut branch 2.6 early.
  • All component repos create 2.6 branch for the release.

Feature Freeze - _Ends Feb 20, 2023

Code Complete - Ends Feb 21, 2022

Release testing - Ends Feb 22, 2022

  • Declare a release candidate build, and provide the instructions with the release candidates for teams on testing (REPLACE_RELEASE-minus-8-days).
  • Stop builds for this version of OpenSearch and/or OpenSearch Dashboards in order to avoid accidental commits going in unknowingly. Restart only if necessary else manually run the build workflow and declare new release candidate.
  • After generate the release candidates, raise PR to lock input manifest refs of both OS and OSD builds with the commit ids from distribution manifest of the release candidates.
  • Sanity Testing (REPLACE_RELEASE-minus-8-days - REPLACE_RELEASE-minus-6-days): Sanity testing and fixing of critical issues found by teams. Teams test their components within the distribution, ensuring integration, backwards compatibility, and perf tests pass.
  • Publish all test results in the comments of this issue.

Performance testing validation - Ends Feb 22, 2022

  • Performance tests do not show a regression.
  • Longevity tests do not show any issues.

Release - _Ends {_Feb 28, 2022}

Post Release

  • Create release tags for each component (Jenkins job name: release-tag-creation).
  • Replace refs in manifests/2.6.0 with tags and remove checks.
  • If this is a major or minor version release, stop building previous patch version.
  • Generate distribution release notes reviewed by PM team for opensearch-build repository.
  • Increment version for Helm Charts (sample PR) for the 2.6.0 release.
  • Increment version for Ansible Charts (sample PR) for the 2.6.0 release.
  • Prepare for next patch release by incrementing patch versions for each component.
  • Update this template with any new or missed steps.
  • Create an issue for a retrospective, solicit feedback, and publish a summary.

Components

Replace with links to all component tracking issues.

Component Untriaged issues Issues tagged with 2.6.0
opensearch-project/OpenSearch#6360
opensearch-project/OpenSearch-Dashboards#3447
opensearch-project/alerting#799
opensearch-project/anomaly-detection#812
opensearch-project/asynchronous-search#238
opensearch-project/reporting#646
opensearch-project/dashboards-visualizations#165
opensearch-project/index-management#688
opensearch-project/job-scheduler#318
opensearch-project/k-NN#770
opensearch-project/performance-analyzer#384
opensearch-project/security-dashboards-plugin#1335
opensearch-project/security#2446
opensearch-project/sql#1357
opensearch-project/observability#1412
opensearch-project/alerting-dashboards-plugin#489
opensearch-project/notifications#619
opensearch-project/index-management-dashboards-plugin#628
opensearch-project/anomaly-detection-dashboards-plugin#427
opensearch-project/cross-cluster-replication#709
opensearch-project/common-utils#360
opensearch-project/geospatial#224
opensearch-project/ml-commons#733
opensearch-project/dashboards-maps#255
opensearch-project/dashboards-observability#287
opensearch-project/dashboards-reporting#63
opensearch-project/security-analytics#333
opensearch-project/neural-search#113
opensearch-project/security-analytics-dashboards-plugin#421
opensearch-project/dashboards-notifications#15
opensearch-project/dashboards-query-workbench#43
opensearch-project/opensearch-dashboards-functional-test#553
opensearch-project/dashboards-search-relevance#157
Legend

Symbol Meaning
🟢 On track with overall release
🟡 Missed last milestone
🔴 Missed multiple milestones

@github-actions github-actions bot added release untriaged Issues that have not yet been triaged v2.6.0 labels Jan 11, 2023
@rishabh6788 rishabh6788 removed the untriaged Issues that have not yet been triaged label Jan 19, 2023
@bbarani bbarani pinned this issue Feb 7, 2023
@Divyaasm Divyaasm unpinned this issue Feb 16, 2023
@gaiksaya gaiksaya pinned this issue Feb 17, 2023
@gaiksaya
Copy link
Member

OpenSearch Dashboard is facing issues to start up due to observability plugin

  '/usr/share/opensearch-dashboards/src/core/server/index.js',\n    '/usr/share/opensearch-dashboards/src/cli/serve/serve.js',\n    '/usr/share/opensearch-dashboards/src/cli/cli.js',\n    '/usr/share/opensearch-dashboards/src/cli/dist.js'\n  ]\n}"}
opensearch-dashboards    |
opensearch-dashboards    |  FATAL  Error: Cannot find module 'performance-now'
opensearch-dashboards    | Require stack:
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/adaptors/notebooks/default_backend.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/adaptors/notebooks/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/routes/notebooks/paraRouter.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/routes/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/plugin.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/plugins/plugin.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/plugins/discovery/plugins_discovery.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/plugins/discovery/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/plugins/plugins_service.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/plugins/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/core/server/index.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/cli/serve/serve.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/cli/cli.js
opensearch-dashboards    | - /usr/share/opensearch-dashboards/src/cli/dist.js

@gaiksaya
Copy link
Member

gaiksaya commented Feb 24, 2023

New release candidates that fixes issues with below components:

  • OpenSearch
  • OpenSearch Dashboards
  • Index Management
  • Ml commons
  • SQL
  • dashboards-observability
  • dashboards-maps

Release candidate 2 testing details

See next few comments for release candidate 3OpenSearch - [Build 7156](https://build.ci.opensearch.org/job/distribution-build-opensearch/7156/) OpenSearch Dashboards - [Build 5520](https://build.ci.opensearch.org/job/distribution-build-opensearch-dashboards/5520/)
  • Use Docker-Compose to setup a cluster

    • Download this gist as docker-compose.yml on your machine
    • Get latest image versions docker-compose pull
    • Start the cluster docker-compose up
  • OpenSearch docker 2.6.0.7156

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:2.6.0.7156 && docker run -it -p 9200:9200 -e “discovery.type=single-node” -e “DISABLE_SECURITY_PLUGIN=true” opensearchstaging/opensearch:2.6.0.7156
      • Connect command curl https://localhost:9200/
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch:2.6.0.7156 && docker run -it -p 9200:9200 -e “discovery.type=single-node” opensearchstaging/opensearch:2.6.0.7156
      • Connect command curl --insecure https://admin:admin@localhost:9200/
  • OpenSearch-Dashboards docker 2.6.0.5520

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.6.0.5520 && docker run -it --network=“host” -e “DISABLE_SECURITY_DASHBOARDS_PLUGIN=true” opensearchstaging/opensearch-dashboards:2.6.0.5520
      • URL https://localhost:5601/
    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.6.0.5520 && docker run -it --network=“host” opensearchstaging/opensearch-dashboards:2.6.0.5520
      • URL https://localhost:5601/
  • Use TARs to deploy OpenSearch Manually

Check how to install opensearch and dashboards on different platforms

@gaiksaya
Copy link
Member

We had rebuild OpenSearch due to k-NN plugin issues. We have triggered a new release candidate only for OpenSearch
https://build.ci.opensearch.org/job/distribution-build-opensearch/7160/

@peterzhuamazon
Copy link
Member

ml dashboards 2.6.0 integTest with ftrepo results.txt

ml dashboards pass.

@gaiksaya
Copy link
Member

New release candidates that fixes issues with below components:

  • Security plugin (due to k-NN config)
  • Asynchronous search (fixed tests)

Release candidate 3 testing details

OpenSearch - Build 7160
OpenSearch Dashboards - Build 5520

  • Use Docker-Compose to setup a cluster

    • Download this gist as docker-compose.yml on your machine
    • Get latest image versions docker-compose pull
    • Start the cluster docker-compose up
  • OpenSearch docker 2.6.0.7160

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:2.6.0.7160 && docker run -it -p 9200:9200 -e “discovery.type=single-node” -e “DISABLE_SECURITY_PLUGIN=true” opensearchstaging/opensearch:2.6.0.7160
      • Connect command curl https://localhost:9200/
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch:2.6.0.7160 && docker run -it -p 9200:9200 -e “discovery.type=single-node” opensearchstaging/opensearch:2.6.0.7160
      • Connect command curl --insecure https://admin:admin@localhost:9200/
  • OpenSearch-Dashboards docker 2.6.0.5520

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.6.0.5520 && docker run -it --network=“host” -e “DISABLE_SECURITY_DASHBOARDS_PLUGIN=true” opensearchstaging/opensearch-dashboards:2.6.0.5520
      • URL https://localhost:5601/
    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.6.0.5520 && docker run -it --network=“host” opensearchstaging/opensearch-dashboards:2.6.0.5520
      • URL https://localhost:5601/
  • Use TARs to deploy OpenSearch Manually

Check how to install opensearch and dashboards on different platforms

@gaiksaya
Copy link
Member

gaiksaya commented Feb 24, 2023

Performance test results for 2.6.0 linux tarball

Product Version Architecture Security Enabled Start Time(PDT) Duration Instance Type Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 arm64 with_security 2/23/23, 6:51 PM 6h 20m 31s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1683.0347211694266 2235.3767651867724 4203.794311523439 7290.1044921875 43023.94140625 44229.926294649165 48394.578125 46128 0 0 337.65208231608074 350.9703480529785 354.98485610961916 360.599666595459 1.7243948817253112 1.738606733083725 1.751881718635559 1510 0 0 95 98 100 100 52 67 86.70999999999998 88 0 954617
2.5.0 arm64 with_security 2/23/23, 1:28 PM 6h 45m 34s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1836.8885674136354 2434.210171357126 4508.794549278847 6948.31787109375 39402.625 41282.72367506761 45056.1484375 46364 0 0 358.4946543375651 374.8652793121338 382.67616064453125 387.61668701171874 1.686248779296875 1.7187442640463508 1.7509675741195678 1510 0 0 96 99 100 100 53 65 75.34999999999997 93 0 981879
+9.14% +8.89% +7.26% -4.69% -8.42% -6.66% -6.9% +0.51% +6.17% +6.81% +7.8% +7.49% -2.21% -1.14% -0.05% 0% +1.05% +1.02% 0% 0% +1.92% -2.99% -13.1% +5.68% +2.86%
Product Version Architecture Security Enabled Start Time(PDT) Duration Instance Type Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 arm64 without_security 2/23/23, 6:46 PM 6h 8m 29s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1599.3872401103763 2143.8051425171298 4033.5009635416627 7060.18017578125 45518.40625 47654.09047942107 51918.875 45884 0 0 337.8011199951172 346.4308171081543 357.8541222229004 361.0412353515625 1.7169495940208435 1.739775425195694 1.7554648756980895 1510 0 0 96 99 100 100 49 65 92.65999999999997 97 0 816082
2.5.0 arm64 without_security 2/23/23, 1:28 PM 6h 4m 30s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1648.5600091795975 2211.895783234262 4186.158430989582 6007.1376953125 44076.78515625 45923.83699724598 50313.49609375 45990 0 0 362.73582089742024 375.63397796630863 379.2197691955567 384.3936912536621 1.682751202583313 1.709676800171534 1.7537832617759705 1510 0 0 95 100 100 100 49 64 90 96 0 816550
+3.07% +3.18% +3.78% -14.92% -3.17% -3.63% -3.09% +0.23% +7.38% +8.43% +5.97% +6.47% -1.99% -1.73% -0.1% 0% -1.04% +1.01% 0% 0% 0% -1.54% -2.87% -1.03% +0.06%
Version Architecture Security Enabled Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 x64 without_security m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2170.40286 2886.07349 5590.27099 9213.98828 34077.00781 35466.31406 40222.62891 46727 0 0 284.25366 292.57317 302.73246 311.67474 1.72737 1.74225 1.75621 1510 0 0 97 99 100 100 50 64 82.85 98 0
2.5.0 X64 Without Security m5.xlarge nyc_taxis/ 2 warmupIterations/ 3 testIterations 2,162.2 2,879.4 5,471 9,189.8 33,906 35,456.2 40,963.7 46,673 - - 280.9 287.9 294.7 313.2 1.726 1.741 1.756 1510 - - 97 99 100 100 50 65 91.79 96 -
Comparison 0.38% 0.23% 2.13% 0.26% -0.50% 4.05% 15.50% 1.18% 1.60% 2.65% -0.49% -0.08% -0.07% -0.01%
Version Architecture Security Enabled Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 x64 with_security m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2327.36801 3087.57067 5828.4058 9607.19922 31703.36328 33938.25352 37719.42578 46913 0 0 286.26946 292.61543 302.75592 313.00728 1.72754 1.73973 1.75321 1510 0 0 97 100 100 100 53 65 81.54 90 0
2.5.0 x64 with_security m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2299.22279 3061.45418 5923.70094 10713.34961 32004.95313 33779.57975 37722.17188 46912 1 0.00002 294.84239 313.2291 326.68552 337.9979 1.72428 1.73966 1.75217 1510 0 0 97 100 100 100 52 66 85.76 93 0
Comparison 1.21% 0.85% -1.64% -11.51% 0.95% 6.55% 11.15% -2.99% -7.04% -7.90% -7.98% -0.19% 0.00% -0.06%

cc: @dblock @CEHENKLE @kotwanikunal @saratvemulapalli

@gaiksaya
Copy link
Member

gaiksaya commented Feb 24, 2023

Performance test results for windows

Version Architecture Security Enabled Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 Windows x64 with-security m5.xlarge nyc_taxis/ 2 warmupIterations/ 3 testIterations 2,090.2 3,323.6 7,910 21,411.1 31,664.4 33,084.5 38,179.4 46,892 0 0 145.8 149.3 157.9 174.5 1.742 1.75 1.764 1,510 0 0 94 97 100 100 56.75 67 81.06 94 0
2.5.0 Windows X64 With Security m5.xlarge nyc_taxis/ 2 warmupIterations/ 3 testIterations 2,477.3 4,864.9 9,265.8 23,008.5 25,974.4 28,740.9 33,497.5 47,188 - - 355.7 576.2 636.4 2,443 0.346 1.06 1.753 1510 - - 89 96 99 100 67 79 91 96 -
Comparison -18.52% -46.37% -17.14% -7.46% -17.97% -9.23% 1.25% -143.96% -285.93% -303.04% -1300.00% -80.14% -39.43% -0.62%
Version Architecture Security Enabled Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.6.0 Windows x64 without-security m5.xlarge nyc_taxis/ 2 warmupIterations/ 3 testIterations 1,940.6 2,960.6 7,619.2 21,441.5 34,287.1 36,095.8 40,747.8 46,697 0 0 139.2 142.1 160.3 175.6 1.742 1.75 1.764 1,510 0 0 93 97 99.76 100 54 67 89.76 94 0
2.5.0 Windows X64 Without Security m5.xlarge nyc_taxis/ 2 warmupIterations/ 3 testIterations 1,961.3 2,958.2 7,725.6 22,610.3 34,154.6 35,885.5 40,500.3 46,725 - - 142.2 146.4 165.9 179.4 1.742 1.751 1.764 1510 - - 94 98 100 100 53 64 86.1 94 -
Comparison -1.07% 0.08% -1.40% -5.45% -0.39% 4.66% 12.20% -2.16% -3.03% -3.49% -2.16% 0.00% 0.06% 0.00%

@gaiksaya
Copy link
Member

Integration test:
ARM64 tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/3839/pipeline
X64 tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/3840/pipeline

Looks like all tests passed except for

  • k-NN (which I believed is expected due to recent change of events )
  • IM (1 test failed for with-security but passed all for without security)

@gaiksaya
Copy link
Member

gaiksaya commented Feb 25, 2023

Windows testing results failures:

Asynchronous search:

with security
2023-02-25 00:23:59 INFO     ===============================================
2023-02-25 00:23:59 INFO     Running integration tests for asynchronous-search with-security
2023-02-25 00:23:59 INFO     ===============================================
2023-02-25 00:23:59 INFO     Executing "bash C:\Users\Administrator\opensearch-build\scripts\default\integtest.sh -b localhost -p 9200 -s true -v 2.6.0" in C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search
2023-02-25 00:31:35 INFO     Recording component test results for asynchronous-search at C:\Users\Administrator\opensearch-build\test-results\baebb64fe73744f4802b864a542e506b\integ-test\asynchronous-search\with-security\test-results
2023-02-25 00:31:35 INFO     Stderr reported for component: asynchronous-search
2023-02-25 00:31:35 INFO     Note: C:\Users\Administrator\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search\src\main\java\org\opensearch\search\asynchronous\transport\TransportAsynchronousSearchStatsAction.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: C:\Users\Administrator\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search\src\test\java\org\opensearch\search\asynchronous\bwc\AsyncSearchBackwardsCompatibilityIT.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0

REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive" -Dtests.seed=EBFAB78FE26ACDA8 -Dtests.security.manager=false -Dtests.locale=pl-PL -Dtests.timezone=Pacific/Gambier -Druntime.java=17

Suite: Test class org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT
2> lut 24, 2023 3:30:43 PM org.opensearch.client.RestClient logResponse
2> WARNING: request [GET https://localhost:9200/_cat/indices?format=json&expand_wildcards=all] returned 1 warnings: [299 OpenSearch-2.6.0-7203a5af21a8a009aece1474446b437a3c674db6 "this request accesses system indices: [.opendistro-asynchronous-search-response], but in a future major version, direct access to system indices will be prevented by default"]
2> REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive" -Dtests.seed=EBFAB78FE26ACDA8 -Dtests.security.manager=false -Dtests.locale=pl-PL -Dtests.timezone=Pacific/Gambier -Druntime.java=17
2> java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([EBFAB78FE26ACDA8:7B7732868BAE3EFE]:0)
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive(ApiParamsValidationIT.java:49)
2> NOTE: leaving temporary files on disk at: C:\Users\Administrator\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search\build\testrun\integTest\temp\org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT_EBFAB78FE26ACDA8-001
2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=167, maxMBSortInHeap=6.4286951600312445, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=pl-PL, timezone=Pacific/Gambier
2> NOTE: Windows Server 2019 10.0 amd64/Eclipse Adoptium 17.0.5 (64-bit)/cpus=16,threads=1,free=355712168,total=536870912
2> NOTE: All tests run in this JVM: [AsynchronousSearchQueryIT, AsynchronousSearchRejectionIT, AsynchronousSearchStatsIT, AsynchronousSearchTaskCancellationIT, DeleteAsynchronousSearchSingleNodeIT, GetAsynchronousSearchSingleNodeIT, MixedOperationSingleNodeIT, SubmitAsynchronousSearchSingleNodeIT, AsynchronousSearchCancellationIT, AsynchronousSearchPartialResponseIT, AsynchronousSearchProgressListenerIT, AsynchronousSearchManagementServiceIT, AsynchronousSearchRequestRoutingIT, ApiParamsValidationIT]

75 tests completed, 1 failed

=== Standard output of node node{::integTest-0} ===

=== Standard error of node node{::integTest-0} ===

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':integTest'.

There were failing tests. See the report at: file:///C:/Users/Administrator/AppData/Local/Temp/2/tmpjvk9ooy4/asynchronous-search/build/reports/tests/integTest/index.html

without security
2023-02-25 00:32:07 INFO     ===============================================
2023-02-25 00:32:07 INFO     Running integration tests for asynchronous-search without-security
2023-02-25 00:32:07 INFO     ===============================================
2023-02-25 00:32:07 INFO     Executing "bash C:\Users\Administrator\opensearch-build\scripts\default\integtest.sh -b localhost -p 9200 -s false -v 2.6.0" in C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search
2023-02-25 00:36:12 INFO     Recording component test results for asynchronous-search at C:\Users\Administrator\opensearch-build\test-results\baebb64fe73744f4802b864a542e506b\integ-test\asynchronous-search\without-security\test-results
2023-02-25 00:36:13 INFO     Stderr reported for component: asynchronous-search
2023-02-25 00:36:13 INFO     OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0

REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive" -Dtests.seed=399761F721F06402 -Dtests.security.manager=false -Dtests.locale=es-CL -Dtests.timezone=Etc/GMT-2 -Druntime.java=17

Suite: Test class org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT
2> feb 25, 2023 2:32:27 AM org.opensearch.client.RestClient logResponse
2> ADVERTENCIA: request [DELETE http://localhost:9200/.opendistro-asynchronous-search-response] returned 1 warnings: [299 OpenSearch-2.6.0-7203a5af21a8a009aece1474446b437a3c674db6 "this request accesses system indices: [.opendistro-asynchronous-search-response], but in a future major version, direct access to system indices will be prevented by default"]
2> REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive" -Dtests.seed=399761F721F06402 -Dtests.security.manager=false -Dtests.locale=es-CL -Dtests.timezone=Etc/GMT-2 -Druntime.java=17
2> java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([399761F721F06402:A91AE4FE48349754]:0)
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT.testSubmitDefaultKeepAlive(ApiParamsValidationIT.java:49)
2> NOTE: leaving temporary files on disk at: C:\Users\Administrator\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search\build\testrun\integTest\temp\org.opensearch.search.asynchronous.restIT.ApiParamsValidationIT_399761F721F06402-001
2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=911, maxMBSortInHeap=6.525117065746059, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=es-CL, timezone=Etc/GMT-2
2> NOTE: Windows Server 2019 10.0 amd64/Eclipse Adoptium 17.0.5 (64-bit)/cpus=16,threads=1,free=461753168,total=536870912
2> NOTE: All tests run in this JVM: [ApiParamsValidationIT]

REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.AsynchronousSearchRestIT.testGetWithKeepAliveUpdate" -Dtests.seed=399761F721F06402 -Dtests.security.manager=false -Dtests.locale=ar-EG -Dtests.timezone=MST7MDT -Druntime.java=17

Suite: Test class org.opensearch.search.asynchronous.restIT.AsynchronousSearchRestIT
2> REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.search.asynchronous.restIT.AsynchronousSearchRestIT.testGetWithKeepAliveUpdate" -Dtests.seed=399761F721F06402 -Dtests.security.manager=false -Dtests.locale=ar-EG -Dtests.timezone=MST7MDT -Druntime.java=17
2> java.lang.AssertionError:
Expected: a value greater than <1677303338003L>
but: <1677303338003L> was equal to <1677303338003L>
at __randomizedtesting.SeedInfo.seed([399761F721F06402:D850833D12EDC9E9]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:964)
at org.junit.Assert.assertThat(Assert.java:930)
at org.opensearch.search.asynchronous.restIT.AsynchronousSearchRestIT.testGetWithKeepAliveUpdate(AsynchronousSearchRestIT.java:135)
2> ??? 24, 2023 5:35:41 ? org.opensearch.client.RestClient logResponse
2> WARNING: request [POST http://localhost:9200/_opendistro/_asynchronous_search?keep_on_completion=true] returned 1 warnings: [299 OpenSearch-2.6.0-7203a5af21a8a009aece1474446b437a3c674db6 "[POST /_opendistro/_asynchronous_search] is deprecated! Use [POST /_plugins/_asynchronous_search] instead."]
2> NOTE: leaving temporary files on disk at: C:\Users\Administrator\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search\build\testrun\integTest\temp\org.opensearch.search.asynchronous.restIT.AsynchronousSearchRestIT_399761F721F06402-001
2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=150, maxMBSortInHeap=5.268587837002168, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=ar-EG, timezone=MST7MDT
2> NOTE: Windows Server 2019 10.0 amd64/Eclipse Adoptium 17.0.5 (64-bit)/cpus=16,threads=1,free=381459648,total=536870912
2> NOTE: All tests run in this JVM: [ApiParamsValidationIT, AsynchronousSearchQueryIT, AsynchronousSearchRejectionIT, AsynchronousSearchStatsIT, AsynchronousSearchTaskCancellationIT, DeleteAsynchronousSearchSingleNodeIT, GetAsynchronousSearchSingleNodeIT, MixedOperationSingleNodeIT, SubmitAsynchronousSearchSingleNodeIT, AsynchronousSearchCancellationIT, AsynchronousSearchPartialResponseIT, AsynchronousSearchProgressListenerIT, AsynchronousSearchManagementServiceIT, AsynchronousSearchRequestRoutingIT, AsynchronousSearchRestIT]

75 tests completed, 2 failed

=== Standard output of node node{::integTest-0} ===

=== Standard error of node node{::integTest-0} ===

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':integTest'.

There were failing tests. See the report at: file:///C:/Users/Administrator/AppData/Local/Temp/2/tmpjvk9ooy4/asynchronous-search/build/reports/tests/integTest/index.html

  • Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

BUILD FAILED in 4m 4s

2023-02-25 00:36:13 INFO Sending SIGKILL to PID 8408
2023-02-25 00:36:13 INFO Process killed with exit code None
2023-02-25 00:36:13 INFO Cleanup C:\Users\ADMINI1\AppData\Local\Temp\2\tmpjvk9ooy4\1\local-test-cluster/* content after the test
2023-02-25 00:36:13 INFO Recording local cluster logs for asynchronous-search with test configuration as without-security at C:\Users\Administrator\opensearch-build\test-results\baebb64fe73744f4802b864a542e506b\integ-test\asynchronous-search\without-security\local-cluster-logs
2023-02-25 00:36:13 INFO Skipping geospatial
2023-02-25 00:36:13 INFO Skipping neural-search
2023-02-25 00:36:13 INFO Skipping anomaly-detection
2023-02-25 00:36:13 INFO Skipping alerting
2023-02-25 00:36:13 INFO Skipping index-management
2023-02-25 00:36:13 INFO Skipping security-analytics
2023-02-25 00:36:13 INFO Skipping cross-cluster-replication
2023-02-25 00:36:13 INFO Removing C:\Users\ADMINI
1\AppData\Local\Temp\2\tmpjvk9ooy4
2023-02-25 00:36:13 WARNING Removing try count: 1/3 for C:\Users\ADMINI1\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search.git\objects\pack\pack-b71d88d048efe1a56fbd365fbc5419b03b7c9c59.idx
2023-02-25 00:36:13 WARNING Removing try count: 1/3 for C:\Users\ADMINI
1\AppData\Local\Temp\2\tmpjvk9ooy4\asynchronous-search.git\objects\pack\pack-b71d88d048efe1a56fbd365fbc5419b03b7c9c59.pack
2023-02-25 00:36:13 ERROR | asynchronous-search | with-security | FAIL |
2023-02-25 00:36:13 ERROR | asynchronous-search | without-security | FAIL |

IM:

With-security
OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0

REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT.test history shard settings" -Dtests.seed=950EECEE4A3D6C76 -Dtests.security.manager=false -Dtests.locale=vi-VN -Dtests.timezone=Australia/Adelaide -Druntime.java=17

Suite: Test class org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT
2> REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT.test history shard settings" -Dtests.seed=950EECEE4A3D6C76 -Dtests.security.manager=false -Dtests.locale=vi-VN -Dtests.timezone=Australia/Adelaide -Druntime.java=17
2> java.lang.AssertionError: Wrong number of shards expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([950EECEE4A3D6C76:66C678D7631F0707]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT$test history shard settings$2.invoke(IndexStateManagementHistoryIT.kt:434)
at org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT$test history shard settings$2.invoke(IndexStateManagementHistoryIT.kt:429)
at org.opensearch.indexmanagement.TestHelpersKt.waitFor(TestHelpers.kt:119)
at org.opensearch.indexmanagement.TestHelpersKt.waitFor$default(TestHelpers.kt:112)
at org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT.test history shard settings(IndexStateManagementHistoryIT.kt:429)
2> NOTE: leaving temporary files on disk at: C:\Users\Administrator\AppData\Local\Temp\2\tmpca9px34w\index-management\build\testrun\integTest\temp\org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT_950EECEE4A3D6C76-001
2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=1936, maxMBSortInHeap=6.103020699295265, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=vi-VN, timezone=Australia/Adelaide
2> NOTE: Windows Server 2019 10.0 amd64/Eclipse Adoptium 17.0.5 (64-bit)/cpus=16,threads=1,free=316938832,total=536870912
2> NOTE: All tests run in this JVM: [IndexManagementIndicesIT, IndexStateManagementSecurityBehaviorIT, RollupSecurityBehaviorIT, SecurityBehaviorIT, TransformSecurityBehaviorIT, ActionRetryIT, ActionTimeoutIT, AliasActionIT, AllocationActionIT, CloseActionIT, DeleteActionIT, ForceMergeActionIT, IndexPolicyActionIT, IndexPriorityActionIT, IndexStateManagementHistoryIT]

332 tests completed, 1 failed, 6 skipped

=== Standard output of node node{::integTest-0} ===

=== Standard error of node node{::integTest-0} ===

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':integTest'.

There were failing tests. See the report at: file:///C:/Users/Administrator/AppData/Local/Temp/2/tmpca9px34w/index-management/build/reports/tests/integTest/index.html

  • Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

BUILD FAILED in 39m 36s

without-security
OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0

REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.indexmanagement.transform.resthandler.RestStopTransformActionIT.test stopping a stopped Transform" -Dtests.seed=3AE2878216465C0D -Dtests.security.manager=false -Dtests.locale=ar-SA -Dtests.timezone=PLT -Druntime.java=17

Suite: Test class org.opensearch.indexmanagement.transform.resthandler.RestStopTransformActionIT
2> REPRODUCE WITH: gradlew ':integTest' --tests "org.opensearch.indexmanagement.transform.resthandler.RestStopTransformActionIT.test stopping a stopped Transform" -Dtests.seed=3AE2878216465C0D -Dtests.security.manager=false -Dtests.locale=ar-SA -Dtests.timezone=PLT -Druntime.java=17
2> java.lang.AssertionError: expected:<{acknowledged=true}> but was:<{acknowledged=false}>
at __randomizedtesting.SeedInfo.seed([3AE2878216465C0D:BB536F5FF53592B4]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.opensearch.indexmanagement.transform.resthandler.RestStopTransformActionIT.test stopping a stopped Transform(RestStopTransformActionIT.kt:45)
2> NOTE: leaving temporary files on disk at: C:\Users\Administrator\AppData\Local\Temp\2\tmpca9px34w\index-management\build\testrun\integTest\temp\org.opensearch.indexmanagement.transform.resthandler.RestStopTransformActionIT_3AE2878216465C0D-001
2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=925, maxMBSortInHeap=6.330201813551298, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=ar-SA, timezone=PLT
2> NOTE: Windows Server 2019 10.0 amd64/Eclipse Adoptium 17.0.5 (64-bit)/cpus=16,threads=1,free=409027640,total=536870912
2> NOTE: All tests run in this JVM: [IndexStateManagementHistoryIT, IndexManagementIndicesIT, ActionRetryIT, ActionTimeoutIT, AliasActionIT, AllocationActionIT, CloseActionIT, DeleteActionIT, ForceMergeActionIT, IndexPolicyActionIT, IndexPriorityActionIT, OpenActionIT, ReadOnlyActionIT, ReadWriteActionIT, ReplicaCountActionIT, RolloverActionIT, RollupActionIT, ShrinkActionIT, SnapshotActionIT, TransitionActionIT, ManagedIndexCoordinatorIT, MigrationServicesIT, ISMTemplateRestAPIIT, IndexStateManagementRestApiIT, RestAddPolicyActionIT, RestChangePolicyActionIT, RestExplainActionIT, RestRemovePolicyActionIT, RestRetryFailedManagedIndexActionIT, ManagedIndexRunnerIT, ValidateDeleteIT, ValidateForceMergeIT, ValidateOpenIT, ValidateReadOnlyIT, ValidateReadWriteIT, ValidateReplicaCountIT, ValidateRolloverIT, RefreshSearchAnalyzerActionIT, RestRefreshSearchAnalyzerActionIT, FieldCapsFilterIT, RollupInterceptorIT, RestDeleteRollupActionIT, RestExplainRollupActionIT, RestGetRollupActionIT, RestIndexRollupActionIT, RestStartRollupActionIT, RestStopRollupActionIT, RollupRunnerIT, SMRunnerIT, RestDeleteSnapshotManagementIT, RestExplainSnapshotManagementIT, RestGetSnapshotManagementIT, RestIndexSnapshotManagementIT, RestStartSnapshotManagementIT, RestStopSnapshotManagementIT, TransformRunnerIT, RestDeleteTransformActionIT, RestExplainTransformActionIT, RestGetTransformActionIT, RestIndexTransformActionIT, RestPreviewTransformActionIT, RestStartTransformActionIT, RestStopTransformActionIT]

316 tests completed, 1 failed, 6 skipped

=== Standard output of node node{::integTest-0} ===

=== Standard error of node node{::integTest-0} ===

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':integTest'.

There were failing tests. See the report at: file:///C:/Users/Administrator/AppData/Local/Temp/2/tmpca9px34w/index-management/build/reports/tests/integTest/index.html

  • Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

BUILD FAILED in 30m 24s

SQL:

 cat stderr.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 148 100 148 0 0 1409 0 --:--:-- --:--:-- --:--:-- 1423
C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpy7pqz4eh\sql\scripts\integtest.sh: line 102: /bin/opensearch-keystore: No such file or directory

@gaiksaya
Copy link
Member

gaiksaya commented Feb 25, 2023

Dashboards with-security on windows:

       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ×  core-opensearch-dashboards/opensear      01:38        5        4        1        -        - │
  │    ch-dashboards/apps/vis_builder/basi                                                         │
  │    c.spec.js                                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:21        4        4        -        -        - │
  │    ch-dashboards/apps/vis_builder/dash                                                         │
  │    board.spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:19        4        4        -        -        - │
  │    ch-dashboards/apps/vis_builder/expe                                                         │
  │    rimental.spec.js                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:14        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/area.spec.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:13        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/bar.spec.js                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:13        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/line.spec.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:12        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/metric.spec.js                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:16        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/table.spec.js                                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:31       34       34        -        -        - │
  │    ch-dashboards/dashboard_sanity_test                                                         │
  │    _spec.js                                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        1ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/1_create_datasource.spec.js                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        0ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/2_datasource_table.spec.js                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        1ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/3_update_datasource.spec.js                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:31        3        3        -        -        - │
  │    acknowledge_alerts_modal_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      02:33        5        5        -        -        - │
  │    alert_spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:29        2        2        -        -        - │
  │    alerts_dashboard_flyout_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/alerting-dashboards-plugin/      04:47        3        -        3        -        - │
  │    bucket_level_monitor_spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/alerting-dashboards-plugin/      04:49        6        3        3        -        - │
  │    cluster_metrics_monitor_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/alerting-dashboards-plugin/      03:39        7        5        2        -        - │
  │    query_level_monitor_spec.js                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/anomaly-detection-dashboard      00:23        1        -        1        -        - │
  │    s-plugin/create_detector_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:27        8        8        -        -        - │
  │    s-plugin/dashboard_spec.js                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/anomaly-detection-dashboard      385ms        5        -        1        -        4 │
  │    s-plugin/detector_configuration_spe                                                         │
  │    c.js                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:26       11       11        -        -        - │
  │    s-plugin/detector_list_spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/anomaly-detection-dashboard      376ms        6        -        1        -        5 │
  │    s-plugin/historical_analysis_spec.j                                                         │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:10        4        4        -        -        - │
  │    s-plugin/overview_spec.js                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:13        2        2        -        -        - │
  │    s-plugin/real_time_results_spec.js                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/anomaly-detection-dashboard      454ms        3        -        1        -        2 │
  │    s-plugin/sample_detector_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      01:52        2        2        -        -        - │
  │    s/documentsLayer.spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      00:33        1        1        -        -        - │
  │    s/import_vector_map_tab.spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      01:34        1        1        -        -        - │
  │    s/opensearchMapLayer.spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/gantt-chart-dashboards/gant      01:19        9        9        -        -        - │
  │    t_ui.spec.js                                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/index-management-dashboards      01:40        4        3        1        -        - │
  │    -plugin/aliases.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:38        4        4        -        -        - │
  │    -plugin/create_index.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:40        9        9        -        -        - │
  │    -plugin/indices_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/index-management-dashboards      02:42        5        4        1        -        - │
  │    -plugin/managed_indices_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/index-management-dashboards      01:50        5        4        1        -        - │
  │    -plugin/policies_spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:15        3        3        -        -        - │
  │    -plugin/reindex_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:12        4        4        -        -        - │
  │    -plugin/rollups_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:50        5        5        -        -        - │
  │    -plugin/split_index.js                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:43        3        3        -        -        - │
  │    -plugin/templates.js                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:57        4        4        -        -        - │
  │    -plugin/transforms_spec.js                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/ml-commons-dashboards/overv        1ms        -        -        -        -        - │
  │    iew_spec.js                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/notifications-dashboards/1_      02:10       11       10        1        -        - │
  │    email_senders_and_groups.spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/notifications-dashboards/2_      05:52       13        9        4        -        - │
  │    channels.spec.js                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/0_      00:14        1        1        -        -        - │
  │    before.spec.js                                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/1_      01:29        8        8        -        -        - │
  │    trace_analytics_dashboard.spec.js                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/2_      00:48        6        6        -        -        - │
  │    trace_analytics_services.spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/3_      00:38        5        5        -        -        - │
  │    trace_analytics_traces.spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/4_      00:56       11       11        -        -        - │
  │    panels.spec.js                                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/5_      01:02        4        4        -        -        - │
  │    event_analytics.spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/6_      03:05       16       16        -        -        - │
  │    notebooks.spec.js                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/7_      01:37        1        1        -        -        - │
  │    app_analytics.spec.js                                                                       │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/8_      135ms        1        1        -        -        - │
  │    after.spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/query-workbench-dashboards/      01:37       20       18        2        -        - │
  │    ui.spec.js                                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/01-creat      04:34       14       14        -        -        - │
  │    e.spec.js                                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/02-edit.      02:14        3        3        -        -        - │
  │    spec.js                                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/03-detai      02:09        6        6        -        -        - │
  │    ls.spec.js                                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/04-downl      01:21        5        5        -        -        - │
  │    oad.spec.js                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/search-relevance-dashboards      00:17        1        1        -        -        - │
  │    /1_query_compare.spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/security-analytics-dashboar      00:05        1        -        1        -        - │
  │    ds-plugin/rules_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/audit_log_spec.js       00:14        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/auth_spec.js            00:07        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/get_started_spec.j      00:16        8        8        -        -        - │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/internalusers_spec      00:17        4        4        -        -        - │
  │    .js                                                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/permissions_spec.j      00:21        5        5        -        -        - │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/roles_spec.js           00:15        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/tenants_spec.js         00:11        2        2        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ×  15 of 66 failed (23%)                  1:15:26      333      298       24        -       11

@gaiksaya
Copy link
Member

Dashboards Without-security on windows:

       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ √  core-opensearch-dashboards/opensear      00:38        5        5        -        -        - │
  │    ch-dashboards/apps/vis_builder/basi                                                         │
  │    c.spec.js                                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:22        4        4        -        -        - │
  │    ch-dashboards/apps/vis_builder/dash                                                         │
  │    board.spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:18        4        4        -        -        - │
  │    ch-dashboards/apps/vis_builder/expe                                                         │
  │    rimental.spec.js                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:13        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/area.spec.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:13        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/bar.spec.js                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:13        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/line.spec.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:12        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/metric.spec.js                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:15        1        1        -        -        - │
  │    ch-dashboards/apps/vis_builder/vis_                                                         │
  │    types/table.spec.js                                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear      00:31       34       34        -        -        - │
  │    ch-dashboards/dashboard_sanity_test                                                         │
  │    _spec.js                                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        1ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/1_create_datasource.spec.js                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        1ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/2_datasource_table.spec.js                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  core-opensearch-dashboards/opensear        1ms        -        -        -        -        - │
  │    ch-dashboards/datasource-management                                                         │
  │    -plugin/3_update_datasource.spec.js                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:28        3        3        -        -        - │
  │    acknowledge_alerts_modal_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      02:29        5        5        -        -        - │
  │    alert_spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:28        2        2        -        -        - │
  │    alerts_dashboard_flyout_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:15        3        3        -        -        - │
  │    bucket_level_monitor_spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/alerting-dashboards-plugin/      01:21        6        6        -        -        - │
  │    cluster_metrics_monitor_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/alerting-dashboards-plugin/      02:33        7        5        2        -        - │
  │    query_level_monitor_spec.js                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:22        1        1        -        -        - │
  │    s-plugin/create_detector_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:26        8        8        -        -        - │
  │    s-plugin/dashboard_spec.js                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:18        5        5        -        -        - │
  │    s-plugin/detector_configuration_spe                                                         │
  │    c.js                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:25       11       11        -        -        - │
  │    s-plugin/detector_list_spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      01:39        6        6        -        -        - │
  │    s-plugin/historical_analysis_spec.j                                                         │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:11        4        4        -        -        - │
  │    s-plugin/overview_spec.js                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:10        2        2        -        -        - │
  │    s-plugin/real_time_results_spec.js                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/anomaly-detection-dashboard      00:16        3        3        -        -        - │
  │    s-plugin/sample_detector_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      01:51        2        2        -        -        - │
  │    s/documentsLayer.spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      00:27        1        1        -        -        - │
  │    s/import_vector_map_tab.spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/custom-import-map-dashboard      01:35        1        1        -        -        - │
  │    s/opensearchMapLayer.spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/gantt-chart-dashboards/gant      01:06        9        9        -        -        - │
  │    t_ui.spec.js                                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:33        4        4        -        -        - │
  │    -plugin/aliases.js                                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:32        4        4        -        -        - │
  │    -plugin/create_index.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:24        9        9        -        -        - │
  │    -plugin/indices_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:33        5        5        -        -        - │
  │    -plugin/managed_indices_spec.js                                                             │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:46        5        5        -        -        - │
  │    -plugin/policies_spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:10        3        3        -        -        - │
  │    -plugin/reindex_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      01:01        4        4        -        -        - │
  │    -plugin/rollups_spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:44        5        5        -        -        - │
  │    -plugin/split_index.js                                                                      │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:37        3        3        -        -        - │
  │    -plugin/templates.js                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/index-management-dashboards      00:46        4        4        -        -        - │
  │    -plugin/transforms_spec.js                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/ml-commons-dashboards/overv        1ms        -        -        -        -        - │
  │    iew_spec.js                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/notifications-dashboards/1_      01:08       11       11        -        -        - │
  │    email_senders_and_groups.spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/notifications-dashboards/2_      04:43       13       10        3        -        - │
  │    channels.spec.js                                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/0_      00:12        1        1        -        -        - │
  │    before.spec.js                                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/observability-dashboards/1_      02:31        8        7        1        -        - │
  │    trace_analytics_dashboard.spec.js                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/2_      00:47        6        6        -        -        - │
  │    trace_analytics_services.spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/observability-dashboards/3_      02:39        5        3        2        -        - │
  │    trace_analytics_traces.spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/4_      00:56       11       11        -        -        - │
  │    panels.spec.js                                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/5_      01:02        4        4        -        -        - │
  │    event_analytics.spec.js                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/6_      02:58       16       16        -        -        - │
  │    notebooks.spec.js                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/7_      01:32        1        1        -        -        - │
  │    app_analytics.spec.js                                                                       │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/observability-dashboards/8_      153ms        1        1        -        -        - │
  │    after.spec.js                                                                               │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ×  plugins/query-workbench-dashboards/      01:37       20       18        2        -        - │
  │    ui.spec.js                                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/01-creat      04:12       14       14        -        -        - │
  │    e.spec.js                                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/02-edit.      01:21        3        3        -        -        - │
  │    spec.js                                                                                     │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/03-detai      01:58        6        6        -        -        - │
  │    ls.spec.js                                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/reports-dashboards/04-downl      01:16        5        5        -        -        - │
  │    oad.spec.js                                                                                 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/search-relevance-dashboards      00:17        1        1        -        -        - │
  │    /1_query_compare.spec.js                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/audit_log_spec.js         1ms        -        -        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/auth_spec.js              1ms        -        -        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/get_started_spec.j        1ms        -        -        -        -        - │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/internalusers_spec        1ms        -        -        -        -        - │
  │    .js                                                                                         │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/permissions_spec.j        1ms        -        -        -        -        - │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/roles_spec.js             1ms        -        -        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ √  plugins/security/tenants_spec.js           0ms        -        -        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ×  5 of 65 failed (8%)                    1:00:54      303      293       10        -        -

@gaiksaya
Copy link
Member

The test case failures were manually signed off by the respective component owners.

@joshuali925
Copy link
Member

@peterzhuamazon
Copy link
Member

Test new repo pipeline with yum:

$ sudo yum install opensearch
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Resolving Dependencies
--> Running transaction check
---> Package opensearch.x86_64 0:2.6.0-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================================================================================================================================================================================================
 Package                                                                       Arch                                                                      Version                                                                    Repository                                                                         Size
============================================================================================================================================================================================================================================================================================================================
Installing:
 opensearch                                                                    x86_64                                                                    2.6.0-1                                                                    opensearch-2.x                                                                    675 M

Transaction Summary
============================================================================================================================================================================================================================================================================================================================
Install  1 Package

Total download size: 675 M
Installed size: 931 M
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/2/opensearch-2.x/packages/opensearch-2.6.0-linux-x64.rpm: Header V4 RSA/SHA512 Signature, key ID 9310d3fc: NOKEY 99% [==================================================================================================================================== ]  38 MB/s | 672 MB  00:00:00 ETA
Public key for opensearch-2.6.0-linux-x64.rpm is not installed
opensearch-2.6.0-linux-x64.rpm                                                                                                                                                                                                                                                                       | 675 MB  00:00:19
Retrieving key from https://artifacts.opensearch.org/publickeys/opensearch.pgp
Importing GPG key 0x9310D3FC:
 Userid     : "OpenSearch project <opensearch@amazon.com>"
 Fingerprint: c5b7 4989 65ef d1c2 924b a9d5 39d3 1987 9310 d3fc
 From       : https://artifacts.opensearch.org/publickeys/opensearch.pgp
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : opensearch-2.6.0-1.x86_64                                                                                                                                                                                                                                                                                1/1
### NOT starting on installation, please execute the following statements to configure opensearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable opensearch.service
### You can start opensearch service by executing
 sudo systemctl start opensearch.service
### Create opensearch demo certificates in /etc/opensearch/
 See demo certs creation log in /var/log/opensearch/install_demo_configuration.log
  Verifying  : opensearch-2.6.0-1.x86_64                                                                                                                                                                                                                                                                                1/1

Installed:
  opensearch.x86_64 0:2.6.0-1

Complete!

@peterzhuamazon
Copy link
Member

$ curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "ip-172-31-28-246.us-west-2.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "CSYwWozfTDG51NdHZ2qM8g",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "rpm",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:57:09.290618503Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

@peterzhuamazon
Copy link
Member

peterzhuamazon commented Feb 28, 2023

APT using automated uploading to production:

$ sudo apt install opensearch opensearch-dashboards
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  opensearch opensearch-dashboards
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 959 MB of archives.
After this operation, 1873 MB of additional disk space will be used.
Get:1 https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 opensearch amd64 2.6.0 [744 MB]
Get:2 https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.x/apt stable/main amd64 opensearch-dashboards amd64 2.6.0 [215 MB]
Fetched 959 MB in 30s (32.3 MB/s)
Selecting previously unselected package opensearch.
(Reading database ... 237176 files and directories currently installed.)
Preparing to unpack .../opensearch_2.6.0_amd64.deb ...
Running OpenSearch Pre-Installation Script
Unpacking opensearch (2.6.0) ...
Selecting previously unselected package opensearch-dashboards.
Preparing to unpack .../opensearch-dashboards_2.6.0_amd64.deb ...
Running OpenSearch-Dashboards Pre-Installation Script
Unpacking opensearch-dashboards (2.6.0) ...
Setting up opensearch (2.6.0) ...
Running OpenSearch Post-Installation Script
### NOT starting on installation, please execute the following statements to configure opensearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable opensearch.service
### You can start opensearch service by executing
 sudo systemctl start opensearch.service
### Create opensearch demo certificates in /etc/opensearch/
 See demo certs creation log in /var/log/opensearch/install_demo_configuration.log
Setting up opensearch-dashboards (2.6.0) ...
Running OpenSearch-Dashboards Post-Installation Script
### NOT starting on installation, please execute the following statements to configure opensearch-dashboards service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable opensearch-dashboards.service
### You can start opensearch-dashboards service by executing
 sudo systemctl start opensearch-dashboards.service
Processing triggers for systemd (245.4-4ubuntu3.19) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
$ sudo systemctl restart opensearch
$ curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "ip-172-31-11-175",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "LzrzolEnS2iQMFMdP3uNeA",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "deb",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:58:38.730482464Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

@bbarani
Copy link
Member

bbarani commented Feb 28, 2023

-> Downloading repository-s3 from opensearch
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.NetPermission setDefaultAuthenticator
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission opensearch.allow_insecure_settings read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3 with folder name repository-s3

@peterzhuamazon
Copy link
Member

Native plugin:

$ sudo ./opensearch-plugin install discovery-ec2
-> Installing discovery-ec2
-> Downloading discovery-ec2 from opensearch
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

@Divyaasm
Copy link
Collaborator

Divyaasm commented Feb 28, 2023

Validation Results for Tarball,Rpm,Yum,Docker

Tar-x64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-15-39.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "73jm2DisQMapm9gtCBQHbA",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "tar",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:57:04.388618985Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
Status_code ->200 
response_text ->epoch      timestamp cluster    status node.total node.data discovered_cluster_manager shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1677611842 19:17:22  opensearch green           1         1                       true      3   3    0    0        0             0                  -                100.0%

Status_code ->200 
response_text ->name                                       component                            version
ip-172-31-4-179.eu-west-1.compute.internal opensearch-alerting                  2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-anomaly-detection         2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-asynchronous-search       2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-cross-cluster-replication 2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-geospatial                2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-index-management          2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-job-scheduler             2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-knn                       2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-ml                        2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-neural-search             2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-notifications             2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-notifications-core        2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-observability             2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-performance-analyzer      2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-reports-scheduler         2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-security                  2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-security-analytics        2.6.0.0
ip-172-31-4-179.eu-west-1.compute.internal opensearch-sql                       2.6.0.0

All tests Pass : There are 4/4 test cases Pass

Same results for the other distributions

Tar-arm64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-40-125.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "X18YQARAQ4OZyJhNZ6iiLA",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "tar",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:58:37.352296474Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
All tests Pass : There are 4/4 test cases Pass

Rpm-x64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-9-28.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "tF6wZU3OTC-YxVLGz_YcTg",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "rpm",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:57:09.290618503Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
All tests Pass : There are 4/4 test cases Pass

Rpm-arm64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-47-158.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "-ls2vYqeSXqA7pZ8kOrMBw",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "rpm",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:58:40.860888760Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
All tests Pass : There are 4/4 test cases Pass

Yum-x64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-41-144.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "Rpy3iK0uQpeorx5JCD-K2A",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "rpm",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:58:40.860888760Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
All tests Pass : There are 4/4 test cases Pass

Yum-arm64:

Status_code ->200 
response_text ->{
  "name" : "ip-172-31-41-144.eu-west-1.compute.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "Rpy3iK0uQpeorx5JCD-K2A",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "rpm",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:58:40.860888760Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
All tests Pass : There are 4/4 test cases Pass

Docker

Status_code ->200
response_text ->{
  "name" : "opensearch-node1",
  "cluster_name" : "opensearch-cluster",
  "cluster_uuid" : "4Qgv6LSoRdWMKHW16274Gg",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "tar",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:57:04.388618985Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Status_code ->200
response_text ->name             component                            version
opensearch-node2 opensearch-alerting                  2.6.0.0
opensearch-node2 opensearch-anomaly-detection         2.6.0.0
opensearch-node2 opensearch-asynchronous-search       2.6.0.0
opensearch-node2 opensearch-cross-cluster-replication 2.6.0.0
opensearch-node2 opensearch-geospatial                2.6.0.0
opensearch-node2 opensearch-index-management          2.6.0.0
opensearch-node2 opensearch-job-scheduler             2.6.0.0
opensearch-node2 opensearch-knn                       2.6.0.0
opensearch-node2 opensearch-ml                        2.6.0.0
opensearch-node2 opensearch-neural-search             2.6.0.0
opensearch-node2 opensearch-notifications             2.6.0.0
opensearch-node2 opensearch-notifications-core        2.6.0.0
opensearch-node2 opensearch-observability             2.6.0.0
opensearch-node2 opensearch-performance-analyzer      2.6.0.0
opensearch-node2 opensearch-reports-scheduler         2.6.0.0
opensearch-node2 opensearch-security                  2.6.0.0
opensearch-node2 opensearch-security-analytics        2.6.0.0
opensearch-node2 opensearch-sql                       2.6.0.0
opensearch-node1 opensearch-alerting                  2.6.0.0
opensearch-node1 opensearch-anomaly-detection         2.6.0.0
opensearch-node1 opensearch-asynchronous-search       2.6.0.0
opensearch-node1 opensearch-cross-cluster-replication 2.6.0.0
opensearch-node1 opensearch-geospatial                2.6.0.0
opensearch-node1 opensearch-index-management          2.6.0.0
opensearch-node1 opensearch-job-scheduler             2.6.0.0
opensearch-node1 opensearch-knn                       2.6.0.0
opensearch-node1 opensearch-ml                        2.6.0.0
opensearch-node1 opensearch-neural-search             2.6.0.0
opensearch-node1 opensearch-notifications             2.6.0.0
opensearch-node1 opensearch-notifications-core        2.6.0.0
opensearch-node1 opensearch-observability             2.6.0.0
opensearch-node1 opensearch-performance-analyzer      2.6.0.0
opensearch-node1 opensearch-reports-scheduler         2.6.0.0
opensearch-node1 opensearch-security                  2.6.0.0
opensearch-node1 opensearch-security-analytics        2.6.0.0
opensearch-node1 opensearch-sql                       2.6.0.0

Status_code ->200
response_text ->epoch      timestamp cluster            status node.total node.data discovered_cluster_manager shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1677611982 19:19:42  opensearch-cluster green           2         2                       true      6   3    0    0        0             0                  -                100.0%
New
[11:54](https://amzn-aws.slack.com/archives/D0413E048L9/p1677614047566219)
2023-02-28 11:19:46 INFO
Status_code ->200 For Dashboards
2023-02-28 11:19:49 INFO     All tests Pass : There are 4/4 test cases Pass

@gaiksaya
Copy link
Member

gaiksaya commented Mar 1, 2023

2.6.0 was released successfully 🎉
Retrospective issue: #3253

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants