Support running tests against Windows artifact

[tianleh]

Inspired by opensearch-project/opensearch-plugins#95 , create an issue to support running tests against Windows artifact.

[tianleh]

cc @peterzhuamazon

[tianleh]

synced up with @peterzhuamazon The execution of the artifact and the test could all happen in the windows server. There won't be Docker for Windows. Peter will check into the Cypress support in Windows next week.

[seraphjiang]

Ideally, functional test should has dependencies on Browser, but not OS.

what does it mean

browser test should run browser test on test agent only required for test
browser test should not run browser test on test server which hosts opensearch or opensearch dashboards

in order to support that, we need to build test infrastructure which could launch test environment only required to run browser test.

unless we are going to certify cross browser and cross OS support (which is only important, but different priority), the test infra should not be blocked by support new OS(hosting environment)

[tianleh]

Ideally, functional test should has dependencies on Browser, but not OS.

I was thinking about running Windows artifact on Windows and then exposing the endpoint. Then the tests can still run in a Linux env. However, this may require a remote endpoint.

Running the artifact and the tests together in the Windows env is more convenient to avoid the remote endpoint. The cost is that we need to have Cypress installed in Windows.

[peterzhuamazon]

Verified we can directly install cypress on windows through npm.

$ ./Cypress.exe --version

9.5.4

$ file Cypress.exe
Cypress.exe: PE32+ executable (GUI) x86-64, for MS Windows

The installed version, unlike arm64, is native.

[peterzhuamazon]

Next step:

• Adding cypress as part of the windows ami
• Run some existing test on windows with cypress.

[peterzhuamazon]

More development and issues in here:

• Add Github workflow for running against Windows latest #359