Added support for testing multiple distributions.

[dblock]

Description

Introduces x-distributions that allows labelling APIs for various distributions and services. I propose we use opensearch.org for the default distribution, aos and aoss for the two Amazon managed services, and others can add their own.

If an API is not present in all distributions one has to list the distributions it is present in. I think it's better than having a negative list so that we can claim support for a well-known list of distributions in the spec and not have piecemeal support.

With this change, npm run test:spec -- --opensearch-distribution=amazon-managed passes against AOS 2.13.

Corrects semver checks for ranges with multiple values (e.g. >= 2.0, < 3).

Issues Resolved

Part of #84.
Closes #475.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Changes Analysis

Commit SHA: 090a038
Comparing To SHA: 83eab43

API Changes

Summary

├─┬Paths
│ └─┬/
│   └─┬GET
│     └─┬Extensions
│       └──[➕] x-distributions-excluded (13:9)
└─┬Components
  ├──[➖] schemas (46999:7)❌ 
  ├──[➕] schemas (47015:7)
  ├─┬nodes._common:ShardIndexingPressureStats
  │ ├──[➕] properties (46749:9)
  │ └─┬total_rejections_breakup_shadow_mode
  │   └──[🔀] $ref (47015:13)❌ 
  ├─┬nodes._common:IoStatDevice
  │ ├──[➕] properties (46346:9)
  │ ├──[➕] properties (46344:9)
  │ ├──[➕] properties (46340:9)
  │ └──[➕] properties (46342:9)
  ├─┬nodes.info:NodeOperatingSystemInfo
  │ ├──[➖] required (47725:11)❌ 
  │ ├──[➖] required (47726:11)❌ 
  │ ├──[➖] required (47728:11)❌ 
  │ └──[➖] required (47723:11)❌ 
  ├─┬nodes.info:NodeJvmInfo
  │ ├──[➖] required (47686:11)❌ 
  │ ├──[➖] required (47692:11)❌ 
  │ ├──[➖] required (47694:11)❌ 
  │ ├──[➖] required (47687:11)❌ 
  │ ├──[➖] required (47689:11)❌ 
  │ ├──[➖] required (47693:11)❌ 
  │ ├──[➖] required (47695:11)❌ 
  │ └─┬using_bundled_jdk
  │   └──[🔀] type (47688:13)❌ 
  ├─┬nodes._common:ShardSearchBackpressureTaskCancellationStats
  │ ├──[➕] properties (46797:9)
  │ └──[➕] properties (46799:9)
  └─┬nodes.info:NodeInfo
    ├──[➖] required (47169:11)❌ 
    ├──[➖] required (47162:11)❌ 
    ├──[➖] required (47165:11)❌ 
    └──[➖] required (47166:11)❌ 

Document Element	Total Changes	Breaking Changes
paths	1	0
components	26	18

• ❌ BREAKING Changes: 18 out of 27
• Modifications: 2
• Removals: 16
• Additions: 9
• Breaking Removals: 16
• Breaking Modifications: 2

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10386946783/artifacts/1811077711

API Coverage

	Before	After	Δ
Covered (%)	524 (51.32 %)	524 (51.32 %)	0 (0 %)
Uncovered (%)	497 (48.68 %)	497 (48.68 %)	0 (0 %)
Unknown	26	26	0

[github-actions]

Spec Test Coverage Analysis

Total	Tested
550	215 (39.09 %)

[nhtruong]

Should we have opensearch.org as the default distribution for the test stories as well since it's the default in the spec?

[Xtansia]

I think having aos and aoss as shortened versions is not immediately grokkable to someone without some AWS specific context. Even as someone with that context having one be a prefix of the other immediately hurts readability. Could we expand these to amazon-managed & amazon-serverless given we have a full form opensearch.org?

We should think ahead for what precedent we set if in future we end up with contributions related to things like Oracle Cloud Infrastructure Search with OpenSearch. (oci, ocisos, oracle, oracle-managed, oci-managed, etc)

[dblock]

Good point. One thought I had is to use URIs, e.g. aws.amazon.com/opensearch-service? WDYT?

[Xtansia]

Could be a bit verbose, AOSS would be aws.amazon.com/opensearch-service/features/serverless presumably.

Either way we'd likely need to define an enum somewhere and validate correctness, as spelling mistakes will be a PITA

[dblock]

I tried to use ajv.addKeyword to define a custom type for the new x-distribution options, but couldn't make it work. Maybe @nhtruong can help me make these values check against an enum on top of this PR.

[nhtruong]

We're only using AJV to validate JSON Schemas in the spec. x-distribution is not used in a schema, but rather an operation. So it should be validated similar to how we validate x-operation-group and other extensions.

In the very near future, we should create a json-schema for the namespace files and another for the category files in the ./spec folder. Even though we currently only require our yaml files to be OpenAPI valid, we actually require a much stricter json-schema for these files:

• x-operation-group is supported but a typo like x-opperation-group should not.
• Someone uses a new and valid OpenAPI keyword that the clients should handle accordingly but it got through the PR review without updating the Client Gen doc.
• #paths is not allowed in category yaml files
• #components/schemas is not allowed in namespace yaml files
• We can enforce validation on the extensions for free through AJV.

[dblock]

#492

[Xtansia]

Should we have opensearch.org as the default distribution for the test stories as well since it's the default in the spec?

I feel like we might really want both an "allow"/"include" and "deny"/"exclude" variant. As in the majority of cases all or most distributions will support an API and just aoss doesn't. So my thought is you'd generally by default want new tests/operations to be automatically opted into for all/new distributions and then whittled down as necessary. (Assuming we get automated testing against the managed distributions)

[dblock]

I've added include and exclude variants, etc., but no value validation in this PR. Please take another look?

We'll need jamesmbourne/aws4-axios#1621 to make serverless work.

[dblock]

Should we have opensearch.org as the default distribution for the test stories as well since it's the default in the spec?

I feel like we might really want both an "allow"/"include" and "deny"/"exclude" variant. As in the majority of cases all or most distributions will support an API and just aoss doesn't. So my thought is you'd generally by default want new tests/operations to be automatically opted into for all/new distributions and then whittled down as necessary. (Assuming we get automated testing against the managed distributions)

On this note. Do you @nhtruong @Xtansia have a (strong) preference for:

1. x-distributions-included, x-distributions-included and distributions: in tests (current state of the PR).
2. ...

   distributions:
      included:
      excluded: 
   

   in tests.
3. ...

   distributions-included:
   distributions-excluded: 
   

   in tests.

[Xtansia]

I think ideally we'd want distributions: {included: [], excluded: []} in tests, could still have distributions: ['foo'] that just shortcuts to distributions: {included: ['foo']}.

There might be better verbiage than (in|ex)cluded (something like supported maybe?) but I haven't come up with any other options I'm 100% happy with so not critical to this PR.

[dblock]

I think ideally we'd want distributions: {included: [], excluded: []} in tests, could still have distributions: ['foo'] that just shortcuts to distributions: {included: ['foo']}.

There might be better verbiage than (in|ex)cluded (something like supported maybe?) but I haven't come up with any other options I'm 100% happy with so not critical to this PR.

Made updates per comments and rebased. We can expand into includes and excludes for the distribution in a future PR.