Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easy to see which plugins are included in a snapshot (3.0/main/2.x) distribution. #5130

Open
dblock opened this issue Oct 24, 2024 · 3 comments
Labels
enhancement New Enhancement

Comments

@dblock
Copy link
Member

dblock commented Oct 24, 2024

Is your feature request related to a problem? Please describe

Coming from opensearch-project/opensearch-api-specification#639 a developer is trying to bump the snapshot build for 3.0 and 2.18 to a newer one. Then tests fail because that newer build does not have the SQL or the ISM plugin.

Describe the solution you'd like

A way to easily find the next "complete" distribution of OpenSearch 3.0 / 2.x / 2.18 / etc., or at least a way to see which plugins are included in any given docker image. I think as a consumer of any docker staging build I'd like to know:

  1. What are the plugins that built successfully and were included in the build.
  2. What are the plugins in the situation described in [Bug]: plugin that build failed but still be assembled #5091, aka didn't build but a previous version was included.
  3. Overall, is this a complete build without failures, meaning a potential beta/demo/release candidate.

Describe alternatives you've considered

Randomly pick new docker staging SHAs and keep retrying.

Additional context

#5091

@prudhvigodithi
Copy link
Collaborator

Today, we include the manifest.yml file in the bundle. For example, with Docker, you can run:

docker run -it --entrypoint bash opensearchstaging/opensearch:2.18.0@sha256:f311da085b968663fadf041bafc78b24dc21556f0726e625caf3560f7323a020 -c "cat /usr/share/opensearch/manifest.yml"

This command will provide all the information about the plugins (along with the specific commits used) that are part of the bundle. We can extend this to include an entry in the YAML file that indicates whether the build was complete and error-free, signaling a potential beta, demo, or release candidate.

Additionally, we could implement logic to include both the head_commit along with the exiting commit_id for each plugin, enabling us to track whether the previous version was included.

Another approach is to use the metrics cluster to store more detailed information for every OpenSearch core snapshot—such as the commit used, plugin details, plugin commit IDs, the head commit of each plugin, and whether any plugin build failed (using an older revision as a result). In this scenario, users could access a dashboard for a more detailed analysis. But the expectation here for the user is to go and see the dashboard.

Adding @getsaurabh02 @peterzhuamazon @gaiksaya

@dblock
Copy link
Member Author

dblock commented Oct 24, 2024

Is it possible to annotate/include text somewhere on dockerhub for these (a description)?

If not, a build dashboard would be very nice!

@dblock
Copy link
Member Author

dblock commented Oct 24, 2024

This command will provide all the information about the plugins (along with the specific commits used) that are part of the bundle. We can extend this to include an entry in the YAML file that indicates whether the build was complete and error-free, signaling a potential beta, demo, or release candidate.

This is helpful. It does require downloading the image, but will save time. I documented it in opensearch-project/opensearch-api-specification#642

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New Enhancement
Projects
Status: 🆕 New
Development

No branches or pull requests

3 participants