[Proposal] Publish snapshots with partial builds

[saratvemulapalli]

Is your feature request related to a problem? Please describe

Our distribution manifest consists of bunch of components and is maintained by different maintainers.
Our build/assemble/publish workflow is sequential and assumes all steps of each workflow to succeed.

As the project grows bigger, few components will consistently fail causing blockers for consumers who depend on snapshots. The stale snapshots are ending with manually kicking off builds with updated manifests.
We have seen multiple problems in the recent weeks:

• [AUTOCUT] OS Distribution Build Failed for security-3.0.0 security#2149
• [BUG] SQL Plugin main branch BWC test faiiled OpenSearch#5815
• Add support for minimum compatible version for Extensions and OpenSearch opensearch-sdk-java#365 (comment)
• [Extensions] Create a proxy ScheduledJobRunner, ScheduledJobParser to invoke corresponding registered Job Detail actions job-scheduler#306 (comment)
  and many more.

Our workflows fail hard when one component fails.

Describe the solution you'd like

Our jenkins workflows should allow fail through and let partial components to be published.
For example:
Our manifest contains OpenSearch, Job Scheduler, security. Its ok to have security to fail the build but we should still continue running though the workflows to publish OpenSearch and JS snapshots.

We can leverage jenkins scripts[1] to smartly identify the successful components and run through all stages of the pipeline for snapshots.

[1] https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/

Describe alternatives you've considered

Wait for snapshots being published :/

Additional context

No response

[ryanbogan]

+1

[bbarani]

@saratvemulapalli Are you referring to Maven artifacts or TAR here? We did add an enhancement to publish TAR snapshots for core even if plugins fail but yet to work on the Maven part. We are looking in to moving the snapshot outside of distribution build job to individual repos as well. You can track the progress here.

[saratvemulapalli]

@bbarani this is for maven artifacts. I like #3185 , our monolithic build should be broken down to make this seamless.

[saratvemulapalli]

I'll close this as we are already taking care of this and move towards publishing artifacts from each repo.