Parallelize stale blobs deletion during snapshot delete

[piyushdaftary]

Signed-off-by: Piyush Daftary pdaftary@amazon.com

Description

Currently during snapshot delete, deletion of unlinked shard level blob is single threaded using SNAPSHOT threadpool. Hence if there is huge number of unlinked shard level blob flies, it will take considerable amount of time to clean them.

Hence I propose to make unlinked shard level blob deletion multi threaded delete the same way we do for cleaning up of stale indices, to speedup the overall snapshot deletion process.

Issues Resolved

#2156

Check List

• [Y] New functionality includes testing.
  • [Y] All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• [Y] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/224/
• CommitID: 0573bc3

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/225/
• CommitID: 0573bc3