Rebase throttling branch with main #4650
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…bat manager" (opensearch-project#4289) * [BUG] Update opensearch-service-x64.exe parameters to //ES for Execute Service. Update opensearch-service-mgr.exe parameters to //ES for Edit Service. Add code comments for the Apache Commons Daemon. Signed-off-by: Alex Burck <me@alexburck.com> * update changelog with pull request link Signed-off-by: Alex Burck <me@alexburck.com> Signed-off-by: Alex Burck <me@alexburck.com>
* Removing dead code in RecoveryTarget. This code in RecoveryTarget is not invoked, all of these methods are implemented by the parent ReplicationTarget with the same behavior. Signed-off-by: Marc Handalian <handalm@amazon.com> * PR Comments. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com>
* Update changelog contribution guide Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Fix reference to pull request Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…4133) This commit prevents a replica shard from being failed in the case that a replication action to a replica is canceled due to the primary shard being closed. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…der from package 'org.opensearch.action.support.master.info' for subclasses (opensearch-project#4307) * Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses Signed-off-by: Tianli Feng <ftianli@amazon.com> * Add changelog Signed-off-by: Tianli Feng <ftianli@amazon.com> Signed-off-by: Tianli Feng <ftianli@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com>
…nsearch-project#4305) * [BUG] Create logs directory before running OpenSearch on Windows Signed-off-by: Alex Burck <me@alexburck.com> * update changlog pr link Signed-off-by: Alex Burck <me@alexburck.com> Signed-off-by: Alex Burck <me@alexburck.com>
…h-project#4240) * Use RemoteSegmentStoreDirectory instead of RemoteDirectory Signed-off-by: Sachin Kale <kalsac@amazon.com>
When publishing Zip POM the groupId value was hard-coded to `org.opensearch.plugin` value which worked fine for existing core plugins but is not convenient for other plugins (such as community plugins maintained in independent repositories). This PR changes the sources of the ZIP publishing groupId value. Specifically, there are two ways to set the value: 1) It is automatically inherited from the Gradle "project.group" 2) It can be manually specified in the ZIP publication POM object This PR also brings a major rework of tests in PublishTests class. Individual testing scenarios are driven by "real" gradle building scripts (utilizing `java-gradle-plugin` gradle plugin). Closes opensearch-project#3692 Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
…ss in testReplicationOnDone test (opensearch-project#4314) * [Segment Replication] testReplicationOnDone Add timeout to allow time for verify call Signed-off-by: Suraj Singh <surajrider@gmail.com> * Update changelog Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add change log entry Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Kartik Ganesh <gkart@amazon.com> Signed-off-by: Kartik Ganesh <gkart@amazon.com>
The package `org.yaml:snakeyaml` before version 1.31 are vulnerable to Denial of Service (DoS) due missing to nested depth limitation for collections. Details at https://nvd.nist.gov/vuln/detail/CVE-2022-25857 Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Fixed label for dependabot PR helper Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Update autocommit workflow for dependabot changelog Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Add version config for dependabot changelog helper Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Support for HTTP/2 (server-side) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Addressing code review comments Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Added HTTP/1.1 channel configuration Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Addressing code review comments Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Update pul request URL in CHANGELOG.md Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…" (opensearch-project#4361) This reverts commit c28221e. Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…n. (opensearch-project#4225) * Segment Replication. Fix Cancellation of replication events. This PR updates segment replication paths to correctly cancel replication events on the primary and replica. In the source service, any ongoing event for a primary that is sending to a replica that shuts down or is promoted as a new primary are cancelled. In the target service, any ongoing event for a replica that is promoted as a new primary or is fetching from a primary that shuts down. It wires up SegmentReplicationSourceService as an IndexEventListener so that it can respond to events and cancel any ongoing transfer state. This change also includes some test cleanup for segment replication to rely on actual components over mocks. Signed-off-by: Marc Handalian <handalm@amazon.com> Fix to not start/stop SegmentReplicationSourceService as a lifecycle component with feature flag off. Signed-off-by: Marc Handalian <handalm@amazon.com> Update logic to properly mark SegmentReplicationTarget as cancelled when cancel initiated by primary. Signed-off-by: Marc Handalian <handalm@amazon.com> Minor updates from self review. Signed-off-by: Marc Handalian <handalm@amazon.com> * Add missing changelog entry. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com>
* Fix token usage for changelog helper Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Add conditional check for dependabot steps Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Add dependency section Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Bug fixes for dependabot changelog verifier Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> * Update the changelog Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…vel client (opensearch-project#4064) * Create and delete PIT search rest layer changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…search-project#4354) * Bump xmlbeans from 5.1.0 to 5.1.1 in /plugins/ingest-attachment Bumps xmlbeans from 5.1.0 to 5.1.1. --- updated-dependencies: - dependency-name: org.apache.xmlbeans:xmlbeans dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…nts (opensearch-project#4352) Overload `generateHistoryOnReplica` to be able to generate only a specific `Engine.Operation.TYPE` operations as required by the `testUpdateSegments` test Signed-off-by: Rabi Panda <adnapibar@gmail.com> Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Added bwc version 2.2.2 * Add changelog Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Kunal Kotwani <kkotwani@amazon.com>
… replica promotion (opensearch-project#4365) * [Segment Replication] Bump segment infos counter before commit during replica promotion Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add changelog entry Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com>
…d primary on onNewCheckpoint on replica (opensearch-project#4363) * [Segment Replication] Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add changelog entry Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments 2 Signed-off-by: Suraj Singh <surajrider@gmail.com> * Test failures Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com>
* Added bwc version 2.3.1 * Add changelog of adding bwc version 2.3.1 Signed-off-by: Tianli Feng <ftianli@amazon.com>
…ication feature (opensearch-project#3993) * [Segment Replication] Add snapshots tests with segment replication enabled Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix spotless failures Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add changelog entry, address review comments, add failover test Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix spotless failures Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments 2 Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com>
…og enabled indices (opensearch-project#4318) Signed-off-by: Ashish Singh <ssashish@amazon.com>
opensearch-project#4241) * Weighted round-robin scheduling policy for shard coordination traffic routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
…h-project#4540) * Add javadocs for example-plugins:custom-settings module Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs for example-plugins:custom-significance-heuristic Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs for example-plugins:custom-suggester Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs to example-plugins:painless-allowlist Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs to example-plugins:rescore Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs to example-plugins:rest-handler Signed-off-by: Daniel Widdis <widdis@gmail.com> * Add javadocs to example-plugins:script-expert-scoring Signed-off-by: Daniel Widdis <widdis@gmail.com> * Remove exclusions for module which aren't giving javadoc errors Signed-off-by: Daniel Widdis <widdis@gmail.com> * CI testing apparently needs to instantiate a no-op class Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com>
…3980) (opensearch-project#4266) Enables geo_bounds aggregation to work with geo_shape field types. This enhancement includes: * Addition of Doc values on the GeoShape Field. * Addition of GeoShape ValueSource level code interfaces for accessing the DocValues. * Addition of Missing Value feature in the GeoShape Aggregations. Signed-off-by: Navneet Verma <navneev@amazon.com>
* Upgrade jackson and snakeyml dependencies Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> * Updated changelog Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
…4547) * Bump com.diffplug.spotless from 6.10.0 to 6.11.0 --- updated-dependencies: - dependency-name: com.diffplug.spotless dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
…pensearch-project#4548) * Bump reactor-core from 3.4.18 to 3.4.23 in /plugins/repository-azure Bumps [reactor-core](https://github.com/reactor/reactor-core) from 3.4.18 to 3.4.23. - [Release notes](https://github.com/reactor/reactor-core/releases) - [Commits](reactor/reactor-core@v3.4.18...v3.4.23) --- updated-dependencies: - dependency-name: io.projectreactor:reactor-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
…nsearch-project#4550) * Bump jempbox from 1.8.16 to 1.8.17 in /plugins/ingest-attachment Bumps jempbox from 1.8.16 to 1.8.17. --- updated-dependencies: - dependency-name: org.apache.pdfbox:jempbox dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
…artyAudit (opensearch-project#4549) * Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit Bumps log4j-core from 2.18.0 to 2.19.0. --- updated-dependencies: - dependency-name: org.apache.logging.log4j:log4j-core dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ensearch-project#4544) Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com>
…perty file (opensearch-project#4568) Signed-off-by: Tianli Feng <ftianli@amazon.com>
…oject#4580) Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com>
…ommissioning (opensearch-project#4084) * Add Executor to decommission node attribute * Decommission service implementation with cluster metadata * Master abdication changes to decommission local awareness leader * Update join validator changes to validate decommissioned node join request Signed-off-by: Rishab Nahata <rnnahata@amazon.com>
…ch-project#4388) * Changes for list all and pit segments Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
* Add PUT api to update shard routing weights Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
…rch-project#4360) A follow-up of PR opensearch-project#4156 that brings in further simplification of the ZIP publication implementation and new tests. It is possible to remove some of the initialization code because the work is already handled by other library under the hood (most likely by NebulaPublishPlugin). For instance the condition to test the `groupId` presence: `if (groupId == null)` was pointless. It was never `true`. If the `project.group` is not defined the publication task fails with an exception, if there is a custom `groupId` value setup in publication config then it overrides the `project.group` as well. Tests are provided to cover these use cases. As for the new tests they cover the following cases: - verify that the task "publishToMavenLocal" gets expected results. The inspiration for this comes from opensearch-project/opensearch-plugin-template-java#35 - verify that applying only the 'opensearch.pluginzip' is enough, because both the NebulaPublish and MavenPublishPlugin plugins are added explicitly and tasks are chained correctly - verify that if the plugin is applied but no publication is defined then a message is printed for the user Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
…t#4629) * Add missing Javadoc tag descriptions in unit tests Using JDK14 (downloaded from https://jdk.java.net/archive/), I tried running ./gradlew check, and received errors like: server/src/test/java/org/opensearch/indices/replication/SegmentReplicationTargetTests.java:440: warning: no description for @throws The build then failed with "warnings found and -Werror specified". I tried again with JDK17 and the build passed. According to DEVELOPER_GUIDE.md, we should be able to build with JDK14, I added the missing Javadoc tag descriptions. Signed-off-by: Michael Froh <froh@amazon.com> * Update CHANGELOG.md Signed-off-by: Michael Froh <froh@amazon.com> Signed-off-by: Michael Froh <froh@amazon.com>
…azure (opensearch-project#4594) * Bump reactor-netty-http in /plugins/repository-azure Bumps [reactor-netty-http](https://github.com/reactor/reactor-netty) from 1.0.18 to 1.0.23. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](reactor/reactor-netty@v1.0.18...v1.0.23) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-http dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…opensearch-project#4592) * Bump jettison from 1.5.0 to 1.5.1 in /plugins/discovery-azure-classic Bumps [jettison](https://github.com/jettison-json/jettison) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/jettison-json/jettison/releases) - [Commits](jettison-json/jettison@jettison-1.5.0...jettison-1.5.1) --- updated-dependencies: - dependency-name: org.codehaus.jettison:jettison dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Related commit: 0e9f74e Signed-off-by: Heemin Kim <heemin@amazon.com> Signed-off-by: Heemin Kim <heemin@amazon.com>
…nAction (opensearch-project#4639) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…to make it possible for the plugins to access associated Netty4HttpChannel / Netty4HttpServerChannel instance (opensearch-project#4638) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
…ranch Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Re basing feature/master-task-throttling branch with main branch
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.