Segment Replication - Add additional unit tests for update & delete ops. #4237

mch2 · 2022-08-16T21:46:09Z

Signed-off-by: Marc Handalian handalm@amazon.com

Description

This adds index shard level tests for update & delete ops while using segrep.

Issues Resolved

related #4216

Check List

[ x] New functionality includes testing.
- [ x] All tests pass
New functionality has been documented.
- New functionality has javadoc added
[x ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-08-16T21:54:52Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1790/
CommitID: 28ce90a458078c437119c03f724c78e00008d3be

github-actions · 2022-08-16T22:33:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1792/
CommitID: 277d4bfce920c248dfd673bee23dd6a7625feca2

dreamer-89 · 2022-08-16T22:04:57Z

server/src/test/java/org/opensearch/index/shard/SegmentReplicationIndexShardTests.java

@@ -52,6 +58,54 @@ public void testReplicationCheckpointNotNullForSegReb() throws IOException {
        closeShards(indexShard);
    }

+    public void testSegmentReplication_Index_Update_Delete() throws Exception {
+        String mappings = "{ \"" + MapperService.SINGLE_MAPPING_NAME + "\": { \"properties\": { \"foo\": { \"type\": \"keyword\"} }}}";


nit: We can remove _doc mapping as mapping types are deprecated.

dreamer-89 · 2022-08-16T22:57:57Z

server/src/test/java/org/opensearch/index/shard/SegmentReplicationIndexShardTests.java

+            for (int i = 0; i < numDocs; i++) {
+                // randomly delete.
+                if (randomBoolean()) {
+                    shards.delete(new DeleteRequest(index.getName()).id(String.valueOf(i)));


nit: We can count number of docs deleted and assert docsAfterDelete.size() == numDocs-deletedDocs

After update & Delete i'm asserting the equality of each doc id not just counts.

dreamer-89 · 2022-08-16T22:59:22Z

@mch2 : You will need to rebase from main

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':distribution:bwc:staged:buildBwcLinuxTar'.
> Building 2.2.0 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/staged/build/bwc/checkout-2.2/distribution/archives/linux-tar/build/distributions/opensearch-min-2.2.0-SNAPSHOT-linux-x64.tar.gz

…perations. Signed-off-by: Marc Handalian <handalm@amazon.com>

Signed-off-by: Marc Handalian <handalm@amazon.com>

github-actions · 2022-08-16T23:42:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/1799/
CommitID: 505765a

mch2 · 2022-08-17T01:09:46Z

org.opensearch.index.fielddata.SortedSetDVStringFieldDataTests > testSortMissingLast FAILED
    java.lang.AssertionError: expected:<1796> but was:<1766>
        at __randomizedtesting.SeedInfo.seed([96E3585621F3ED7A:7E9CD7DF9752B87B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:633)
        at

github-actions · 2022-08-17T01:36:44Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/1807/
CommitID: 505765a

codecov-commenter · 2022-08-17T01:37:39Z

Codecov Report

Merging #4237 (505765a) into main (5f2e66b) will decrease coverage by 0.10%.
The diff coverage is 92.50%.

@@             Coverage Diff              @@
##               main    #4237      +/-   ##
============================================
- Coverage     70.78%   70.67%   -0.11%     
+ Complexity    57218    57121      -97     
============================================
  Files          4605     4606       +1     
  Lines        274695   274707      +12     
  Branches      40228    40228              
============================================
- Hits         194441   194154     -287     
- Misses        63955    64289     +334     
+ Partials      16299    16264      -35

Impacted Files	Coverage Δ
...main/java/org/opensearch/common/lucene/Lucene.java	`66.02% <ø> (-1.28%)`	⬇️
...c/main/java/org/opensearch/index/IndexService.java	`74.31% <0.00%> (-0.17%)`	⬇️
...index/codec/PerFieldMappingPostingFormatCodec.java	`64.28% <ø> (ø)`
...g/opensearch/index/engine/EngineConfigFactory.java	`88.63% <ø> (ø)`
...earch/index/translog/WriteOnlyTranslogManager.java	`60.00% <ø> (ø)`
...s/replication/SegmentReplicationSourceHandler.java	`87.71% <ø> (-0.81%)`	⬇️
...org/opensearch/index/shard/IndexShardTestCase.java	`94.04% <ø> (ø)`
...n/java/org/opensearch/index/engine/NoOpEngine.java	`63.88% <80.00%> (-0.90%)`	⬇️
...va/org/opensearch/index/engine/ReadOnlyEngine.java	`72.59% <80.00%> (+0.20%)`	⬆️
server/src/main/java/org/opensearch/Version.java	`79.81% <100.00%> (+0.09%)`	⬆️
... and 490 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…ps. (opensearch-project#4237) * Segment Replication - Add additional unit tests for update & delete operations. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix spotless. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com>

…aining segment replication changes (#4243) * [Segment Replication] Checkpoint Replay on Replica Shard (#3658) * Adding Latest Recevied checkpoint, replay checkpoint logic along with tests Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * [Segment Replication] Wire up segment replication with peer recovery and add ITs. (#3743) * Add null check when computing max segment version. With segment replication enabled it is possible Lucene does not set the SegmentInfos min segment version, leaving the default value as null. Signed-off-by: Marc Handalian <handalm@amazon.com> * Update peer recovery to set the translogUUID of replicas to the UUID generated on the primary. This change updates the UUID when the translog is created to the value stored in the passed segment userdata. This is to ensure during failover scenarios that the replica can be promoted and not have a uuid mismatch with the value stored in user data. Signed-off-by: Marc Handalian <handalm@amazon.com> * Wire up Segment Replication under the feature flag. This PR wires up segment replication and adds some initial integration tests. Signed-off-by: Marc Handalian <handalm@amazon.com> * Add test to ensure replicas use primary translog uuid with segrep. Signed-off-by: Marc Handalian <handalm@amazon.com> * Update SegmentReplicationIT to assert previous commit points are valid and SegmentInfos can be built. Fix nitpicks in PR feedback. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix test with Assert.fail to include a message. Signed-off-by: Marc Handalian <handalm@amazon.com> * Disable shard idle with segment replication. (#4118) This change disables shard idle when segment replication is enabled. Primary shards will only push out new segments on refresh, we do not want to block this based on search behavior. Replicas will only refresh on an externally provided SegmentInfos, so we do not want search requests to hang waiting for a refresh. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix isAheadOf logic for ReplicationCheckpoint comparison (#4112) Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix waitUntil refresh policy for segrep enabled indices. (#4137) Signed-off-by: Marc Handalian <handalm@amazon.com> * Add IndexShard#getLatestReplicationCheckpoint behind segrep enable feature flag (#4163) * Add IndexShard#getLatestReplicationCheckpoint behind segrep enable feature flag Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comment. Move tests to SegmentReplicationIndexShardTests Signed-off-by: Suraj Singh <surajrider@gmail.com> * Add segrep enbaled index settings in TargetServiceTests, SourceHandlerTests Signed-off-by: Suraj Singh <surajrider@gmail.com> * [Segment Replication] Fix OngoingSegmentReplications to key by allocation ID instead of DiscoveryNode. (#4182) * Fix OngoingSegmentReplications to key by allocation ID instead of DiscoveryNode. This change fixes segrep to work with multiple shards per node by keying ongoing replications on allocation ID. This also updates cancel methods to ensure state is properly cleared on shard cancel. Signed-off-by: Marc Handalian <handalm@amazon.com> * Clean up cancel methods. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> * [Bug] [Segment Replication] Update store metadata recovery diff logic to ignore missing files causing exception (#4185) * Update Store for segment replication dif Signed-off-by: Poojita Raj <poojiraj@amazon.com> Signed-off-by: Poojita Raj <poojiraj@amazon.com> * Update recoveryDiff logic to ingore missing files causing exception on replica during copy Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Poojita Raj <poojiraj@amazon.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Poojita Raj <poojiraj@amazon.com> * [Segment Replication] Adding PrimaryMode check before publishing checkpoint and processing a received checkpoint. (#4157) * Adding PrimaryMode check before publishing checkpoint. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Applying spotless check Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Moving segrep specific tests to SegmentReplicationIndexShardTests. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Adding logic and tests for rejecting checkpoints if shard is in PrimaryMode. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Applying ./gradlew :server:spotlessApply. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Applying ./gradlew :server:spotlessApply Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Changing log level to warn in shouldProcessCheckpoint() of IndexShard.java class. Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Removing unnecessary lazy logging in shouldProcessCheckpoint(). Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * [Segment Replication] Wait for documents to replicate to replica shards (#4236) * [Segment Replication] Add thread sleep to account for replica lag in delete operations test Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments, assertBusy on doc count rather than sleep Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> * Segment Replication - Remove unnecessary call to markAllocationIdAsInSync. (#4224) This PR Removes an unnecessary call to markAllocationIdAsInSync on the primary shard when replication events complete. Recovery will manage this initial call. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> * Segment Replication - Add additional unit tests for update & delete ops. (#4237) * Segment Replication - Add additional unit tests for update & delete operations. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix spotless. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Poojita Raj <poojiraj@amazon.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Poojita Raj <poojiraj@amazon.com>

mch2 requested review from a team and reta as code owners August 16, 2022 21:46

dreamer-89 approved these changes Aug 16, 2022

View reviewed changes

dreamer-89 reviewed Aug 16, 2022

View reviewed changes

mch2 added 2 commits August 16, 2022 16:18

Segment Replication - Add additional unit tests for update & delete o…

5b937a5

…perations. Signed-off-by: Marc Handalian <handalm@amazon.com>

Fix spotless.

505765a

Signed-off-by: Marc Handalian <handalm@amazon.com>

mch2 force-pushed the units branch from 277d4bf to 505765a Compare August 16, 2022 23:18

dreamer-89 mentioned this pull request Aug 17, 2022

[BUG] Flakey test failure in SortedSetDVStringFieldDataTests testSortMissingLast #4238

Closed

dreamer-89 merged commit 3a97d4c into opensearch-project:main Aug 17, 2022

Rishikesh1159 mentioned this pull request Aug 17, 2022

[Backport 2.x] [Segment Replication] Backport all PR's containing remaining segment replication changes #4243

Merged

mch2 deleted the units branch January 11, 2023 18:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Segment Replication - Add additional unit tests for update & delete ops. #4237

Segment Replication - Add additional unit tests for update & delete ops. #4237

mch2 commented Aug 16, 2022

github-actions bot commented Aug 16, 2022

github-actions bot commented Aug 16, 2022

dreamer-89 Aug 16, 2022

dreamer-89 Aug 16, 2022

mch2 Aug 16, 2022

dreamer-89 commented Aug 16, 2022 •

edited

Loading

github-actions bot commented Aug 16, 2022

mch2 commented Aug 17, 2022

github-actions bot commented Aug 17, 2022

codecov-commenter commented Aug 17, 2022

Segment Replication - Add additional unit tests for update & delete ops. #4237

Segment Replication - Add additional unit tests for update & delete ops. #4237

Conversation

mch2 commented Aug 16, 2022

Description

Issues Resolved

Check List

github-actions bot commented Aug 16, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 16, 2022

Gradle Check (Jenkins) Run Completed with:

dreamer-89 Aug 16, 2022

Choose a reason for hiding this comment

dreamer-89 Aug 16, 2022

Choose a reason for hiding this comment

mch2 Aug 16, 2022

Choose a reason for hiding this comment

dreamer-89 commented Aug 16, 2022 • edited Loading

github-actions bot commented Aug 16, 2022

Gradle Check (Jenkins) Run Completed with:

mch2 commented Aug 17, 2022

github-actions bot commented Aug 17, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Aug 17, 2022

Codecov Report

dreamer-89 commented Aug 16, 2022 •

edited

Loading