Handle `ChangeFeedProcessorContext` for full-fidelity changefeed. #36715

jeet1995 · 2023-09-12T18:24:38Z

The PR introduces public API changes to handle a ChangeFeedProcessorContext along with the change feed batch of documents.

`ChangeFeedProcessorBuilder#handleAllVersionsAndDeletesChanges`

API contract

public ChangeFeedProcessorBuilder handleAllVersionsAndDeletesChanges(BiConsumer<List<ChangeFeedProcessorItem>, ChangeFeedProcessorContext> biConsumer);

Using the API

// initialize the feedContainer & leaseContainer
CosmosAsyncContainer feedContainer, leaseContainer;
        
// build changeFeedProcessorOptions as needed
ChangeFeedProcessorOptions changeFeedProcessorOptions;

ChangeFeedProcessor changeFeedProcessor = new ChangeFeedProcessorBuilder()
    .hostName(hostName)
    .handleAllVersionsAndDeletesChanges((docs, context) -> {
         for (ChangeFeedProcessorItem doc : docs) {
              // process docs based on business logic
         }
         String leaseToken = context.getLeaseToken();
         // process leaseToken based on business logic
    })
    .feedContainer(feedContainer)
    .leaseContainer(leaseContainer)
    .options(changeFeedProcessorOptions)
    .buildChangeFeedProcessor();

azure-sdk · 2023-09-20T18:02:51Z

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-cosmos

…dCFPContext

sdk/cosmos/azure-cosmos/CHANGELOG.md

sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/ChangeFeedProcessorContext.java

…dCFPContext # Conflicts: # sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/changefeed/epkversion/FullFidelityChangeFeedProcessorTest.java # sdk/cosmos/azure-cosmos/CHANGELOG.md

jeet1995 · 2023-09-24T01:36:38Z

/azp run java - cosmos - tests

azure-pipelines · 2023-09-24T01:36:56Z

Azure Pipelines successfully started running 1 pipeline(s).

…dCFPContext

jeet1995 · 2023-09-25T22:55:24Z

/azp run java - cosmos - tests

azure-pipelines · 2023-09-25T22:55:37Z

Azure Pipelines successfully started running 1 pipeline(s).

...test/java/com/azure/cosmos/rx/changefeed/epkversion/FullFidelityChangeFeedProcessorTest.java

sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/ChangeFeedProcessorBuilder.java

jeet1995 · 2023-09-26T13:57:54Z

/azp run java - cosmos - tests

azure-pipelines · 2023-09-26T13:58:09Z

Azure Pipelines successfully started running 1 pipeline(s).

jeet1995 · 2023-09-26T17:42:39Z

/azp run java - cosmos - tests

azure-pipelines · 2023-09-26T17:42:52Z

Azure Pipelines successfully started running 1 pipeline(s).

kushagraThapar

LGTM, great work @jeet1995 , thank you!

…dCFPContext

jeet1995 · 2023-09-27T17:28:38Z

/azp run java - cosmos - tests

azure-pipelines · 2023-09-27T17:28:52Z

Azure Pipelines successfully started running 1 pipeline(s).

jeet1995 · 2023-09-28T00:30:09Z

/check-enforcer override

jeet1995 · 2023-09-28T00:42:45Z

There is a flakiness issue with a few tests in ProactiveConnectionManagementTests and ClientMetricsTests - fixes are being tracked

Adding preliminary changes to handle ChangeFeedProcessorContext.

40e9ee1

github-actions bot added the Cosmos label Sep 12, 2023

jeet1995 added 22 commits September 13, 2023 09:02

Adding preliminary changes to handle ChangeFeedProcessorContext.

9a63a4f

Fixing CI errors.

ec376c0

Fixing CI errors.

94de515

Adding handleChanges which accepts ChangeFeedProcessorContext.

15303ca

Removed code comments.

40407cc

Made handleChanges with context as package-private.

d3f4381

Refactorings.

fc0e8a6

Refactorings.

2fff7ad

Adding tests.

5eac046

Adding tests.

95a5323

Adding tests.

2f63510

Adding tests.

d01e9a6

Adding tests.

9212f9c

Adding tests.

1c189e0

Adding tests.

8f7f03a

Adding validations.

15c08b2

Adding javadoc.

dda726d

Removed type parameter from ChangeFeedProcessorContext.

43f26d4

Updated javadoc.

947b919

Fixing compilation errors.

c5dbee5

Fixing compilation errors.

2a65035

Fixing javadoc.

a346e07

jeet1995 added 3 commits September 20, 2023 16:31

Merge branch 'main' of github.com:jeet1995/azure-sdk-for-java into Ad…

809aa73

…dCFPContext

Fixing javadoc.

59822a2

Reverting changes.

29c6b53

jeet1995 changed the title ~~[No review]: Handle ChangeFeedProcessorContext.~~ Handle ChangeFeedProcessorContext for full-fidelity changefeed. Sep 20, 2023

jeet1995 marked this pull request as ready for review September 20, 2023 23:03

Updated javadoc.

d6bf8cb

kushagraThapar reviewed Sep 21, 2023

View reviewed changes

jeet1995 added 5 commits September 21, 2023 18:36

Reacting to review comments.

11eb5fb

Attempt at fixing CI error.

34f34e3

Attempt at fixing CI pipeline.

3cfe22a

Attempt at fixing CI pipeline.

9220e26

Merge branch 'main' of github.com:jeet1995/azure-sdk-for-java into Ad…

732dfa6

…dCFPContext # Conflicts: # sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/changefeed/epkversion/FullFidelityChangeFeedProcessorTest.java # sdk/cosmos/azure-cosmos/CHANGELOG.md

jeet1995 added 3 commits September 25, 2023 16:27

Merge branch 'main' of github.com:jeet1995/azure-sdk-for-java into Ad…

31299e4

…dCFPContext

Updating javadoc.

4cb59f8

Merge branch 'main' of github.com:jeet1995/azure-sdk-for-java into Ad…

1243a37

…dCFPContext

xinlian12 reviewed Sep 25, 2023

View reviewed changes

...test/java/com/azure/cosmos/rx/changefeed/epkversion/FullFidelityChangeFeedProcessorTest.java Outdated Show resolved Hide resolved

xinlian12 reviewed Sep 26, 2023

View reviewed changes

sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/ChangeFeedProcessorBuilder.java Outdated Show resolved Hide resolved

Addressing review comments.

2861131

Addressing review comments.

33066e4

kushagraThapar approved these changes Sep 26, 2023

View reviewed changes

jeet1995 added 2 commits September 27, 2023 13:26

Merge branch 'main' of github.com:jeet1995/azure-sdk-for-java into Ad…

dd9a1f1

…dCFPContext

Updated CHANGELOG.md.

86a763e

jeet1995 merged commit 59f37b6 into Azure:main Sep 28, 2023
56 of 58 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle `ChangeFeedProcessorContext` for full-fidelity changefeed. #36715

Handle `ChangeFeedProcessorContext` for full-fidelity changefeed. #36715

jeet1995 commented Sep 12, 2023 •

edited

Loading

azure-sdk commented Sep 20, 2023

jeet1995 commented Sep 24, 2023

azure-pipelines bot commented Sep 24, 2023

jeet1995 commented Sep 25, 2023

azure-pipelines bot commented Sep 25, 2023

jeet1995 commented Sep 26, 2023

azure-pipelines bot commented Sep 26, 2023

jeet1995 commented Sep 26, 2023

azure-pipelines bot commented Sep 26, 2023

kushagraThapar left a comment

jeet1995 commented Sep 27, 2023

azure-pipelines bot commented Sep 27, 2023

jeet1995 commented Sep 28, 2023

jeet1995 commented Sep 28, 2023

Handle ChangeFeedProcessorContext for full-fidelity changefeed. #36715

Handle ChangeFeedProcessorContext for full-fidelity changefeed. #36715

Conversation

jeet1995 commented Sep 12, 2023 • edited Loading

ChangeFeedProcessorBuilder#handleAllVersionsAndDeletesChanges

azure-sdk commented Sep 20, 2023

jeet1995 commented Sep 24, 2023

azure-pipelines bot commented Sep 24, 2023

jeet1995 commented Sep 25, 2023

azure-pipelines bot commented Sep 25, 2023

jeet1995 commented Sep 26, 2023

azure-pipelines bot commented Sep 26, 2023

jeet1995 commented Sep 26, 2023

azure-pipelines bot commented Sep 26, 2023

kushagraThapar left a comment

Choose a reason for hiding this comment

jeet1995 commented Sep 27, 2023

azure-pipelines bot commented Sep 27, 2023

jeet1995 commented Sep 28, 2023

jeet1995 commented Sep 28, 2023

Handle `ChangeFeedProcessorContext` for full-fidelity changefeed. #36715

Handle `ChangeFeedProcessorContext` for full-fidelity changefeed. #36715

jeet1995 commented Sep 12, 2023 •

edited

Loading

`ChangeFeedProcessorBuilder#handleAllVersionsAndDeletesChanges`