Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka connector loop retry #2792

Merged
merged 12 commits into from
Jul 17, 2024
Merged

Kafka connector loop retry #2792

merged 12 commits into from
Jul 17, 2024

Conversation

chillleader
Copy link
Member

Description

Adds retries to the Kafka connector consumer loop.

Previously, when the consumer threw an exception, it just stopped processing until the process is redeployed. The new default is to retry this as long as the connector is active, with a backoff of 30 seconds.

Related issues

closes #2750

@chillleader chillleader self-assigned this Jun 25, 2024
@chillleader chillleader marked this pull request as ready for review June 25, 2024 18:04
@chillleader chillleader requested a review from a team as a code owner June 25, 2024 18:04
johnBgood
johnBgood previously approved these changes Jun 26, 2024
Copy link
Collaborator

@johnBgood johnBgood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice!

johnBgood
johnBgood previously approved these changes Jun 26, 2024
Copy link
Collaborator

@johnBgood johnBgood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 minor comment only 👍

sbuettner
sbuettner previously approved these changes Jun 27, 2024
@sbuettner sbuettner force-pushed the kafka-connector-loop-retry branch from c813afa to 9e0022e Compare June 27, 2024 09:02
@sbuettner sbuettner enabled auto-merge June 27, 2024 09:02
@sbuettner sbuettner added this pull request to the merge queue Jun 27, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 27, 2024
@sbuettner sbuettner added this pull request to the merge queue Jul 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 2, 2024
@sbuettner sbuettner added this pull request to the merge queue Jul 2, 2024
@sbuettner sbuettner removed this pull request from the merge queue due to a manual request Jul 2, 2024
@sbuettner sbuettner force-pushed the kafka-connector-loop-retry branch from 9e0022e to d1718a6 Compare July 2, 2024 09:24
@sbuettner sbuettner added this pull request to the merge queue Jul 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 2, 2024
@sbuettner
Copy link
Contributor

@chillleader This one might require e2e test fixes.

@chillleader
Copy link
Member Author

@sbuettner this should have fixed the test

@chillleader chillleader requested a review from johnBgood July 15, 2024 16:25
@sbuettner sbuettner enabled auto-merge July 17, 2024 14:27
@sbuettner sbuettner force-pushed the kafka-connector-loop-retry branch from 47ce498 to 9575a75 Compare July 17, 2024 14:27
@sbuettner sbuettner added this pull request to the merge queue Jul 17, 2024
Merged via the queue into main with commit 807071e Jul 17, 2024
9 checks passed
@sbuettner sbuettner deleted the kafka-connector-loop-retry branch July 17, 2024 14:56
@chillleader
Copy link
Member Author

/backport

Copy link
Contributor

Backport failed for release/8.3, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/8.3
git worktree add -d .worktree/backport-2792-to-release/8.3 origin/release/8.3
cd .worktree/backport-2792-to-release/8.3
git switch --create backport-2792-to-release/8.3
git cherry-pick -x 807071e2ae74a81e73d2689505a131e1cdd8bb7e

Copy link
Contributor

Backport failed for release/8.4, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/8.4
git worktree add -d .worktree/backport-2792-to-release/8.4 origin/release/8.4
cd .worktree/backport-2792-to-release/8.4
git switch --create backport-2792-to-release/8.4
git cherry-pick -x 807071e2ae74a81e73d2689505a131e1cdd8bb7e

Copy link
Contributor

Backport failed for release/8.5, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/8.5
git worktree add -d .worktree/backport-2792-to-release/8.5 origin/release/8.5
cd .worktree/backport-2792-to-release/8.5
git switch --create backport-2792-to-release/8.5
git cherry-pick -x 807071e2ae74a81e73d2689505a131e1cdd8bb7e

chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Jul 25, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting

(cherry picked from commit 807071e)
chillleader added a commit that referenced this pull request Aug 7, 2024
* fix(kafka): retry the main consumer loop with 30 seconds backoff

* reduce test wait duration

* lint

* switch to failsafe

* set number of retries to -1 (unlimited)

* use constant for retry number

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* try fixing the integration test

* formatting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kafka inbound connector stops processing messages on any error
3 participants