Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuring kafka producer max.message.size #2445

Merged
merged 6 commits into from
Sep 26, 2023

Conversation

francoisferrand
Copy link
Contributor

  • Allow configuring kafka producer's max request size
  • Properly validate oplogPopulator params
  • Remove conf/Config.js, not used in 8.x branches

kafka producer max request size used to be hard-coded in backbeat, even though it is closely linked to a server setting hard-coded in ZKOP.
Goal is now to make it configurable in backbeat, so that ZKOP can propagate the value everywhere and make it configurable via CR.

Note that we do not propagate the value to individual kafka-connect connectors: ZKOP will instead set the default values for all connectors in the kafka-connect cluster.

Issue: BB-436

Seems to be a remnant from Backbeat 7.x, config has been moved to `lib/`
in 8.x and should thus be removed from there.

Issue: BB-436
@bert-e
Copy link
Contributor

bert-e commented Sep 11, 2023

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

Will be set globally in kafka-connect (via ZKOP), and allows custom
values to be (manually) set on each connector if ever needed: as any
"extra" config will be preserved by oplog populator.

Issue: BB-436
@scality scality deleted a comment from bert-e Sep 21, 2023
@bert-e
Copy link
Contributor

bert-e commented Sep 21, 2023

Integration data created

I have created the integration data for the additional destination branches.

  • this pull request will merge improvement/BB-436 into
    development/8.6
  • w/8.7/improvement/BB-436 will be merged into development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

@bert-e
Copy link
Contributor

bert-e commented Sep 21, 2023

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@francoisferrand
Copy link
Contributor Author

/approve

@bert-e
Copy link
Contributor

bert-e commented Sep 26, 2023

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Sep 26, 2023

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

Please check the status of the associated issue BB-436.

Goodbye francoisferrand.

@bert-e bert-e merged commit 4587c06 into development/8.6 Sep 26, 2023
@bert-e bert-e deleted the improvement/BB-436 branch September 26, 2023 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants