Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bucket-processor : do not retry the whole listing range when their is an error on individual objects #2447

Merged
merged 5 commits into from
Oct 13, 2023
Merged

Bucket-processor : do not retry the whole listing range when their is an error on individual objects #2447

merged 5 commits into from
Oct 13, 2023

Conversation

francoisferrand
Copy link
Contributor

@francoisferrand francoisferrand commented Sep 29, 2023 •
edited
Loading

Misc tweaks to lifecycle, which should avoid duplicated processing.

  • Properly handle async calls to _applyTransitionRule
  • Do not retry whole batch when a single entry fails

Issue: BB-437

@bert-e
Copy link
Contributor

bert-e commented Sep 29, 2023

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@nicolas2bert
Copy link
Contributor

Thanks, could it target development/7.10 branch?

@francoisferrand francoisferrand changed the title Lifecycle fixups Bucket-processor : do not retry the whole listing range when their is an error on individual objects Oct 11, 2023
@scality scality deleted a comment from bert-e Oct 11, 2023
@scality scality deleted a comment from bert-e Oct 11, 2023
@scality scality deleted a comment from bert-e Oct 11, 2023
@bert-e
Copy link
Contributor

bert-e commented Oct 11, 2023

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@francoisferrand
Properly handle async calls to _applyTransitionRule
cf621ce 
This esnures we don't process more entries than the expected concurrency
at the same time.

Issue: BB-437
@francoisferrand
Do not retry whole batch when a single entry fails
98c7f7d 
We don't want to retry the _whole_ list if only a single entry fails, so
we possibly retry each entry individually, and ignore errors. Ignoring
error is not too bad anyway, the entry will be picked up again on next
lifecycle run.

Issue: BB-437
@francoisferrand
gha: Bump setup-node to v3
989e565 
Issue: BB-437
@francoisferrand
Bucket-processor: limit the amount of retries
b1688b6 
Limit the total amount of retries when processing a range, to keep the
processing time of the kafka message under check (--> avoid rebalance),
yet keep some resiliance to temporary errors.

It does not really matter that we don't retry a lot, since lfiecycle
will be retried later.

Aslo, propagate the error from _applyTransitionRule() so it may get
retried.

Issue: BB-437
@francoisferrand
Better handle errors on individual objects in lifecycle v2
da37533 
And let `_compareRulesToList()` retry just a little bit, to limit impact/delay
while ensuring message gets processed timely.

Issue: BB-437
@francoisferrand
Copy link
Contributor Author

/approve

@bert-e
Copy link
Contributor

bert-e commented Oct 13, 2023

Integration data created

I have created the integration data for the additional destination branches.

  • this pull request will merge bugfix/BB-437 into
    development/8.6
  • w/8.7/bugfix/BB-437 will be merged into development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Oct 13, 2023

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Oct 13, 2023

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

Please check the status of the associated issue BB-437.

Goodbye francoisferrand.

@bert-e bert-e merged commit da37533 into development/8.6 Oct 13, 2023
7 checks passed
@bert-e bert-e deleted the bugfix/BB-437 branch October 13, 2023 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benzekrimaha benzekrimaha benzekrimaha approved these changes

@rachedbenmustapha rachedbenmustapha rachedbenmustapha approved these changes

@nicolas2bert nicolas2bert nicolas2bert approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@francoisferrand @bert-e @nicolas2bert @rachedbenmustapha @benzekrimaha