Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move restore-expiration processing to transition processor #2467

Merged
merged 2 commits into from
Oct 27, 2023

Conversation

francoisferrand
Copy link
Contributor

The conductor checks the lag of the object-processor to detect if the previous lifecycle listing has been processed, i.e. status has been persisted for all triggered operations.

Consequently it must check the bucket-processor lag (e.g. is listing terminated), but also the data-mover topic (for transitions) and object- processor (for expirations). It may check the cold-archive topics, but it is not needed since the object is immediately updated with a "transition in process" flag.

However, it must not check topics/consumers which are not directly linked to the lifecycle listing: in particular the transition-processor, which handles the end of transitions (update status, requeue...).

To be consistent and avoid unexpected throtlling, restored object expiration messages must be ignored by the conductor: hence we move processing to the transition processor.

Issue: BB-449

The conductor checks the lag of the object-processor to detect if the
previous lifecycle listing has been processed, i.e. status has been
persisted for all triggered operations.

Consequently it must check the bucket-processor lag (e.g. is listing
terminated), but also the data-mover topic (for transitions) and object-
processor (for expirations). It may check the cold-archive topics, but
it is not needed since the object is immediately updated with a
"transition in process" flag.

However, it must not check topics/consumers which are not directly
linked to the lifecycle listing: in particular the transition-processor,
which handles the end of transitions (update status, requeue...).

To be consistent and avoid unexpected throtlling, restored object
expiration messages must be ignored by the conductor: hence we move
processing to the transition processor.

Issue: BB-449
@francoisferrand
Copy link
Contributor Author

/approve

Base automatically changed from bugfix/BB-440 to development/8.6 October 27, 2023 16:30
@bert-e
Copy link
Contributor

bert-e commented Oct 27, 2023

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@bert-e
Copy link
Contributor

bert-e commented Oct 27, 2023

Integration data created

I have created the integration data for the additional destination branches.

  • this pull request will merge bugfix/BB-449 into
    development/8.6
  • w/8.7/bugfix/BB-449 will be merged into development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Oct 27, 2023

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Oct 27, 2023

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

Please check the status of the associated issue BB-449.

Goodbye francoisferrand.

@bert-e bert-e merged commit 3cba5de into development/8.6 Oct 27, 2023
4 checks passed
@bert-e bert-e deleted the bugfix/BB-449 branch October 27, 2023 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants