Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give corpus pruning enough time to complete #4396

Merged
merged 4 commits into from
Nov 12, 2024
Merged

Give corpus pruning enough time to complete #4396

merged 4 commits into from
Nov 12, 2024

Conversation

jonathanmetzman
Copy link
Collaborator

@jonathanmetzman jonathanmetzman commented Nov 11, 2024

Corpus pruning can take up to 24 hours to complete, this is more than the default of 6 hours.

@@ -33,7 +33,8 @@

_local = threading.local()

MAX_DURATION = f'{60 * 60 * 6}s'
DEFAULT_MAX_RUN_DURATION = f'{6 * 60 * 60}s'
CORPUS_PRUNING_MAX_RUN_DURATION = f'{24 * 60 * 60}s'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just use

TASK_LEASE_SECONDS_BY_COMMAND = {
as the source of truth instead?

@jonathanmetzman jonathanmetzman merged commit 20997be into master Nov 12, 2024
7 checks passed
@jonathanmetzman jonathanmetzman deleted the maxdur branch November 12, 2024 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants