Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add deadline for rpcClient calls #994

Merged

Conversation

DiegoTavares
Copy link
Collaborator

Calls are frequently getting locked, setting a timeout is strongly recommended on the grpc documentation.

  • Change rqd cache arguments
  • Increase the cache size to reduce cacheMiss (Currently at 50%) and reduce expiration to avoid too many channels idle.
  • upgrade grpc minor version as there are some issues reported on the previous version related to timeouts on rpc calls.

Sorting jobs only by priority causes a situation where low priority jobs can get starved by a constant flow of high priority jobs.
The new formula adds a modifier to the sorting rank to take into account the number of cores the job is requesting and also the number of days the job is waiting on the queue.
Priorities numbers over 200 will mostly override the formula and work as a priority only based scheduling.
sort = priority + (100 * (1 - (job.cores/job.int_min_cores))) + (age in days)

Besides that, also take layer_int_cores_min into account when filtering folder_resourse limitations to avoid allocating more cores than the folder limits.

(cherry picked from commit 566411aeeddc60983a30eabe121fd03263d05525)
Calls are frequently getting locked, setting a timeout is strongly recommended
on the grpc documentation.

Change rqd cache arguments

Increasing the cache size to reduce cacheMiss (Currently at 50%) and
reduce expiration to avoid too many channels idle.
@DiegoTavares DiegoTavares merged commit 87bcf7a into AcademySoftwareFoundation:master Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants