Skip to content

Commit

Permalink
doc: improve package doc and add a note about unbounded task results
Browse files Browse the repository at this point in the history
Signed-off-by: Robin Hahling <robin.hahling@gw-computing.net>
  • Loading branch information
rolinh committed Nov 4, 2021
1 parent 2534c95 commit 780bee2
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 2 deletions.
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,19 @@
[![Go Report Card](https://goreportcard.com/badge/github.com/cilium/workerpool)](https://goreportcard.com/report/github.com/cilium/workerpool)

Package workerpool implements a concurrency limiting worker pool. Worker
routines are spawned on demand as tasks are submitted.
routines are spawned on demand as tasks are submitted; up to the configured
limit of concurrent workers.

When the limit of concurrently running workers is reached, submitting a task
blocks until a worker is able to pick it up. This behavior is intentional as it
prevents from accumulating tasks which could grow unbounded. Therefore, it is
the responsibility of the caller to queue up tasks if that's the intended
behavior.

One caveat is that while the number of concurrently running workers is limited,
task results are not and they accumulate until they are collected. Therefore,
if a large number of tasks can be expected, the workerpool should be
periodically drained (e.g. every 10k tasks).

This package is mostly useful when tasks are CPU bound and spawning too many
routines would be detrimental to performance. It features a straightforward API
Expand Down
14 changes: 13 additions & 1 deletion workerpool.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,19 @@
// limitations under the License.

// Package workerpool implements a concurrency limiting worker pool.
// Worker routines are spawned on demand as tasks are submitted.
// Worker routines are spawned on demand as tasks are submitted; up to the
// configured limit of concurrent workers.
//
// When the limit of concurrently running workers is reached, submitting a task
// blocks until a worker is able to pick it up. This behavior is intentional as
// it prevents from accumulating tasks which could grow unbounded. Therefore,
// it is the responsibility of the caller to queue up tasks if that's the
// intended behavior.
//
// One caveat is that while the number of concurrently running workers is
// limited, task results are not and they accumulate until they are collected.
// Therefore, if a large number of tasks can be expected, the workerpool should
// be periodically drained (e.g. every 10k tasks).
package workerpool

import (
Expand Down

0 comments on commit 780bee2

Please sign in to comment.