Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue #3589 #3622

Merged
merged 7 commits into from
May 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 12 additions & 20 deletions docs/migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ We can divide the parameter `k` into the following:

- `Either[Throwable, A] => Unit` - the callback that will complete or fail this effect when called. This is identical as in CE2.
- `=> F[...]` (outer effect) - the effect of registering the callback. This would be e.g. `delay { window.setTimeout(() => cb(...)) }`.
- `Option[F[Unit]]` - an optional effect that will run if the action is canceled. Passing `None` here is equivalent to `Some(F.unit)`
- `Option[F[Unit]]` - an optional effect that will run if the action is canceled. Passing `None` here makes the whole `async(...)` uncancelable.

The most similar method to this in CE2 would be `Concurrent.cancelableF`:

Expand Down Expand Up @@ -217,7 +217,7 @@ Please refer to each library's appropriate documentation/changelog to see how to

| Cats Effect 2.x | Cats Effect 3 | Notes |
| --------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------- |
| `Blocker.apply` | - | blocking pool is [provided by runtime](#where-does-the-blocking-pool-come-from) |
| `Blocker.apply` | - | blocking is [provided by runtime](#how-does-blocking-work) |
| `Blocker#delay` | `Sync[F].blocking`, `Sync[F].interruptible`, `Sync[F].interruptibleMany` | `Blocker` was removed |
| `Blocker(ec).blockOn(fa)`, `Blocker.blockOnK` | [see notes](#no-blockon) | |

Expand All @@ -242,28 +242,27 @@ It is now possible to make the blocking task interruptible using [`Sync`](./type

```scala mdoc
val programInterruptible =
Sync[IO].interruptible(println("hello Sync blocking!"))
Sync[IO].interruptible(println("hello interruptible blocking!"))
```

If we require our operation to be more sensitive to cancelation we can use `interruptibleMany`.
The difference between `interruptible` and `interruptibleMany` is that in case of cancelation
`interruptibleMany` will repeatedly attempt to interrupt until the blocking operation completes or exits,
on the other hand using `interruptible` the interrupt will be attempted only once.

#### Where Does The Blocking Pool Come From?
#### How Does Blocking Work?

The blocking thread pool, similarly to the compute pool, is provided in `IORuntime` when you run your `IO`.
For other effect systems it could be a `Runtime` or `Scheduler`, etc. You can learn more about CE3 [schedulers](./schedulers.md) and [the thread model in comparison to CE2's](./thread-model.md).
Support for blocking actions is provided by `IORuntime`. The runtime provides the blocking threads as needed.
(For other effect systems it could be a `Runtime` or `Scheduler`, etc.)
You can learn more about CE3 [schedulers](./schedulers.md) and [the thread model in comparison to CE2's](./thread-model.md).

```scala mdoc
val runtime = cats.effect.unsafe.IORuntime.global

def showThread() = java.lang.Thread.currentThread().getName()

IO.blocking(showThread())
.product(
IO(showThread())
)
IO(showThread())
.product(IO.blocking(showThread()))
.unsafeRunSync()(runtime)
```

Expand Down Expand Up @@ -439,15 +438,13 @@ You can get an instance of it with `Dispatcher.parallel[F]` (or `sequential[F]`)

```scala
object Dispatcher {
def parallel[F[_]](implicit F: Async[F]): Resource[F, Dispatcher[F]]
def sequential[F[_]](implicit F: Async[F]): Resource[F, Dispatcher[F]]
def parallel[F[_]](await: Boolean = false)(implicit F: Async[F]): Resource[F, Dispatcher[F]]
def sequential[F[_]](await: Boolean = false)(implicit F: Async[F]): Resource[F, Dispatcher[F]]
}
```

> Note: keep in mind the shape of that method: the resource is related to the lifecycle of all tasks you run with a dispatcher.
> When this resource is closed, **all its running tasks are canceled**.
>
> This [might be configurable](https://github.com/typelevel/cats-effect/issues/1881) in a future release.
> When this resource is closed, **all its running tasks are canceled or joined** (depending on the `await` parameter).

Creating a `Dispatcher` is relatively lightweight, so you can create one even for each task you execute, but sometimes it's worth keeping a `Dispatcher` alive for longer.
To find out more, see [its docs](./std/dispatcher.md).
Expand Down Expand Up @@ -709,11 +706,6 @@ For `Clock`, see [the relevant part of the guide](#clock).

Similarly to `Clock`, `Timer` has been replaced with a lawful type class, `Temporal`. Learn more in [its documentation](./typeclasses/temporal.md).

### Tracing

Currently, improved stack traces are not implemented.
There is currently [work in progress](https://github.com/typelevel/cats-effect/pull/1763) to bring them back.

## Test Your Application

If you followed this guide, all your dependencies are using the 3.x releases of Cats Effect, your code compiles and your tests pass,
Expand Down
17 changes: 8 additions & 9 deletions docs/schedulers.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,10 @@ This is true across both the JVM and JavaScript, and while it seems intuitive th

`IO` programs and fibers are ultimately executed on JVM threads, which are themselves mapped directly to kernel threads and, ultimately (when scheduled), to processors. Determining the optimal method of mapping a real-world, concurrent application down to kernel-level threads is an extremely complex problem, and the details of *why* Cats Effect has chosen the particular strategies it employs are discussed later in this section.

As an executive overview though, `IO` leverages three independent thread pools to evaluate programs:
As an executive overview though, `IO` leverages these mostly independent thread pools to evaluate programs:

- **A work-stealing pool for *computation***, consisting of exactly the same number of `Thread`s as there are hardware processors (minimum: 2)
- **A single-threaded schedule dispatcher**, consisting of a single maximum-priority `Thread` which dispatches `sleep`s with high precision
- **An unbounded blocking pool**, defaulting to zero `Thread`s and allocating as-needed (with caching and downsizing) to meet demand for *blocking* operations
- **An unbounded blocking pool**, defaulting to zero `Thread`s and allocating as-needed (with caching and downsizing) to meet demand for *blocking* operations (technically this separate thread pool is built into the work-stealing pool, and handled transparently)

The *vast* majority of any program will execute on the work-stealing pool. Shifting computation between these pools is done transparently to the user, and for the most part, you don't even need to know they are there.

Expand All @@ -39,19 +38,19 @@ Blocking operations are, in practice, unavoidable. Some of this has to do with t

The JVM uses just such a native library call *within the `URL#equals` comparison*, and that library call blocks until the DNS resolution has been completed, because the underlying DNS client within every major OS is *itself* blocking and uses blocking network I/O. So this means that every time you innocently call `==` on `URL`s (or if you're unlucky enough to use them as `Map` keys), you're blocking a thread for as long as it takes for your DNS server to respond with an answer (or non-answer).

Another excellent example is file I/O (such as `FileInputStream`). Filesystems are also a good example of exceptionally old software designed for a previous age being brought into the modern world, and to that end, they are almost universally blocking. Windows local NTFS mounts are a notable exception to this paradigm, but all macOS filesystem operations are blocking at the OS level, as are (in practice) nearly all Linux filesystem operations (io_uring is changing this, but it's still exceptionally new and not yet universally available, seeing its first public release in May 2019).
Another excellent example is file I/O (such as `FileInputStream`). Filesystems are also a good example of exceptionally old software designed for a previous age being brought into the modern world, and to that end, they are almost universally blocking. Windows local NTFS mounts are a notable exception to this paradigm, but all macOS filesystem operations are blocking at the OS level, as are (in practice) nearly all Linux filesystem operations.

File I/O is effectively unavoidable in any application, but it too means blocking a thread while the kernel fishes out the bytes your application has requested.

Clearly, we cannot simply pretend this problem does not exist. This is why the `blocking` and `interruptible`/`interruptibleMany` functions on `IO` are so important: they declare to the `IO` runtime that the effect in question will block a thread, and it *must* be shifted to the blocking worker pool:
Clearly, we cannot simply pretend this problem does not exist. This is why the `blocking` and `interruptible`/`interruptibleMany` functions on `IO` are so important: they declare to the `IO` runtime that the effect in question will block a thread, and it *must* be shifted to a blocking thread:

```scala
IO.blocking(url1 == url2) // => IO[Boolean]
```

In the above, `URL#equals` effect (along with its associated blocking DNS resolution) is moved off of the precious thread-stealing compute pool and onto an unbounded blocking pool. This worker thread then blocks (which causes the kernel to remove it from the processor) for as long as necessary to complete the DNS resolution, after which it returns and completes the boolean comparison. As soon as this effect completes, `IO` *immediately* shifts the fiber back to the work-stealing compute pool, ensuring all of the benefits are applied and the blocking worker thread can be returned to its pool.
In the above, the `URL#equals` effect (along with its associated blocking DNS resolution) is moved off of the precious thread-stealing compute pool and onto a blocking thread. This worker thread then blocks (which causes the kernel to remove it from the processor) for as long as necessary to complete the DNS resolution, after which it returns and completes the boolean comparison. After this effect completes, `IO` shifts the fiber back to the work-stealing compute pool, ensuring all of the benefits are applied and the blocking worker thread can be reused.

This scheduling dance is handled for you entirely automatically, and the only extra work which must be performed by you, the user, is explicitly declaring your thread-blocking effects as `blocking` or `interruptible` (the `interruptible` implementation is similar to `blocking` except that it also utilizes thread interruption to allow fiber cancelation to function even within thread-blocking effects).
This scheduling dance is handled for you entirely automatically, and the only extra work which must be performed by you, the user, is explicitly declaring your thread-blocking effects as `blocking` or `interruptible`/`interruptibleMany` (the `interruptible`/`interruptibleMany` implementation is similar to `blocking` except that it also utilizes thread interruption to allow fiber cancelation to function even within thread-blocking effects).

## JavaScript

Expand Down Expand Up @@ -83,5 +82,5 @@ For the most part, you should never have to worry about any of this. Just unders
## Scala Native

The [Scala Native](https://github.com/scala-native/scala-native) runtime is [single-threaded](https://scala-native.org/en/latest/user/lang.html#multithreading), similarly to ScalaJS. That's why the `IO#unsafeRunSync` is not available.
Be careful with `IO.blocking(...)` as it blocks the thread since there is no dedicated blocking thread pool.
For more in-depth details, see the [article](https://typelevel.org/blog/2022/09/19/typelevel-native.html#how-does-it-work) with explanations of how the Native runtime works.
Be careful with `IO.blocking(...)` as it blocks the thread since there is no dedicated blocking thread pool.
For more in-depth details, see the [article](https://typelevel.org/blog/2022/09/19/typelevel-native.html#how-does-it-work) with explanations of how the Native runtime works.
16 changes: 8 additions & 8 deletions docs/thread-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ id: thread-model
title: Thread Model
---

Cats effect is a powerful tool for creating and reasoning about highly
Cats Effect is a powerful tool for creating and reasoning about highly
concurrent systems. However, to utilize it effectively it is necessary to
understand something about the underlying thread model and how fibers are
scheduled on top of it.

This section also discusses cats effect 2 as the comparison is instructive and
This section also discusses Cats Effect 2 as the comparison is instructive and
many of our users are already at least somewhat familiar with it.

## High-level goals
Expand Down Expand Up @@ -104,8 +104,8 @@ it is quite different between CE2 and CE3.
### Fiber blocking (previously "Semantic Blocking")

Of course, we do also need the ability to tell fibers to wait for conditions to
be fulfilled. If we can't call thread blocking operations (eg Java/Scala builtin
locks, semaphores, etc) then what can we do? It seems we need a notion of
be fulfilled. If we can't call thread blocking operations (e.g., Java/Scala builtin
locks, semaphores, etc.) then what can we do? It seems we need a notion of
_semantic_ blocking, where the execution of a fiber is suspended and control of
the thread it was running on is yielded. This concept is called "fiber blocking" in cats-effect.

Expand All @@ -115,7 +115,7 @@ Cats effect provides various APIs which have these semantics, such as
whereas `IO.sleep` is only fiber blocking.

The building block for arbitrary semantic blocking is `Deferred`, which is a
purely functional promise that can only be completed once
purely functional promise that can only be completed once.

```scala
trait Deferred[F[_], A] {
Expand All @@ -126,15 +126,15 @@ trait Deferred[F[_], A] {
```

`Deferred#get` is fiber blocking until `Deferred#complete` is called and
cats effect provides many more fiber blocking abstractions like
Cats Effect provides many more fiber blocking abstractions like
semaphores that are built on top of this.

## Summary thus far

So we've seen that best performance is achieved when we dedicate use of the compute pool
to evaluating `IO` fiber runloops and ensure that we shift _all_ blocking operations
to a separate blocking threadpool. We've also seen that many things do not need to
block a thread at all - cats effect provides fiber blocking abstractions for waiting
block a thread at all cats effect provides fiber blocking abstractions for waiting
for arbitrary conditions to be satisfied. Now it's time to see the details of how we achieve
this in cats effect 2 and 3.

Expand Down Expand Up @@ -294,7 +294,7 @@ object Main extends IOApp.WithContext {
The good news is that CE3 fixes these things and makes other things nicer as well! :)
Notably, `ContextShift` and `Blocker` are no more.

### Spawn
### Async

CE3 introduces a re-designed typeclass `Async`

Expand Down