New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rt: add LocalRuntime #6808

Merged

Noah-Kennedy merged 32 commits into master from noah/localruntime

Oct 12, 2024

Contributor

Noah-Kennedy commented Aug 30, 2024

This change adds LocalRuntime, a new unstable runtime type which cannot be transferred across thread boundaries and supports spawn_local when called from the thread which owns the runtime.

The initial set of docs for this are iffy. Documentation is absent right now at the module level, with the docs for the LocalRuntime struct itself being somewhat duplicative of those for the Runtime type. This can probably be addressed later as stabilization nears.

This API has a few interesting implementation details:

because it was considered beneficial to reuse the same Handle as the normal runtime, it is possible to call spawn_local from a runtime context while on a different thread from the one which drives the runtime and owns it. This forces us to check the thread ID before attempting a local spawn.
An empty LocalOptions struct is passed into the build_local method in order to build the runtime. This will eventually have stuff in it like hooks.

Relates to #6739.

Noah-Kennedy requested review from carllerche and Darksonn

August 30, 2024 16:54

github-actions bot added R-loom-current-thread R-loom-multi-thread R-loom-multi-thread-alt labels

Noah-Kennedy self-assigned this

Noah-Kennedy mentioned this pull request

Add a LocalRuntime for replacing Runtime+LocalSet #6739

Open

taikulawo commented Sep 2, 2024

Does the #[tokio::main] and #[tokio::test] support start as LocalRuntime?

Contributor Author

Noah-Kennedy commented Sep 2, 2024

Does the #[tokio::main] and #[tokio::test] support start as LocalRuntime?

No, not in this PR.

I scoped this change to adding the initial implementation of LocalRuntime and its associated set of core APIs.

I'd like to make these APIs available before we start looking at how to support LocalRuntime in the macros, and stabilize the runtime independently before we stabilize its usage semantics in our macros.

Darksonn added A-tokio M-runtime labels

Noah-Kennedy mentioned this pull request

Proposal: Deprecate Tokio's LocalSet #6741

Open

Darksonn reviewed

View reviewed changes

tokio/src/runtime/builder.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/scheduler/current_thread/mod.rs Show resolved Hide resolved

tokio/src/runtime/task/list.rs Show resolved Hide resolved

yotamofek reviewed

View reviewed changes

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

This comment was marked as off-topic.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

Noah-Kennedy requested a review from Darksonn

September 10, 2024 15:43

Darksonn reviewed

View reviewed changes

tokio/src/runtime/handle.rs Show resolved Hide resolved

tokio/src/runtime/local_runtime/options.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/task/local.rs Outdated Show resolved Hide resolved

ThinkChaos reviewed

View reviewed changes

tokio/src/runtime/scheduler/current_thread/mod.rs Outdated Show resolved Hide resolved

Noah-Kennedy force-pushed the noah/localruntime branch from c03338d to 67ff600 Compare

October 2, 2024 20:09

Darksonn reviewed

View reviewed changes

tokio/src/runtime/builder.rs Outdated Show resolved Hide resolved

tokio/src/runtime/handle.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs

Comment on lines +19 to +22

		/// This runtime is incompatible with `LocalSet`. You should not attempt to drive a `LocalSet` within a
		/// `LocalRuntime`.

Contributor

Darksonn Oct 4, 2024

Why? What happens if you try?

Contributor Author

Noah-Kennedy Oct 4, 2024

LocalRuntime will intercept spawn_local calls within the localset context, and get the events.

I didn't want to specify the exact behavior here, in case we make changes such that this panics or similar.

tokio/tests/rt_local.rs Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

tokio/src/runtime/local_runtime/runtime.rs Outdated Show resolved Hide resolved

Noah-Kennedy requested a review from Darksonn

October 4, 2024 16:11

Darksonn approved these changes

View reviewed changes

Contributor

Darksonn left a comment

LGTM assuming hds's comment is fixed.

Noah-Kennedy added 6 commits

October 12, 2024 09:38


          rt: add LocalRuntime

5b87bc6

This change adds LocalRuntime, a new unstable runtime type which cannot be transferred across thread boundaries and supports spawn_local when called from the thread which owns the runtime.

The initial set of docs for this are iffy. Documentation is absent right now at the module level, with the docs for the LocalRuntime struct itself being somewhat duplicative of those for the `Runtime` type. This can probably be addressed later as stabilization nears.

This API has a few interesting implementation details:
- because it was considered beneficial to reuse the same Handle as the normal runtime, it is possible to call spawn_local from a runtime context while on a different thread from the one which drives the runtime and owns it. This forces us to check the thread ID before attempting a local spawn.
- An empty LocalOptions struct is passed into the build_local method in order to build the runtime. This will eventually have stuff in it like hooks.

Relates to #6739.


          fix oversite with tokio_unstable gates leading to compile failure

1ba8bc0


          appease clippy

e9ca7b8


          fix paths

439c084


          fix localset spawn

20e7bda


          no rustc, i actually want it this way to prevent this from accidental…

5aa073b

…ly getting broken in the future

Noah-Kennedy and others added 23 commits

October 12, 2024 09:38


          i dislike having spellcheck even more

d7fa83e


          fix typo

29d72ba

Co-authored-by: Yotam Ofek <yotam.ofek@gmail.com>


          get rid of spawn, improve docs for spawn_local

0dcfcd9


          Update tokio/src/runtime/local_runtime/runtime.rs

662e37e

Co-authored-by: Alice Ryhl <aliceryhl@google.com>


          Update tokio/src/runtime/local_runtime/runtime.rs

17a3862

Co-authored-by: Alice Ryhl <aliceryhl@google.com>


          take LocalOptions by reference in build_local

11715ac


          make LocalOptions !Send + !Sync


          fix safety docs

073a049


          cleanup spawn_blocking docs

fdb4976


          cleanup runtime docs

db4d554


          cleanup runtime docs

0e6b6a4


          hopefully this fixes docs

30721a5


          add tests

8c45993


          "spelling"

40e69ed


          i am going to become the joker

4b58bdb


          hah, spellcheck doesnt even catch my actual grammar issues


          Update tokio/src/runtime/local_runtime/runtime.rs

6fc68fb

Co-authored-by: Alice Ryhl <aliceryhl@google.com>


          Update tokio/src/runtime/local_runtime/runtime.rs

8e414e6

Co-authored-by: Alice Ryhl <aliceryhl@google.com>


          add panics section to build_local

20b4c92


          send big futures to the heap in release mode

bf5e5f9


          document that handle.enter() allows spawn_local with LocalRuntime

85f2c36


          document that only blocking tasks leak

017adc9


          update with tracing task size stuff

613dcd6

Noah-Kennedy force-pushed the noah/localruntime branch from a1fd7fe to 613dcd6 Compare

October 12, 2024 14:51


          we stabilized id!

92cafd6

Noah-Kennedy merged commit 512e9de into master

87 checks passed

Noah-Kennedy deleted the noah/localruntime branch

October 12, 2024 15:39

mox692 mentioned this pull request

chore: prepare Tokio v1.41.0 #6917

Merged

andrewdavidmackenzie mentioned this pull request

build(deps): bump tokio from 1.40.0 to 1.41.0 andrewdavidmackenzie/flow#2329

Merged

kodiakhq bot pushed a commit to pdylanross/fatigue that referenced this pull request


          chore: Bump tokio from 1.40.0 to 1.41.0 (#340)

7e6ff54

Bumps tokio from 1.40.0 to 1.41.0.

Release notes
Sourced from tokio's releases.

Tokio v1.41.0
1.41.0 (Oct 22th, 2024)
Added

metrics: stabilize global_queue_depth (#6854, #6918)
net: add conversions for unix SocketAddr (#6868)
sync: add watch::Sender::sender_count (#6836)
sync: add mpsc::Receiver::blocking_recv_many (#6867)
task: stabilize Id apis (#6793, #6891)

Added (unstable)

metrics: add H2 Histogram option to improve histogram granularity (#6897)
metrics: rename some histogram apis (#6924)
runtime: add LocalRuntime (#6808)

Changed

runtime: box futures larger than 16k on release mode (#6826)
sync: add #[must_use] to Notified (#6828)
sync: make watch cooperative (#6846)
sync: make broadcast::Receiver cooperative (#6870)
task: add task size to tracing instrumentation (#6881)
wasm: enable cfg_fs for wasi target (#6822)

Fixed

net: fix regression of abstract socket path in unix socket (#6838)

Documented

io: recommend OwnedFd with AsyncFd (#6821)
io: document cancel safety of AsyncFd methods (#6890)
macros: render more comprehensible documentation for join and try_join (#6814, #6841)
net: fix swapped examples for TcpSocket::set_nodelay and TcpSocket::nodelay (#6840)
sync: document runtime compatibility (#6833)

#6793: tokio-rs/tokio#6793
#6808: tokio-rs/tokio#6808
#6810: tokio-rs/tokio#6810
#6814: tokio-rs/tokio#6814
#6821: tokio-rs/tokio#6821
#6822: tokio-rs/tokio#6822
#6826: tokio-rs/tokio#6826
#6828: tokio-rs/tokio#6828
#6833: tokio-rs/tokio#6833
#6836: tokio-rs/tokio#6836
#6838: tokio-rs/tokio#6838
#6840: tokio-rs/tokio#6840


... (truncated)


Commits

01e04da chore: prepare Tokio v1.41.0 (#6917)
92ccade runtime: fix stability feature flags for docs (#6909)
fbfeb9a metrics: rename *_poll_count_* to *_poll_time_* (#6924)
da745ff metrics: add H2 Histogram option to improve histogram granularity (#6897)
ce1c74f metrics: fix deadlock in injection_queue_depth_multi_thread test (#6916)
28c9a14 metrics: rename injection_queue_depth to global_queue_depth (#6918)
32e0b43 ci: freeze FreeBSD and wasm-unknown-unknown on rustc 1.81 (#6911)
1656d8e sync: add mpsc::Receiver::blocking_recv_many (#6867)
c9e998e ci: print the correct sort order of the dictionary on failure (#6905)
512e9de rt: add LocalRuntime (#6808)
Additional commits viewable in compare view




Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

ThinkChaos ThinkChaos left review comments

yotamofek yotamofek left review comments

hds hds left review comments

Darksonn Darksonn approved these changes

carllerche Awaiting requested review from carllerche

Labels

A-tokio M-runtime R-loom-current-thread R-loom-multi-thread R-loom-multi-thread-alt