Fix unnecessary zeroing for AsyncRead implementors #2525

MikailBag · 2020-05-12T09:55:15Z

sfackler · 2020-05-12T11:05:21Z

tokio/src/io/async_read.rs

+    for x in buf {
+        // we could use safe method x.write(0) here, but it is unstable.
+        // Safety: as_mut_ptr() returns valid writeable pointer.
+        unsafe {


*x = MaybeUninit::new(0);

I've updated both places.

MikailBag · 2020-05-12T15:46:38Z

How should I feature-gate prepare_uninitialized_buffer_std_read()?
I don't know which features should be used.

UPD: I fixed this problem by making that function pub. I think it's reasonable, because this function is useful to every type that wraps Read internally.

Darksonn

This seems reasonable to me.

Darksonn · 2020-05-18T20:17:30Z

Oh, I didn't see that this exposes prepare_uninitialized_buffer_std_read. Is that really that useful?

MikailBag · 2020-05-18T21:42:26Z

I am not sure it is useful for outside users. I will make it private, then.

hawkw · 2020-05-18T23:23:42Z

tokio/src/io/async_read.rs

+/// Implements `AsyncRead::prepare_uninitialized_buffer` in a such a way
+/// that is valid for `R` requirements.
+/// Should be used to override `AsyncRead::prepare_uninitialized_buffer`
+/// for any AsyncRead impl which wraps an `std::io::Read` instance.


If this is going to be publicly exposed (and I'm not convinced it needs to be), it would be nice to have a more detailed comment explaining what it does and why users would want to use it...

MikailBag · 2020-05-19T15:50:53Z

I made that function private again, but I can't solve #[cfg] problems. These problems are definitely related to tests. I can't use plain cfg-s because tests want to override it. I can't use cfg_* macros as well because we need OR of cfg_io_blocking and cfg_io_driver, and nested macros effect in AND.

MikailBag · 2020-05-19T15:52:06Z

Alternatively, I can remove all feature-gates, and annotate that function with #[allow(dead_code)], but I fear it would be very poor code style.

Darksonn · 2020-05-21T09:57:15Z

What's the issue with tests? I don't expect tests to successfully run without all features turned on. Anyway, if you can't find another way to fix it, I'm okay with #[allow(dead_code)].

MikailBag · 2020-05-21T13:26:58Z

I expanded prepare_uninitialized_buffer_std_read documentation a bit and annotated with allow(dead_code). I think PR is now finished (except for awful commit history). Can I squash it now?

Darksonn · 2020-05-21T13:28:32Z

No, please leave the history intact. GitHub will ensure that the commit that ends up on master is properly squashed.

If you want to provide an expanded commit message to the commit on master, you can push an empty commit with the description, and I'll make sure it ends up as the description on the squashed commit.

MikailBag · 2020-05-21T14:21:28Z

I think I finally understood why CI is failing.
test fs_file_mocked mocks all tokio::io APIs.
It include!s src/fs/file.rs and src/io/blocking.rs, and imports all other tokio::io items. However, tokio::io::prepare_uninitialized_buffer_std_read is not imported because it is pub(crate). So, when Blocking implementation is compiled in context of fs_file_mocked test (which has no access to private crate items), it can not find crate::io::prepare_uninitialized_buffer_std_read, and build fails.
I don't have good solution for this problem.
I can mark prepare_uninitialized_buffer_std_read as pub and #[doc(hidden)] with comment discouraging using this function, e.g. "this function is marked public for usage in tests. It is not part of API and should not be used outside of Tokio".
I can confirm that with this change cargo test --all-features succeeds (previsouly fs_file_mocked test would not compile).
I will now commit this change; I hope CI will finally pass :)

Darksonn · 2020-05-21T14:29:45Z

Maybe just delete the function and copy its body into blocking?

MikailBag · 2020-05-21T16:20:05Z

Thanks, I implemented your approach, it really looks better that doc(hidden) public item.

MikailBag · 2020-05-21T16:41:00Z

I was told in Discord that PollEvented will likely be excluded from Tokio 0.3, and Read::initializer is unlikely to be stabilized before that.
It means all machinery introduced in this PR will not bring any benefits, so I'm going to rollback prepare_uninitialized_buffer_std_read function at all.
Changes to concrete AsyncRead implementors are still in place.

- docs: misc improvements (#2572, #2658, #2663, #2656, #2647, #2630, #2487, #2621, #2624, #2600, #2623, #2622, #2577, #2569, #2589, #2575, #2540, #2564, #2567, #2520, #2521, #2572, #2493) - rt: allow calls to `block_on` inside calls to `block_in_place` that are themselves inside `block_on` (#2645) - net: fix non-portable behavior when dropping `TcpStream` `OwnedWriteHalf` (#2597) - io: improve stack usage by allocating large buffers on directly on the heap (#2634) - io: fix unsound pin projection in `AsyncReadExt::read_buf` and `AsyncWriteExt::write_buf` (#2612) - io: fix unnecessary zeroing for `AsyncRead` implementors (#2525) - io: Fix `BufReader` not correctly forwarding `poll_write_buf` (#2654) - coop: returning `Poll::Pending` no longer decrements the task budget (#2549) - io: little-endian variants of `AsyncReadExt` and `AsyncWriteExt` methods (#1915) - io: fix panic in `AsyncReadExt::read_line` (#2541) - task: add [`tracing`] instrumentation to spawned tasks (#2655) - sync: allow unsized types in `Mutex` and `RwLock` (via `default` constructors) (#2615) - net: add `ToSocketAddrs` implementation for `&[SocketAddr]` (#2604) - fs: add `OpenOptionsExt` for `OpenOptions` (#2515) - fs: add `DirBuilder` (#2524) [`tracing`]: https://crates.io/crates/tracing Signed-off-by: Eliza Weisman <eliza@buoyant.io>

# 0.2.22 (July 2!, 2020) ### Fixes - docs: misc improvements (#2572, #2658, #2663, #2656, #2647, #2630, #2487, #2621, #2624, #2600, #2623, #2622, #2577, #2569, #2589, #2575, #2540, #2564, #2567, #2520, #2521, #2493) - rt: allow calls to `block_on` inside calls to `block_in_place` that are themselves inside `block_on` (#2645) - net: fix non-portable behavior when dropping `TcpStream` `OwnedWriteHalf` (#2597) - io: improve stack usage by allocating large buffers on directly on the heap (#2634) - io: fix unsound pin projection in `AsyncReadExt::read_buf` and `AsyncWriteExt::write_buf` (#2612) - io: fix unnecessary zeroing for `AsyncRead` implementors (#2525) - io: Fix `BufReader` not correctly forwarding `poll_write_buf` (#2654) - io: fix panic in `AsyncReadExt::read_line` (#2541) ### Changes - coop: returning `Poll::Pending` no longer decrements the task budget (#2549) ### Added - io: little-endian variants of `AsyncReadExt` and `AsyncWriteExt` methods (#1915) - task: add [`tracing`] instrumentation to spawned tasks (#2655) - sync: allow unsized types in `Mutex` and `RwLock` (via `default` constructors) (#2615) - net: add `ToSocketAddrs` implementation for `&[SocketAddr]` (#2604) - fs: add `OpenOptionsExt` for `OpenOptions` (#2515) - fs: add `DirBuilder` (#2524) [`tracing`]: https://crates.io/crates/tracing Signed-off-by: Eliza Weisman <eliza@buoyant.io>

Fix unnecessary zeroing for AsyncRead implementors

6a21dec

MikailBag force-pushed the async-read-uninit branch from 7a98b67 to 6a21dec Compare May 12, 2020 10:01

sfackler reviewed May 12, 2020

View reviewed changes

MikailBag added 2 commits May 12, 2020 14:22

Less unsafe

572d443

Fix typo

70ed766

Darksonn added A-tokio Area: The main tokio crate C-enhancement Category: A PR with an enhancement or bugfix. M-fs Module: tokio/fs M-io Module: tokio/io M-process Module: tokio/process T-performance Topic: performance and benchmarks labels May 12, 2020

try to fix features

1bd2348

MikailBag added 3 commits May 13, 2020 19:02

Expose prepare_uninitialized_buffer_std_read

a00fc0c

Fix bug

28ad620

Cleanup

87169e4

MikailBag marked this pull request as ready for review May 13, 2020 16:23

Darksonn approved these changes May 16, 2020

View reviewed changes

talchas mentioned this pull request May 18, 2020

read_to_end rezeroes the internal buffer many times #2544

Closed

hawkw reviewed May 18, 2020

View reviewed changes

MikailBag added 5 commits May 19, 2020 17:46

[fixup] unpub, attempt 1

3c153aa

attempt=2

777597a

attempt=3

caa170d

attempt=4

a256072

attempt=5

b634498

allow(dead_code)

2cb3ea8

MikailBag added 2 commits May 21, 2020 16:48

Fix allow on use

e8eeb85

Expose the function for tests

fa9299d

MikailBag added 2 commits May 21, 2020 18:37

switch to better approach

65dc798

Fix zeroing in Chain

549860f

Rollback useless machinery

aeb924a

Darksonn approved these changes May 21, 2020

View reviewed changes

Darksonn merged commit 1e54a35 into tokio-rs:master May 21, 2020

MikailBag deleted the async-read-uninit branch May 21, 2020 18:21

jensim pushed a commit to jensim/tokio that referenced this pull request Jun 7, 2020

io: remove zeroing for AsyncRead implementors (tokio-rs#2525)

1514b63

hawkw mentioned this pull request Jul 20, 2020

chore: prepare to release 0.2.22 #2672

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix unnecessary zeroing for AsyncRead implementors #2525

Fix unnecessary zeroing for AsyncRead implementors #2525

MikailBag commented May 12, 2020

sfackler May 12, 2020

MikailBag May 12, 2020

MikailBag commented May 12, 2020 •

edited

Loading

Darksonn left a comment

Darksonn commented May 18, 2020

MikailBag commented May 18, 2020

hawkw May 18, 2020

MikailBag commented May 19, 2020

MikailBag commented May 19, 2020

Darksonn commented May 21, 2020

MikailBag commented May 21, 2020

Darksonn commented May 21, 2020 •

edited

Loading

MikailBag commented May 21, 2020

Darksonn commented May 21, 2020

MikailBag commented May 21, 2020

MikailBag commented May 21, 2020 •

edited

Loading

Fix unnecessary zeroing for AsyncRead implementors #2525

Fix unnecessary zeroing for AsyncRead implementors #2525

Conversation

MikailBag commented May 12, 2020

sfackler May 12, 2020

Choose a reason for hiding this comment

MikailBag May 12, 2020

Choose a reason for hiding this comment

MikailBag commented May 12, 2020 • edited Loading

Darksonn left a comment

Choose a reason for hiding this comment

Darksonn commented May 18, 2020

MikailBag commented May 18, 2020

hawkw May 18, 2020

Choose a reason for hiding this comment

MikailBag commented May 19, 2020

MikailBag commented May 19, 2020

Darksonn commented May 21, 2020

MikailBag commented May 21, 2020

Darksonn commented May 21, 2020 • edited Loading

MikailBag commented May 21, 2020

Darksonn commented May 21, 2020

MikailBag commented May 21, 2020

MikailBag commented May 21, 2020 • edited Loading

MikailBag commented May 12, 2020 •

edited

Loading

Darksonn commented May 21, 2020 •

edited

Loading

MikailBag commented May 21, 2020 •

edited

Loading