Size snapshot

[sunfishcode]

This introduces Read::size_snapshot() and it modifies read_to_end to allocate exactly the number of required bytes up front, and read exactly the number of bytes at once, when possible.

size_snapshot() is not treated as a hint. In the common case, this allows it to do a single read syscall (compared to two in #45928, with the second one to confirm that EOF is reached), which roughly makes up for doing an extra syscall to get the size, and to request exactly the right size buffer (compared to requesting the size + 1 in #45928).

The main downside of this patch compared to #45928 is that it is possible to observe subtle behavior differences due to obtaining the size up front rather than waiting until read returns 0 in some specialized cases. That said, they all involve read_to_end racing with some independent writer, and the new consequences are just that read_to_end might return more of the old data than it should, or less of the new data than it should, if one knows enough about the underlying OS to reason about what "should" be possible.

This patch also depends on metadata() being reliable and not memoized. On OS's I'm familiar with it's reliable (when the size isn't 0). And in the current codebase, it's not memoized.

EDITED: #45928 now requests size_hint + 1 bytes, fixing the main performance difference.

[rust-highfive]

r? @aidanhs

(rust_highfive has picked a reviewer for you, use r? to override)

[SimonSapin]

As mentioned in #45928, a file could grow after its size was obtained and before the final read.

[sunfishcode]

If the file grows after its size is obtained, it's already a race whether read_to_end sees the extra bytes, because if gets to the old EOF before the write happens, it'll stop and it won't see the write.

[aidanhs]

r? @sfackler

[SimonSapin]

Except for slightly different implementation strategy this is mostly the same as #45928. Both should be discussed together.

[aidanhs]

ping @sfackler for review! Also pinging on IRC

[aidanhs]

Oh, also @sunfishcode I see there are merge conflicts to resolve. I'll keep this as to-review though since it sounds like this may be a multipart discussion.

[sunfishcode]

Merge conflicts now resolved.

[shepmaster]

I'm going to make an executive decision and mark this and #45928 as waiting on a team decision as it seems that they cannot both exist together. Let me know if you disagree.

[sfackler]

The @rust-lang/libs team talked about this and #45928 during our triage meeting and decided that we're not looking to expand the surface area of Read in this way just yet. Instead, we can modify the implementation of fs::read to do a length lookup which should cover 99% of the use cases for this kind of thing.