Compiler panics when disk is full #115298
Comments
rustbot
added
the
needs-triage
workingjubilee
added
the
T-compiler
Noratrieb
added
I-ICE
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Posting a wall of AI generated text is very unhelpful here. Please refrain from doing so in the future. |
|
This ICE has been reported at least 9 times now; the issues above contain a total of 7 reports, then there's this issue and @kpreid posted about running into this on the community Discord. I do not know why we've seen a recent uptick in reports of this ICE, but as far as I can tell the root cause is this PR (which predates all the reports): #94732, which subtly changes the API of It looks to me like this tripping hazard wasn't noticed at the time, which is hardly surprising considering that this is along an error path and all the reports we have will result in unsuccessful compilation even if the assertion isn't tripped. I am going to try to make this not ICE anymore. The current implementation of |
…=<try> Simplify/Optimize FileEncoder FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing. So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written. The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are now small. Fixes rust-lang#115298 Fixes rust-lang#114671 Fixes rust-lang#108100 Fixes rust-lang#106787
…pkin Simplify/Optimize FileEncoder FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing. So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written. The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small. Fixes rust-lang/rust#115298 Fixes rust-lang/rust#114671 Fixes rust-lang/rust#114045 Fixes rust-lang/rust#108100 Fixes rust-lang/rust#106787
|
It might be good, in addition to fixing this specific error, for the compiler to estimate a "safe" disk space requirement before starting its operations. It could at least issue a warning before initiating or completely fail and proceed only if the user decides to force it. It could also issue a warning at the end of a compilation when disk space is running low, something like: "WARN: Low disk space, this may affect future compilations." |
|
You're welcome to open a feature request issue for this. I do not think we can do such estimation accurately enough to have an acceptable false positive rate, so I suspect we would get reports of the warning firing when it shouldn't when people compile on systems with a small disk. If we actually issue such a thing as a warning, we will probably break a lot of CI that uses |
Well, I suggested it. As expected, the idea was poorly received. Here is the link for record and cross-reference: Check disk space before compilation. |
…pkin Simplify/Optimize FileEncoder FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing. So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written. The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small. Fixes rust-lang/rust#115298 Fixes rust-lang/rust#114671 Fixes rust-lang/rust#114045 Fixes rust-lang/rust#108100 Fixes rust-lang/rust#106787
…pkin Simplify/Optimize FileEncoder FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing. So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written. The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small. Fixes rust-lang/rust#115298 Fixes rust-lang/rust#114671 Fixes rust-lang/rust#114045 Fixes rust-lang/rust#108100 Fixes rust-lang/rust#106787
When compiling my project, the disk filled up and I got:
I’m assuming this is a compiler bug, if only because the error message is not helpful. I haven’t included an example because I don’t think the specific code being compiled matters here.
The text was updated successfully, but these errors were encountered: