Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

thread 'rustc' panicked at 'assertion failed: pos.get() <= self.position()', compiler\rustc_metadata\src\rmeta\encoder.rs #108100

Closed
GregoryConrad opened this issue Feb 15, 2023 · 6 comments · Fixed by #115542
Labels
C-bug Category: This is a bug. E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@GregoryConrad
Copy link

GregoryConrad commented Feb 15, 2023

This just started getting thrown in my CI earlier today on 1.67.1 stable. Using cargo-xwin to build for x86_64-pc-windows-msvc on a windows-latest GitHub Actions runner. Failed to compile crate lindera-ko-dic:

(Ignore the out of space error at the end of the CI; that is a bit after the error in question was raised. See the "build the binaries" step.)

2023-02-15T21:47:09.8887505Z thread 'rustc' panicked at 'assertion failed: pos.get() <= self.position()', compiler\rustc_metadata\src\rmeta\encoder.rs:445:9
2023-02-15T21:47:09.8888239Z stack backtrace:
2023-02-15T21:47:09.8888479Z 
2023-02-15T21:47:09.8888537Z 
2023-02-15T21:47:10.7410389Z    0:     0x7ff838009de2 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha63cfe4b75b3ba72
2023-02-15T21:47:10.7415214Z    1:     0x7ff838045d0b - core::fmt::write::h5f1cd682e23942b6
2023-02-15T21:47:10.7419700Z 
2023-02-15T21:47:10.7420325Z    2:     0x7ff837ffca2a - <std::io::IoSliceMut as core::fmt::Debug>::fmt::h6b970ebe775b9627
2023-02-15T21:47:10.7420671Z 
2023-02-15T21:47:10.7421046Z    3:     0x7ff838009b2b - std::sys::common::alloc::realloc_fallback::h413616b22f3f3818
2023-02-15T21:47:10.7421655Z    4:     0x7ff83800d459 - std::panicking::default_hook::h0c26739d8a69e80d
2023-02-15T21:47:10.7421942Z 
2023-02-15T21:47:10.7422246Z    5:     0x7ff83800d0db - std::panicking::default_hook::h0c26739d8a69e80d
2023-02-15T21:47:10.7428029Z 
2023-02-15T21:47:10.7428476Z    6:     0x7ff8164cd16d - rustc_driver[6d4b93d6bcfc3289]::describe_lints
2023-02-15T21:47:10.7428746Z 
2023-02-15T21:47:10.7429133Z    7:     0x7ff83800ddc0 - std::panicking::rust_panic_with_hook::hdf07e8fcc7859773
2023-02-15T21:47:10.7429833Z    8:     0x7ff83800dadb - <std::panicking::begin_panic_handler::StrPanicPayload as core::panic::BoxMeUp>::get::hac35d15c76f5304d
2023-02-15T21:47:10.7434974Z    9:     0x7ff83800aadf - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha63cfe4b75b3ba72
2023-02-15T21:47:10.7435593Z   10:     0x7ff83800d7d0 - rust_begin_unwind
2023-02-15T21:47:10.7436138Z   11:     0x7ff83807bb85 - core::panicking::panic_fmt::hd2b8fa31b060140f
2023-02-15T21:47:10.7436414Z 
2023-02-15T21:47:10.7437017Z   12:     0x7ff83807bc7c - core::panicking::panic::h2d50e3e44ac775d8
2023-02-15T21:47:10.7437292Z 
2023-02-15T21:47:10.7438133Z   13:     0x7ff815f5e814 - <rustc_metadata[154eba34594de74d]::creader::CStore>::from_tcx
2023-02-15T21:47:10.7444406Z   14:     0x7ff814943b57 - rustc_metadata[154eba34594de74d]::rmeta::encoder::encode_metadata
2023-02-15T21:47:10.7444739Z 
2023-02-15T21:47:10.7445128Z   15:     0x7ff81492e4da - rustc_plugin_impl[ce6d0be28df6f001]::load::load_plugins
2023-02-15T21:47:10.7445699Z   16:     0x7ff814942fce - rustc_metadata[154eba34594de74d]::rmeta::encoder::encode_metadata
2023-02-15T21:47:10.7446513Z 
2023-02-15T21:47:10.7446938Z   17:     0x7ff814947573 - rustc_metadata[154eba34594de74d]::fs::encode_and_write_metadata
2023-02-15T21:47:10.7447224Z 
2023-02-15T21:47:10.7452819Z   18:     0x7ff813cdca56 - rustc_interface[aed5c8d6751baeab]::passes::start_codegen
2023-02-15T21:47:10.7453370Z 
2023-02-15T21:47:10.7453469Z 
2023-02-15T21:47:10.7453926Z   19:     0x7ff813cdb459 - rustc_interface[aed5c8d6751baeab]::passes::prepare_outputs
2023-02-15T21:47:10.7454219Z 
2023-02-15T21:47:10.7454674Z   20:     0x7ff813cc84c8 - <rustc_interface[aed5c8d6751baeab]::queries::Queries>::ongoing_codegen
2023-02-15T21:47:10.7454972Z 
2023-02-15T21:47:10.7455028Z 
2023-02-15T21:47:10.7455465Z   21:     0x7ff813c9d0c0 - rustc_driver[6d4b93d6bcfc3289]::args::arg_expand_all
2023-02-15T21:47:10.7455753Z 
2023-02-15T21:47:10.7460527Z   22:     0x7ff813c8d176 - <unknown>
2023-02-15T21:47:10.7460958Z   23:     0x7ff813c82b7d - <unknown>
2023-02-15T21:47:10.7461524Z   24:     0x7ff83801fe9c - std::sys::windows::thread::Thread::new::h92ec69467f05d131
2023-02-15T21:47:10.7461803Z 
2023-02-15T21:47:10.7462101Z   25:     0x7ff851e24de0 - BaseThreadInitThunk
2023-02-15T21:47:10.7462345Z 
2023-02-15T21:47:10.7462844Z   26:     0x7ff852ede40b - RtlUserThreadStart
2023-02-15T21:47:10.7468807Z 
2023-02-15T21:47:10.7468892Z 
2023-02-15T21:47:10.7469136Z error: internal compiler error: unexpected panic
2023-02-15T21:47:10.7469405Z 
2023-02-15T21:47:10.7469460Z 
2023-02-15T21:47:10.7469692Z note: the compiler unexpectedly panicked. this is a bug.
2023-02-15T21:47:10.7469959Z 
2023-02-15T21:47:10.7470499Z note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
2023-02-15T21:47:10.7470873Z 
2023-02-15T21:47:10.7471212Z note: rustc 1.67.1 (d5a82bbd2 2023-02-07) running on x86_64-pc-windows-msvc
2023-02-15T21:47:10.7474686Z 
2023-02-15T21:47:10.7475189Z note: compiler flags: --crate-type lib -C opt-level=3 -C embed-bitcode=no -C linker=lld-link -C linker-flavor=lld-link
2023-02-15T21:47:10.7475550Z 
2023-02-15T21:47:10.7475781Z note: some of the compiler flags provided by cargo are hidden
2023-02-15T21:47:10.7476045Z 
2023-02-15T21:47:10.7476227Z query stack during panic:
2023-02-15T21:47:10.7476446Z 
2023-02-15T21:47:10.7476606Z end of query stack
2023-02-15T21:47:10.7476772Z 
2023-02-15T21:47:10.8093668Z error: could not compile `lindera-ko-dic`
2023-02-15T21:47:10.8094446Z 
2023-02-15T21:47:10.8100818Z warning: build failed, waiting for other jobs to finish...
@GregoryConrad GregoryConrad added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 15, 2023
@TaKO8Ki TaKO8Ki added the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Feb 20, 2023
@cjgillot
Copy link
Contributor

cjgillot commented Mar 5, 2023

I'm not sure the out of space error is unrelated. Did you experience the same ICE outside of that CI run?
We already had instances of assertions triggering in metadata or incremental caches because of truncated files.

@GregoryConrad
Copy link
Author

GregoryConrad commented Mar 5, 2023

@cjgillot as I interpreted the logs, the out of memory error happened later in the CI job, but maybe it was only reported later. I suppose it is entirely possible that it is related, now that I think of it, and just wasn't reported by the rust compiler.

Did you experience the same ICE outside of that CI run?

I actually only experienced that error for several days in a row, and haven't seen it again for some weeks now. I never tried compiling locally since I don't have a Windows computer. Considering I have noticed other issues related to the available storage space on CI sometimes fluctuating, that could be why.

We already had instances of assertions triggering in metadata or incremental caches because of truncated files.

Perhaps this is caused by out of storage space then.

@CeNiEi
Copy link

CeNiEi commented May 31, 2023

Hello. I just got the same panic, while building a project. Mine is most probably due to space shortage, as the compiler makes it very explicit.

error: failed to write /var/folders/qr/w92rrf2x5pn_ny17y7kljt300000gn/T/rustcBr1XIs/lib.rmeta: No space left on device (os error 28)

error: could not compile `ndarray` due to previous error
warning: build failed, waiting for other jobs to finish...
thread 'rustc' panicked at 'assertion failed: pos.get() <= self.position()', compiler/rustc_metadata/src/rmeta/encoder.rs:435:9
LLVM ERROR: IO failure on output stream: No space left on device
error: could not compile `rayon-core`
LLVM ERROR: IO failure on output stream: No space left on device
error: failed to run custom build command for `ring v0.16.20`

Bactrace:

thread 'rustc' panicked at 'assertion failed: pos.get() <= self.position()', compiler/rustc_metadata/src/rmeta/encoder.rs:435:9
stack backtrace:
   0:        0x102850468 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h105074e3d85f800b
   1:        0x1028a3388 - core::fmt::write::h34766cf8fff7af1e
   2:        0x1028461e0 - std::io::Write::write_fmt::hd64c4cf6e7adea59
   3:        0x10285027c - std::sys_common::backtrace::print::h2a6828a537036cf9
   4:        0x102852d04 - std::panicking::default_hook::{{closure}}::h4e82ce6ccef941b2
   5:        0x102852a5c - std::panicking::default_hook::h29f62f8795c5cb00
   6:        0x10a92574c - rustc_driver_impl[4da67b3d18d5a97a]::DEFAULT_HOOK::{closure#0}::{closure#0}
   7:        0x1028533fc - std::panicking::rust_panic_with_hook::h19862cbd0fbda7ba
   8:        0x1028531b4 - std::panicking::begin_panic_handler::{{closure}}::h3f3626935e1669fe
   9:        0x102850888 - std::sys_common::backtrace::__rust_end_short_backtrace::h5054ef52bd507d0a
  10:        0x102852f50 - _rust_begin_unwind
  11:        0x1028ce608 - core::panicking::panic_fmt::h7e47e10600a90221
  12:        0x1028ce678 - core::panicking::panic::h2f410888133670e4
  13:        0x10de92184 - <rustc_span[9bec1e8d1a20d9df]::hygiene::HygieneEncodeContext>::encode::<(&mut rustc_metadata[9b8298dffc3c5cc]::rmeta:
:encoder::EncodeContext, &mut rustc_metadata[9b8298dffc3c5cc]::rmeta::table::TableBuilder<u32, core[446d629763b4aa]::option::Option<rustc_metada
ta[9b8298dffc3c5cc]::rmeta::LazyValue<rustc_span[9bec1e8d1a20d9df]::hygiene::SyntaxContextData>>>, &mut rustc_metadata[9b8298dffc3c5cc]::rmeta::
table::TableBuilder<rustc_span[9bec1e8d1a20d9df]::hygiene::ExpnIndex, core[446d629763b4aa]::option::Option<rustc_metadata[9b8298dffc3c5cc]::rmet
a::LazyValue<rustc_span[9bec1e8d1a20d9df]::hygiene::ExpnData>>>, &mut rustc_metadata[9b8298dffc3c5cc]::rmeta::table::TableBuilder<rustc_span[9be
c1e8d1a20d9df]::hygiene::ExpnIndex, core[446d629763b4aa]::option::Option<rustc_metadata[9b8298dffc3c5cc]::rmeta::LazyValue<rustc_span[9bec1e8d1a
20d9df]::hygiene::ExpnHash>>>), <rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::EncodeContext>::encode_hygiene::{closure#0}, <rustc_metadata[9
b8298dffc3c5cc]::rmeta::encoder::EncodeContext>::encode_hygiene::{closure#1}>
  14:        0x10df0ec30 - <rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::EncodeContext>::encode_crate_root
  15:        0x10df1a070 - rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::encode_metadata_impl
  16:        0x10de80364 - rustc_data_structures[a49ab2b4a1c20397]::sync::join::<rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::encode_metadat
a::{closure#0}, rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::encode_metadata::{closure#1}, (), ()>
  17:        0x10df198d8 - rustc_metadata[9b8298dffc3c5cc]::rmeta::encoder::encode_metadata
  18:        0x10de898e0 - rustc_metadata[9b8298dffc3c5cc]::fs::encode_and_write_metadata
  19:        0x10a9c2918 - rustc_interface[812bb1defcb7896d]::passes::start_codegen
  20:        0x10a9c46c4 - <rustc_middle[a12d4e31e9c7c7b2]::ty::context::GlobalCtxt>::enter::<<rustc_interface[812bb1defcb7896d]::queries::Queri
es>::ongoing_codegen::{closure#0}::{closure#0}, core[446d629763b4aa]::result::Result<alloc[f251364f33aae5a4]::boxed::Box<dyn core[446d629763b4aa
]::any::Any>, rustc_span[9bec1e8d1a20d9df]::ErrorGuaranteed>>
  21:        0x10a9e9a58 - <rustc_interface[812bb1defcb7896d]::queries::Queries>::ongoing_codegen
  22:        0x10a971b70 - rustc_span[9bec1e8d1a20d9df]::with_source_map::<core[446d629763b4aa]::result::Result<(), rustc_span[9bec1e8d1a20d9df]
::ErrorGuaranteed>, rustc_interface[812bb1defcb7896d]::interface::run_compiler<core[446d629763b4aa]::result::Result<(), rustc_span[9bec1e8d1a20d
9df]::ErrorGuaranteed>, rustc_driver_impl[4da67b3d18d5a97a]::run_compiler::{closure#1}>::{closure#0}::{closure#0}>
  23:        0x10a933f84 - std[57ecbc5b2c1792f]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[812bb1defcb7896d]::util::
run_in_thread_pool_with_globals<rustc_interface[812bb1defcb7896d]::interface::run_compiler<core[446d629763b4aa]::result::Result<(), rustc_span[9
bec1e8d1a20d9df]::ErrorGuaranteed>, rustc_driver_impl[4da67b3d18d5a97a]::run_compiler::{closure#1}>::{closure#0}, core[446d629763b4aa]::result::
Result<(), rustc_span[9bec1e8d1a20d9df]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[446d629763b4aa]::result::Result<(), rustc_span[9bec1e
8d1a20d9df]::ErrorGuaranteed>>
  24:        0x10a935d20 - <<std[57ecbc5b2c1792f]::thread::Builder>::spawn_unchecked_<rustc_interface[812bb1defcb7896d]::util::run_in_thread_poo
l_with_globals<rustc_interface[812bb1defcb7896d]::interface::run_compiler<core[446d629763b4aa]::result::Result<(), rustc_span[9bec1e8d1a20d9df]:
:ErrorGuaranteed>, rustc_driver_impl[4da67b3d18d5a97a]::run_compiler::{closure#1}>::{closure#0}, core[446d629763b4aa]::result::Result<(), rustc_
span[9bec1e8d1a20d9df]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[446d629763b4aa]::result::Result<(), rustc_span[9bec1e8d1a20d9df]::Erro
rGuaranteed>>::{closure#1} as core[446d629763b4aa]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  25:        0x10285ba24 - std::sys::unix::thread::Thread::new::thread_start::h5db5d66cd91ffb33
  26:        0x1a352ffa8 - __pthread_joiner_wake

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.69.0 (84c898d65 2023-04-16) running on aarch64-apple-darwin

note: compiler flags: --crate-type lib -C embed-bitcode=no -C split-debuginfo=unpacked -C debug-assertions=off

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

@dullbananas
Copy link

@Manishearth
Copy link
Member

Just hit this here: https://github.com/unicode-org/icu4x/actions/runs/5465410649/jobs/9948816731 . Not yet clear why.

@saethlin
Copy link
Member

saethlin commented Sep 4, 2023

Duplicate of #115298

@saethlin saethlin marked this as a duplicate of #115298 Sep 4, 2023
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 11, 2023
…=<try>

Simplify/Optimize FileEncoder

FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing.

So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written.

The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are now small.

Fixes rust-lang#115298
Fixes rust-lang#114671
Fixes rust-lang#108100
Fixes rust-lang#106787
@bors bors closed this as completed in 3223b0b Sep 20, 2023
github-actions bot pushed a commit to rust-lang/miri that referenced this issue Sep 21, 2023
…pkin

Simplify/Optimize FileEncoder

FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing.

So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written.

The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small.

Fixes rust-lang/rust#115298
Fixes rust-lang/rust#114671
Fixes rust-lang/rust#114045
Fixes rust-lang/rust#108100
Fixes rust-lang/rust#106787
lnicola pushed a commit to lnicola/rust-analyzer that referenced this issue Apr 7, 2024
…pkin

Simplify/Optimize FileEncoder

FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing.

So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written.

The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small.

Fixes rust-lang/rust#115298
Fixes rust-lang/rust#114671
Fixes rust-lang/rust#114045
Fixes rust-lang/rust#108100
Fixes rust-lang/rust#106787
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this issue Apr 27, 2024
…pkin

Simplify/Optimize FileEncoder

FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing.

So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written.

The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small.

Fixes rust-lang/rust#115298
Fixes rust-lang/rust#114671
Fixes rust-lang/rust#114045
Fixes rust-lang/rust#108100
Fixes rust-lang/rust#106787
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants