Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid span created when using format_args with modified literal span and multibyte chars #106191

Closed
Noratrieb opened this issue Dec 27, 2022 · 0 comments · Fixed by #106195 or #106505
Closed

Invalid span created when using format_args with modified literal span and multibyte chars #106191

Noratrieb opened this issue Dec 27, 2022 · 0 comments · Fixed by #106195 or #106505
Assignees
@Noratrieb
Labels
C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@Noratrieb
Copy link
Member

Noratrieb commented Dec 27, 2022 •
edited
Loading

Code

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c9a64a485c010845e037c3f1fd2248ff

fn main() {
    format_args!(r#concat!("¡        {"));
}

Or with a proc-macro:
boom.rs

extern crate proc_macro;

use proc_macro::{Delimiter, Group, Ident, Literal, Punct, Spacing, Span, TokenStream, TokenTree};
use std::iter::FromIterator;
#[proc_macro]
pub fn fmt(input: TokenStream) -> TokenStream {
    let mut s = Literal::string("{");
    s.set_span(input.into_iter().next().unwrap().span());
    TokenStream::from_iter([
        TokenTree::from(Ident::new("format", Span::call_site())),
        TokenTree::from(Punct::new('!', Spacing::Alone)),
        TokenTree::from(Group::new(Delimiter::Parenthesis, TokenTree::from(s).into())),
    ])
}

broken.rs

fn main() {
    boom::fmt!("¡");
}

$ rustc boom.rs --crate-type proc-macro
$ rustc broken.rs --extern boom=libboom.so

Meta

rustc --version --verbose:

<version>

Error output

thread '<unnamed>' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', compiler/rustc_span/src/lib.rs:1700:17
Backtrace 

thread '<unnamed>' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', compiler/rustc_span/src/lib.rs:1700:17
stack backtrace:
   0:     0x7f171d7f19a0 - std::backtrace_rs::backtrace::libunwind::trace::he615646ea344481f
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x7f171d7f19a0 - std::backtrace_rs::backtrace::trace_unsynchronized::h6ea8eaac68705b9c
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f171d7f19a0 - std::sys_common::backtrace::_print_fmt::h7ac486a935ce0bf7
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x7f171d7f19a0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1b5a095d3db2e28f
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x7f171d84d98e - core::fmt::write::h445545b92224a1cd
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
   5:     0x7f171d7e1b15 - std::io::Write::write_fmt::h55a43474c6520b00
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
   6:     0x7f171d7f1765 - std::sys_common::backtrace::_print::h65d20526fdb736b0
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
   7:     0x7f171d7f1765 - std::sys_common::backtrace::print::h6555fbe12a1cc41b
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
   8:     0x7f171d7f456f - std::panicking::default_hook::{{closure}}::hbdf58083140e7ac6
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
   9:     0x7f171d7f42aa - std::panicking::default_hook::haef8271c56b74d85
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
  10:     0x7f171d7f4d78 - std::panicking::rust_panic_with_hook::hfd45b6b6c12d9fa5
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
  11:     0x7f171d7f4ad1 - std::panicking::begin_panic_handler::{{closure}}::hf591e8609a75bd4b
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:577:13
  12:     0x7f171d7f1e4c - std::sys_common::backtrace::__rust_end_short_backtrace::h81899558795e4ff7
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
  13:     0x7f171d7f4832 - rust_begin_unwind
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
  14:     0x7f171d84a373 - core::panicking::panic_fmt::h4235fa9b4675b332
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
  15:     0x7f171d84a44d - core::panicking::panic::h9ced3cf2f605ba6a
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:115:5
  16:     0x7f171f12d6bd - <rustc_span[14998722174c1bca]::source_map::SourceMap>::lookup_char_pos
  17:     0x7f1720017822 - <rustc_errors[21897ed46328f955]::emitter::EmitterWriter>::get_multispan_max_line_num
  18:     0x7f172000f3e8 - <rustc_errors[21897ed46328f955]::emitter::EmitterWriter as rustc_errors[21897ed46328f955]::emitter::Emitter>::emit_diagnostic
  19:     0x7f172000d0e1 - <rustc_errors[21897ed46328f955]::json::Diagnostic>::from_errors_diagnostic
  20:     0x7f172000c68c - <rustc_errors[21897ed46328f955]::json::JsonEmitter as rustc_errors[21897ed46328f955]::emitter::Emitter>::emit_diagnostic
  21:     0x7f171fa467da - <rustc_errors[21897ed46328f955]::HandlerInner>::emit_diagnostic
  22:     0x7f171fa46191 - <rustc_errors[21897ed46328f955]::Handler>::emit_diagnostic
  23:     0x7f171fa01b6d - <rustc_errors[21897ed46328f955]::ErrorGuaranteed as rustc_errors[21897ed46328f955]::diagnostic_builder::EmissionGuarantee>::diagnostic_builder_emit_producing_guarantee
  24:     0x7f171fcb4353 - rustc_builtin_macros[e234c542650290a3]::format::make_format_args
  25:     0x7f171fca92f9 - rustc_builtin_macros[e234c542650290a3]::format::expand_format_args_impl
  26:     0x7f171f3e6f44 - <rustc_expand[e35988b3db587ce3]::expand::MacroExpander>::fully_expand_fragment
  27:     0x7f171fd5101d - <rustc_expand[e35988b3db587ce3]::expand::MacroExpander>::expand_crate
  28:     0x7f171fd4fd64 - <rustc_session[87e5e219eb43521c]::session::Session>::time::<core[b97a30f8df81432d]::result::Result<rustc_ast[ef781098e3b1703c]::ast::Crate, rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_interface[ee1a3f92e887e004]::passes::configure_and_expand::{closure#1}>
  29:     0x7f171fd15d56 - rustc_interface[ee1a3f92e887e004]::passes::configure_and_expand
  30:     0x7f171fd0e524 - <rustc_interface[ee1a3f92e887e004]::queries::Queries>::expansion
  31:     0x7f171fd0be6c - <rustc_interface[ee1a3f92e887e004]::interface::Compiler>::enter::<rustc_driver[9c4183344b2d0066]::run_compiler::{closure#1}::{closure#2}, core[b97a30f8df81432d]::result::Result<core[b97a30f8df81432d]::option::Option<rustc_interface[ee1a3f92e887e004]::queries::Linker>, rustc_errors[21897ed46328f955]::ErrorGuaranteed>>
  32:     0x7f171fd03ac2 - rustc_span[14998722174c1bca]::with_source_map::<core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_interface[ee1a3f92e887e004]::interface::run_compiler<core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_driver[9c4183344b2d0066]::run_compiler::{closure#1}>::{closure#0}::{closure#1}>
  33:     0x7f171fd035b9 - <scoped_tls[23afcff80b89ba49]::ScopedKey<rustc_span[14998722174c1bca]::SessionGlobals>>::set::<rustc_interface[ee1a3f92e887e004]::interface::run_compiler<core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_driver[9c4183344b2d0066]::run_compiler::{closure#1}>::{closure#0}, core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>>
  34:     0x7f171fd02bc8 - std[bbad73ae434e23e5]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[ee1a3f92e887e004]::util::run_in_thread_pool_with_globals<rustc_interface[ee1a3f92e887e004]::interface::run_compiler<core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_driver[9c4183344b2d0066]::run_compiler::{closure#1}>::{closure#0}, core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>>
  35:     0x7f171fd028ec - <<std[bbad73ae434e23e5]::thread::Builder>::spawn_unchecked_<rustc_interface[ee1a3f92e887e004]::util::run_in_thread_pool_with_globals<rustc_interface[ee1a3f92e887e004]::interface::run_compiler<core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>, rustc_driver[9c4183344b2d0066]::run_compiler::{closure#1}>::{closure#0}, core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[b97a30f8df81432d]::result::Result<(), rustc_errors[21897ed46328f955]::ErrorGuaranteed>>::{closure#1} as core[b97a30f8df81432d]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  36:     0x7f17216ac6a3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h4273f95ec44459b3
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
  37:     0x7f17216ac6a3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h70f28fa4ddc269e5
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
  38:     0x7f17216ac6a3 - std::sys::unix::thread::Thread::new::thread_start::h85a9c16b988e2bd0
                               at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17
  39:     0x7f171d6d0609 - start_thread
  40:     0x7f171d5f3133 - clone
  41:                0x0 - <unknown>
error: could not compile `playground`

After some local testing I added some code to validate that spans were created with their lo and hi at char boundaries and got this:

thread 'rustc' panicked at 'start of span not on char boundary', compiler/rustc_span/src/span_encoding.rs:98:25
validate backtrace 
thread 'rustc' panicked at 'start of span not on char boundary', compiler/rustc_span/src/span_encoding.rs:98:25
stack backtrace:
   0: rust_begin_unwind
             at /home/nilsh/projects/rust/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /home/nilsh/projects/rust/library/core/src/panicking.rs:64:14
   2: <rustc_span::span_encoding::Span>::new::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_span/src/span_encoding.rs:98:25
   3: <scoped_tls::ScopedKey<rustc_span::SessionGlobals>>::with::<<rustc_span::span_encoding::Span>::new::{closure#0}, ()>
             at /home/nilsh/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:171:13
   4: rustc_span::with_session_globals::<(), <rustc_span::span_encoding::Span>::new::{closure#0}>
             at /home/nilsh/projects/rust/compiler/rustc_span/src/lib.rs:150:5
   5: <rustc_span::span_encoding::Span>::new
             at /home/nilsh/projects/rust/compiler/rustc_span/src/span_encoding.rs:93:13
   6: <rustc_span::span_encoding::Span>::from_inner
             at /home/nilsh/projects/rust/compiler/rustc_span/src/lib.rs:882:9
   7: rustc_builtin_macros::format::make_format_args
             at /home/nilsh/projects/rust/compiler/rustc_builtin_macros/src/format.rs:221:13
   8: rustc_builtin_macros::format::expand_format_args_impl
             at /home/nilsh/projects/rust/compiler/rustc_builtin_macros/src/format.rs:858:38
   9: <rustc_expand::expand::MacroExpander>::expand_invoc
             at /home/nilsh/projects/rust/compiler/rustc_expand/src/expand.rs:662:38
  10: <rustc_expand::expand::MacroExpander>::fully_expand_fragment
             at /home/nilsh/projects/rust/compiler/rustc_expand/src/expand.rs:475:62
  11: <rustc_expand::expand::MacroExpander>::expand_crate
             at /home/nilsh/projects/rust/compiler/rustc_expand/src/expand.rs:402:21
  12: rustc_interface::passes::configure_and_expand::{closure#1}::{closure#1}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/passes.rs:335:50
  13: <rustc_data_structures::profiling::VerboseTimingGuard>::run::<rustc_ast::ast::Crate, rustc_interface::passes::configure_and_expand::{closure#1}::{closure#1}>
             at /home/nilsh/projects/rust/compiler/rustc_data_structures/src/profiling.rs:723:9
  14: <rustc_session::session::Session>::time::<rustc_ast::ast::Crate, rustc_interface::passes::configure_and_expand::{closure#1}::{closure#1}>
             at /home/nilsh/projects/rust/compiler/rustc_session/src/utils.rs:10:9
  15: rustc_interface::passes::configure_and_expand::{closure#1}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/passes.rs:335:21
  16: <rustc_data_structures::profiling::VerboseTimingGuard>::run::<core::result::Result<rustc_ast::ast::Crate, rustc_errors::ErrorGuaranteed>, rustc_interface::passes::configure_and_expand::{closure#1}>
             at /home/nilsh/projects/rust/compiler/rustc_data_structures/src/profiling.rs:723:9
  17: <rustc_session::session::Session>::time::<core::result::Result<rustc_ast::ast::Crate, rustc_errors::ErrorGuaranteed>, rustc_interface::passes::configure_and_expand::{closure#1}>
             at /home/nilsh/projects/rust/compiler/rustc_session/src/utils.rs:10:9
  18: rustc_interface::passes::configure_and_expand
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/passes.rs:287:13
  19: <rustc_interface::queries::Queries>::expansion::{closure#0}::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/queries.rs:180:17
  20: <rustc_interface::passes::boxed_resolver::BoxedResolver>::access::<<rustc_interface::queries::Queries>::expansion::{closure#0}::{closure#0}, core::result::Result<rustc_ast::ast::Crate, rustc_errors::ErrorGuaranteed>>
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/passes.rs:135:13
  21: <rustc_interface::queries::Queries>::expansion::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/queries.rs:179:25
  22: <core::option::Option<core::result::Result<(alloc::rc::Rc<rustc_ast::ast::Crate>, alloc::rc::Rc<core::cell::RefCell<rustc_interface::passes::boxed_resolver::BoxedResolver>>, alloc::rc::Rc<rustc_lint::context::LintStore>), rustc_errors::ErrorGuaranteed>>>::get_or_insert_with::<<rustc_interface::queries::Queries>::expansion::{closure#0}>
             at /home/nilsh/projects/rust/library/core/src/option.rs:1590:49
  23: <rustc_interface::queries::Query<(alloc::rc::Rc<rustc_ast::ast::Crate>, alloc::rc::Rc<core::cell::RefCell<rustc_interface::passes::boxed_resolver::BoxedResolver>>, alloc::rc::Rc<rustc_lint::context::LintStore>)>>::compute::<<rustc_interface::queries::Queries>::expansion::{closure#0}>
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/queries.rs:38:9
  24: <rustc_interface::queries::Queries>::expansion
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/queries.rs:168:9
  25: rustc_driver::run_compiler::{closure#1}::{closure#2}
             at /home/nilsh/projects/rust/compiler/rustc_driver/src/lib.rs:355:13
  26: <rustc_interface::interface::Compiler>::enter::<rustc_driver::run_compiler::{closure#1}::{closure#2}, core::result::Result<core::option::Option<rustc_interface::queries::Linker>, rustc_errors::ErrorGuaranteed>>
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/queries.rs:380:19
  27: rustc_driver::run_compiler::{closure#1}
             at /home/nilsh/projects/rust/compiler/rustc_driver/src/lib.rs:306:22
  28: rustc_interface::interface::run_compiler::<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/interface.rs:327:21
  29: rustc_span::with_source_map::<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_interface::interface::run_compiler<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}::{closure#0}>
             at /home/nilsh/projects/rust/compiler/rustc_span/src/lib.rs:1023:5
  30: rustc_interface::interface::run_compiler::<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/interface.rs:321:13
  31: <scoped_tls::ScopedKey<rustc_span::SessionGlobals>>::set::<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}, core::result::Result<(), rustc_errors::ErrorGuaranteed>>
             at /home/nilsh/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
  32: rustc_span::create_session_globals_then::<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_interface::interface::run_compiler<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}>
             at /home/nilsh/projects/rust/compiler/rustc_span/src/lib.rs:111:5
  33: rustc_interface::util::run_in_thread_pool_with_globals::<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_errors::ErrorGuaranteed>, rustc_driver::run_compiler::{closure#1}>::{closure#0}, core::result::Result<(), rustc_errors::ErrorGuaranteed>>::{closure#0}::{closure#0}
             at /home/nilsh/projects/rust/compiler/rustc_interface/src/util.rs:145:38
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.68.0-dev running on x86_64-unknown-linux-gnu

query stack during panic:
end of query stack
@Noratrieb Noratrieb added I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. C-bug Category: This is a bug. labels Dec 27, 2022
@Noratrieb Noratrieb self-assigned this Dec 27, 2022
This was referenced Dec 27, 2022
Open
Closed
Closed
Merged
@bors bors closed this as completed in 0c0b403 Dec 29, 2022
@VorfeedCanal VorfeedCanal mentioned this issue Jan 3, 2023
Closed
@Noratrieb Noratrieb mentioned this issue Jan 5, 2023
Merged
@Noratrieb Noratrieb mentioned this issue Jan 18, 2023
Merged
bors added a commit to rust-lang-ci/rust that referenced this issue Jan 18, 2023
@bors
Auto merge of rust-lang#107041 - Nilstrieb:back-to-being-clueless-whe…
8b11574 
…ther-it-really-is-a-literal, r=compiler-errors

Revert "Improve heuristics whether `format_args` string is a source literal"

This reverts commit e6c02aa (from rust-lang#106195).

Keeps the code improvements from the PR and the test (as a known-bug).

Works around rust-lang#106408 while a proper fix is discussed more thoroughly in rust-lang#106505, as proposed by `@tmandry.`

Reopens rust-lang#106191

r? compiler-errors
@Noratrieb Noratrieb mentioned this issue Jan 23, 2023
Closed
@bors bors closed this as completed in 2e7034e Mar 14, 2023
@compiler-errors compiler-errors mentioned this issue May 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Noratrieb Noratrieb

Labels
C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
2 participants
@compiler-errors @Noratrieb