rustc produces invalid IR with -O -g #28947
Comments
jdm
added
the
A-debuginfo
|
I am able to reproduce this on my native linux system. I have not yet tried in a VM yet. Its a pretty bad show-stopper for Linux work (at least if one wants to use |
|
JFYI, I started looking into this. |
|
Update: Now I have reproduced it on OS X. I am not sure where my earlier attempt to reproduce went wrong. Anyway, this heightens my desire to get this fixed considerably... cc @nrc |
|
@dotdash have you made much progress, e.g. have you made a reduced test case, or have insight into what's wrong? I am wondering in part because I am willing to spend time looking at this, but I do not want to waste or duplicate effort. |
|
Not much progress yet. For some reason, the 2015-10-13 12:01 GMT+02:00 Felix S Klock II notifications@github.com:
|
|
Something weird is going on. |
|
Reproduced on a hardened gentoo linux (here's all you need to know about the system: http://dpaste.com/1DQ0EFB.txt ) as follows: rust commit tested: 294ef5b In trying to follow sanxiyn's comment above, changing the above RUSTFLAGS to: No idea where to stick that EDIT1: It actually went past it with EDIT2: I'm still getting the error that maybe I meant to use Does rustc maybe have a whitelisted llvm-args that's allowing to passthru ? Or how does it fail to use my llvm(3.7.0) which clearly does have that but maybe I got the EDIT4: ok, the reason EDIT5: tested with git llvm (3.8.0) and the error still happens: latest rust commit tested 1ad1b7d |
pnkfelix
added
I-nominated
T-compiler
|
triage: P-medium |
|
I would really like to see a narrowed test case for this bug. I've tried to start working my way down from libsyntax but keep getting interrupted (or flummoxed -- my usual tools for this like |
|
I tried stripping most "unrelated" functions from the generated IR for libsyntax, but even that made the bug disappear at some point. |
|
(at some point we may have to ask ourselves if we are willing to revert the lowering PR, if only to allow us to finally set up gating bors via the buildbot with |
|
tldr; I don't know how helpful this is but, using In other words, this causes the issue Last tested commit beeaea4, with $ ./configure --prefix=/home/zazdxscf/build/1nonpkgs/rust/usr/local --disable-rpath --enable-manage-submodules --disable-clang --enable-ccache --enable-dist-host-only --disable-valgrind --disable-helgrind --disable-valgrind-rpass --python=/usr/bin/python2 --enable-optimize --enable-optimize-cxx --enable-optimize-llvm --enable-debug --disable-debuginfo --enable-debug-assertions --enable-debuginfo-tests --enable-llvm-assertions --enable-debug-jemalloc --disable-llvm-version-check --disable-local-rust --llvm-root=/usr
...
$ make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 'RUSTFLAGS=--verbose -Z verbose -Z print-link-args -Z debug-llvm '
...
time: 20.675; rss: 703MB translation
time: 7.623; rss: 402MB llvm function passes
time: 154.266; rss: 482MB llvm module passes
time: 61.107; rss: 516MB codegen passes
time: 0.007; rss: 482MB codegen passes
//so it passed with --disable-debuginfo
//next: changed --disable-debuginfo into --enable-debuginfo to make it fail
...
$ make clean
...
$ ./configure --prefix=/home/zazdxscf/build/1nonpkgs/rust/usr/local --disable-rpath --enable-manage-submodules --disable-clang --enable-ccache --enable-dist-host-only --disable-valgrind --disable-helgrind --disable-valgrind-rpass --python=/usr/bin/python2 --enable-optimize --enable-optimize-cxx --enable-optimize-llvm --enable-debug --enable-debuginfo --enable-debug-assertions --enable-debuginfo-tests --enable-llvm-assertions --enable-debug-jemalloc --disable-llvm-version-check --disable-local-rust --llvm-root=/usr
...
$ time make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 'RUSTFLAGS=--verbose -Z verbose -Z print-link-args -Z debug-llvm '
...
rustc: x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax
time: 0.391; rss: 58MB parsing
time: 0.127; rss: 59MB configuration 1
time: 0.000; rss: 59MB recursion limit
time: 0.009; rss: 59MB gated macro checking
time: 0.000; rss: 59MB crate injection
time: 0.019; rss: 61MB macro loading
time: 0.000; rss: 61MB plugin loading
time: 0.000; rss: 61MB plugin registration
time: 1.831; rss: 94MB expansion
time: 0.036; rss: 94MB complete gated feature checking 1
time: 0.322; rss: 94MB configuration 2
time: 0.166; rss: 94MB maybe building test harness
time: 0.154; rss: 94MB prelude injection
time: 0.021; rss: 94MB checking that all macro invocations are gone
time: 0.037; rss: 94MB complete gated feature checking 2
time: 0.196; rss: 103MB assigning node ids and indexing ast
time: 0.024; rss: 103MB external crate/lib resolution
time: 0.044; rss: 103MB language item collection
time: 0.696; rss: 148MB resolution
time: 0.033; rss: 143MB lifetime resolution
time: 0.000; rss: 143MB looking for entry point
time: 0.021; rss: 143MB looking for plugin registrar
time: 0.263; rss: 158MB region resolution
time: 0.022; rss: 158MB loop checking
time: 0.024; rss: 158MB static item recursion checking
time: 0.151; rss: 161MB type collecting
time: 0.044; rss: 161MB variance inference
time: 0.455; rss: 202MB coherence checking
time: 23.707; rss: 308MB type checking
time: 4.946; rss: 319MB const checking
time: 0.198; rss: 320MB privacy checking
time: 0.005; rss: 320MB stability index
time: 0.067; rss: 320MB intrinsic checking
time: 0.051; rss: 320MB effect checking
time: 1.123; rss: 320MB match checking
time: 0.176; rss: 327MB liveness checking
time: 9.925; rss: 327MB borrow checking
time: 5.309; rss: 327MB rvalue checking
time: 0.064; rss: 327MB reachability checking
time: 0.152; rss: 327MB death checking
time: 0.178; rss: 327MB stability checking
time: 0.000; rss: 327MB unused lib feature checking
time: 0.801; rss: 327MB lint checking
time: 0.000; rss: 327MB resolving dependency formats
time: 25.456; rss: 928MB translation
time: 9.374; rss: 631MB llvm function passes
time: 201.226; rss: 740MB llvm module passes
!dbg attachment points at wrong subprogram for function
!156127 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156111, file: !156111, line: 38, type: !42675, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69395E, templateParams: !1032, variables: !156128)
i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69395E
tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.857, i64 0, metadata !156506, metadata !189686), !dbg !765837
!765837 = !DILocation(line: 113, scope: !156500, inlinedAt: !765838)
!765839 = distinct !DILexicalBlock(scope: !765840, file: !156111, line: 129, column: 56)
!159769 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156111, file: !156111, line: 129, type: !159770, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1032, variables: !159772)
LLVM ERROR: Broken function found, compilation aborted!
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax] Error 1
real 9m11.276s
user 8m21.940s
sys 0m40.597sSo, replacing If it matters, Registered Targets: Now that I think about it, it's actually in the issue's title $ time x86_64-unknown-linux-gnu/stage0/bin/rustc -g --emit llvm-bc src/libsyntax/lib.rs
real 1m23.926s
user 1m20.543s
sys 0m3.040s
$ time opt -O2 -disable-output syntax.bc
real 4m24.718s
user 4m23.393s
sys 0m0.653s
//oh wait, what? this worked this time?! WHAT? :) without -verify-each and with --enable-debuginfo that was left from above.
//I don't get it; is it my 3.8.0 llc ? I don't remember if I used this llc version to test this
...Going to try using llvm that comes with rust next.
EDIT1: Actually I was wrong, it DOES USE CCACHE! (I was just looking in the wrong ccache folder! with ccache -s) |
|
Note that rustc generates different IR even pre-optimization when using |
|
@sanxiyn |
|
tl;dr it now fails with Very nice, thanks dotdash for the help. While I basically have no idea what I'm doing, I can now reproduce (again) sanxiyn's comment by adding $ time LD_LIBRARY_PATH='/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/lib' x86_64-unknown-linux-gnu/stage0/bin/rustc -O -g --emit llvm-bc src/libsyntax/lib.rs
real 4m45.443s
user 4m41.333s
sys 0m3.127s
$ time opt -O2 -disable-output syntax.bc
!dbg attachment points at wrong subprogram for function
!154953 = distinct !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 38, type: !42660, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E, templateParams: !1032, variables: !154954)
i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E
tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.763, i64 0, metadata !155332, metadata !188522), !dbg !882290
!882290 = !DILocation(line: 113, scope: !155326, inlinedAt: !882291)
!882292 = distinct !DILexicalBlock(scope: !882293, file: !154937, line: 129, column: 56)
!158595 = distinct !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 129, type: !158596, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1032, variables: !158598)
opt: syntax.bc: error: input module is broken!
real 0m12.978s
user 0m12.263s
sys 0m0.657s
$ time opt -O2 -verify-each -disable-output syntax.bc
!dbg attachment points at wrong subprogram for function
!154953 = distinct !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 38, type: !42660, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E, templateParams: !1032, variables: !154954)
i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E
tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.763, i64 0, metadata !155332, metadata !188522), !dbg !882290
!882290 = !DILocation(line: 113, scope: !155326, inlinedAt: !882291)
!882292 = distinct !DILexicalBlock(scope: !882293, file: !154937, line: 129, column: 56)
!158595 = distinct !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 129, type: !158596, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1032, variables: !158598)
opt: syntax.bc: error: input module is broken!
real 0m12.416s
user 0m11.690s
sys 0m0.673s
//yup, fails with -verify-each too (also tested by adding all other -verify* options, still no change)
//even with the llvm that comes with rust, still fails with -verify-each unless I'm doing it wrong and it uses stuff from the svn llvm that I've installed
$ time LD_LIBRARY_PATH='/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/llvm/Release+Asserts/lib' x86_64-unknown-linux-gnu/llvm/Release+Asserts/bin/opt -O2 -verify-each -disable-output syntax.bc
!dbg attachment points at wrong subprogram for function
!154953 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 38, type: !42660, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E, templateParams: !1032, variables: !154954)
i64 (%closure.1235*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.68893E
tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.763, i64 0, metadata !155332, metadata !188522), !dbg !882290
!882290 = !DILocation(line: 113, scope: !155326, inlinedAt: !882291)
!882292 = distinct !DILexicalBlock(scope: !882293, file: !154937, line: 129, column: 56)
!158595 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !154937, file: !154937, line: 129, type: !158596, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1032, variables: !158598)
x86_64-unknown-linux-gnu/llvm/Release+Asserts/bin/opt: syntax.bc: error: input module is broken!
real 0m14.177s
user 0m13.450s
sys 0m0.630s
$ opt --version
LLVM (http://llvm.org/):
LLVM version 3.8.0svn
Optimized build.
Default target: x86_64-pc-linux-gnu
Host CPU: (unknown)
//so what I wanted to try before was not using -g to see if the issue disappears and it does:
$ time LD_LIBRARY_PATH='/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/lib' x86_64-unknown-linux-gnu/stage0/bin/rustc -O --emit llvm-bc src/libsyntax/lib.rs
real 3m40.865s
user 3m37.717s
sys 0m2.530s
$ time opt -O2 -disable-output syntax.bc
real 1m29.530s
user 1m28.343s
sys 0m0.910s
EDIT: dotdash's previous comment explains why -verify-each doesn't work anymore. |
|
It fails with |
|
Felix raised the quesiton with me on IRC if we consider this On Thu, Oct 15, 2015 at 02:04:03PM -0700, Felix S Klock II wrote:
|
|
But doesn't that mean that in the case of NOT passing $ time x86_64-unknown-linux-gnu/stage0/bin/rustc -g --emit llvm-bc -Csave-temps src/libsyntax/lib.rs
real 1m27.220s
user 1m22.487s
sys 0m4.070s
$ la *.bc
-rw-r--r-- 1 zazdxscf zazdxscf 25854804 16.10.2015 16:20 syntax.0.bc
-rw-r--r-- 1 zazdxscf zazdxscf 24978784 16.10.2015 16:20 syntax.0.no-opt.bc
-rw-r--r-- 1 zazdxscf zazdxscf 25854804 16.10.2015 16:20 syntax.bc
-rw-r--r-- 1 zazdxscf zazdxscf 3356724 16.10.2015 16:20 syntax.metadata.bc
$ time opt -O2 -disable-output syntax.0.no-opt.bc
real 4m21.647s
user 4m20.130s
sys 0m0.747s
$ time opt -O2 -disable-output syntax.bc
real 4m22.069s
user 4m20.267s
sys 0m1.027s
Here's with -O What's curious though, is that previously I was able to reproduce this: #28947 (comment) Is there a way to see what command(s) rustc invokes to generate that EDIT: also note the almost 5 meg difference between the two |
|
I can now confirm that #28947 (comment) can only be reproduced with Therefore, if rustc has llvm code embeded in it(I don't know this) then a recompile(with llvm 3.8.0 that is) should fix this, right? the stage 0 rustc that gets downloaded I mean (which currently was this: rustc 1.4.0-dev (1af31d4 2015-08-11) ) So, the issue should be gone with a rustc that EDIT: the 3.8.0 llvm was at commit: f657b6395ac5c9fa57a3d4c871d7f32c565b11fe |
|
I too am affected. EDIT:
This is not an option, really, because it would delay landing |
|
In my attempt to compile myself a rustc for stage0 to use it to recompile rust with More exactly, when compiling rust with $ ./configure --prefix=/home/zazdxscf/build/1nonpkgs/rust/usr/local --disable-rpath --enable-manage-submodules --disable-clang --enable-ccache --enable-dist-host-only --disable-valgrind --disable-helgrind --disable-valgrind-rpass --python=/usr/bin/python2 --enable-optimize --enable-optimize-cxx --enable-optimize-llvm --enable-debug --disable-debuginfo --enable-debug-assertions --enable-debuginfo-tests --enable-llvm-assertions --enable-debug-jemalloc --disable-local-rust --disable-llvm-version-check --llvm-root=/usr
...
//this is a rerun after the error happened already:
$ make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 'RUSTFLAGS=--verbose -Z verbose -Z print-link-args'
cfg: version 1.5.0-dev (24228fee1 2015-10-16)
cfg: build triple x86_64-unknown-linux-gnu
cfg: host triples x86_64-unknown-linux-gnu
cfg: target triples x86_64-unknown-linux-gnu
cfg: enabling debug assertions (CFG_ENABLE_DEBUG_ASSERTIONS)
cfg: host for x86_64-unknown-linux-gnu is x86_64
cfg: os for x86_64-unknown-linux-gnu is unknown-linux-gnu
cfg: good valgrind for x86_64-unknown-linux-gnu is 1
cfg: using CC=ccache gcc (CFG_CC)
cfg: disabling valgrind run-pass tests
compile: x86_64-unknown-linux-gnu/rustllvm/RustWrapper.o
/home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/RustWrapper.cpp: In function 'LLVMOpaqueMetadata* LLVMDIBuilderCreateSubroutineType(DIBuilderRef, LLVMMetadataRef, LLVMMetadataRef)':
/home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/RustWrapper.cpp:325:56: error: no matching function for call to 'llvm::DIBuilder::createSubroutineType(llvm::DIFile*, llvm::DITypeRefArray)'
DITypeRefArray(unwrap<MDTuple>(ParameterTypes))));
^
In file included from /home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/rustllvm.h:56:0,
from /home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/RustWrapper.cpp:11:
/usr/include/llvm/IR/DIBuilder.h:382:23: note: candidate: llvm::DISubroutineType* llvm::DIBuilder::createSubroutineType(llvm::DITypeRefArray, unsigned int)
DISubroutineType *createSubroutineType(DITypeRefArray ParameterTypes,
^
/usr/include/llvm/IR/DIBuilder.h:382:23: note: no known conversion for argument 1 from 'llvm::DIFile*' to 'llvm::DITypeRefArray'
/home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/RustWrapper.cpp: In function 'LLVMOpaqueMetadata* LLVMDIBuilderCreateVariable(DIBuilderRef, unsigned int, LLVMMetadataRef, const char*, LLVMMetadataRef, unsigned int, LLVMMetadataRef, bool, unsigned int, int64_t*, unsigned int, unsigned int)':
/home/zazdxscf/build/1nonpkgs/rust/rust/src/rustllvm/RustWrapper.cpp:505:26: error: 'class llvm::DIBuilder' has no member named 'createLocalVariable'
return wrap(Builder->createLocalVariable(Tag,
^
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/rustllvm.mk:60: recipe for target 'x86_64-unknown-linux-gnu/rustllvm/RustWrapper.o' failed
make: *** [x86_64-unknown-linux-gnu/rustllvm/RustWrapper.o] Error 1
real 0m8.069s
user 0m6.697s
sys 0m1.237sShould I maybe try to find out which commit(s) on top of 3.7.0 actually fixes the originally reported issue? instead of trying to make rust compile for 3.8.0 (which I know has the issue fixed) [the issue I'm referring to, in case I'm not being clear, is Hmm, apparently this: llvm-mirror/llvm@9b38085 |
|
Re @nikomatsakis "how many programs are affected by this problem", can we use Crater run to find out? (If so, I also hope this may provide a smaller test case.) |
|
Bisecting LLVM against I said "seems", because fixing We also should nominate the patch for LLVM 3.7.1. |
|
Thanks so much sanxiyn for finding that commit! It would've taken me days to learn how to find it, not having the experience and know how. The same commit in git is here: llvm-mirror/llvm@0f8344c LLVM people did say they cherry-picked commits for 3.7.0, so maybe they already did pick (or will easily do so) this commit for 3.7.1 |
|
The patch is not included in LLVM 3.7.0 release and is not included in release_37 branch, so it is not the case they already picked it. |
|
For 3.7.1 I meant, but you're right it's not already picked (although some others are) https://github.com/llvm-mirror/llvm/commits/release_37 EDIT: I've tried to locally backport that patch to the llvm submodule(the llvm that comes with rust) but don't really know how (keeps getting reset to HEAD, and when it doesn't it fails by not finding some 'utility' include file) so I recompiled my system llvm 3.7.0 with the patch, but something's still failing when compiling rust: I guess I'll wait until rust's llvm gets that patch. |
|
Thanks for identifying the commit that fixed this issue upstream. I'm going to cherry-pick that commit to our LLVM fork and create a PR to update to that and disable -g in stage0 until we have a new snapshot. |
When compiling rustc with "--enable-debug --enable-optimize" the stage0 compiler fails to build libsyntax with
This started with #28857, but it looks to me like that just happened to trigger a preexisting bug.
The text was updated successfully, but these errors were encountered: