Segfault in JuliaParser

[Keno]

@carnaval and I were dissecting a segfault in JuliaParser (+my custom changes) after pulling latest julia master. This is what I've found since:

The cause is a corrupt pointer value being passed to jl_type_error_rt.

Using, rr, the execution trace leading up to this point is:

movabs $0x7f57ab3f76d0,%rax
# rcx is 0 here
callq  *%rax #julia_take_token_23569
# rcx is 0xddacc4a6af0d4800
mov    -0x1f0(%rbp),%rax
cmp    $0x0,%rax
jne <>
mov    %rbx,%rdx
add    $0xffffffffffffb880,%rdx
movabs $0x7f57ab3cc00c,%rdi
movabs $0x7f57ab50e00d,%rsi 
movabs $0x7f59b1554802,%rax
callq *%rax # to jl_type_error_rt

At this point got in jl_type_rror_rt is 0xddacc4a6af0d4800. So far I haven't looked at anything else, so it may still be that rcx is somehow used by llvm to pass the return value, meaning the problem is elsewhere, but the assembly trace does look suspicious.

[Keno]

Relevant LLVM IR: https://gist.github.com/Keno/cb751c61e3fc7b3037c9

[Keno]

My working theory is that we're in

if258:                                            ; preds = %ok.260
  %598 = call %jl_value_t* @julia_take_token_23684(%jl_value_t* %1) #1
  %599 = load volatile %jl_value_t*, %jl_value_t** %9, align 8
  %600 = icmp eq %jl_value_t* %599, null
  br i1 %600, label %err262, label %ok.263

and are jumping to the wrong basic block

err262:                                           ; preds = %if258
  call void @jl_undefined_var_error(%jl_value_t* inttoptr (i64 140220473766056 to %jl_value_t*))
  unreachable

[carnaval]

yep it's definitely not a type check. it's still weird since the nullcheck should happen on the predecessor BB so as soon as it does the cmp it's already wrong, even before the branch.

the code generator got really confused there

[Keno]

yep it's definitely not a type check. it's still weird since the nullcheck should happen on the predecessor BB so as soon as it does the cmp it's already wrong, even before the branch.

Not sure what you mean

[carnaval]

I just don't think it can be a labeling problem (~ got confused somehow between the two jump targets) since the cmp instruction is already wrong there

[carnaval]

nevermind, read the first block wrong. I thought the typecheck was the intended thing to do.

carry on :)

[vtjnash]

if you suspect llvm, it seems worthwhile to me to try implementing -O0 mode

[Keno]

I updated the gist with the annotated assembly code of the object file. Specifically, we jump to .Ltmp744 from LBB27_142. Now to track this through the LLVM IR and figure out when when this happens.

[Keno]

I'm having a hard time reproducing the generated assembly outside of julia itself. I tried:

cat backup2-after.bc | ./julia-master/usr/bin/llc -debug-pass=Structure -O3 -mcpu=westmere -mattr=-sse4a,-avx512bw,+cx16,-tbm,-adx,-fma4,-avx512vl,-prfchw,-bmi2,-avx512pf,-fsgsbase,-avx,-avx512cd,-rtm,+popcnt,-fma,-bmi,+aes,-rdrnd,+sse4.1,+sse4.2,-avx2,-avx512er,+sse,-lzcnt,+pclmul,-avx512f,-f16c,+ssse3,+mmx,+cmov,-xop,-rdseed,-movbe,-hle,-sha,+sse2,+sse3,-avx512dq -code-model=large -relocation-model=pic -fast-isel -O3 -o - -enable-tail-merge=0 - > indirect2.s

I can however reproduce by loading the module into julia and then writing it to assembly. Any ideas why that might be?

[Keno]

Hmm, running our passes, dumping out to bitcode and then running the rest of the passes does seem to not reproduce the bug. Very odd!

[Keno]

Turns out it's our fault not the backend. The IR was misleading because it generates the correct thing when we regenerate it, but generates undefs the first time around. Will have to figure out why.

[Keno]

Ok, this is starting to look like a type inference bug, probably somewhat complicated by the fact that the function in question calls itself. When I ran code_typed though, everything inferred correctly, it's only during codegen, that things go bad. cc @JeffBezanson

[Keno]

https://gist.github.com/Keno/9103e343bbc9a66231c2 compares the inferred AST we're looking at during codegen and the one obtained by starting a fresh julia session and calling code_typed.

[Keno]

@JeffBezanson The function in question is here: https://github.com/JuliaLang/JuliaParser.jl/blob/kf/loctrack/src/parser.jl#L1685. You should be able to reproduce everything by checking out the kf/loctrack branch of julia parser. You might also need various other packages of mine, but they should all be on github in appropriate versions.

[Keno]

Also, you might want to apply #14967, before trying to reproduce, otherwise it's very difficult to actually see the bug where it appears.

[Keno]

The suspicious behavior seems to start at

:ex = Expr(:call, JuliaParser.Parser.parse_RtoL, ps::JuliaParser.Parser.ParseState, ts::JuliaParser.Lexer.TokenStream{JuliaParser.Lexer.SourceLocToken}, JuliaParser.Parser.parse_cond, JuliaParser.Parser.EQ_OPS)::Union{},

[Keno]

Fixed by #15300.