Illegal instruction when calling Math.exp()

[tberthe]

I'm using io.js on a Raspberry Pi (Raspbian) and calling Math.exp() generates an illegal instruction.
I have tryed many packages: iojs-v1.2.0-linux-armv6l.tar.gz, iojs-v1.4.1-linux-armv6l.tar.gz and iojs-v1.6.4-linux-armv6l.tar.gz
For example, a simple file with:

var v = Math.exp(-0.5);

produces this error.

[bnoordhuis]

I'd like to know what instruction it's tripping on. Can you try this?

$ gdb --args iojs -e 'Math.exp(-0.5)'
> run
# wait for the illegal instruction trap
> disassemble $pc,$pc+32

[tberthe]

$ gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e 'Math.exp(-0.5)'
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs...done.
(gdb) r
Starting program: /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e Math.exp\(-0.5\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x003abda0 in _armv7_neon_probe ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda0 to 0x3abdc0:
=> 0x003abda0 <_armv7_neon_probe+0>:    vorr    q15, q15, q15
   0x003abda4 <_armv7_neon_probe+4>:    bx      lr
   0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
End of assembler dump.

But io.js has a strange behavior since calling only gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs and then run also produces the illegal instruction whereas calling ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs produces the error only when entering the call to 'Math.exp()' ...

[bnoordhuis]

Oh, I forgot that openssl does feature detection. Can you repeat the above (enter continue after disassembling) until you hit the final SIGILL?

[tberthe]

$ gdb --args ~/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e 'Math.exp(-0.5)'
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs...done.
(gdb) r
Starting program: /home/pi/interpreters/iojs-v1.6.4-linux-armv6l/bin/iojs -e Math.exp\(-0.5\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x003abda0 in _armv7_neon_probe ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda0 to 0x3abdc0:
=> 0x003abda0 <_armv7_neon_probe+0>:    vorr    q15, q15, q15
   0x003abda4 <_armv7_neon_probe+4>:    bx      lr
   0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
End of assembler dump.
(gdb) continue
Continuing.

Program received signal SIGILL, Illegal instruction.
0x003abda8 in _armv7_tick ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x3abda8 to 0x3abdc8:
=> 0x003abda8 <_armv7_tick+0>:  mrc     15, 0, r0, cr9, cr13, {0}
   0x003abdac <_armv7_tick+4>:  bx      lr
   0x003abdb0 <OPENSSL_atomic_add+0>:   ldrex   r2, [r0]
   0x003abdb4 <OPENSSL_atomic_add+4>:   add     r3, r2, r1
   0x003abdb8 <OPENSSL_atomic_add+8>:   strex   r2, r3, [r0]
   0x003abdbc <OPENSSL_atomic_add+12>:  cmp     r2, #0
   0x003abdc0 <OPENSSL_atomic_add+16>:  bne     0x3abdb0 <OPENSSL_atomic_add>
   0x003abdc4 <OPENSSL_atomic_add+20>:  mov     r0, r3
End of assembler dump.
(gdb) continue
Continuing.
[New Thread 0xb6ca7450 (LWP 29165)]
[New Thread 0xb64a7450 (LWP 29166)]
[New Thread 0xb5ca7450 (LWP 29167)]
[New Thread 0xb54a7450 (LWP 29168)]

Program received signal SIGILL, Illegal instruction.
0x42783070 in ?? ()
(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x42783070 to 0x42783090:
=> 0x42783070:  movt    r12, #16368     ; 0x3ff0
   0x42783074:  vmov.32 d3[1], r12
   0x42783078:  vadd.f64        d1, d1, d3
   0x4278307c:  lsr     r4, r5, #11
   0x42783080:  ldr     r12, [pc, #68]  ; 0x427830cc
   0x42783084:  and     r5, r5, r12
   0x42783088:  ldr     r12, [pc, #64]  ; 0x427830d0
   0x4278308c:  add     r4, r4, r12
End of assembler dump.```

[bnoordhuis]

Ah, movt again? I'm not sure how that can happen, that should have been fixed as of 297cadb. Can you post the contents of /proc/cpuinfo? What happens when you pass --noenable_movw_movt on the command line?

[tberthe]

Of course:

$ cat /proc/cpuinfo
processor       : 0
model name      : ARMv6-compatible processor rev 7 (v6l)
Features        : swp half thumb fastmult vfp edsp java tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2708
Revision        : 000e
Serial          : 00000000afbc086e

Using --noenable_movw_movt does not change anything: illegal instruction and the gdb output is the same.

[bnoordhuis]

Thanks. I can't think of a plausible reason why it's failing.

@rvagg Can I get (or do I still have) access to the rpi1 buildbot?

[silverwind]

Confirmed on my RPi1, RPi2 is fine.

[silverwind]

Might have a partial stack there:

(gdb) bt
#0  0x2b481070 in ?? ()
#1  0x00422bb0 in v8::internal::fast_exp(double) ()
#2  0x00722df8 in v8::internal::Runtime_MathExpRT(int, v8::internal::Object**, v8::internal::Isolate*) ()
#3  0x2eb08090 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

[bnoordhuis]

Ah, interesting. What happens when you run with --nofast_math?

[silverwind]

Works:

$ iojs --nofast_math -p "Math.exp(-1)"
0.36787944117144233

[bnoordhuis]

Okay, good to know. Just to be sure, is the offending instruction in the top stack frame a movt for you as well or is it e.g. a vmov?

[silverwind]

Looks pretty much the same as @tberthe's:

(gdb) disassemble $pc,$pc+32
Dump of assembler code from 0x2b481070 to 0x2b481090:
=> 0x2b481070:  movt    r12, #16368 ; 0x3ff0
   0x2b481074:  vmov.32 d3[1], r12
   0x2b481078:  vadd.f64    d1, d1, d3
   0x2b48107c:  lsr r4, r5, #11
   0x2b481080:  ldr r12, [pc, #68]  ; 0x2b4810cc
   0x2b481084:  and r5, r5, r12
   0x2b481088:  ldr r12, [pc, #64]  ; 0x2b4810d0
   0x2b48108c:  add r4, r4, r12

This is from a core dump. For some reason, I can't get it to crash while live debugging, I blame it on my shell.

[silverwind]

Tried all Math function, also crashes on

• Math.tanh(.5)
• Math.cosh(.5)

Math.exp() crashes without an argument, above two need a number argument to crash.

[silverwind]

The other two functions show the exact same crash signature, so they're just calling into fast_exp.

[silverwind]

Also, current v8 options:

$ iojs --v8-options | head -2
target arm v6 vfp2 hard
ARMv7=0 VFP3=0 VFP32DREGS=0 NEON=0 SUDIV=0 UNALIGNED_ACCESSES=0
MOVW_MOVT_IMMEDIATE_LOADS=0 COHERENT_CACHE=0 USE_EABI_HARDFLOAT=1

[bnoordhuis]

The code in upstream V8 seems to be identical so it's probably best to file an issue in the V8 bug tracker.

[silverwind]

This is how far I got. Still interested in finding the cause myself, but v8 guys might be faster to find this ;)

[bnoordhuis]

V8 issue: https://code.google.com/p/v8/issues/detail?id=4019

/cc @Fishrock123 - this and the --nofast_math workaround should probably be called out in the release notes.

[Fishrock123]

@bnoordhuis You mean in future release notes? Sure.

[rvagg]

@bnoordhuis still want to play with this?

ssh iojs@iojs.org -p 2224

The network is slow on these but there is a reference clone of io.js so you can use --reference git/io.js.reference/ when you're cloning to speed it up significantly. ccache is also used and should be well primed for builds so you can PATH=/usr/lib/ccache/:$PATH to use that.

[rvagg]

(updated comment to be ssh iojs@iojs.org -p 2224, I have iojs-www as an ssh alias sorry)

[silverwind]

@bnoordhuis let me know if above ssh works for you 😉

[bnoordhuis]

Someone from ARM promised to look at it today so I'll wait for his findings.

@silverwind Go ahead and delete my account on your rpi. And thanks again. :-)

[silverwind]

@bnoordhuis thanks 😉

I think it's worth to float a patch for this until the fix lands in v8.

[brendanashworth]

Is this fixed (and can be closed) or are we waiting for the v8 update on this?

[bnoordhuis]

We're working around it now by disabling fast math. The core issue (v8:4019) is still unfixed.

[indutny]

Is it still happening?

[silverwind]

Yes, of course:

$ iojs --fast-math -p "Math.exp(-1)"
[1]    2773 illegal hardware instruction (core dumped)  iojs --fast-math -p "Math.exp(-1)"

Patch has landed it seems:

https://chromium.googlesource.com/v8/v8/+/81703350bbb9923d211fe5b79e90bd458b0916e2%5E!/#F0

[trevnorris]

For reference, the first V8 version that patch is available is 4.5.24.

[ronkorving]

So this has been resolved now?

[targos]

Yes, it has been resolved by the V8 4.5 upgrade. Fast math is back with 074315f