Reinterpreting bit pattern as Float64 corrupts it on 32-bit Windows

[nalimilan]

Reinterpreting a bit pattern as Float64 leads to its corruption on 32-bit Windows, both on git master and on 0.4.6. This doesn't happen on 64-bit Windows, nor on 32-bit Linux.

Simplest reproducer:

julia> reinterpret(UInt64, reinterpret(Float64, 0x7ff00000000007a2))
0x7ff80000000007a2

Putting the value in an array fixes the problem:

julia> reinterpret(UInt64, reinterpret(Float64, [0x7ff00000000007a2]))
1-element Array{UInt64,1}:
 0x7ff00000000007a2

[nalimilan]

On 0.4.6 32-bit Windows, the same corruption also occurs in this example:

julia> b=IOBuffer([0x7f, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x07, 0xa2])
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=f
alse, size=8, maxsize=Inf, ptr=1, mark=-1)

julia> x=map(ntoh, read(b, Float64, 1))
1-element Array{Float64,1}:
 NaN

julia> reinterpret(UInt64, x)
1-element Array{UInt64,1}:
 0x7ff80000000007a2

On 0.5, this no longer happens (I get 0x7ff00000000007a2).

EDIT: funny fact: 0.4.6 was consistent in the corruption, so that the DataFrames code parsing R data files worked, while in 0.5 it fails because of the discrepancy. See JuliaData/DataFrames.jl#1006.

[yuyichao]

Same issue as #18019 (comment) As noted in that issue, this is a hardware/abi limit and there's little(nothing?) we can do to fix this.

[vtjnash]

Note that this isn't a hardware limit: this behavior is formally specified by the IEEE spec that this bit should get set when the value is interpreted as a float. (although the spec says nothing about whether compiler optimizations may remove the presence of the potentially costly bitcasts). Additionally, according to the spec, all values 0x7ff00xxx... and 0xfffxxx... are equivalent and indistinguishable (and, to be pedantic, never equal). By analogy, 0x1_23456789 can't round-trip from UInt64 -> UInt32 -> UInt64, since all values are mapped mod 2^32. That is also not a bug, since there is no guarantee that you can round-trip through any change of format.

[nalimilan]

Can we fix the discrepancy between reinterpreting a scalar and reinterpreting an array, though? On 0.4.6, the "corruption" also happened with arrays, but that's no longer the case in 0.5. This is confusing and doesn't looks like it follows the IEEE spec.

[yuyichao]

Not really, no matter what the ieee spec says, the value is corrupted whenever it hits the x87 stack which is really hard to avoid when it's part of the ABI.

[nalimilan]

Yeah, but what I'm missing is why it isn't corrupted when reinterpreting an array on 0.5, while it was corrupted on 0.4. I would find it better to always corrupt, so at least it would be consistent.

[yuyichao]

Special case these two cases might be possible (but hard and almost certainly add overhead to unrelated cases), but not when this appears in other context so I don't think it's useful at all.

[vtjnash]

My point above is that this isn't corruption. It is well-defined what should happen to the value, it is simply undefined when it should happen. A combination of compiler optimizations and platform ABI is permitted to pick a behavior based on whatever is more efficient.

[nalimilan]

Yeah, that's why I used quotes around "corruption". Anyway, if it's not possible to make this happen in all cases, probably not worth spending time on adjusting some of them.