mingw: abi change on recent gcc

[klutzy]

Win32's cdecl callconv is different from linux's cdecl on returning large structs. For example (from test suite),

struct TwoU64s {
    uint64_t one;
    uint64_t two;
};

TwoU64s rust_dbg_extern_identity_TwoU64s(TwoU64s u) {
    return u;
}

On linux or mingw (gcc <=4.6), the function ends with:

ret    $0x4

This means callee cleans stack for extra space. However, msvc (cl.exe) treats it differently:

ret

then caller is expected to clean 4 bytes.
This implies it will hurt stack if mingw caller calls msvc function.
It is a known issue, and is reported at gcc bugzilla and at mingw.

However, mingw's recent gcc follows win32 convention. (mingw-w64's gcc 4.8 also does.)
This is good for msvc compatibility, however it causes make check-fast failure on latest mingw:

task <unnamed> failed at 'assertion failed: `(left == right) &&
(right == left)`
(left: `t_317::TwoU64s{one: 98784247808u64, two: 257698037760u64}`,
right: `t_317::TwoU64s{one: 22u64, two: 94489280535u64}`)',
C:\home\stone\rust-vanilla\src\test\run-pass\extern-pass-TwoU64s-ref.rs:27
make: *** [i686-pc-mingw32/test/run_pass_stage2_driver-i686-pc-mingw32.out]
Error 101

Relevant codes are: extern-pass-TwoU64s-ref.rs, rust_test_helpers.cpp.

(Well, to build rust with latest mingw, you need some hacks: 1, 2.)

I think we should follow msvc's calling convention on Win32. It implies that we (and bors) have to upgrade mingw later.

[zokier]

I tested this with clang and mingw, with bad results. It seems to be broken there too, I submitted bug to llvm. Hopefully anything they are going to do would help us solve our issue too.

https://gist.github.com/zokier/7464584

http://llvm.org/bugs/show_bug.cgi?id=17924

[klutzy]

Seems that LLVM fixed it: http://llvm.org/bugs/show_bug.cgi?id=18034
When this is fixed, buildbot will need mingw update before (#8598) re-enabling tests.

[klutzy]

We migrated to mingw-w64 with gcc 4.8. needstest!

[klutzy]

@volth I've just downloaded nightly installer and confirmed that it works well on latest mingw (gcc 4.8.1).
Usually the crash happens if wrong mingw dll is chosen. Could you check if your $PATH has other mingw dependency? If you run which libgcc_s_dw-2.dll on msys shell, it should show /mingw/bin/libgcc_s_dw-2.dll, but if you installed other mingw-bundled software (e.g. GHC) it can introduce dll hell.
Also, could you run gdb rustc.exe and type run? If the crash is reproduced, it shows where it goes wrong and backtrace command provides stack trace.

[klutzy]

(If you still have the problem, you'd be better to open new issue; I'm going to close this issue soon since the "abi" issue" is fixed.)

[klutzy]

(Also, we recommend to use mingw-w64 instead of mingw: https://github.com/mozilla/rust/wiki/Using-Rust-on-Windows)