Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic in new version when cross-compiled to Windows. #470

Closed
John-Nagle opened this issue Feb 15, 2023 · 22 comments
Closed

Panic in new version when cross-compiled to Windows. #470

John-Nagle opened this issue Feb 15, 2023 · 22 comments

Comments

@John-Nagle
Copy link
Contributor

Existing program using new Rend3, rev = "6dd847f", works when built on LInux with a Linux target, but if built for a cross-compiled Windows target, crashes under Wine 7.0. This program worked cross platform with version rev = "a972b0ff0a98bb79b2b1c8812e3a823840a5e02e"

Reproduce by:

git clone https://github.com/John-Nagle/ui-mock.git
cd ui-mock
cargo build --target x86_64-pc-windows-gnu --examples

cd to x86 target dir
wine ui-mock.exe

wine ui-mock.exe

0104:err:ntlm:ntlm_LsaApInitializePackage no NTLM support, expect problems
Proj dirs data local dir: "C:\\users\\john\\Local Settings\\Application Data\\animats\\ui-mock\\data"
Log path: "C:\\users\\john\\Local Settings\\Application Data\\animats\\ui-mock\\data\\log.txt"
0104:fixme:system:EnableNonClientDpiScaling (000000000001004A): stub
0084:fixme:imm:ImeSetActiveContext (0x3a7b0, 0): stub
0104:fixme:imm:ImeSetActiveContext (0x261270, 1): stub
0084:fixme:imm:ImmReleaseContext (0000000000010020, 000000000003A7B0): stub
0104:fixme:imm:ImmReleaseContext (000000000001004A, 0000000000261270): stub
0104:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
0104:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
0104:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/john/.cargo/git/checkouts/rend3-e03f89403de3386a/6dd847f/rend3/src/shader.rs:63:60
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Code from shader.rs is:

pub fn add_shaders_embed<T: RustEmbed>(&mut self, prefix: &str) {
        for file in T::iter() {
            let contents = String::from_utf8(T::get(&file).unwrap().data.into_owned()).unwrap();
            self.files.insert(format!("{prefix}/{file}"), contents);
        }
    }

Tried to get a backtrace, but in this environment the unwinder blew up and went into a loop that produced a long stream of the same error, then a crash.

@cwfitzgerald
Copy link
Member

cwfitzgerald commented Feb 15, 2023

If compiled in release, does it work?

@John-Nagle
Copy link
Contributor Author

Compiling various options...

@John-Nagle
Copy link
Contributor Author

Reproduced this problem with rend3's "cube" example:

wine cube.exe
00cc:fixme:system:EnableNonClientDpiScaling (0000000000010050): stub
00cc:fixme:imm:ImeSetActiveContext (0x25f160, 1): stub
00cc:fixme:imm:ImmReleaseContext (0000000000010050, 000000000025F160): stub
0080:fixme:imm:ImeSetActiveContext (0x3a8b0, 0): stub
0080:fixme:imm:ImmReleaseContext (0000000000010020, 000000000003A8B0): stub
00cc:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
[2023-02-15T04:06:39Z ERROR wgpu_hal::dx12::instance] Failed to create IDXGIFactoryMedia: 0x80004
002
00cc:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
00cc:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', rend3/src/shader.rs:63:60
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

That's in debug mode. Release mode next.

@John-Nagle
Copy link
Contributor Author

John-Nagle commented Feb 15, 2023

Compile/link of Rend3 examples failed in release mode cross-compiling to "--target x86_64-pc-windows-gnu --release":

   Compiling rend3-imgui-example v0.3.0 (/home/john/projects/rend3/examples/imgui)
error: linking with `x86_64-w64-mingw32-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/john/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/john/.cargo/bin:/home/john/bin:/home/john/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin" VSLANG="1033" "x86_64-w64-mingw32-gcc" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "/home/john/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/rsbegin.o" "/tmp/rustch4o5tc/symbols.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.40lp5brinm9q8dzr.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.0.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.1.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.2.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.3.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.4.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.5.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-pc-windows-gnu/release/deps/imgui-06a790c73f2ba56d.addr2line-c9e87f35e53199ed.addr2line.1bf14959-cgu.6.rcgu.o.rcgu.o" "/home/john/projects/rend3/target/x86_64-

Linker in a loop or something.

@cwfitzgerald
Copy link
Member

cwfitzgerald commented Feb 15, 2023

Try just building a single package without imgui - imgui may not compile under mingw

@John-Nagle
Copy link
Contributor Author

It used to work. Hang on, I seem to have somehow gotten Rust nightly installed.

@John-Nagle
Copy link
Contributor Author

OK. Had to reproduce on a machine with more memory.

Using current Rust stable on Ubuntu 20.04 LTS and 22.04 LTS.

  • Building Rend3 for the Linux target works fine.
  • Building Rend3 for target x86_64-pc-windows-gnu in debug mode compiles, but the executable for cube.exe under Wine fails at an unwrap as mentioned above.
  • Building Rend3 for target x86_64-pc-windows-gnu in release mode fails for all targets in the link step, with thousands of lines of error messages.
  • Building my entire viewer for target target x86_64-pc-windows-gnu in release mode using rend3 rev f2b7df4 from months ago produces a running program that runs under Wine. At least for a while, until rend3/src/graph/graph.rs panics with the message "Outdated".

@John-Nagle
Copy link
Contributor Author

Upgraded to Wine 8. Same error, but better messages: This was cross-compiled in debug mode.

> wine --version
wine-8.0

> wine cube.exe
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:build_joystick_report_descriptor More than 8 absolute axes found, ignoring.
0110:fixme:system:EnableNonClientDpiScaling (0000000000010052): stub
0058:fixme:imm:ImeSetActiveContext (0000000000010026, 0): stub
0058:fixme:imm:ImmReleaseContext (0000000000010020, 0000000000010026): stub
0110:fixme:imm:ImeSetActiveContext (000000000001004E, 1): stub
0110:fixme:imm:ImmReleaseContext (0000000000010052, 000000000001004E): stub
0110:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
[2023-02-15T21:06:09Z ERROR wgpu_hal::dx12::instance] Failed to create IDXGIFactoryMedia: 0x80004002
0110:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
0110:fixme:dxgi:dxgi_factory_EnumAdapterByGpuPreference Ignoring GPU preference 0x2.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', rend3/src/shader.rs:63:60
stack backtrace:
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023EFF040) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F02D10) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0A460) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0A6F0) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0BAD0) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0BC60) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0BE70) in ctx(0000000023E708C0,L"cube")
0110:fixme:dbghelp_dwarf:dwarf2_parse_udt_type Unhandled Tag type 0x33 at debug_info(abbrev:000000002F7F0660,symt:0000000023F0C420) in ctx(0000000023E708C0,L"cube")
(17,000 more lines like this)

This version of Wine, like Wine 7.0, will run my full program which uses rend3 rev f2b7df4

So, somewhere in the last few months of Rend3 development, cross-compilation was broken.

@John-Nagle
Copy link
Contributor Author

Any progress? Able to reproduce this?

@cwfitzgerald
Copy link
Member

Just tested this out. I do get the weird linking errors in release with mingw - I have no idea what cause this as all other platforms link fine. They seem to all be errors linking with other parts of rust, which shouldn't be able to happen using just rust code.

Cube debug example runs fine natively on windows though, so it seems like it's something about wine or the act of cross compilation that's messing it up. I don't have a way of testing wine (as I don't have any linux machines right now).

So the error seems to be from the shader loader code not being able to find the shader files. Could you add a dbg!(file) on line 63 (before the unwrap, inside the for loop) to see what the files it's trying to open are? I have a hunch this is because of cross-compilation.

@John-Nagle
Copy link
Contributor Author

Ah. Added appropriate "expect" calls, and got

thread 'main' panicked at 'Unable to get shader file rend3/mipmap.wgsl', rend3/src/shader.rs:64:18

OK, so Rend3, at least in these demos, is looking in for "mipmap.wsgl" and not finding it. Rend3 is using the "rust_embed" crate to embed additional files in the executable, and somehow that didn't work. Let me dig into rust_embed vs Wine and see how that works.

rust-embed says that, "Debug builds will read the data from the file system at runtime". I'm trying to figure out which components assume what is supposed to be where. The trouble in debug mode is somewhere in that space.

@John-Nagle
Copy link
Contributor Author

John-Nagle commented Feb 20, 2023

It's a bug in either Wine or cross-compile or rust-embed. I built the examples for rust-embed, and example "basic" works on Unix but fails under Wine.

wine basic.exe
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:build_joystick_report_descriptor More than 8 absolute axes found, ignoring.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', examples/basic.rs:8:45
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Exactly the same problem. the "unwrap()" is for get("index.html").unwrap().

That's in debug mode. Release mode, where the file actually gets included in the executable, succeeds.

@cwfitzgerald
Copy link
Member

I think the issue is that rust-embed is calling Path::canonicalize on these paths, but as it is running it as the host (linux) not the cross-compilation target (windows), it produces a folder path which uses /path/to/non-wine/file which isnt' a valid path under wine.

@John-Nagle
Copy link
Contributor Author

That's reasonably likely. Paths under Wine are kind of strange, because you can access both the Windows and Linux file systems. I filed a bug report on rust-embed..

As for the linker bug with the megabyte-sized error message, I posted this on the Rust forums. It seems to have something to do with core::sync::atomic_load, but the error messages make little sense. Maybe someone else has seen this before. atomic_load is the sort of thing that would have a different implementation in release mode.

@John-Nagle
Copy link
Contributor Author

Possibly related: two bugs from last year involving atomic_load causing similar linker errors.

@John-Nagle
Copy link
Contributor Author

See also Rust Bug #98302. This looks likely, and it's way down in LLVM.

@cwfitzgerald
Copy link
Member

Yup, that looks like exactly it - at least you can work around it with fat lto

@John-Nagle
Copy link
Contributor Author

Yes,
lto = "fat"
got Rend3's tests to compile and run cross-platform.

@John-Nagle
Copy link
Contributor Author

Re rust-embed crash:

Right. Under Wine, you can get at the Linux file system, but it's the Z drive. Names relative to the current directory still work, and getting prefixes from the Windows environment works, but what rust-embed is storing is a canonical Unix path meaningful only in the original environment.

It looks like you already fixed this problem with "wasm" targets. I see the code in Rend3's Cargo.toml:

[target.'cfg(target_arch = "wasm32")'.dependencies]
# Needed to embed shaders in the binary on wasm
rust-embed = { version = "6", features = ["interpolate-folder-path", "debug-embed"] }

That's a fix for the same problem I'm having - cross compiling to a different OS. It seems "debug-embed" needs to be enabled for all cross-compiles, not just the "wasm" target". It's cross-compiling that's the problem. (Can you write cfg(target_arch != source_arch) in Cargo files? Probably not. That's the general idea, though.) Minimum support needed is the ability to request feature debug-embed for rust-embed when using Rend3. Can't currently do that as a user of Rend3 without modifying the sources.

@cwfitzgerald
Copy link
Member

Thankfully, you can actually enable features as the end user - if you take a dependency on rust-embed yourself and enable the features, the feature will be enabled for everyone, including rend3.

Unfortunately there's no good signal for the true condition: the program will be run on a different (logical) machine than the one it is compiled on. Being compiled in debug is comorbid, but not always the case (as we can clearly see).

Ultimately, embedding files is the desired behavior for all users of rend3, as they can't directly modify our shaders. But users may want their own shaders to be read directly for ease of iteration.

@John-Nagle
Copy link
Contributor Author

Thankfully, you can actually enable features as the end user - if you take a dependency on rust-embed yourself and enable the features, the feature will be enabled for everyone, including rend3.

OK, that should work. I misunderstood "feature unification". I was thinking that would load two versions of rust-embed. But apparently not.

OK, so now I have workarounds for all these problems and can move forward again.

@John-Nagle
Copy link
Contributor Author

Closing for now. Workarounds available, and the problems are not in Rend3 but in lower levels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants