zig cc can fail with extremely long argument list #10881

sin-ack · 2022-02-13T18:06:29Z

Zig Version

0.10.0-dev.699+55fa349ad

Steps to Reproduce

Note that this isn't really a problem for most purposes as nobody should be hitting the ARG_MAX limit in normal use.

Generate a very long argument list, check your actual maximum with getconf ARG_MAX (on my system, it's 2097152)
Call zig cc with such an argument list, i.e.: zig cc test.c -o test -DAAAAAAAAA...

Expected Behavior

Compile to proceed as usual.

Actual Behavior

info: Usage: zig [command] [options]

Commands:

  build            Build project from build.zig
  init-exe         Initialize a `zig build` application in the cwd
  init-lib         Initialize a `zig build` library in the cwd

  ast-check        Look for simple compile errors in any set of files
  build-exe        Create executable from source or object files
  build-lib        Create library from source or object files
  build-obj        Create object from source or object files
  fmt              Reformat Zig source into canonical form
  run              Create executable and run immediately
  test             Create and run a test build
  translate-c      Convert C code to Zig code

  ar               Use Zig as a drop-in archiver
  cc               Use Zig as a drop-in C compiler
  c++              Use Zig as a drop-in C++ compiler
  dlltool          Use Zig as a drop-in dlltool.exe
  lib              Use Zig as a drop-in lib.exe
  ranlib           Use Zig as a drop-in ranlib

  env              Print lib path, std path, cache directory, and version
  help             Print this help and exit
  libc             Display native libc paths file or validate one
  targets          List available compilation targets
  version          Print version number and exit
  zen              Print Zen of Zig and exit

General Options:

  -h, --help       Print command-specific usage

error: unknown command: @/tmp/response-8e9edb.txt

The text was updated successfully, but these errors were encountered:

sin-ack · 2022-02-13T18:08:51Z

Remarks: This happens because such an argument lists triggers a behavior in Clang to invoke subprograms with clang @/tmp/response-foobar.txt (Clang will actually read the argument list if a file is passed like this). The problem is that argv[0] is zig when the zig binary is used for C compilation, so this fails. I haven't been able to catch the argument list in action yet so I can't say whether it contains the right argument list for zig clang to work or something else.

schmee · 2022-02-13T18:12:05Z

Duplicate of #10693?

sin-ack · 2022-02-13T18:17:51Z

Perhaps, seems like #9654 didn't fix this case.

slimsag · 2022-02-20T21:07:59Z

This is not a duplicate of #10693.

This may be due to shelling out to lld after the response file is read/expanded by zig cc itself.

It's easier to tell where the cause is on Windows because you get a stacktrace: Create a file foo with omg.c omg.c omg.c omg.c repeated until it's 64 KiB. Then create an empty omg.c file, and run it zig cc "@tmp":

PS C:\Users\slimsag\Desktop\mach2\gpu-dawn> zig cc "@tmp"
error.Unexpected: GetLastError(206): The filename or extension is too long.

C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\debug.zig:531:53: 0x7ff60d2ca8bb in std.debug.writeCurrentStackTraceWindows (zig.obj)
    const n = windows.ntdll.RtlCaptureStackBackTrace(0, addr_buf.len, @ptrCast(**anyopaque, &addr_buf), null);
                                                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\debug.zig:511:45: 0x7ff60d008cc6 in std.debug.writeCurrentStackTrace (zig.obj)
        return writeCurrentStackTraceWindows(out_stream, debug_info, tty_config, start_addr);
                                            ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\debug.zig:125:31: 0x7ff60cf265ee in std.debug.dumpCurrentStackTrace (zig.obj)
        writeCurrentStackTrace(stderr, debug_info, detectTTYConfig(), start_addr) catch |err| {
                              ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:2026:40: 0x7ff60d23296f in std.os.windows.unexpectedError (zig.obj)
        std.debug.dumpCurrentStackTrace(null);
                                       ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:1606:49: 0x7ff60d5b9f4f in std.os.windows.CreateProcessW (zig.obj)
            else => |err| return unexpectedError(err),
                                                ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:952:34: 0x7ff60d43b509 in std.child_process.windowsCreateProcess (zig.obj)
    return windows.CreateProcessW(
                                 ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:842:29: 0x7ff60d22fdca in std.child_process.ChildProcess::std.child_process.ChildProcess.spawnWindows (zig.obj)
        windowsCreateProcess(app_path_w.ptr, cmd_line_w.ptr, envp_ptr, cwd_w_ptr, &siStartInfo, &piProcInfo) catch |no_path_err| {
                            ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:132:37: 0x7ff60d22a55a in std.child_process.ChildProcess::std.child_process.ChildProcess.spawn (zig.obj)
            return self.spawnWindows();
                                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:139:23: 0x7ff60d0289c9 in std.child_process.ChildProcess::std.child_process.ChildProcess.spawnAndWait (zig.obj)
        try self.spawn();
                      ^
C:\Users\slimsag\Desktop\zig\src\link\Coff.zig:1381:48: 0x7ff60d0f97ab in link.Coff.linkWithLLD (zig.obj)
                const term = child.spawnAndWait() catch |err| {
                                               ^
C:\Users\slimsag\Desktop\zig\src\link\Coff.zig:840:32: 0x7ff60d0ef809 in link.Coff.flush (zig.obj)
        return self.linkWithLLD(comp);
                               ^
C:\Users\slimsag\Desktop\zig\src\link.zig:613:70: 0x7ff60d0ec659 in link.File::link.File.flush (zig.obj)
            .coff => return @fieldParentPtr(Coff, "base", base).flush(comp),
                                                                     ^
C:\Users\slimsag\Desktop\zig\src\Compilation.zig:2187:28: 0x7ff60d0b1ef2 in Compilation.flush (zig.obj)
    try comp.bin_file.flush(comp); // This is needed before reading the error flags.
                           ^
C:\Users\slimsag\Desktop\zig\src\Compilation.zig:2158:27: 0x7ff60d0a86b4 in Compilation.update (zig.obj)
            try comp.flush();
                          ^
C:\Users\slimsag\Desktop\zig\src\main.zig:3048:20: 0x7ff60d02609c in main.updateModule (zig.obj)

                   ^
C:\Users\slimsag\Desktop\zig\src\main.zig:2735:17: 0x7ff60cf3d684 in main.buildOutputType (zig.obj)
    };
                ^
C:\Users\slimsag\Desktop\zig\src\main.zig:226:31: 0x7ff60cf20116 in main.mainArgs (zig.obj)
        return buildOutputType(gpa, arena, args, .cc);
                              ^
C:\Users\slimsag\Desktop\zig\src\stage1.zig:48:24: 0x7ff60cf1f88d in main (zig.obj)
        stage2.mainArgs(gpa, arena, args) catch unreachable;
                       ^
Unable to dump stack trace: InvalidDebugInfo
error(link): unable to spawn C:\Users\slimsag\Desktop\zig\zig-out\bin\zig.exe: Unexpected
thread 14620 panic: attempt to unwrap error: UnableToSpawnSelf
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:139:9: 0x7ff60d0289ec in std.child_process.ChildProcess::std.child_process.ChildProcess.spawnAndWait (zig.obj)
        try self.spawn();
        ^
C:\Users\slimsag\Desktop\zig\src\link\Coff.zig:1383:21: 0x7ff60d0f981a in link.Coff.linkWithLLD (zig.obj)
                    return error.UnableToSpawnSelf;
                    ^
C:\Users\slimsag\Desktop\zig\src\link\Coff.zig:840:9: 0x7ff60d0ef81c in link.Coff.flush (zig.obj)
        return self.linkWithLLD(comp);
        ^
C:\Users\slimsag\Desktop\zig\src\link.zig:613:22: 0x7ff60d0ec66c in link.File::link.File.flush (zig.obj)
            .coff => return @fieldParentPtr(Coff, "base", base).flush(comp),
                     ^
C:\Users\slimsag\Desktop\zig\src\Compilation.zig:2187:5: 0x7ff60d0b1f11 in Compilation.flush (zig.obj)
    try comp.bin_file.flush(comp); // This is needed before reading the error flags.
    ^
C:\Users\slimsag\Desktop\zig\src\Compilation.zig:2158:13: 0x7ff60d0a876f in Compilation.update (zig.obj)
            try comp.flush();
            ^
C:\Users\slimsag\Desktop\zig\src\main.zig:3048:5: 0x7ff60d0260c4 in main.updateModule (zig.obj)

    ^
C:\Users\slimsag\Desktop\zig\src\main.zig:2737:21: 0x7ff60cf3d8e9 in main.buildOutputType (zig.obj)
    updateModule(gpa, comp, hook) catch |err| switch (err) {
                    ^
C:\Users\slimsag\Desktop\zig\src\main.zig:226:9: 0x7ff60cf20139 in main.mainArgs (zig.obj)
        return buildOutputType(gpa, arena, args, .cc);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2381:9: 0x7ff60cfb7998 in std.os.mkdirat (zig.obj)
        return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\fs.zig:1270:9: 0x7ff60cfb788a in std.fs.Dir::std.fs.Dir.makeDir (zig.obj)
        try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:136:35: 0x7ff60d29040b in std.os.windows.OpenFile (zig.obj)
        .OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
                                  ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2450:21: 0x7ff60cfb7aa0 in std.os.mkdiratW (zig.obj)
        else => |e| return e,
                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2381:9: 0x7ff60cfb7998 in std.os.mkdirat (zig.obj)
        return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\fs.zig:1270:9: 0x7ff60cfb788a in std.fs.Dir::std.fs.Dir.makeDir (zig.obj)
        try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:136:35: 0x7ff60d29040b in std.os.windows.OpenFile (zig.obj)
        .OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
                                  ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2450:21: 0x7ff60cfb7aa0 in std.os.mkdiratW (zig.obj)
        else => |e| return e,
                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2381:9: 0x7ff60cfb7998 in std.os.mkdirat (zig.obj)
        return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\fs.zig:1270:9: 0x7ff60cfb788a in std.fs.Dir::std.fs.Dir.makeDir (zig.obj)
        try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:136:35: 0x7ff60d29040b in std.os.windows.OpenFile (zig.obj)
        .OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
                                  ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2450:21: 0x7ff60cfb7aa0 in std.os.mkdiratW (zig.obj)
        else => |e| return e,
                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2381:9: 0x7ff60cfb7998 in std.os.mkdirat (zig.obj)
        return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\fs.zig:1270:9: 0x7ff60cfb788a in std.fs.Dir::std.fs.Dir.makeDir (zig.obj)
        try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:136:35: 0x7ff60d29040b in std.os.windows.OpenFile (zig.obj)
        .OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
                                  ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2450:21: 0x7ff60cfb7aa0 in std.os.mkdiratW (zig.obj)
        else => |e| return e,
                    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os.zig:2381:9: 0x7ff60cfb7998 in std.os.mkdirat (zig.obj)
        return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\fs.zig:1270:9: 0x7ff60cfb788a in std.fs.Dir::std.fs.Dir.makeDir (zig.obj)
        try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
        ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:2028:5: 0x7ff60d23297e in std.os.windows.unexpectedError (zig.obj)
    return error.Unexpected;
    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\os\windows.zig:1606:27: 0x7ff60d5b9f5c in std.os.windows.CreateProcessW (zig.obj)
            else => |err| return unexpectedError(err),
                          ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:952:5: 0x7ff60d43b51c in std.child_process.windowsCreateProcess (zig.obj)
    return windows.CreateProcessW(
    ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:843:52: 0x7ff60d23001d in std.child_process.ChildProcess::std.child_process.ChildProcess.spawnWindows (zig.obj)
            if (no_path_err != error.FileNotFound) return no_path_err;
                                                   ^
C:\zig-windows-x86_64-0.10.0-dev.862+539bb8a2d\lib\std\child_process.zig:132:13: 0x7ff60d22a56d in std.child_process.ChildProcess::std.child_process.ChildProcess.spawn (zig.obj)
            return self.spawnWindows();
            ^
Unable to dump stack trace: MissingDebugInfo

jcmoyer · 2022-03-22T20:38:47Z

I believe I ran into this on windows while trying to compile fftw with cmake using CMAKE_C_COMPILER=zig cc. It fails on the final link:

D:\Scratch\fftw-3.3.10\build>zig cc -DWITH_OUR_MALLOC -O3 -DNDEBUG   -shared -o Release\libfftw3f.dll -Wl,--out-implib,Release\libfftw3f.dll.a -Wl,--major-image-version,3,--minor-image-version,0 @CMakeFiles\fftw3f.Release.rsp
error(link): unable to spawn D:\Scratch\zig-install\bin\zig.exe: Unexpected
error: UnableToSpawnSelf
ninja: build stopped: subcommand failed.

fftw3f.Release.rsp is 46K. zig 0.10.0-dev.1516+b74f29241

I was able to work around the problem by disabling library features until cmake stopped using a response file.

calavera · 2022-04-16T05:00:39Z

I bumped into this problem too on a Windows build, and I really have no idea how to reproduce it, because it only happens in CI. If anyone is interested in taking a look to see if there is a clear fix, this is the full log:

https://ci.appveyor.com/project/AWSSAMCLI/aws-lambda-builders/builds/43252407/job/9n3ebh8fb8khkjw8

lost22git · 2022-09-27T12:40:39Z

Same problem

pkova · 2024-05-04T09:42:31Z

Fixed by #19850

calavera · 2024-05-04T19:52:54Z

Is the fix going to be in 0.13.0, or do you have plans to include it in a 0.12.1 version at some point?

calavera · 2024-06-10T01:45:01Z

I've confirmed that this issue doesn't happen on 0.13.0 with the release of #19850 🎊

@andrewrk I think this issue can be closed as resolved.

sin-ack added the bug Observed behavior contradicts documented or intended behavior label Feb 13, 2022

Vexu added the zig cc Zig as a drop-in C compiler feature label Feb 14, 2022

Vexu added this to the 0.10.0 milestone Feb 14, 2022

calavera mentioned this issue Apr 19, 2022

Build command fails in some Windows installations cargo-lambda/cargo-lambda#77

Closed

LemonBoy mentioned this issue May 6, 2022

Macro defines lose double-quotes if sum of all source filenames + macros exceed certain length #11595

Closed

messense mentioned this issue Jul 10, 2022

Pass linker arguments on Windows with a linker @-file rust-cross/cargo-zigbuild#46

Closed

andrewrk added the contributor friendly This issue is limited in scope and/or knowledge of Zig internals. label Sep 14, 2022

andrewrk modified the milestones: 0.10.0, 0.11.0 Sep 14, 2022

andrewrk modified the milestones: 0.11.0, 0.11.1 Jun 19, 2023

Gankra mentioned this issue Aug 11, 2023

windows zigcc-x86_64-unknown-linux-musl.bat param too long rust-cross/cargo-zigbuild#129

Closed

andrewrk modified the milestones: 0.11.1, 0.12.0, 0.13.0 Jan 29, 2024

mlugg closed this as completed Jun 10, 2024

mlugg modified the milestones: 0.14.0, 0.13.0 Jun 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

zig cc can fail with extremely long argument list #10881

zig cc can fail with extremely long argument list #10881

sin-ack commented Feb 13, 2022

sin-ack commented Feb 13, 2022

schmee commented Feb 13, 2022

sin-ack commented Feb 13, 2022

slimsag commented Feb 20, 2022 •

edited

Loading

jcmoyer commented Mar 22, 2022 •

edited

Loading

calavera commented Apr 16, 2022

lost22git commented Sep 27, 2022

pkova commented May 4, 2024

calavera commented May 4, 2024

calavera commented Jun 10, 2024

zig cc can fail with extremely long argument list #10881

zig cc can fail with extremely long argument list #10881

Comments

sin-ack commented Feb 13, 2022

Zig Version

Steps to Reproduce

Expected Behavior

Actual Behavior

sin-ack commented Feb 13, 2022

schmee commented Feb 13, 2022

sin-ack commented Feb 13, 2022

slimsag commented Feb 20, 2022 • edited Loading

jcmoyer commented Mar 22, 2022 • edited Loading

calavera commented Apr 16, 2022

lost22git commented Sep 27, 2022

pkova commented May 4, 2024

calavera commented May 4, 2024

calavera commented Jun 10, 2024

slimsag commented Feb 20, 2022 •

edited

Loading

jcmoyer commented Mar 22, 2022 •

edited

Loading