Nim compiler hangs for certain C/C++ compiler errors

[genotrance]

Attached is a test.zip file with a simple script and some dependency files:

{.compile: "HanjaDic.cxx".}
{.passC: "-DWIN32_LEAN_AND_MEAN -I.".}

This issue was found while wrapping Scintilla on Windows. I was unable to create a simpler example so all the code does is compiles the HanjaDic.cxx file which fails due to -DWIN32_LEAN_AND_MEAN being set. Without the -D, it works fine. If you directly run the Nim generated g++ command, it works fine and shows the g++ compiler errors on the console.

If you run it within Nim, it hangs forever. It seems the execProcesses() call has some race condition such that the child process is waiting forever. If you kill the child process in task manager, Nim immediately shows the errors on the console. Using CTRL-C on Nim simply kills the process tree and nothing is displayed.

This works just fine in 0.17.2. After that, there were several changes made to execProcesses() which are suspect in this issue.

cc @cheatfate who worked on those changes.

[jyapayne]

@genotrance, I encountered a similar issue when trying to build libnx on Windows. The thing that solved it for me was disabling warnings with the -w flag set in the passC args. Maybe you could try that as a workaround?

[genotrance]

I just tried -w but it didn't work. In fact, it seems -w is already passed to gcc.

I worked around it by removing ospaths from nimgen since it wasn't really needed but not everyone would know what to do and a workaround in code may not always be possible.

[cheatfate]

I'm sorry for long delay, but could you please give me information about

• Windows version/arch.
• MinGW version/arch.
• Nim version/arch.

I also need somewhere gisted C files you are trying to compile, this will also help.

CC @genotrance, @jyapayne.

[jyapayne]

@genotrance, ah, then it must be a different issue than what I encountered.

[cheatfate]

Ok, i was able to reproduce it, investigating...

[treeform]

I am also having issue with MS VC++. But second recompile usually works. I think it has some thing to do with file "locking" on windows thing. I think i used to get an error like "Can't write to file because another program is using it", but now it just locks up? A second recompile usually works.

[cheatfate]

In this particular case GCC produces big amount of colored output into stderr channel, and because execProcesses trying to hide executed programs output without any reading from pipes, it can be happens that stderr pipe is got overflowed.
Because stderr is overflowed, gcc got stuck on blocking call on write to stderr. And Nim compiler is waiting for process to exit.

[cheatfate]

And this is not regression, it was present here always, the only difference is compiler being used.

[jyapayne]

@cheatfate, is this complete? A PR for this would be awesome when you get the chance!

[treeform]

I found that, if you kill the process on windows it will spit out the errors, allowing you to continue development.

[stefantalpalaru]

Same problem on Linux, while trying to get C compiler warnings: not only is the gcc output completely hidden but, if there's too much of it, Nim hangs.

Simple replication setup:

nim.cfg

@if release:
  nimcache = "nimcache/release/$projectName"
@else:
  nimcache = "nimcache/debug/$projectName"
@end

@if cwarnings:
  @if windows:
    gcc.options.always = "-Wall -mno-ms-bitfields"
  @else:
    gcc.options.always = "-Wall"
  @end
@end

test1.nim

template t1(body: untyped): untyped =
  body
  body
  body
  body
  body

t1(t1(t1(t1(t1(echo "foo")))))

This will hang both the nim compiler and gcc:

nim c -f --verbosity:2 -d:cwarnings test1.nim

(run ps -e -o pid,command | grep gcc in another terminal tab to verify, then kill nim with Ctrl+C)

Amount of gcc output:

$ gcc -c -Wall -I/usr/share/nim/lib -o nimcache/debug/test1/test1.c.o nimcache/debug/test1/test1.c 2>&1 | wc
  18754  143769 1811834

[zevv]

Even simpler replication setup:

• Generate t.c with 2000 @s in it: yes @ | head -2000 >t.c

One liner t.nim:

{.compile:"t.c".}

nim c t.nim now hangs.

Process tree detail ─nim(13797)───sh(13804)───gcc(13805)───cc1(13806)

strace shows cc1 is stuck in write() complaining about the C errors, while gcc, sh, and nim, are all wait()ing on the children to finish.

The easiest fix would probably involve writing the compiler stdout/stderr to temp files, although this will probably have performance impact.

[exelotl]

Hey, wondering if anyone has looked into this recently?

I wanted to ditch the clunky makefile in my project, but right now I basically can't have Nim take care of the C compilation stage because it hangs as soon as I get a few warnings or errors.

[treeform]

I had these issues all the time before, but they have been fixed for me. I have not had a hanging issue in like 4 months.