Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dune build -w stops with epipe after running for a while #6870

Closed
copy opened this issue Jan 12, 2023 · 0 comments · Fixed by #7319
Closed

dune build -w stops with epipe after running for a while #6870

copy opened this issue Jan 12, 2023 · 0 comments · Fixed by #7319
Labels
bug
Milestone
3.8.0

Comments

@copy
Copy link

copy commented Jan 12, 2023

Expected Behavior

dune build -w should keep running forever.

Actual Behavior

dune build -w stops after a while with exit code 141 (epipe).

Reproduction

Difficult, see additional information

Specifications

  • Version of dune (output of dune --version): 3.6.1
  • Version of ocaml (output of ocamlc --version): 5.0.0 (with musl)
  • Operating system (distribution and version): Linux 6.1.3

Additional information

It took about 20 minutes to reproduce in a big project. I ran dune under strace, and the relevant output is (the first epipe happens in a write syscall to _build/.rpc/dune):

26909 12:24:30.544092 write(11<UNIX-STREAM:[133385->136255,"_build/.rpc/dune"]>, "((2:id((4:poll(4:auto1:0))(1:i1:"..., 115 <unfinished ...>
24172 12:24:30.544101 <... madvise resumed>) = 0
26909 12:24:30.544104 <... write resumed>) = -1 EPIPE (Broken pipe)
26909 12:24:30.544112 futex(0x7f3cf94a27ec, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
24172 12:24:30.544118 futex(0x7f3cf94a27ec, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
26909 12:24:30.544122 <... futex resumed>) = -1 EAGAIN (Resource temporarily unavailable)
24172 12:24:30.544125 <... futex resumed>) = 0
24172 12:24:30.544132 mkdir("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/byte", 0777 <unfinished ...>
26909 12:24:30.544138 futex(0x7f3cf94a289c, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
24172 12:24:30.544141 <... mkdir resumed>) = -1 EEXIST (File exists)
24172 12:24:30.544151 madvise(0x7f3cf8647000, 4096, MADV_FREE) = 0
24172 12:24:30.544166 madvise(0x7f3cf8649000, 4096, MADV_FREE) = 0
24172 12:24:30.544180 stat("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/byte/dune__exe__Config.cmi", {st_mode=S_IFREG|0444, st_size=425, ...}) = 0
24172 12:24:30.544196 stat("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/byte/dune__exe__Config.cmo", {st_mode=S_IFREG|0444, st_size=3913, ...}) = 0
24172 12:24:30.544213 stat("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/byte/dune__exe__Config.cmt", {st_mode=S_IFREG|0444, st_size=8219, ...}) = 0
24172 12:24:30.544229 madvise(0x7f3cf864f000, 4096, MADV_FREE) = 0
24172 12:24:30.544246 mkdir("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/native", 0777) = -1 EEXIST (File exists)
24172 12:24:30.544266 madvise(0x7f3cf84eb000, 4096, MADV_FREE) = 0
24172 12:24:30.544282 stat("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/native/dune__exe__Config.cmx", {st_mode=S_IFREG|0444, st_size=2539, ...}) = 0
24172 12:24:30.544298 stat("_build/default/duniverse/eqaf.0.9/config/.config.eobjs/native/dune__exe__Config.o", {st_mode=S_IFREG|0444, st_size=8192, ...}) = 0
24172 12:24:30.544313 madvise(0x7f3cf84ed000, 4096, MADV_FREE) = 0
24172 12:24:30.544419 mkdir("_build/default/duniverse/base64.3.5.0/config", 0777) = -1 EEXIST (File exists)
24172 12:24:30.544439 madvise(0x7f3cf904b000, 4096, MADV_FREE) = 0
24172 12:24:30.544456 madvise(0x7f3cf904d000, 4096, MADV_FREE) = 0
24172 12:24:30.544471 stat("_build/default/duniverse/base64.3.5.0/config/config.exe", {st_mode=S_IFREG|0555, st_size=3656232, ...}) = 0
24172 12:24:30.544489 madvise(0x7f3cf8882000, 4096, MADV_FREE) = 0
24172 12:24:30.544509 mkdir("_build/default/duniverse/base64.3.5.0/config", 0777) = -1 EEXIST (File exists)
24172 12:24:30.544528 madvise(0x7f3cf8884000, 4096, MADV_FREE) = 0
24172 12:24:30.544542 madvise(0x7f3cf8eda000, 4096, MADV_FREE) = 0
24172 12:24:30.544558 stat("_build/default/duniverse/base64.3.5.0/config/which-unsafe-file", {st_mode=S_IFREG|0444, st_size=16, ...}) = 0
24172 12:24:30.544576 madvise(0x7f3cf8edc000, 4096, MADV_FREE) = 0
24172 12:24:30.544594 open("_build/default/duniverse/base64.3.5.0/config/which-unsafe-file", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>
24172 12:24:30.544617 fcntl(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, F_SETFD, FD_CLOEXEC) = 0
24172 12:24:30.544637 lseek(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, 0, SEEK_CUR) = 0
24172 12:24:30.544654 lseek(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, 0, SEEK_END) = 16
24172 12:24:30.544670 lseek(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, 0, SEEK_SET) = 0
24172 12:24:30.544685 read(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, "unsafe_stable.ml", 65536) = 16
24172 12:24:30.544702 read(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>, "", 65536) = 0
24172 12:24:30.544718 close(12<_build/default/duniverse/base64.3.5.0/config/which-unsafe-file>) = 0
24172 12:24:30.544751 mkdir("_build/default/duniverse/eqaf.0.9/config", 0777) = -1 EEXIST (File exists)
24172 12:24:30.544771 madvise(0x7f3cf810a000, 4096, MADV_FREE) = 0
24172 12:24:30.544791 stat("_build/default/duniverse/eqaf.0.9/config/config.exe", {st_mode=S_IFREG|0555, st_size=3656456, ...}) = 0
24172 12:24:30.544810 madvise(0x7f3cf8bf3000, 4096, MADV_FREE) = 0
24172 12:24:30.544830 mkdir("_build/default/duniverse/eqaf.0.9/config", 0777) = -1 EEXIST (File exists)
24172 12:24:30.544848 madvise(0x7f3cf8bf5000, 4096, MADV_FREE) = 0
24172 12:24:30.544861 madvise(0x7f3cf9041000, 4096, MADV_FREE) = 0
24172 12:24:30.544875 stat("_build/default/duniverse/eqaf.0.9/config/which-unsafe-file", {st_mode=S_IFREG|0444, st_size=16, ...}) = 0
24172 12:24:30.544892 madvise(0x7f3cf8647000, 4096, MADV_FREE) = 0
24172 12:24:30.544909 open("_build/default/duniverse/eqaf.0.9/config/which-unsafe-file", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>
24172 12:24:30.544927 fcntl(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, F_SETFD, FD_CLOEXEC) = 0
24172 12:24:30.544946 lseek(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, 0, SEEK_CUR) = 0
24172 12:24:30.544961 lseek(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, 0, SEEK_END) = 16
24172 12:24:30.544976 lseek(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, 0, SEEK_SET) = 0
24172 12:24:30.544991 read(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, "unsafe_stable.ml", 65536) = 16
24172 12:24:30.545007 read(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>, "", 65536) = 0
24172 12:24:30.545022 close(12<_build/default/duniverse/eqaf.0.9/config/which-unsafe-file>) = 0
24172 12:24:30.545051 futex(0x7f3cf94c589c, FUTEX_WAKE_PRIVATE, 1) = 1
26908 12:24:30.545061 <... futex resumed>) = 0
24172 12:24:30.545064 futex(0x7f3cf94a289c, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
26908 12:24:30.545068 futex(0x7f3cf94c583c, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
24172 12:24:30.545072 <... futex resumed>) = 1
26909 12:24:30.545074 <... futex resumed>) = 0
24172 12:24:30.545077 futex(0x7f3cf94c583c, FUTEX_WAKE_PRIVATE, 1) = 1
26909 12:24:30.545087 futex(0x1cd4354, FUTEX_WAIT_PRIVATE, 2147483664, NULL <unfinished ...>
26908 12:24:30.545090 <... futex resumed>) = 0
24172 12:24:30.545093 futex(0x1cd4354, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
26909 12:24:30.545096 <... futex resumed>) = -1 EAGAIN (Resource temporarily unavailable)
24172 12:24:30.545100 <... futex resumed>) = 0
26908 12:24:30.545102 futex(0x1cd4354, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
24172 12:24:30.545106 write(11<UNIX-STREAM:[133385->136255,"_build/.rpc/dune"]>, "((2:id((4:poll(4:auto1:0))(1:i1:"..., 115 <unfinished ...>
26909 12:24:30.545112 futex(0x7f3cf94a283c, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
24172 12:24:30.545117 <... write resumed>) = -1 EPIPE (Broken pipe)
26908 12:24:30.545120 <... futex resumed>) = 0
24172 12:24:30.545123 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=24172, si_uid=1000} ---
26908 12:24:30.545128 munmap(0x7f3cf9746000, 36864) = ?
24177 12:24:30.545141 <... rt_sigtimedwait resumed> <unfinished ...>) = ?
24179 12:24:30.545146 <... select resumed> <unfinished ...>) = ?
24173 12:24:30.545148 <... nanosleep resumed> <unfinished ...>) = ?
26909 12:24:30.545151 <... futex resumed>) = ?
26908 12:24:30.545205 +++ killed by SIGPIPE +++
24179 12:24:30.545209 +++ killed by SIGPIPE +++
26909 12:24:30.545211 +++ killed by SIGPIPE +++
24178 12:24:30.545212 <... futex resumed>) = ?
24177 12:24:30.545214 +++ killed by SIGPIPE +++
24176 12:24:30.545216 <... select resumed> <unfinished ...>) = ?
24174 12:24:30.545218 <... select resumed> <unfinished ...>) = ?
24173 12:24:30.545220 +++ killed by SIGPIPE +++
24178 12:24:30.545255 +++ killed by SIGPIPE +++
24174 12:24:30.545299 +++ killed by SIGPIPE +++
24176 12:24:30.564746 +++ killed by SIGPIPE +++
24172 12:24:30.564758 +++ killed by SIGPIPE +++
@copy copy mentioned this issue Jan 12, 2023
Closed
@rgrinberg rgrinberg added the bug label Jan 12, 2023
richardlford pushed a commit to ivg/dune that referenced this issue Feb 28, 2023
@rgrinberg @richardlford
fix: ignore sigpipe
4e989f7 
Dune would disconnect on writes to clients that suddenly disconnected
because of a sigpipe. We ignore sigpipe and manually shutdown the
scheduler

fixes ocaml#6870

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
anmonteiro pushed a commit to anmonteiro/dune that referenced this issue Feb 28, 2023
@rgrinberg @anmonteiro
fix: ignore sigpipe
bad7e46 
Dune would disconnect on writes to clients that suddenly disconnected
because of a sigpipe. We ignore sigpipe and manually shutdown the
scheduler

fixes ocaml#6870

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
anmonteiro pushed a commit to anmonteiro/dune that referenced this issue Feb 28, 2023
@rgrinberg @anmonteiro
fix: ignore sigpipe
b304507 
Dune would disconnect on writes to clients that suddenly disconnected
because of a sigpipe. We ignore sigpipe and manually shutdown the
scheduler

fixes ocaml#6870

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
rgrinberg added a commit that referenced this issue Mar 6, 2023
@rgrinberg
fix: ignore sigpipe
de4a24a 
Dune would disconnect on writes to clients that suddenly disconnected
because of a sigpipe. We ignore sigpipe and manually shutdown the
scheduler

fixes #6870

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

<!-- ps-id: 2d62e5a9-470f-4df8-8f45-cbdd035142ea -->
@rgrinberg rgrinberg mentioned this issue Mar 6, 2023
Closed
@rgrinberg rgrinberg reopened this Mar 6, 2023
@rgrinberg rgrinberg added this to the 3.8.0 milestone Mar 6, 2023
rgrinberg added a commit that referenced this issue Mar 6, 2023
@rgrinberg
fix: ignore sigpipe
205b4f6 
Dune would disconnect on writes to clients that suddenly disconnected
because of a sigpipe. We ignore sigpipe and manually shutdown the
scheduler

fixes #6870

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

<!-- ps-id: 2d62e5a9-470f-4df8-8f45-cbdd035142ea -->
@rgrinberg rgrinberg linked a pull request Mar 15, 2023 that will close this issue
fix(rpc): ignore sigpipe on linux #7319
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
3.8.0
Development

Successfully merging a pull request may close this issue.

fix(rpc): ignore sigpipe on linux ocaml/dune
fix: ignore sigpipe ocaml/dune
2 participants
@rgrinberg @copy