Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build:deploy fails with EMFILE: too many open files #2654

Closed
XhmikosR opened this issue Oct 9, 2019 · 4 comments · Fixed by #3700
Closed

build:deploy fails with EMFILE: too many open files #2654

XhmikosR opened this issue Oct 9, 2019 · 4 comments · Fixed by #3700

Comments

@XhmikosR
Copy link
Contributor

XhmikosR commented Oct 9, 2019 •
edited
Loading

I'm opening this for ideas about solving the EMFILE: too many open files errors that happen when running the full build on CI.

It seems the OS on CI has a maximum limit of 4096 open files which we pass. I tried using graceful-fs, but the error still happens.

We'll need to run the full build at some point when we have linkinator run for all locales.
@XhmikosR
Copy link
Contributor Author

Perhaps we should use async queue?

Any help welcome. If we go with the async queue solution #2633 will become obsolete I think.

@Trott
Copy link
Member

Trott commented Oct 10, 2019

@nodejs/website

@phillipj
Copy link
Member

You were thinking about using an async queue to not build all the locales at one, right? If so that sounds like a good idea 👍

It would ofcourse make the build take a bit longer, but I don't see that as an issue in practise. Sounds much more important and valuable to be able to verify a full build succeeds in CI, than be concerned about the time it takes.

@XhmikosR
Copy link
Contributor Author

Any help welcome. We need to see how much slower it'll be also.

@SEWeiTung SEWeiTung mentioned this issue Apr 3, 2020
Closed
XhmikosR pushed a commit that referenced this issue Apr 3, 2020
@XhmikosR
Run the build in parallel
7170894 
For the issue (#2654),
because we have a "foreach" run in sequence so that it will take quite
a long time to run. Now according to XhmikosR's idea, this is a fixture
for that by "Promise.all" to run it in parallel.
XhmikosR pushed a commit that referenced this issue Oct 27, 2020
@XhmikosR
Run the build in parallel
85bfb98 
For the issue (#2654),
because we have a "foreach" run in sequence so that it will take quite
a long time to run. Now according to XhmikosR's idea, this is a fixture
for that by "Promise.all" to run it in parallel.
XhmikosR pushed a commit that referenced this issue Oct 27, 2020
@XhmikosR
Run the build in parallel
efcb2f6 
For the issue (#2654),
because we have a "foreach" run in sequence so that it will take quite
a long time to run. Now according to XhmikosR's idea, this is a fix
for that by "Promise.all" to run it in parallel.
XhmikosR pushed a commit that referenced this issue Feb 14, 2021
@XhmikosR
Run the build in parallel
8887a5d 
For the issue (#2654),
because we have a "foreach" run in sequence so that it will take quite
a long time to run. Now according to XhmikosR's idea, this is a fix
for that by "Promise.all" to run it in parallel.
@XhmikosR XhmikosR mentioned this issue Feb 14, 2021
Merged
XhmikosR pushed a commit that referenced this issue Apr 30, 2021
@XhmikosR
Run the build in parallel
ec35bbb 
For the issue (#2654),
because we have a "foreach" run in sequence so that it will take quite
a long time to run. Now according to XhmikosR's idea, this is a fix
for that by "Promise.all" to run it in parallel.
@soloinovator soloinovator mentioned this issue Dec 14, 2022
Open
@snyk-bot snyk-bot mentioned this issue Dec 24, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add graceful-fs. nodejs/nodejs.org
3 participants
@XhmikosR @Trott @phillipj