Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

actionlint causes pipe: too many open files or hangs when checking so many files #3

Closed
rhysd opened this issue Jul 15, 2021 · 0 comments
Labels
bug Something isn't working

Comments

@rhysd
Copy link
Owner

rhysd commented Jul 15, 2021

Environment

  • actionlint: 3c86c33
  • OS: macOS 10.15
  • Go 1.16

Repro

Check many workflows or large workflow:

actionlint ./testdata/realworld/*.yml

Expected behavior

actionlint runs as usual; runs without any output or reports some errors in workflows.

Actual behavior

actionlint dies with message:

`/usr/local/bin/shellcheck --norc -f json -x --shell bash -e SC1091,SC2194 -` did not run successfully while checking script at line:463,col:9: pipe: too many open files: error while checking ./testdata/realworld/diem_diem_.github_workflows_ci-test.yml

or hangs.

Investigation

actionlint runs checks in parallel thanks to Go's goroutine:

  • each files are checked in parallel
  • processes for external linters to check each run: are run in parallel

This works fine and makes actionlint much faster when the target size is small. But running so many processes in parallel consumes OS resource (in this case, it hit limit for number of files to open). And running process sometimes hangs on macOS though I don't know the reason.

@rhysd rhysd added the bug Something isn't working label Jul 15, 2021
@rhysd rhysd closed this as completed in ba6fc4e Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant