Skip to content
/ rfcs Public
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Lint files in parallel if many files exist #42

Closed
wants to merge 16 commits into from
Prev Previous commit
Next Next commit
add a table for clarification
  • Loading branch information
mysticatea committed Oct 15, 2019
commit 8d352aa8b46b902216c76ec897925e6d02b8fc27
15 changes: 14 additions & 1 deletion designs/2019-worker-threads/README.md
Original file line number Diff line number Diff line change
@@ -14,7 +14,11 @@ Linting logic uses CPU heavily, so parallel linting by threads may reduce spent

## Detailed Design

**PoC**: [eslint/eslint#very-rough-worker-threads-poc](https://github.com/eslint/eslint/tree/very-rough-worker-threads-poc/lib/eslint)
- **PoC Source Code**: [eslint/eslint#very-rough-worker-threads-poc](https://github.com/eslint/eslint/tree/very-rough-worker-threads-poc/lib/eslint)
- **PoC Installation**:
```
npm install eslint/eslint#very-rough-worker-threads-poc
```

This RFC has two steps.

@@ -45,6 +49,15 @@ concurrency = Math.min(os.cpus().length, Math.ceil(targetFiles.length / 128))

This means that ESLint does linting in the main thread if the number of target files is less than [128](#constants) in order to avoid the overhead of multithreading. But ESLint does linting with using worker threads automatically if target files are many.

For example, if there are 4 processors, `--concurrency=auto` behaves like below. This means that each worker lints [128](#constants)/2 files in the mean at least. Because the cost of multithreading is larger than speed up if each worker lints only a few files.
mysticatea marked this conversation as resolved.
Show resolved Hide resolved

| The number of files | What executes linting |
| :------------------ | :-------------------- |
| 1 - 128 | the main thread |
| 129 - 256 | 2 workers |
| 257 - 384 | 3 workers |
| 385 - ∞ | 4 workers |
mysticatea marked this conversation as resolved.
Show resolved Hide resolved

If `--concurrency` option is present along with the following options, ESLint throws a fatal error.

- `--stdin`