Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Improve parallelism of yarn test #30033

Merged
merged 23 commits into from
Jun 22, 2024
Merged

Conversation

poteto
Copy link
Member

@poteto poteto commented Jun 21, 2024

Stack from ghstack (oldest at bottom):

This PR adds parallelism similar to our existing circleci setup for
running yarn tests with the various test params. It does this by
sharding tests into $SHARD_COUNT number of groups, then spawning a job
for each of them and using jest's built in --shard option.

Effectively this means that the job will spawn an additional (where n
is the number of test params)

n * $SHARD_COUNT number of jobs to run tests in parallel

for a total of n + (n * $SHARD_COUNT) jobs. This does mean the
GitHub UI at the bottom of each PR gets longer and unfortunately it's
not sorted in any way as far as I can tell. But if something goes wrong
it should still be easy to find out what the problem is.

The PR also changes the ci argument for jest-cli to be an enum instead
so the tests use all available workers in GitHub actions. This will have
to live around for a bit until we can fully migrate off of circleci.

[ghstack-poisoned]
Copy link

vercel bot commented Jun 21, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
react-compiler-playground ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 22, 2024 4:31pm

poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions

ghstack-source-id: 6ba49ccc0fbb42f4b0528231a055fdc282017898
Pull Request resolved: #30033
@facebook-github-bot facebook-github-bot added CLA Signed React Core Team Opened by a member of the React Core Team labels Jun 21, 2024
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions

ghstack-source-id: 66fe7f0dbcc3207a6b8ca8fff65162baf22148d9
Pull Request resolved: #30033
@react-sizebot
Copy link

react-sizebot commented Jun 21, 2024

Comparing: 454fb35...e4b9888

Critical size changes

Includes critical production bundles, as well as any change greater than 2%:

Name +/- Base Current +/- gzip Base gzip Current gzip
oss-stable/react-dom/cjs/react-dom.production.js = 6.68 kB 6.68 kB = 1.83 kB 1.83 kB
oss-stable/react-dom/cjs/react-dom-client.production.js = 497.93 kB 497.93 kB = 89.26 kB 89.26 kB
oss-experimental/react-dom/cjs/react-dom.production.js = 6.69 kB 6.69 kB = 1.83 kB 1.83 kB
oss-experimental/react-dom/cjs/react-dom-client.production.js = 502.75 kB 502.75 kB = 89.96 kB 89.96 kB
facebook-www/ReactDOM-prod.classic.js = 597.17 kB 597.17 kB = 105.33 kB 105.33 kB
facebook-www/ReactDOM-prod.modern.js = 571.52 kB 571.52 kB = 101.27 kB 101.27 kB
test_utils/ReactAllWarnings.js Deleted 62.88 kB 0.00 kB Deleted 15.69 kB 0.00 kB

Significant size changes

Includes any change greater than 0.2%:

Expand to show
Name +/- Base Current +/- gzip Base gzip Current gzip
test_utils/ReactAllWarnings.js Deleted 62.88 kB 0.00 kB Deleted 15.69 kB 0.00 kB

Generated by 🚫 dangerJS against e4b9888

[ghstack-poisoned]
@poteto poteto changed the title [ci] Remove jest maxWorkers arg when running in github action [ci] Improve parallelism of yarn test Jun 21, 2024
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

When `ci === 'github'` also increase maxConcurrency and enable
workerThreads

ghstack-source-id: 7297ed89271c27746e636583f38c392dd777a369
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js

ghstack-source-id: f8dee9a6e859d1e9544f870277de231e2b7a9261
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js
- Chunk test files

ghstack-source-id: 376cef60af3d97485eb6b336a1e8ad995e892c7b
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js
- Chunk test files

ghstack-source-id: 730499dbbe2de8aa48b351f61706d4b1df58e362
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js
- Chunk test files

ghstack-source-id: 76a509584521d6d86466f267a891bf6d509ecc5a
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js
- Chunk test files

ghstack-source-id: cee93fe1531b35ddb309d21865f67cc89a805696
Pull Request resolved: #30033
[ghstack-poisoned]
poteto added a commit that referenced this pull request Jun 21, 2024
Changes the `ci` argument to be an enum instead so the tests use all
available workers in GitHub actions.

- When `ci === 'github'` also increase maxConcurrency to 10
- Increase timeout of ReactMultiChildText-test.js
- Chunk test files

ghstack-source-id: 6c9b2f8344fd7159a73acf7c199f56c0b48ab6a4
Pull Request resolved: #30033
poteto added 2 commits June 21, 2024 15:10
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
poteto added 2 commits June 22, 2024 12:23
[ghstack-poisoned]
[ghstack-poisoned]
@poteto poteto merged commit e4b9888 into gh/poteto/27/base Jun 22, 2024
159 checks passed
poteto added a commit that referenced this pull request Jun 22, 2024
This PR adds parallelism similar to our existing circleci setup for
running yarn tests with the various test params. It does this by
sharding tests into `$SHARD_COUNT` number of groups, then spawning a job
for each of them and using jest's built in `--shard` option.

Effectively this means that the job will spawn an additional (where `n`
is the number of test params)

`n * $SHARD_COUNT` number of jobs to run tests in parallel

for a total of `n + (n * $SHARD_COUNT)` jobs. This does mean the
GitHub UI at the bottom of each PR gets longer and unfortunately it's
not sorted in any way as far as I can tell. But if something goes wrong
it should still be easy to find out what the problem is.

The PR also changes the `ci` argument for jest-cli to be an enum instead
so the tests use all available workers in GitHub actions. This will have
to live around for a bit until we can fully migrate off of circleci.

ghstack-source-id: 08f2d16353aeca7cd15655d5ce2b7963a12507a7
Pull Request resolved: #30033
@poteto poteto deleted the gh/poteto/27/head branch June 22, 2024 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed React Core Team Opened by a member of the React Core Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants