Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ability to run script on all workspaces #6244

Merged
merged 2 commits into from
Aug 13, 2018

Conversation

kwelch
Copy link
Contributor

@kwelch kwelch commented Aug 8, 2018

Summary
Added the ability to run a script on every workspace in a project.
yarn workspaces run script

Motivation was to be able to run build or test across an entire monorepo during a CI process. While it was possible to run a script on a workspace individually by name yarn workspace workspace-1 script. This required you to know and list all workspaces.

Test plan
I added more tests that ensure that the commands are being spawned properly. Since most of the code was copied from functioning parts of the project, I felt this was sufficient.

Update
Based on feedback I validated that arguments can be passed.

Here were the commands I ran.

  • cd __tests__/fixtures/workspace/run-basic
  • yarn workspace workspace-1 add jest
  • yarn workspace workspace-2 add jest
  • yarn workspace workspace-1 run jest --all --> sanity check
  • yarn workspaces run jest --all

While jest doesn't pass because there are no tests, you can see from the output the arguments are passed properly.
Here are the spawned child commands

$ /Users/kwelch/_git/yarn/__tests__/fixtures/workspace/run-basic/packages/workspace-child-2/node_modules/.bin/jest --all
$ /Users/kwelch/_git/yarn/__tests__/fixtures/workspace/run-basic/packages/workspace-child-1/node_modules/.bin/jest --all

@buildsize
Copy link

buildsize bot commented Aug 8, 2018

File name Previous Size New Size Change
yarn-[version].noarch.rpm 920.27 KB 920.33 KB 69 bytes (0%)
yarn-[version].js 4.02 MB 4.02 MB 2.37 KB (0%)
yarn-legacy-[version].js 4.18 MB 4.19 MB 2.56 KB (0%)
yarn-v[version].tar.gz 925.2 KB 925.23 KB 26 bytes (0%)
yarn_[version]all.deb 675.45 KB 675.31 KB -146 bytes (0%)

@arcanis
Copy link
Member

arcanis commented Aug 8, 2018

Looks good, thanks! How are options handled? Will yarn workspaces run jest --all work? It seems like the --all will get swallowed by the CLI parsing.

We have some mechanisms to avoid this, can you check whether it could be easily done here?: https://github.com/yarnpkg/yarn/blob/master/src/cli/index.js#L183-L199

@@ -42,3 +47,17 @@ test('workspaces info should list the workspaces', (): Promise<void> => {
});
});
});

test('workspaces run should spawn command for each workspace', (): Promise<void> => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arcanis

I thought test would cover the passing of the arguments, or am I missing something. It sounds like I may need to either leave run in the arg list or add an additional -- for the user?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with those tests is that when you call runWorkspaces, it directly invokes the command, without going through our CLI code. If it did, you would see that the --flag1 flag would be interpreted as a command-line option for the yarn workspaces run command, and wouldn't actually be passed to the argument array.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the additional testing I did and added to the description may cover the case you are mentioning. I was able to validate that --all does properly get passed to the subsequent sub-processes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arcanis Can you confirm if my additional testing steps are adequate?

@kwelch
Copy link
Contributor Author

kwelch commented Aug 8, 2018

@arcanis

I updated the description with some more testing that I ran to verify that the command was properly invoked with the correct arguments passed.

kwelch added a commit to kwelch/yarn-website that referenced this pull request Aug 8, 2018
);
}

await Promise.all(childProcesses);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind if we execute the processes synchronously, at least for a first iteration? Something to keep in mind is that Yarn currently shouldn't be executed concurrently on the same cache - so running it in parallel here might cause cache corruption issues by default 😕

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know. That makes total sense!

@arcanis arcanis merged commit 53a6483 into yarnpkg:master Aug 13, 2018
@kwelch kwelch deleted the workspaces-run branch August 14, 2018 01:41
@donaldpipowitch
Copy link
Contributor

Thank you! Can't wait to test this 😍

@dsifford
Copy link

dsifford commented Sep 27, 2018

Any plans for documentation updates for this one?

Edit: Also, quickly trying this out, it looks like the behavior is to fail when all workspaces don't have the "script" being ran... Why not just skip over the ones that don't have it?

@jahed
Copy link

jahed commented Sep 29, 2018

RE: "Why not just skip over the ones that don't have it?"

Similar to how test runners fail when no tests are found, if yarn did skip non-existent scripts, it may cause unintended errors (typos, forgetfulness). So failing is a safer default.

Also, to me, workspaces run <script> says "run <script> on all my workspaces", where behaviour in npm run, yarn run and yarn workspace <workspace> run has always been to fail if <script> doesn't exist.

Something like --ignore-missing as a flag would then say "run <script> on all my workspaces, and ignore the ones that don't have it".

Personally, I prefer explicit per-workspace "ignores" than implicit ones. So in a monorepo, I'd expect scripts being run globally from the top level to exist in every package for consistency. You can easily add a script to a workspace to say something like "echo 'No tests.'" if there's nothing to do and it makes your intentions explicit.

@kwelch
Copy link
Contributor Author

kwelch commented Oct 1, 2018

To echo @jahed, the intent was to match workspace run, but for all workspaces so failing on missing was intended.

@dsifford, as for docs. I already have a PR up for it. Looks like it needs to be merged, I will post there. yarnpkg/website#855

Haroenv pushed a commit to yarnpkg/website that referenced this pull request Oct 1, 2018
@dsifford
Copy link

dsifford commented Oct 1, 2018

Got it, thanks.

Just for sake of discussion, why not have yarn workspaces run run scripts defined in the top-level package.json on all workspaces as fallback behavior? I can imagine a situation where it might become tedious to add an explicit run script to all workspaces when you are managing a bunch of workspaces at a time.

Is there any use-case for top-level scripts currently?

I guess it could become weird/buggy if workspaces shared a script with the same name as the top-level script though...

@arcanis
Copy link
Member

arcanis commented Oct 1, 2018

Too magical imo. Once we implement nested workspaces, "top-level scripts" will become even more blurry.

I like Yarn scripts being simple, because they're easy to reason about.

@kwelch
Copy link
Contributor Author

kwelch commented Oct 2, 2018

I agree with above. IMO, the power of workspaces is that any workspace could be removed and put into their own repo without breaking anything.

Kumljevx1 added a commit to Kumljevx1/yarn-website that referenced this pull request Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants