Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split package entrypoint selection from task dependency resolution. #912

Closed
gsoltis opened this issue Mar 21, 2022 · 1 comment · Fixed by #918
Closed

Split package entrypoint selection from task dependency resolution. #912

gsoltis opened this issue Mar 21, 2022 · 1 comment · Fixed by #918
Assignees
@gsoltis

Comments

@gsoltis
Copy link
Contributor

gsoltis commented Mar 21, 2022

Describe the feature you'd like to request

If a depends on b and turbo.json has specifies ^build for test task dependencies, and I run turbo run test --filter=a, turbo should build b then test a. turbo should not run test on b.

Describe the solution you'd like

Split the notion of package entrypoints from the graph of packages to be considered when resolving task dependencies. Currently, packages that are not selected are removed from the dependency graph before resolving task dependencies. This means that either b won't be built, or the task (test) will also be applied to b if --include-dependencies is specified.

I would like to instead apply the task to all of the selected entrypoints, and then use the full package-dependency graph to resolve pipeline dependencies.

Describe alternatives you've considered

Passing --include-dependencies usually works, but it is not a guarantee. Further, it will almost certainly do extra work, and in the worst case (e.g. watch tasks) may be incompatible.
@gsoltis gsoltis self-assigned this Mar 21, 2022
@gsoltis gsoltis mentioned this issue Mar 21, 2022
Closed
@jaredpalmer
Copy link
Contributor

Let's fix as part of --filter

@gsoltis gsoltis mentioned this issue Mar 21, 2022
Merged
@kodiakhq kodiakhq bot closed this as completed in #918 Mar 23, 2022
kodiakhq bot pushed a commit that referenced this issue Mar 23, 2022
Stop removing unreferenced packages before task graph construction (#918
b5a6f58 
)

Fixes #912 

In this PR:
 * Stop removing unreferenced packages from the package dependency graph before building the task graph.
 * Remove the implied `--include-dependencies` if `--scope` and `--since` are both set. It should no longer be necessary.
 * Remove the test for `--scope` and `--since` implying `--include-dependencies`
 * Add a new scheduler test that mimics the way `run` now behaves.
 * Add a debugging profile for doing a dry run against the kitchen sink example.

Future work includes being able to write a test in `run_test.go` to get the whole end-to-end graph build, but we don't quite have the infrastructure for that yet (also required for follow-on work to #911) 

Note that this is technically a breaking change. Previously, if `a` depends on `b`, and the `"test"` task depends on `"^build"` the following would be true:
 * `turbo run test --scope=a` would test `a` but not build `b`
 * `turbo run test --scope=a --include-dependencies` would build `b` and test `a` *AND* `b`.

With this change, `turbo run test --scope=a` will now build `b` and test `a`. The addition of tasks running for unselected packages is new. While this seems like more work being done, in practice I expect users to be able to move from the second example above to the first, ultimately resulting in less work being done: `b#test` will no longer need to run to get `a#test` to run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gsoltis gsoltis

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Stop removing unreferenced packages before task graph construction vercel/turborepo
2 participants
@gsoltis @jaredpalmer