Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes check a USES_ENVIRONMENTS goal #17315

Merged
merged 6 commits into from
Oct 25, 2022

Conversation

chrisjrn
Copy link
Contributor

@chrisjrn chrisjrn commented Oct 21, 2022

check will now run each CheckRequest in each specified environment per the field_sets in the CheckRequest. It looks like the remainder of the check infrastructure already successfully handles dealing with multiple CheckRequests.

See #17129

(cherry picked from commit 15f3f36)
src/python/pants/core/goals/check.py Outdated Show resolved Hide resolved
for _ in request.field_sets
)
grouped = itertools.groupby(
zip(request_for_each_environment_name, environment_names), lambda x: x[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than zipping, consider emitting both the request and environment name in request_for_each_environment_name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A zip is required somewhere because it's not possible to get both the EnvironmentName and the requests into the same object. That said, I've tidied up the relevant code so it's a bit clearer what's going on. New commit incoming.

src/python/pants/core/goals/check.py Outdated Show resolved Hide resolved
src/python/pants/core/goals/check.py Outdated Show resolved Hide resolved
@chrisjrn chrisjrn added the category:internal CI, fixes for not-yet-released features, etc. label Oct 24, 2022
@chrisjrn chrisjrn requested a review from stuhood October 24, 2022 16:31
Comment on lines +210 to +222
environment_names = await MultiGet(
Get(
EnvironmentName,
EnvironmentNameRequest,
EnvironmentNameRequest.from_field_set(field_set),
)
for (_, field_set) in request_to_field_set
)

request_to_env_name = {
(request, env_name)
for (request, _), env_name in zip(request_to_field_set, environment_names)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a request has multiple field sets, this might end up computing multiple environment names for one request, which will overwrite one another in this dict.

This code probably needs to either partition by environment name or error for that case? See #17328 on that topic: currently mypy internally partitions, but check itself does not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a literal for a set of (request, env_name) tuples, not a dict, so it'll run each request in each environment for which it is valid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was considering whether this rule should actually re-construct each request with only the field sets that match each EnvironmentName, but for now, this is probably fine?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was considering whether this rule should actually re-construct each request with only the field sets that match each EnvironmentName, but for now, this is probably fine?

Yea, that would effectively be partitioning, á la #17328. I expect that that will be necessary in the medium term.

@chrisjrn chrisjrn requested a review from stuhood October 24, 2022 20:10
Copy link
Member

@stuhood stuhood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this behavior is a little bit strange, and that #17328 will be necessary. But #17328 is probably a sufficient tracking issue for this. So: #shipit

@chrisjrn chrisjrn merged commit 2aa289a into pantsbuild:main Oct 25, 2022
@chrisjrn chrisjrn deleted the chrisjrn/17129-check branch October 25, 2022 14:59
@stuhood stuhood mentioned this pull request Oct 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants