Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Factors lockfile metadata code into class that supports multiple languages as well as versions #14116

Merged
merged 7 commits into from
Jan 12, 2022
Merged

Factors lockfile metadata code into class that supports multiple languages as well as versions #14116

merged 7 commits into from
Jan 12, 2022

Conversation

chrisjrn
Copy link
Contributor

@chrisjrn chrisjrn commented Jan 7, 2022

This is in aid of #13373. This factors out the LockfileMetadata base class into a language-agnostic form, which will allow changes to the JVM lockfile format in a future PR.

As far as I can tell, the existing lockfile metadata tests adequately cover this new refactoring for now.

Factors lockfile metadata code into class that supports multiple lang…
167ecbf 
…uages as well as versions

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Eric-Arellano
Eric-Arellano reviewed Jan 7, 2022
View reviewed changes
Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you make any changes to the code, or it's solely mechanically moving things? If you made changes, can you please annotate where?

.env Outdated Show resolved Hide resolved
@chrisjrn
Copy link
Contributor Author

chrisjrn commented Jan 7, 2022 via email

removes vscode .env file
abf337f 
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@chrisjrn
Copy link
Contributor Author

Sorry, missed the headline comment in the email thread and only saw the item in the thread!

Functionality is generally unchanged. The notable structural changes are:

  • The concrete Python metadata classes are now prefixed with Python
  • The concrete Python metadata classes now subclass PythonLockfileMetadata
  • LockfileMetadata is now language-agnostic, and lives in core/util_rules.

Functionally:

  • Concrete metadata classes are now annotated with a scope (currently just PYTHON, soon JVM) as well as a version
  • The _lockfile_metadata_version decorator is now returned by a factory function (lockfile_metadata_registrar) that captures the scope
  • _header_dict does things in three levels of subclassing rather than two

Christopher Neugebauer added 2 commits January 10, 2022 08:50
Merge remote-tracking branch 'origin/main' into chrisjrn/13373-lockfi…
1ec3de3 
…le-metadata-refactor

# Conflicts:
#	src/python/pants/backend/python/util_rules/lockfile_metadata.py
maybe-fix mypy complaints in 3.7
fae9b59 
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Christopher Neugebauer and others added 3 commits January 11, 2022 11:03
@stuhood
Update src/python/pants/core/util_rules/lockfile_metadata.py
14e58a0 
Co-authored-by: Stu Hood <stuhood@gmail.com>
@stuhood
Update src/python/pants/core/util_rules/lockfile_metadata.py
5d8862b 
Co-authored-by: Stu Hood <stuhood@gmail.com>
Merge remote-tracking branch 'origin/main' into chrisjrn/13373-lockfi…
5c09f9c 
…le-metadata-refactor

# Conflicts:
#	src/python/pants/backend/python/goals/lockfile.py
#	src/python/pants/backend/python/subsystems/python_tool_base.py
@chrisjrn chrisjrn merged commit dd454a6 into pantsbuild:main Jan 12, 2022
@chrisjrn chrisjrn deleted the chrisjrn/13373-lockfile-metadata-refactor branch January 12, 2022 17:09
illicitonion added a commit to illicitonion/pants that referenced this pull request Jan 22, 2022
@illicitonion
Prep for 2.10.0.dev2
bcb22d6 
Internal changes:

* upgrade to Rust v1.58.0 ([pantsbuild#14174](pantsbuild#14174))

* [internal] fix typos in codegen registration ([pantsbuild#14172](pantsbuild#14172))

* Remove per-language indirection for formatter plugins. ([pantsbuild#14166](pantsbuild#14166))

* Pulls `Coordinate` and `ArtifactRequirements` out into a separate file to avoid a circuilar import later ([pantsbuild#14164](pantsbuild#14164))

* Factors lockfile metadata code into class that supports multiple languages as well as versions ([pantsbuild#14116](pantsbuild#14116))

* Adds `.env` file to make vscode auto-imports work correctly out of the box ([pantsbuild#14130](pantsbuild#14130))

* [internal] Rename classes for `generate_lockfiles.py` for clarity ([pantsbuild#14218](pantsbuild#14218))

* [internal] Fix bad Find+Replace ([pantsbuild#14213](pantsbuild#14213))

* [internal] Bump libc from 0.2.106 to 0.2.112 in /src/rust/engine ([pantsbuild#14206](pantsbuild#14206))

* [internal] Bump tempfile from 3.2.0 to 3.3.0 in /src/rust/engine ([pantsbuild#14202](pantsbuild#14202))

* [internal] Bump criterion from 0.3.3 to 0.3.5 in /src/rust/engine ([pantsbuild#14205](pantsbuild#14205))

* [internal] Bump walkdir from 2.3.1 to 2.3.2 in /src/rust/engine ([pantsbuild#14204](pantsbuild#14204))

* [internal] Bump generic-array from 0.14.4 to 0.14.5 in /src/rust/engine ([pantsbuild#14203](pantsbuild#14203))

* [internal] Remove the minimum bucket size of batching to improve stability. ([pantsbuild#14210](pantsbuild#14210))

* [internal] Make `JvmLockfileRequest` generic ([pantsbuild#14201](pantsbuild#14201))

* [internal] add comment re clippy issue ([pantsbuild#14188](pantsbuild#14188))

* [internal] rename some codegen scopes to put language first ([pantsbuild#14187](pantsbuild#14187))

* [internal] Check for ambiguity when running `generate-lockfiles` ([pantsbuild#14178](pantsbuild#14178))

* [internal] Change JVM lockfile format ([pantsbuild#14175](pantsbuild#14175))

* [internal] go: rewrite third party package analysis to not use `go list` ([pantsbuild#14157](pantsbuild#14157))

* [internal] Hotfix `fmt` crashing on non-formattable targets ([pantsbuild#14168](pantsbuild#14168))

* [internal] Simplify `core/goals/package.py` filesystem API usage ([pantsbuild#14162](pantsbuild#14162))

* [internal] java/thrift: register union that was not registered ([pantsbuild#14159](pantsbuild#14159))

* [internal] Bump arc-swap from 1.3.0 to 1.5.0 in /src/rust/engine ([pantsbuild#14148](pantsbuild#14148))

* [internals] Bump tokio-rustls from 0.22.0 to 0.23.2 in /src/rust/engine ([pantsbuild#14149](pantsbuild#14149))

* [internal] Bump num_cpus from 1.13.0 to 1.13.1 in /src/rust/engine ([pantsbuild#14150](pantsbuild#14150))

* [internal] Bump tokio-util from 0.6.7 to 0.6.9 in /src/rust/engine ([pantsbuild#14151](pantsbuild#14151))

* [internal] Bump os_pipe from 0.9.2 to 1.0.0 in /src/rust/engine ([pantsbuild#14152](pantsbuild#14152))

* [internal] Introduce new `BuiltinGoal` subsystem type. ([pantsbuild#13991](pantsbuild#13991))

* [internal] Upgrade python type stubs packages ([pantsbuild#14143](pantsbuild#14143))

* [internal] Bump strum_macros from 0.20.1 to 0.23.1 in /src/rust/engine ([pantsbuild#14141](pantsbuild#14141))

* [internal] Make `generate-lockfiles` goal generic ([pantsbuild#14122](pantsbuild#14122))

* [internal] Bump errno from 0.2.7 to 0.2.8 in /src/rust/engine ([pantsbuild#14137](pantsbuild#14137))

* [internal] Bump colored from 1.9.3 to 2.0.0 in /src/rust/engine ([pantsbuild#14138](pantsbuild#14138))

* [internal] Bump crossbeam-channel from 0.4.4 to 0.5.0 in /src/rust/engine ([pantsbuild#14139](pantsbuild#14139))

* [internal] Bump reqwest from 0.11.4 to 0.11.9 in /src/rust/engine ([pantsbuild#14135](pantsbuild#14135))

* [internal] Further tweak dependabot config ([pantsbuild#14132](pantsbuild#14132))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] No need for two f-strings/two strings. ([pantsbuild#14119](pantsbuild#14119))
@illicitonion illicitonion mentioned this pull request Jan 22, 2022
Merged
illicitonion added a commit to illicitonion/pants that referenced this pull request Jan 22, 2022
@illicitonion
Prep for 2.10.0.dev2
6071ee3 
Internal changes:

* upgrade to Rust v1.58.0 ([pantsbuild#14174](pantsbuild#14174))

* [internal] fix typos in codegen registration ([pantsbuild#14172](pantsbuild#14172))

* Remove per-language indirection for formatter plugins. ([pantsbuild#14166](pantsbuild#14166))

* Pulls `Coordinate` and `ArtifactRequirements` out into a separate file to avoid a circuilar import later ([pantsbuild#14164](pantsbuild#14164))

* Factors lockfile metadata code into class that supports multiple languages as well as versions ([pantsbuild#14116](pantsbuild#14116))

* Adds `.env` file to make vscode auto-imports work correctly out of the box ([pantsbuild#14130](pantsbuild#14130))

* [internal] Rename classes for `generate_lockfiles.py` for clarity ([pantsbuild#14218](pantsbuild#14218))

* [internal] Fix bad Find+Replace ([pantsbuild#14213](pantsbuild#14213))

* [internal] Bump libc from 0.2.106 to 0.2.112 in /src/rust/engine ([pantsbuild#14206](pantsbuild#14206))

* [internal] Bump tempfile from 3.2.0 to 3.3.0 in /src/rust/engine ([pantsbuild#14202](pantsbuild#14202))

* [internal] Bump criterion from 0.3.3 to 0.3.5 in /src/rust/engine ([pantsbuild#14205](pantsbuild#14205))

* [internal] Bump walkdir from 2.3.1 to 2.3.2 in /src/rust/engine ([pantsbuild#14204](pantsbuild#14204))

* [internal] Bump generic-array from 0.14.4 to 0.14.5 in /src/rust/engine ([pantsbuild#14203](pantsbuild#14203))

* [internal] Remove the minimum bucket size of batching to improve stability. ([pantsbuild#14210](pantsbuild#14210))

* [internal] Make `JvmLockfileRequest` generic ([pantsbuild#14201](pantsbuild#14201))

* [internal] add comment re clippy issue ([pantsbuild#14188](pantsbuild#14188))

* [internal] rename some codegen scopes to put language first ([pantsbuild#14187](pantsbuild#14187))

* [internal] Check for ambiguity when running `generate-lockfiles` ([pantsbuild#14178](pantsbuild#14178))

* [internal] Change JVM lockfile format ([pantsbuild#14175](pantsbuild#14175))

* [internal] go: rewrite third party package analysis to not use `go list` ([pantsbuild#14157](pantsbuild#14157))

* [internal] Hotfix `fmt` crashing on non-formattable targets ([pantsbuild#14168](pantsbuild#14168))

* [internal] Simplify `core/goals/package.py` filesystem API usage ([pantsbuild#14162](pantsbuild#14162))

* [internal] java/thrift: register union that was not registered ([pantsbuild#14159](pantsbuild#14159))

* [internal] Bump arc-swap from 1.3.0 to 1.5.0 in /src/rust/engine ([pantsbuild#14148](pantsbuild#14148))

* [internals] Bump tokio-rustls from 0.22.0 to 0.23.2 in /src/rust/engine ([pantsbuild#14149](pantsbuild#14149))

* [internal] Bump num_cpus from 1.13.0 to 1.13.1 in /src/rust/engine ([pantsbuild#14150](pantsbuild#14150))

* [internal] Bump tokio-util from 0.6.7 to 0.6.9 in /src/rust/engine ([pantsbuild#14151](pantsbuild#14151))

* [internal] Bump os_pipe from 0.9.2 to 1.0.0 in /src/rust/engine ([pantsbuild#14152](pantsbuild#14152))

* [internal] Introduce new `BuiltinGoal` subsystem type. ([pantsbuild#13991](pantsbuild#13991))

* [internal] Upgrade python type stubs packages ([pantsbuild#14143](pantsbuild#14143))

* [internal] Bump strum_macros from 0.20.1 to 0.23.1 in /src/rust/engine ([pantsbuild#14141](pantsbuild#14141))

* [internal] Make `generate-lockfiles` goal generic ([pantsbuild#14122](pantsbuild#14122))

* [internal] Bump errno from 0.2.7 to 0.2.8 in /src/rust/engine ([pantsbuild#14137](pantsbuild#14137))

* [internal] Bump colored from 1.9.3 to 2.0.0 in /src/rust/engine ([pantsbuild#14138](pantsbuild#14138))

* [internal] Bump crossbeam-channel from 0.4.4 to 0.5.0 in /src/rust/engine ([pantsbuild#14139](pantsbuild#14139))

* [internal] Bump reqwest from 0.11.4 to 0.11.9 in /src/rust/engine ([pantsbuild#14135](pantsbuild#14135))

* [internal] Further tweak dependabot config ([pantsbuild#14132](pantsbuild#14132))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] No need for two f-strings/two strings. ([pantsbuild#14119](pantsbuild#14119))
illicitonion added a commit to illicitonion/pants that referenced this pull request Jan 22, 2022
@illicitonion
Prep for 2.10.0.dev2
237e3e1 
Internal changes:

* upgrade to Rust v1.58.0 ([pantsbuild#14174](pantsbuild#14174))

* [internal] fix typos in codegen registration ([pantsbuild#14172](pantsbuild#14172))

* Pulls `Coordinate` and `ArtifactRequirements` out into a separate file to avoid a circuilar import later ([pantsbuild#14164](pantsbuild#14164))

* Factors lockfile metadata code into class that supports multiple languages as well as versions ([pantsbuild#14116](pantsbuild#14116))

* Adds `.env` file to make vscode auto-imports work correctly out of the box ([pantsbuild#14130](pantsbuild#14130))

* [internal] Rename classes for `generate_lockfiles.py` for clarity ([pantsbuild#14218](pantsbuild#14218))

* [internal] Fix bad Find+Replace ([pantsbuild#14213](pantsbuild#14213))

* [internal] Bump libc from 0.2.106 to 0.2.112 in /src/rust/engine ([pantsbuild#14206](pantsbuild#14206))

* [internal] Bump tempfile from 3.2.0 to 3.3.0 in /src/rust/engine ([pantsbuild#14202](pantsbuild#14202))

* [internal] Bump criterion from 0.3.3 to 0.3.5 in /src/rust/engine ([pantsbuild#14205](pantsbuild#14205))

* [internal] Bump walkdir from 2.3.1 to 2.3.2 in /src/rust/engine ([pantsbuild#14204](pantsbuild#14204))

* [internal] Bump generic-array from 0.14.4 to 0.14.5 in /src/rust/engine ([pantsbuild#14203](pantsbuild#14203))

* [internal] Remove the minimum bucket size of batching to improve stability. ([pantsbuild#14210](pantsbuild#14210))

* [internal] Make `JvmLockfileRequest` generic ([pantsbuild#14201](pantsbuild#14201))

* [internal] add comment re clippy issue ([pantsbuild#14188](pantsbuild#14188))

* [internal] rename some codegen scopes to put language first ([pantsbuild#14187](pantsbuild#14187))

* [internal] Check for ambiguity when running `generate-lockfiles` ([pantsbuild#14178](pantsbuild#14178))

* [internal] Change JVM lockfile format ([pantsbuild#14175](pantsbuild#14175))

* [internal] go: rewrite third party package analysis to not use `go list` ([pantsbuild#14157](pantsbuild#14157))

* [internal] Hotfix `fmt` crashing on non-formattable targets ([pantsbuild#14168](pantsbuild#14168))

* [internal] Simplify `core/goals/package.py` filesystem API usage ([pantsbuild#14162](pantsbuild#14162))

* [internal] java/thrift: register union that was not registered ([pantsbuild#14159](pantsbuild#14159))

* [internal] Bump arc-swap from 1.3.0 to 1.5.0 in /src/rust/engine ([pantsbuild#14148](pantsbuild#14148))

* [internals] Bump tokio-rustls from 0.22.0 to 0.23.2 in /src/rust/engine ([pantsbuild#14149](pantsbuild#14149))

* [internal] Bump num_cpus from 1.13.0 to 1.13.1 in /src/rust/engine ([pantsbuild#14150](pantsbuild#14150))

* [internal] Bump tokio-util from 0.6.7 to 0.6.9 in /src/rust/engine ([pantsbuild#14151](pantsbuild#14151))

* [internal] Bump os_pipe from 0.9.2 to 1.0.0 in /src/rust/engine ([pantsbuild#14152](pantsbuild#14152))

* [internal] Introduce new `BuiltinGoal` subsystem type. ([pantsbuild#13991](pantsbuild#13991))

* [internal] Upgrade python type stubs packages ([pantsbuild#14143](pantsbuild#14143))

* [internal] Bump strum_macros from 0.20.1 to 0.23.1 in /src/rust/engine ([pantsbuild#14141](pantsbuild#14141))

* [internal] Make `generate-lockfiles` goal generic ([pantsbuild#14122](pantsbuild#14122))

* [internal] Bump errno from 0.2.7 to 0.2.8 in /src/rust/engine ([pantsbuild#14137](pantsbuild#14137))

* [internal] Bump colored from 1.9.3 to 2.0.0 in /src/rust/engine ([pantsbuild#14138](pantsbuild#14138))

* [internal] Bump crossbeam-channel from 0.4.4 to 0.5.0 in /src/rust/engine ([pantsbuild#14139](pantsbuild#14139))

* [internal] Bump reqwest from 0.11.4 to 0.11.9 in /src/rust/engine ([pantsbuild#14135](pantsbuild#14135))

* [internal] Further tweak dependabot config ([pantsbuild#14132](pantsbuild#14132))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] python: use immutable_input_digests for protobuf codegen ([pantsbuild#13885](pantsbuild#13885))

* [internal] No need for two f-strings/two strings. ([pantsbuild#14119](pantsbuild#14119))

* `LockfileMetadata` - Replace flaky `_header_dict()` mechanism with more robust `header_attrs` mechanism ([pantsbuild#14229](pantsbuild#14229))
illicitonion added a commit that referenced this pull request Jan 22, 2022
@illicitonion
Prep for 2.10.0.dev2 (#14236)
4113608 
Internal changes:

* upgrade to Rust v1.58.0 ([#14174](#14174))

* [internal] fix typos in codegen registration ([#14172](#14172))

* Pulls `Coordinate` and `ArtifactRequirements` out into a separate file to avoid a circuilar import later ([#14164](#14164))

* Factors lockfile metadata code into class that supports multiple languages as well as versions ([#14116](#14116))

* Adds `.env` file to make vscode auto-imports work correctly out of the box ([#14130](#14130))

* [internal] Rename classes for `generate_lockfiles.py` for clarity ([#14218](#14218))

* [internal] Fix bad Find+Replace ([#14213](#14213))

* [internal] Bump libc from 0.2.106 to 0.2.112 in /src/rust/engine ([#14206](#14206))

* [internal] Bump tempfile from 3.2.0 to 3.3.0 in /src/rust/engine ([#14202](#14202))

* [internal] Bump criterion from 0.3.3 to 0.3.5 in /src/rust/engine ([#14205](#14205))

* [internal] Bump walkdir from 2.3.1 to 2.3.2 in /src/rust/engine ([#14204](#14204))

* [internal] Bump generic-array from 0.14.4 to 0.14.5 in /src/rust/engine ([#14203](#14203))

* [internal] Remove the minimum bucket size of batching to improve stability. ([#14210](#14210))

* [internal] Make `JvmLockfileRequest` generic ([#14201](#14201))

* [internal] add comment re clippy issue ([#14188](#14188))

* [internal] rename some codegen scopes to put language first ([#14187](#14187))

* [internal] Check for ambiguity when running `generate-lockfiles` ([#14178](#14178))

* [internal] Change JVM lockfile format ([#14175](#14175))

* [internal] go: rewrite third party package analysis to not use `go list` ([#14157](#14157))

* [internal] Hotfix `fmt` crashing on non-formattable targets ([#14168](#14168))

* [internal] Simplify `core/goals/package.py` filesystem API usage ([#14162](#14162))

* [internal] java/thrift: register union that was not registered ([#14159](#14159))

* [internal] Bump arc-swap from 1.3.0 to 1.5.0 in /src/rust/engine ([#14148](#14148))

* [internals] Bump tokio-rustls from 0.22.0 to 0.23.2 in /src/rust/engine ([#14149](#14149))

* [internal] Bump num_cpus from 1.13.0 to 1.13.1 in /src/rust/engine ([#14150](#14150))

* [internal] Bump tokio-util from 0.6.7 to 0.6.9 in /src/rust/engine ([#14151](#14151))

* [internal] Bump os_pipe from 0.9.2 to 1.0.0 in /src/rust/engine ([#14152](#14152))

* [internal] Introduce new `BuiltinGoal` subsystem type. ([#13991](#13991))

* [internal] Upgrade python type stubs packages ([#14143](#14143))

* [internal] Bump strum_macros from 0.20.1 to 0.23.1 in /src/rust/engine ([#14141](#14141))

* [internal] Make `generate-lockfiles` goal generic ([#14122](#14122))

* [internal] Bump errno from 0.2.7 to 0.2.8 in /src/rust/engine ([#14137](#14137))

* [internal] Bump colored from 1.9.3 to 2.0.0 in /src/rust/engine ([#14138](#14138))

* [internal] Bump crossbeam-channel from 0.4.4 to 0.5.0 in /src/rust/engine ([#14139](#14139))

* [internal] Bump reqwest from 0.11.4 to 0.11.9 in /src/rust/engine ([#14135](#14135))

* [internal] Further tweak dependabot config ([#14132](#14132))

* [internal] python: use immutable_input_digests for protobuf codegen ([#13885](#13885))

* [internal] python: use immutable_input_digests for protobuf codegen ([#13885](#13885))

* [internal] No need for two f-strings/two strings. ([#14119](#14119))

* `LockfileMetadata` - Replace flaky `_header_dict()` mechanism with more robust `header_attrs` mechanism ([#14229](#14229))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stuhood stuhood stuhood approved these changes

@Eric-Arellano Eric-Arellano Awaiting requested review from Eric-Arellano

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@chrisjrn @stuhood @Eric-Arellano