-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: parse BUILD.bzel
to determine whether a commit that only changed BUILD.bazel
is a qualified commit
#2937
Changes from 11 commits
a49b574
378ae55
eb6674d
22208fa
c2907bc
db56ad8
23e66d5
2a5cf36
1052244
c7ab267
c98066d
06176fa
8228434
d4128c4
1e73898
00e96e4
0ebbf03
fd7318f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,8 @@ | |
from enum import Enum | ||
from typing import Optional | ||
from git import Commit, Repo | ||
|
||
from library_generation.model.gapic_inputs import parse_build_str | ||
from library_generation.model.generation_config import GenerationConfig | ||
from library_generation.model.library_config import LibraryConfig | ||
from library_generation.utils.utilities import sh_util | ||
|
@@ -77,6 +79,7 @@ def get_changed_libraries(self) -> Optional[list[str]]: | |
Returns a unique, sorted list of library name of changed libraries. | ||
None if there is a repository level change, which means all libraries | ||
in the current_config will be generated. | ||
|
||
:return: library names of change libraries. | ||
""" | ||
if ChangeType.REPO_LEVEL_CHANGE in self.change_to_libraries: | ||
|
@@ -143,11 +146,23 @@ def __create_qualified_commit( | |
:return: qualified commits. | ||
""" | ||
libraries = set() | ||
file_change_num = len(commit.stats.files.keys()) | ||
for file in commit.stats.files.keys(): | ||
if file.endswith("BUILD.bazel"): | ||
continue | ||
versioned_proto_path = find_versioned_proto_path(file) | ||
if versioned_proto_path in proto_paths: | ||
# Skip a commit if the only changed file is `BUILD.bazel` | ||
# and the change doesn't contain fields used in library | ||
# generation, e.g., transport, rest_numeric_enum. | ||
if ( | ||
file.endswith("BUILD.bazel") | ||
and file_change_num == 1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What if a commit contains changes to multiple There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch, I need to make some changes when a commit contains multiple There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. I changed the logic to qualify commits that adding Commits that changed |
||
and ( | ||
not ConfigChange.__is_qualified_build_change( | ||
commit=commit, build_file_path=file | ||
) | ||
) | ||
): | ||
continue | ||
# Even though a commit usually only changes one | ||
# library, we don't want to miss generating a | ||
# library because the commit may change multiple | ||
|
@@ -156,3 +171,31 @@ def __create_qualified_commit( | |
if len(libraries) == 0: | ||
return None | ||
return QualifiedCommit(commit=commit, libraries=libraries) | ||
|
||
@staticmethod | ||
def __is_qualified_build_change(commit: Commit, build_file_path: str) -> bool: | ||
""" | ||
Checks if the given commit containing a BUILD.bazel change is a | ||
qualified commit. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add another sentence explaining that the function parses the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
|
||
The commit is a qualified commit if the | ||
:class:`library_generation.model.gapic_inputs.GapicInputs` objects | ||
parsed from the commit and its parent are different, since there are | ||
changes in fields that used in library generation. | ||
|
||
:param commit: a GitHub commit object. | ||
:param build_file_path: the path of the BUILD.bazel | ||
:return: True if the commit is a qualified commit; False otherwise. | ||
""" | ||
versioned_proto_path = find_versioned_proto_path(build_file_path) | ||
build = str((commit.tree / build_file_path).data_stream.read()) | ||
parent_commit = commit.parents[0] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I know we are getting it from There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe. However, there might be multiple commits changed a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Can you elaborate on this with an example? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I don't have a concrete example but now I understand your original question. Yes, the parent commit contains the last commit touching this file. |
||
# If BUILD.bazel doesn't exist in the parent commit, i.e., it is added | ||
# in the current commit, `parent_commit.tree / build_file_path` will | ||
# raise KeyError. | ||
# However, there's unlikely that a BUILD.bazel is the only file that | ||
# added in a commit. | ||
parent_build = str((parent_commit.tree / build_file_path).data_stream.read()) | ||
inputs = parse_build_str(build, versioned_proto_path) | ||
parent_inputs = parse_build_str(parent_build, versioned_proto_path) | ||
return inputs != parent_inputs |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -208,6 +208,29 @@ def test_get_qualified_commits_success(self): | |
qualified_commits[2].commit.hexsha, | ||
) | ||
|
||
def test_get_qualified_commits_build_only_commit_returns_a_commit(self): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's not clear to me that what this test is testing just based on the name, I guess this is for the basic scenario that a commit contains There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed the test name to |
||
baseline_commit = "45694d2bad602c52170096072d325aa644d550e5" | ||
latest_commit = "758f0d1217d9c7fe398aa5efb1057ce4b6409e55" | ||
config_change = ConfigChange( | ||
change_to_libraries={}, | ||
baseline_config=ConfigChangeTest.__get_a_gen_config( | ||
googleapis_commitish=baseline_commit | ||
), | ||
current_config=ConfigChangeTest.__get_a_gen_config( | ||
googleapis_commitish=latest_commit, | ||
libraries=[ | ||
ConfigChangeTest.__get_a_library_config( | ||
library_name="container", | ||
gapic_configs=[GapicConfig(proto_path="google/container/v1")], | ||
) | ||
], | ||
), | ||
) | ||
# one commit between latest_commit and baseline_commit which only | ||
# changed BUILD.bazel. | ||
# this commit changed `rest_numeric_enums`. | ||
self.assertTrue(len(config_change.get_qualified_commits()) == 1) | ||
|
||
def test_get_qualified_commits_build_only_commit_returns_empty_list(self): | ||
baseline_commit = "bdda0174f68a738518ec311e05e6fd9bbe19cd78" | ||
latest_commit = "c9a5050ef225b0011603e1109cf53ab1de0a8e53" | ||
|
@@ -228,8 +251,35 @@ def test_get_qualified_commits_build_only_commit_returns_empty_list(self): | |
) | ||
# one commit between latest_commit and baseline_commit which only | ||
# changed BUILD.bazel. | ||
# this commit didn't change fields used in library generation. | ||
self.assertTrue(len(config_change.get_qualified_commits()) == 0) | ||
|
||
def test_get_qualified_commits_new_client_commit_returns_a_commit(self): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same question here, the test name is not clear. Also since we are using real commits for the tests, it makes the problem worse that it's not easy for developers to see the test data, but that's a different problem to solve. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed the test name to |
||
baseline_commit = "d007ca1b3cc820651530d44d5388533047ae1414" | ||
latest_commit = "05d889e7dfe087fc2ddc9de9579f01d4e1c2f35e" | ||
config_change = ConfigChange( | ||
change_to_libraries={}, | ||
baseline_config=ConfigChangeTest.__get_a_gen_config( | ||
googleapis_commitish=baseline_commit | ||
), | ||
current_config=ConfigChangeTest.__get_a_gen_config( | ||
googleapis_commitish=latest_commit, | ||
libraries=[ | ||
ConfigChangeTest.__get_a_library_config( | ||
library_name="cloudcontrolspartner", | ||
gapic_configs=[ | ||
GapicConfig( | ||
proto_path="google/cloud/cloudcontrolspartner/v1" | ||
) | ||
], | ||
) | ||
], | ||
), | ||
) | ||
# one commit between latest_commit and baseline_commit which added | ||
# google/cloud/cloudcontrolspartner/v1. | ||
self.assertTrue(len(config_change.get_qualified_commits()) == 1) | ||
|
||
@staticmethod | ||
def __get_a_gen_config( | ||
googleapis_commitish="", libraries: list[LibraryConfig] = None | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we use
endswith
instead of==
? Any special cases we need to handle?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The file is a relative path starting from
google
, e.g.,google/cloud/aiplatform/v1/BUILD.bazel
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. A nit improvement, rename
file
tofile_path
, I thought it is afile_name
that is onlyBUILD.bazel
.