Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reintroduce create_skipfile.py script for gerrit #3008

Merged
merged 1 commit into from
Nov 10, 2020

Conversation

gyorb
Copy link
Contributor

@gyorb gyorb commented Nov 5, 2020

The script was removed in c1626bb
but for the usecase where the user does not want to analyze
all the source files in the review it is needed.

@gyorb gyorb added this to the release 6.15.0 milestone Nov 5, 2020
@gyorb gyorb requested a review from dkrupp as a code owner November 5, 2020 16:42
import re


def create_skipfile(files_changed, skipfile):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inside the common library there is already a similar function to get the list of changed files:

def __get_changed_files(changed_file_path: Union[None, str]) -> List[str]:
"""Return a list of changed files.
Process the given gerrit changed file object and return a list of
file paths which changed.
The file can contain some garbage values at start, so we use regex
to find a json object.
"""
changed_files = []
if not changed_file_path or not os.path.exists(changed_file_path):
return changed_files
with open(changed_file_path,
encoding='utf-8',
errors='ignore') as changed_file:
content = changed_file.read()
# The file can contain some garbage values at start, so we use
# regex search to find a json object.
match = re.search(r'\{[\s\S]*\}', content)
if not match:
return changed_files
for filename in json.loads(match.group(0)):
if "/COMMIT_MSG" in filename:
continue
changed_files.append(filename)
return changed_files

Because you created a module in this PR I recommend you to move this function under this tool so we can use this in two places and we can avoid code duplication:

  • you can use this __get_changed_files function to create skip file in this tool.
  • and you can use this function in the common module.

#
# -------------------------------------------------------------------------
"""
Converts Gerrit Review changed files list to CodeChecker skipfile.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the structure of our other tools:

  • create a readme in the root directory of the tool which describe what this tool does, how you can use it and maybe an example.
  • create a codechecker_skiplist directory which will be a module and you move this file also to this directory. So in the CodeChecker we can import and use it.
  • I recommend you to rename this file because from the path (tools/skiplist/create_skipfile.py ) for example I can't figure out that this tool is for gerrit integration. So for example the gerrit_changed_files_to_skipfile or something similar would be a better name choice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved it under the scripts directory as we discussed this will not be a separate tool yet.
I thought about moving it under the bin directory but in that case the create_commands.py script needs to be further complicated and I'm not sure we want that.

Makefile Outdated Show resolved Hide resolved
export CC_SKIPFILE="$WORKSPACE/skipfile"

# gerrit_changed_files_to_skipfile.py script can be found in the bin directory in the CodeChecker package.
python gerrit_changed_files_to_skipfile.py $CC_CHANGED_FILES $CC_SKIPFILE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not we need to use python3 binary here? For example in the Makefiles we are using python3 binary.

Copy link
Contributor Author

@gyorb gyorb Nov 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the jenkins environment has only python2?

@@ -0,0 +1,54 @@
# -------------------------------------------------------------------------
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# -------------------------------------------------------------------------
#!/usr/bin/env python3
# -------------------------------------------------------------------------

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I thought about it but the script is python2/python3 compatible now, with that change we limit it to python3 only do we want that?

Makefile Outdated
@@ -91,7 +91,10 @@ package_statistics_collector: build_statistics_collector package_dir_structure
cd $(CC_BUILD_DIR) && \
ln -sf ../lib/python3/codechecker_statistics_collector/cli.py bin/post-process-stats

package: package_dir_structure set_git_commit_template package_plist_to_html package_tu_collector package_report_converter package_report_hash package_merge_clang_extdef_mappings package_statistics_collector
package_gerrit_skiplist:
cp -rp scripts/gerrit_changed_files_to_skipfile.py $(CC_BUILD_DIR)/bin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we specify the interpreter in the python file below we can copy it without the .py extension so this command can be called easily.
Also other commands in the bin directory doesn't have extension so it would be more consistent and the user doesn't have to call this script through the python binary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the same reason I did not this change what I described in my previous answer. I thought about it but the script is python2/python3 compatible now, with that change we limit it to python3 only do we want that?

The script was removed in c1626bb
but for the usecase where the user does not want to analyze
all the source files in the review it is needed.
Copy link
Contributor

@csordasmarton csordasmarton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@csordasmarton csordasmarton merged commit 616f593 into Ericsson:master Nov 10, 2020
@gyorb gyorb deleted the gerrit-skiplist branch November 18, 2020 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants