Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deduplicate paths between old and new PYTHONPATH #15648

Closed

Conversation

ericsong
Copy link
Contributor

@ericsong ericsong commented Jun 9, 2022

Resolves #15649 by deduplicating the PYTHONPATH after the old and new one has been joined.

@google-cla
Copy link

google-cla bot commented Jun 9, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@ericsong ericsong closed this Jun 9, 2022
@ericsong ericsong reopened this Jun 9, 2022
@ericsong
Copy link
Contributor Author

ericsong commented Jun 9, 2022

Not sure how to run rerun the cla/google check but I have since signed the CLA (with the correct email this time).

@ckolli5
Copy link

ckolli5 commented Jun 9, 2022

Hello @ericsong, I rerun the cla/google check and it is passed. Thanks!

@ckolli5 ckolli5 added team-Rules-Python Native rules for Python awaiting-review PR is awaiting review from an assigned reviewer labels Jun 9, 2022
@comius comius self-requested a review June 10, 2022 06:23
@comius
Copy link
Contributor

comius commented Jun 10, 2022

I have a question. Is there a way to do this deduplication one level higher, that is when stub parameters are being substituted? Bazel has a lot of machinery to do this, and it might be nicer if the duplicates never get into the template.

@ericsong
Copy link
Contributor Author

I haven't tried it out yet but with that approach, we would be swapping it out here ?

I'm not sure if that would be possible since the duplicates are between that and the PYTHONPATH at runtime. We wouldn't know what old_python_path would be when the python script is being generated.

@ericsong
Copy link
Contributor Author

@comius thoughts on the above?

Comment on lines 304 to 310
old_python_path = os.environ.get('PYTHONPATH')
python_path = os.pathsep.join(python_path_entries)
if old_python_path:
python_path += os.pathsep + old_python_path

python_path = os.pathsep.join(Deduplicate(python_path.split(os.pathsep)))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current implementation with joining, splitting, joining, is not the most readable.

Perhaps rewrite this block, by adding old_python_path to python_path_entries and then deduplicate and join with path separator.

@comius
Copy link
Contributor

comius commented Jun 20, 2022

I haven't tried it out yet but with that approach, we would be swapping it out here ?

I'm not sure if that would be possible since the duplicates are between that and the PYTHONPATH at runtime. We wouldn't know what old_python_path would be when the python script is being generated.

Thanks @ericsong. Since it's coming from the environment, it in fact isn't possible to move it one layer higher.

@ericsong
Copy link
Contributor Author

thanks for the review @comius . I made some changes per your comment.

@ericsong
Copy link
Contributor Author

@comius anything else you'd like for me to change?

@ShreeM01 ShreeM01 removed the awaiting-review PR is awaiting review from an assigned reviewer label Sep 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Rules-Python Native rules for Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Duplicate paths in PYTHONPATH when calling py_binary targets
4 participants