Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for musllinux #315

Merged
merged 1 commit into from
Aug 24, 2021
Merged

Add support for musllinux #315

merged 1 commit into from
Aug 24, 2021

Conversation

mayeut
Copy link
Member

@mayeut mayeut commented Jul 17, 2021

This aims to modify less things than #313 at first to ease the review.
Once there are sufficient tests, the added complexity of #313 might be something to look into as it might be a good step forward to cross-repairing/cross-checking wheels.

@lkollar, thanks for the work in #313 (and in manylinux). What do you think of doing this in 2 steps as mentioned above?

  1. add support for musllinux without extensively modifying the code base for manylinux
  2. once we can test everything properly, refactor policy loading/management

@codecov
Copy link

codecov bot commented Jul 17, 2021

Codecov Report

Merging #315 (ce52d0a) into master (b2ea792) will decrease coverage by 2.21%.
The diff coverage is 65.68%.

❗ Current head ce52d0a differs from pull request most recent head e34bb79. Consider uploading reports for the commit e34bb79 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #315      +/-   ##
==========================================
- Coverage   91.28%   89.07%   -2.22%     
==========================================
  Files          20       23       +3     
  Lines        1102     1199      +97     
  Branches      237      250      +13     
==========================================
+ Hits         1006     1068      +62     
- Misses         54       82      +28     
- Partials       42       49       +7     
Impacted Files Coverage Δ
auditwheel/lddtree.py 87.17% <29.41%> (-7.15%) ⬇️
auditwheel/policy/__init__.py 82.72% <44.11%> (-17.28%) ⬇️
auditwheel/libc.py 87.50% <87.50%> (ø)
auditwheel/musllinux.py 93.54% <93.54%> (ø)
auditwheel/error.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d47670...e34bb79. Read the comment docs.

@lkollar
Copy link
Contributor

lkollar commented Jul 26, 2021

@lkollar, thanks for the work in #313 (and in manylinux). What do you think of doing this in 2 steps as mentioned above?

  1. add support for musllinux without extensively modifying the code base for manylinux
  2. once we can test everything properly, refactor policy loading/management

Sounds good to me. TBH I wasn't completely happy with how the policy loading ended up looking and I agree that it would be better to come back to this and properly refactor it later. I'll close my PR in favour of this one.

@mayeut mayeut mentioned this pull request Jul 31, 2021
6 tasks
@mayeut mayeut force-pushed the musllinux-simple branch 2 times, most recently from 0a00272 to 66a0783 Compare August 1, 2021 11:44
@mayeut mayeut marked this pull request as ready for review August 1, 2021 12:09
@mayeut mayeut requested a review from lkollar August 1, 2021 12:09
@mayeut
Copy link
Member Author

mayeut commented Aug 1, 2021

@lkollar, I split the PR in 2:

I think it's now good enough for review even though, as stated earlier, a refactor of the whole policy selection/loading shall be reworked later on.

assert len(ld_musl) <= 1
if len(ld_musl) == 0:
ldpaths['conf'] = [
root + '/lib',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the prefix should be taken into account here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. I guess it won't matter until cross-repairing is a thing.
PS: for glibc, some folders are appended without either root or prefix.

ldpaths['conf'].append(root + ldpath_stripped)
else:
# Load up /etc/ld.so.conf.
ldpaths['conf'] = parse_ld_so_conf(root + prefix + '/etc/ld.so.conf',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ldpaths['conf'] = parse_ld_so_conf(root + prefix + '/etc/ld.so.conf',
ldpaths['conf'] = parse_ld_so_conf(str(root_prefix / 'etc/ld.so.conf'),

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not touch the glibc code path for now. We can get this cleaned up later.

auditwheel/lddtree.py Outdated Show resolved Hide resolved
ldpath_stripped = ldpath.strip()
if ldpath_stripped == "":
continue
ldpaths['conf'].append(root + ldpath_stripped)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefix is not taken into account here either but I'm not actually sure how the prefix logic is supposed to work in this function so maybe it's fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be:

Suggested change
ldpaths['conf'].append(root + ldpath_stripped)
ldpaths['conf'].append(str(root_prefix / ldpath_stripped))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as earlier, it won't matter until cross-repairing is a thing.
It mimics parse_ld_so_conf where only root is taken into account.
The prefix seems to only be used for the configuration on glibc so I did the same on musl for now.

Detect the running libc at startup & select the correct policy file based on this.
For musllinux, we select only one musllinux policy depending on the running version of musl.
@mayeut
Copy link
Member Author

mayeut commented Aug 22, 2021

@lkollar,

are my answers enough for you to finish the review ?

@mayeut mayeut merged commit 24fddf5 into pypa:master Aug 24, 2021
@mayeut mayeut deleted the musllinux-simple branch August 24, 2021 18:39
@mayeut mayeut mentioned this pull request Aug 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants