Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lld linker to the centos, alpine and crossbuild images #446

Merged
merged 5 commits into from
Jul 14, 2021

Conversation

janvorli
Copy link
Member

@janvorli janvorli commented May 5, 2021

We will be switching runtime build to using lld linker, so we need it in the images that we use to build it.

@janvorli janvorli requested a review from mthalman May 5, 2021 00:43
@janvorli janvorli self-assigned this May 5, 2021
@@ -20,6 +20,7 @@ RUN apk add --no-cache \
libtool \
libunwind-dev \
linux-headers \
lld \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still use it to build runtime for Alpine and we don't have equivalent one for 3.10 yet. Would you like me to add the 3.10 one in this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the 3.11, as the 3.10 is EOL too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine 3.11 is EOL in Nov 2021 so we'll just have to do it all over again then. It'd be preferable to just jump to the latest, Alpine 3.13 to buy us an extra year.

@mthalman
Copy link
Member

mthalman commented May 5, 2021

Looks like the build is failing when executing src/ubuntu/18.04/cross/arm64-alpine/hooks/pre-build

ERROR: unsatisfiable constraints:
  so:libLLVM-11.so (missing):
    required by: lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
                 lldb-11.1.0-r1[so:libLLVM-11.so]
  so:libclang-cpp.so.11.1 (missing):
    required by:
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]
                 lldb-11.1.0-r1[so:libclang-cpp.so.11.1]

@mthalman
Copy link
Member

mthalman commented May 5, 2021

@janvorli
Copy link
Member Author

janvorli commented May 6, 2021

Thanks Matt, I'll fix the timeout and look into the LLVM issue.

@janvorli
Copy link
Member Author

janvorli commented Jul 9, 2021

@mthalman do you understand the failures in the CI? I am not sure what's the issue. I can see a lot of "keyserver receive failed" messages, but I am not sure if that's the real issue here.

@mthalman
Copy link
Member

mthalman commented Jul 9, 2021

I think the keyserver error message is relevant. Perhaps something has been deprecated in there? I think those commands were lifted from the Node Dockerfiles. Here's the Node 10 Dockerfile: https://github.com/nodejs/docker-node/blob/node-core/10/alpine/Dockerfile. Would be good to compare that to what we have.

The Ubuntu build is failing when running the src/ubuntu/18.04/cross/s390x/hooks/pre-build script. It's attempting to install the libomp5 package for the s390x arch but no such package exists for that arch: https://packages.ubuntu.com/search?suite=bionic&arch=s390x&searchon=names&keywords=libomp. I guess it must have at one point?

@janvorli
Copy link
Member Author

janvorli commented Jul 9, 2021

@mthalman it seems that node is actually having the same problem with the key servers, see the recent issue nodejs/node#39114. Looks like the keys are gone from the key servers.

@janvorli
Copy link
Member Author

janvorli commented Jul 9, 2021

A linked issue had a fix, I've made the change in this PR according to that.
nodejs/docker-node#1500

@janvorli janvorli closed this Jul 12, 2021
@janvorli janvorli reopened this Jul 12, 2021
@janvorli
Copy link
Member Author

@mthalman the centos-7-graph amd64 leg has failed with the following while building image from src/centos/7/rpmpkg/Dockerfile:

2021-07-12T22:59:37.2657333Z Building native extensions.  This could take a while...
2021-07-12T22:59:52.5655495Z �[91mERROR:  Error installing fpm:
2021-07-12T22:59:52.5656272Z 	git requires Ruby version >= 2.3.

The initial part of the dockerfile installs Ruby 2.0. Any idea what could have caused this issue?
It doesn't seem the centos dockerfiles were touched in a relevant way recently and my change also doesn't do anything with Ruby.

@mthalman
Copy link
Member

@mthalman the centos-7-graph amd64 leg has failed with the following while building image from src/centos/7/rpmpkg/Dockerfile:

2021-07-12T22:59:37.2657333Z Building native extensions.  This could take a while...
2021-07-12T22:59:52.5655495Z �[91mERROR:  Error installing fpm:
2021-07-12T22:59:52.5656272Z 	git requires Ruby version >= 2.3.

The initial part of the dockerfile installs Ruby 2.0. Any idea what could have caused this issue?

CentOS 7 only includes Ruby 2.0. The latest versions of git are dependent on Ruby >=2.3, which is being pulled in as a dependency of fpm. This is the same kind of issue that we dealt with earlier with ffi (#323). To resolve this, you can pin git to a version that doesn't require Ruby 2.3:

To resolve this, add this

git:1.7.0 \

to

&& gem install --no-document \
ffi:1.12.2 \
fpm \

@janvorli
Copy link
Member Author

Great, thank you! Let's hope it was the last tweak needed.

@janvorli
Copy link
Member Author

Ah, yet another keyserver issue, this time for Alpine 3.13.

@mthalman
Copy link
Member

Encountered an error in the Ubuntu build. Here's the last few relevant lines from the log:

2021-07-13T18:48:26.9769680Z cc1plus: warning: -Wabi won't warn about anything [-Wabi]
2021-07-13T18:48:26.9770522Z cc1plus: note: -Wabi warns about differences from the most up-to-date ABI, which is also used by default
2021-07-13T18:48:26.9771351Z cc1plus: note: use e.g. -Wabi=11 to warn about changes from GCC 7
2021-07-13T18:48:26.9772119Z /usr/bin/install: cannot stat 'clearcap.map': No such file or directory
2021-07-13T18:48:26.9772859Z make[2]: [install-clearcap-map] Error 1 (ignored)
2021-07-13T18:48:26.9773687Z --2021-07-13 18:48:19--  https://pkgsrc.joyent.com/packages/SmartOS/2020Q1/x86_64/All/icu-64.2nb2.tgz
2021-07-13T18:48:26.9774330Z Resolving pkgsrc.joyent.com (pkgsrc.joyent.com)... 64.30.129.74
2021-07-13T18:48:26.9774920Z Connecting to pkgsrc.joyent.com (pkgsrc.joyent.com)|64.30.129.74|:443... connected.
2021-07-13T18:48:26.9775498Z HTTP request sent, awaiting response... 301 Moved Permanently
2021-07-13T18:48:26.9776374Z Location: https://us-east.manta.joyent.com/pkgsrc/public/packages/SmartOS/2020Q1/x86_64/All/icu-64.2nb2.tgz [following]
2021-07-13T18:48:26.9777405Z --2021-07-13 18:48:19--  https://us-east.manta.joyent.com/pkgsrc/public/packages/SmartOS/2020Q1/x86_64/All/icu-64.2nb2.tgz
2021-07-13T18:48:26.9778365Z Resolving us-east.manta.joyent.com (us-east.manta.joyent.com)... 64.30.136.21, 64.30.136.19, 64.30.136.18, ...
2021-07-13T18:48:26.9779309Z Connecting to us-east.manta.joyent.com (us-east.manta.joyent.com)|64.30.136.21|:443... connected.
2021-07-13T18:48:26.9779953Z HTTP request sent, awaiting response... 500 Internal Server Error

I'm going to retry the build.

@mthalman
Copy link
Member

Hmm, the dotnet-buildtools-prereqs-docker-eng build ran into the same error. This occurs when executing the src/ubuntu/18.04/cross/illumos/hooks/pre-build script.

@am11
Copy link
Member

am11 commented Jul 13, 2021

I can take a look at illumos failure (pkgsrc url is having some trouble apparently).

@am11
Copy link
Member

am11 commented Jul 13, 2021

I have discussed with upstream on IRC (libera.chat #joyent), there is a temporary issue in manta which is being investigated.

Meanwhile, we have a workaround for it to switch to Germany mirror which is intact. @janvorli, could you please add --use-mirror at the end of

$SCRIPTPATH/../../../../build-scripts/build-rootfs.sh ubuntu-18.04 illumos x64

I will keep an eye on the server status and switch it back to primary once the issue is resolved.

@janvorli
Copy link
Member Author

@am11 I'll add commit with that change once the current legs complete so that I know if there is anything else to fix. The build of the llvm stuff in centos takes a very long time.
Btw, what handles the --use-mirror option? Maybe you know it, but the $SCRIPTPATH/../../../../build-scripts/build-rootfs.sh is not our well known script, but rather a wrapper to that one.

@janvorli
Copy link
Member Author

Seems like we'll need to make this script pass through the extra argument to the build-rootfs.sh from arcade.

@am11
Copy link
Member

am11 commented Jul 13, 2021

The upstream issue has been resolved. So we don't need this PR to be blocked by that. However, they did mentioned that dotnet is producing a lot of traffic for their server, so we can either parmanently switch to the mirror or switch to OpenIndiana packages (rather than SmartOS ones).

@janvorli
Copy link
Member Author

Great, finally the CI is green.

@mthalman mthalman merged commit 9b5bbc2 into dotnet:main Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants