Skip to content
This repository has been archived by the owner on Nov 21, 2018. It is now read-only.

linux-cross: use glibc-2.14/gcc-4.8 for the arm toolchain #69

Merged
merged 5 commits into from
Mar 13, 2016
Merged

linux-cross: use glibc-2.14/gcc-4.8 for the arm toolchain #69

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2a8a424
linux-cross: use glibc-2.14/gcc-4.8 for the arm toolchain
Mar 10, 2016
2004de6
rename build_arm_compiler to build_compiler
Mar 12, 2016
3b19b3a
merge some commands
Mar 12, 2016
65cdb54
add docs and bump arm gcc to 4.9
Mar 12, 2016
c12d174
change armv6 as the default arch level for both toolchains
Mar 12, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 30 additions & 2 deletions slaves/linux-cross/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ RUN apt-get install -y --force-yes --no-install-recommends \
curl make git wget file \
python-dev python-pip stunnel \
g++ gcc libc6-dev \
gcc-4.7-arm-linux-gnueabi libc6-dev-armel-cross \
gcc-4.7-arm-linux-gnueabihf libc6-dev-armhf-cross \
gcc-4.8-aarch64-linux-gnu libc6-dev-arm64-cross \
gcc-4.8-powerpc-linux-gnu libc6-dev-powerpc-cross \
gcc-4.8-powerpc64le-linux-gnu libc6-dev-ppc64el-cross \
Expand Down Expand Up @@ -38,6 +36,36 @@ RUN pip install buildbot-slave
RUN groupadd -r rustbuild && useradd -r -g rustbuild rustbuild
RUN mkdir /buildslave && chown rustbuild:rustbuild /buildslave

# Install arm cross compiler
# NOTE crosstool-ng can't be executed by root so we execute it under the rustbuild user. /x-tools
# is the crosstool-ng output directory and /build is the crosstool-ng build directory so both must
# be writable by rustbuild
WORKDIR /build
COPY linux-cross/build_toolchain_root.sh /build/
RUN /bin/bash build_toolchain_root.sh && \
mkdir /x-tools && \
chown rustbuild:rustbuild /build && \
chown rustbuild:rustbuild /x-tools
COPY linux-cross/build_toolchain.sh \
linux-cross/arm-linux-gnueabi.config \
linux-cross/arm-linux-gnueabihf.config \
/build/
USER rustbuild
RUN /bin/bash build_toolchain.sh arm-linux-gnueabi && \
/bin/bash build_toolchain.sh arm-linux-gnueabihf
USER root
RUN rm -rf /build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've found that not having a bunch of intermediate artifacts can hugely reduce the size of docker containers, could this be done as part of each step? Some examples I've done in the past are:

RUN /bin/bash build_arm_toolchain.sh arm-linux-gnueabi && rm -rf /build

(or something like that)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've found that not having a bunch of intermediate artifacts can hugely reduce the size of docker containers

Great tip! I'll try to condense these commands as much as possible.


RUN \
for f in `ls /x-tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-*`; do \
g=`basename $f`; \
ln -vs $f /usr/bin/`echo $g | sed -e 's/-unknown//'`; \
done && \
for f in `ls /x-tools/arm-unknown-linux-gnueabihf/bin/arm-unknown-linux-gnueabihf-*`; do \
g=`basename $f`; \
ln -vs $f /usr/bin/`echo $g | sed -e 's/-unknown//'`; \
done

# When running this container, startup buildbot
WORKDIR /buildslave
USER rustbuild
Expand Down
102 changes: 102 additions & 0 deletions slaves/linux-cross/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# `linux-cross`

This image is used to cross compile libstd/rustc to targets that run linux but are not the
`x86_64-unknown-linux-gnu` triple which is the "host" triple.

To cross compile libstd/rustc we need a C cross toolchain: a cross gcc and a cross compiled libc.
For some targets, we use crosstool-ng to build these toolchains ourselves instead of using the ones
packaged for Ubuntu because:

- We can control the glibc version of the toolchain. In particular, we can lower its version as much
as possible, this lets us generate libstd/rustc binaries that run in systems with old glibcs.
- We can create toolchains for targets that don't have an equivalent package available in Ubuntu.

crosstool-ng uses a `.config` file, generated via a menuconfig interface, to specify the target,
glibc version, etc. of the toolchain to build. Because this menuconfig interface requires user
intervention we store pre-generated `.config` files in this repository to keep the `docker build`
command free of user intervention.

The next section explains how to generate a `.config` file for a new target, and the one after that
contains the changes, on top of the default toolchain configuration, used to generate the `.config`
files stored in this repository.

## Generating a `.config` file

If you have a `linux-cross` image lying around you can use that and skip the next two steps.

- First we spin up a container and copy `build_toolchain_root.sh` into it. All these steps are
outside the container:

```
# Note: We use ubuntu:15.10 because that's the "base" of linux-cross Docker image
$ docker run -it ubuntu:15.10 bash
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cfbec05ed730 ubuntu:15.10 "bash" 16 seconds ago Up 15 seconds drunk_murdock
$ docker cp build_toolchain_root.sh drunk_murdock:/
```

- Then inside the container we build crosstool-ng by simply calling the bash script we copied in the
previous step:

```
$ bash build_toolchain_root.sh
```

- Now, inside the container run the following command to configure the toolchain. To get a clue of
which options need to be changed check the next section and come back.

```
$ ct-ng menuconfig
```

- Finally, we retrieve the `.config` file from the container and give it a meaningful name. This is
done outside the container.

```
$ docker drunk_murdock:/.config arm-linux-gnueabi.config
```

- Now you can shutdown the container or repeat the two last steps to generate a new `.config` file.

## Toolchain configuration

Changes on top of the default toolchain configuration used to generate the `.config` files in this
directory. The changes are formatted as follows:

```
$category > $option = $value -- $comment
```

## `arm-linux-gnueabi.config`

For targets: `arm-unknown-linux-gnueabi`

- Path and misc options > Prefix directory = /x-tools/${CT_TARGET}
- Target options > Target Architecture = arm
- Target options > Architecture level = armv5t -- (*)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirming, but this is still armv5t instead of armv6?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same with armv7-a below instead of armv6)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although now that I think about it I'm not sure it matters all that much. This largely only affects glibc, and we're not distributing glibc, so...

I thought you concluded, from this comment, that this default architecture level doesn't affect the C libraries we build, apart from glibc, because we always pass the -march flag to gcc when compiling them. But, perhaps, my guess about your conclusion was wrong!

I'm OK with changing both to armv6 though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah sorry I shall clarify.

So in general this probably doesn't matter if it's too old. This option affects basically only how the support libraries are compiled, like libgcc and glibc. We don't ship glibc, so if you use a different one at runtime which is appropriately compiled, then you'll just naturally pick up the optimizations. Now we do ship some things compiled by gcc, like jemalloc/libbacktrace, but we compile them at rust-build-time which means we can change the codegen there (to armv6). I think that we'll pull in some bits from libgcc statically, however, which here would be compiled with armv5 but only only run on armv6 systems. So with that in mind, it's probably best to compile with armv6.

I realized, though, that this probably matters a lot for C++. We do ship all of libstdc++ as we link it statically to the compiler for the compilers that we ship. So if we start shipping ARM compilers we can perhaps benefit quite a bit from using armv6 here instead of armv5 (because libstdc++ in theory will be faster).

Most of this rationale is the same for armv7 below, although there is probably definitely matters. We're linking to libgcc with Rust and pulling in some bits statically most likely, and if they're armv7 bits and running on an armv6 machine we're likely to run into problems.

So all in all it's probably just best to make sure these are always the same, hopefully it won't bite us later!

- Target options > Floating point = software (no FPU) -- (*)
- Operating System > Target OS = linux
- Operating System > Linux kernel version = 3.2.72 -- Precise kernel
- C-library > glibc version = 2.14.1
- C compiler > gcc version = 4.9.3
- C compiler > C++ = ENABLE -- to cross compile LLVM

## `arm-linux-gnueabihf.config`

For targets: `arm-unknown-linux-gnueabihf`, `armv7-unknown-linux-gnueabihf`

- Path and misc options > Prefix directory = /x-tools/${CT_TARGET}
- Target options > Target Architecture = arm
- Target options > Architecture level = armv7-a -- (*)
- Target options > Use specific FPU = vfpv3-d16 -- (*)
- Target options > Floating point = hardware (FPU) -- (*)
- Target options > Default instruction set mode (thumb) -- (*)
- Operating System > Target OS = linux
- Operating System > Linux kernel version = 3.2.72 -- Precise kernel
- C-library > glibc version = 2.14.1
- C compiler > gcc version = 4.9.3
- C compiler > C++ = ENABLE -- to cross compile LLVM

(*) These options have been selected to match the configuration of the arm toolchains shipped with
Ubuntu 15.10
Loading