Rollup of 7 pull requests

CONTRIBUTING.md

@@ -97,33 +97,38 @@ system internals, try asking in [`#rust-internals`][pound-rust-internals].
 
 Before you can start building the compiler you need to configure the build for
 your system. In most cases, that will just mean using the defaults provided
-for Rust. Configuring involves invoking the `configure` script in the project
-root.
+for Rust.
 
-```
-./configure
-```
+To change configuration, you must copy the file `src/bootstrap/config.toml.example`
+to `config.toml` in the directory from which you will be running the build, and
+change the settings provided.
+
+There are large number of options provided in this config file that will alter the
+configuration used in the build process. Some options to note:
 
-There are large number of options accepted by this script to alter the
-configuration used later in the build process. Some options to note:
+#### `[llvm]`:
+- `ccache = true` - Use ccache when building llvm
 
-- `--enable-debug` - Build a debug version of the compiler (disables optimizations,
-    which speeds up compilation of stage1 rustc)
-- `--enable-optimize` - Enable optimizations (can be used with `--enable-debug`
-    to make a debug build with optimizations)
-- `--disable-valgrind-rpass` - Don't run tests with valgrind
-- `--enable-clang` - Prefer clang to gcc for building dependencies (e.g., LLVM)
-- `--enable-ccache` - Invoke clang/gcc with ccache to re-use object files between builds
-- `--enable-compiler-docs` - Build compiler documentation
+#### `[build]`:
+- `compiler-docs = true` - Build compiler documentation
 
-To see a full list of options, run `./configure --help`.
+#### `[rust]`:
+- `debuginfo = true` - Build a compiler with debuginfo
+- `optimize = false` - Disable optimizations to speed up compilation of stage1 rust
+
+For more options, the `config.toml` file contains commented out defaults, with
+descriptions of what each option will do.
+
+Note: Previously the `./configure` script was used to configure this
+project. It can still be used, but it's recommended to use a `config.toml`
+file. If you still have a `config.mk` file in your directory - from
+`./configure` - you may need to delete it for `config.toml` to work.
 
 ### Building
 
-Although the `./configure` script will generate a `Makefile`, this is actually
-just a thin veneer over the actual build system driver, `x.py`. This file, at
-the root of the repository, is used to build, test, and document various parts
-of the compiler. You can execute it as:
+The build system uses the `x.py` script to control the build process. This script
+is used to build, test, and document various parts of the compiler. You can
+execute it as:
 
 ```sh
 python x.py build
@@ -185,6 +190,9 @@ To learn about all possible rules you can execute, run:
 python x.py build --help --verbose
 ```
 
+Note: Previously `./configure` and `make` were used to build this project.
+They are still available, but `x.py` is the recommended build system.
+
 ### Useful commands
 
 Some common invocations of `x.py` are:
@@ -235,8 +243,8 @@ feature. We use the 'fork and pull' model described there.
 
 Please make pull requests against the `master` branch.
 
-Compiling all of `make check` can take a while. When testing your pull request,
-consider using one of the more specialized `make` targets to cut down on the
+Compiling all of `./x.py test` can take a while. When testing your pull request,
+consider using one of the more specialized `./x.py` targets to cut down on the
 amount of time you have to wait. You need to have built the compiler at least
 once before running these will work, but that’s only one full build rather than
 one each time.
@@ -307,7 +315,7 @@ To find documentation-related issues, sort by the [A-docs label][adocs].
 
 [adocs]: https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AA-docs
 
-In many cases, you don't need a full `make doc`. You can use `rustdoc` directly
+In many cases, you don't need a full `./x.py doc`. You can use `rustdoc` directly
 to check small fixes. For example, `rustdoc src/doc/reference.md` will render
 reference to `doc/reference.html`. The CSS might be messed up, but you can
 verify that the HTML is right.

README.md

@@ -35,15 +35,15 @@ Read ["Installing Rust"] from [The Book].
 3. Build and install:
 
     ```sh
-    $ ./configure
-    $ make && sudo make install
+    $ ./x.py build && sudo ./x.py dist --install
     ```
 
-    > ***Note:*** Install locations can be adjusted by passing a `--prefix`
-    > argument to `configure`. Various other options are also supported – pass
-    > `--help` for more information on them.
+    > ***Note:*** Install locations can be adjusted by copying the config file
+    > from `./src/bootstrap/config.toml.example` to `./config.toml`, and
+    > adjusting the `prefix` option under `[install]`. Various other options are
+    > also supported, and are documented in the config file.
 
-    When complete, `sudo make install` will place several programs into
+    When complete, `sudo ./x.py dist --install` will place several programs into
     `/usr/local/bin`: `rustc`, the Rust compiler, and `rustdoc`, the
     API-documentation tool. This install does not include [Cargo],
     Rust's package manager, which you may also want to build.
@@ -59,7 +59,6 @@ for interop with software produced by Visual Studio use the MSVC build of Rust;
 for interop with GNU software built using the MinGW/MSYS2 toolchain use the GNU
 build.
 
-
 #### MinGW
 
 [MSYS2][msys2] can be used to easily build Rust on Windows:
@@ -94,11 +93,10 @@ build.
                mingw-w64-x86_64-gcc
    ```
 
-4. Navigate to Rust's source code (or clone it), then configure and build it:
+4. Navigate to Rust's source code (or clone it), then build it:
 
    ```sh
-   $ ./configure
-   $ make && make install
+   $ ./x.py build && ./x.py dist --install
    ```
 
 #### MSVC
@@ -114,13 +112,6 @@ shell with:
 > python x.py build
 ```
 
-If you're running inside of an msys shell, however, you can run:
-
-```sh
-$ ./configure --build=x86_64-pc-windows-msvc
-$ make && make install
-```
-
 Currently building Rust only works with some known versions of Visual Studio. If
 you have a more recent version installed the build system doesn't understand
 then you may need to force rustbuild to use an older version. This can be done
@@ -131,13 +122,43 @@ CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\vcvars64.
 python x.py build
 ```
 
+#### Specifying an ABI
+
+Each specific ABI can also be used from either environment (for example, using
+the GNU ABI in powershell) by using an explicit build triple. The available
+Windows build triples are:
+- GNU ABI (using GCC)
+    - `i686-pc-windows-gnu`
+    - `x86_64-pc-windows-gnu`
+- The MSVC ABI
+    - `i686-pc-windows-msvc`
+    - `x86_64-pc-windows-msvc`
+
+The build triple can be specified by either specifying `--build=ABI` when
+invoking `x.py` commands, or by copying the `config.toml` file (as described
+in Building From Source), and modifying the `build` option under the `[build]`
+section.
+
+### Configure and Make
+
+While it's not the recommended build system, this project also provides a
+configure script and makefile (the latter of which just invokes `x.py`).
+
+```sh
+$ ./configure
+$ make && sudo make install
+```
+
+When using the configure script, the generated config.mk` file may override the
+`config.toml` file. To go back to the `config.toml` file, delete the generated
+`config.mk` file.
+
 ## Building Documentation
 
 If you’d like to build the documentation, it’s almost the same:
 
 ```sh
-$ ./configure
-$ make docs
+$ ./x.py doc
 ```
 
 The generated documentation will appear in a top-level `doc` directory,

src/ci/docker/cross/Dockerfile

@@ -3,12 +3,12 @@ FROM ubuntu:16.04
 RUN apt-get update && apt-get install -y --no-install-recommends \
   g++ \
   make \
+  ninja-build \
   file \
   curl \
   ca-certificates \
-  python2.7 \
+  python2.7-dev \
   git \
-  cmake \
   sudo \
   xz-utils \
   zlib1g-dev \
@@ -19,7 +19,15 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   bzip2 \
   patch \
   libssl-dev \
-  pkg-config
+  pkg-config \
+  swig \
+  libedit-dev \
+  libncurses5-dev
+
+# CMake 3.8.0 is the first version with official support for Fuchsia,
+# which is needed to build the Fuchsia toolchain.
+RUN curl -L https://cmake.org/files/v3.8/cmake-3.8.0-rc1-Linux-x86_64.tar.gz | \
+      tar xzf - -C /usr/local --strip-components=1
 
 RUN curl -o /usr/local/bin/sccache \
       https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-02-24-sccache-x86_64-unknown-linux-gnu && \
@@ -31,6 +39,10 @@ RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-ini
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 WORKDIR /tmp
+COPY shared.sh /tmp/
+
+COPY build-fuchsia.sh /tmp/
+RUN ./build-fuchsia.sh
 
 COPY build-rumprun.sh /tmp/
 RUN ./build-rumprun.sh
@@ -65,10 +77,18 @@ ENV TARGETS=$TARGETS,arm-unknown-linux-musleabi
 ENV TARGETS=$TARGETS,arm-unknown-linux-musleabihf
 ENV TARGETS=$TARGETS,armv7-unknown-linux-musleabihf
 ENV TARGETS=$TARGETS,sparc64-unknown-linux-gnu
+ENV TARGETS=$TARGETS,x86_64-unknown-fuchsia
+ENV TARGETS=$TARGETS,aarch64-unknown-fuchsia
 
 ENV CC_mipsel_unknown_linux_musl=mipsel-openwrt-linux-gcc \
     CC_mips_unknown_linux_musl=mips-openwrt-linux-gcc \
-    CC_sparc64_unknown_linux_gnu=sparc64-linux-gnu-gcc
+    CC_sparc64_unknown_linux_gnu=sparc64-linux-gnu-gcc \
+    AR_x86_64_unknown_fuchsia=x86_64-unknown-fuchsia-ar \
+    CC_x86_64_unknown_fuchsia=x86_64-unknown-fuchsia-clang \
+    CXX_x86_64_unknown_fuchsia=x86_64-unknown-fuchsia-clang++ \
+    AR_aarch64_unknown_fuchsia=aarch64-unknown-fuchsia-ar \
+    CC_aarch64_unknown_fuchsia=aarch64-unknown-fuchsia-clang \
+    CXX_aarch64_unknown_fuchsia=aarch64-unknown-fuchsia-clang++
 
 # Suppress some warnings in the openwrt toolchains we downloaded
 ENV STAGING_DIR=/tmp

src/ci/docker/cross/build-fuchsia.sh

@@ -0,0 +1,114 @@
+#!/bin/bash
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+source shared.sh
+
+# Download sources
+SRCS=(
+  "https://fuchsia.googlesource.com/magenta magenta ac69119"
+  "https://fuchsia.googlesource.com/third_party/llvm llvm 5463083"
+  "https://fuchsia.googlesource.com/third_party/clang llvm/tools/clang 4ff7b4b"
+  "https://fuchsia.googlesource.com/third_party/lld llvm/tools/lld fd465a3"
+  "https://fuchsia.googlesource.com/third_party/lldb llvm/tools/lldb 6bb11f8"
+  "https://fuchsia.googlesource.com/third_party/compiler-rt llvm/runtimes/compiler-rt 52d4ecc"
+  "https://fuchsia.googlesource.com/third_party/libcxx llvm/runtimes/libcxx e891cc8"
+  "https://fuchsia.googlesource.com/third_party/libcxxabi llvm/runtimes/libcxxabi f0f0257"
+  "https://fuchsia.googlesource.com/third_party/libunwind llvm/runtimes/libunwind 50bddc1"
+)
+
+fetch() {
+  mkdir -p $2
+  pushd $2 > /dev/null
+  curl -sL $1/+archive/$3.tar.gz | tar xzf -
+  popd > /dev/null
+}
+
+for i in "${SRCS[@]}"; do
+  fetch $i
+done
+
+# Build toolchain
+cd llvm
+mkdir build
+cd build
+hide_output cmake -GNinja \
+  -DFUCHSIA_SYSROOT=${PWD}/../../magenta/third_party/ulib/musl \
+  -C ../tools/clang/cmake/caches/Fuchsia.cmake \
+  ..
+hide_output ninja stage2-distribution
+hide_output ninja stage2-install-distribution
+cd ../..
+
+# Build sysroot
+rm -rf llvm/runtimes/compiler-rt
+./magenta/scripts/download-toolchain
+
+build_sysroot() {
+  local arch="$1"
+
+  case "${arch}" in
+    x86_64) tgt="magenta-pc-x86-64" ;;
+    aarch64) tgt="magenta-qemu-arm64" ;;
+  esac
+
+  hide_output make -C magenta -j$(getconf _NPROCESSORS_ONLN) $tgt
+  dst=/usr/local/${arch}-unknown-fuchsia
+  mkdir -p $dst
+  cp -r magenta/build-${tgt}/sysroot/include $dst/
+  cp -r magenta/build-${tgt}/sysroot/lib $dst/
+
+  cd llvm
+  mkdir build-runtimes-${arch}
+  cd build-runtimes-${arch}
+  hide_output cmake -GNinja \
+    -DCMAKE_C_COMPILER=clang \
+    -DCMAKE_CXX_COMPILER=clang++ \
+    -DCMAKE_AR=/usr/local/bin/llvm-ar \
+    -DCMAKE_RANLIB=/usr/local/bin/llvm-ranlib \
+    -DCMAKE_INSTALL_PREFIX= \
+    -DLLVM_MAIN_SRC_DIR=${PWD}/.. \
+    -DLLVM_BINARY_DIR=${PWD}/../build \
+    -DLLVM_ENABLE_WERROR=OFF \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DLLVM_INCLUDE_TESTS=ON \
+    -DCMAKE_SYSTEM_NAME=Fuchsia \
+    -DCMAKE_C_COMPILER_TARGET=${arch}-fuchsia \
+    -DCMAKE_CXX_COMPILER_TARGET=${arch}-fuchsia \
+    -DUNIX=1 \
+    -DLIBCXX_HAS_MUSL_LIBC=ON \
+    -DLIBCXXABI_USE_LLVM_UNWINDER=ON \
+    -DCMAKE_SYSROOT=${dst} \
+    -DCMAKE_C_COMPILER_FORCED=TRUE \
+    -DCMAKE_CXX_COMPILER_FORCED=TRUE \
+    -DLLVM_ENABLE_LIBCXX=ON \
+    -DCMAKE_EXE_LINKER_FLAGS="-nodefaultlibs -lc" \
+    -DCMAKE_SHARED_LINKER_FLAGS="$(clang --target=${arch}-fuchsia -print-libgcc-file-name)" \
+    ../runtimes
+  hide_output env DESTDIR="${dst}" ninja install
+  cd ../..
+}
+
+build_sysroot "x86_64"
+build_sysroot "aarch64"
+
+rm -rf magenta llvm
+
+for arch in x86_64 aarch64; do
+  for tool in clang clang++; do
+    cat >/usr/local/bin/${arch}-unknown-fuchsia-${tool} <<EOF
+#!/bin/sh
+${tool} --target=${arch}-unknown-fuchsia --sysroot=/usr/local/${arch}-unknown-fuchsia "\$@"
+EOF
+    chmod +x /usr/local/bin/${arch}-unknown-fuchsia-${tool}
+  done
+  ln -s /usr/local/bin/llvm-ar /usr/local/bin/${arch}-unknown-fuchsia-ar
+done

src/ci/docker/cross/shared.sh

@@ -0,0 +1,25 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+hide_output() {
+  set +x
+  on_err="
+echo ERROR: An error was encountered with the build.
+cat /tmp/build.log
+exit 1
+"
+  trap "$on_err" ERR
+  bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
+  PING_LOOP_PID=$!
+  "$@" &> /tmp/build.log
+  trap - ERR
+  kill $PING_LOOP_PID
+  set -x
+}