auto merge of #13513 : alexcrichton/rust/up-llvm, r=brson

This is a bit of an interesting upgrade to LLVM. Upstream LLVM has started using C++11 features, so they require a C++11 compiler to build. I've updated all the bots to have a C++11 compiler, and they appear to be building LLVM successfully:

* Linux bots - I added gcc/g++ 4.7 (good enough)
* Android bots - same as the linux ones
* Mac bots - I installed the most recent command line tools for Lion which gives us clang 3.2, but LLVM wouldn't build unless it was explicitly asked to link to `libc++` instead of `libstdc++`. This involved tweaking `mklldeps.py` and the `configure` script to get things to work out
* Windows bots - mingw-w64 has gcc 4.8.1 which is sufficient for building LLVM (hurray!)
* BSD bots - I updated FreeBSD to 10.0 which brought with it a relevant version of clang.

The largest fallout I've seen so far is that the test suite doesn't work at all on FreeBSD 10. We've already stopped gating on FreeBSD due to #13427 (we used to be on freebsd 9), so I don't think this puts us in too bad of a situation. I will continue to attempt to fix FreeBSD and the breakage on there.

The LLVM update brings with it all of the recently upstreamed LLVM patches. We only have one local patch now which is just an optimization, and isn't required to use upstream LLVM. I want to maintain compatibility with LLVM 3.3 and 3.4 while we can, and this upgrade is keeping us up to date with the 3.5 release. Once 3.5 is release we will in theory no longer require a bundled LLVM.

configure

@@ -388,6 +388,7 @@ opt ccache 0 "invoke gcc/clang via ccache to reuse object files between builds"
 opt local-rust 0 "use an installed rustc rather than downloading a snapshot"
 opt pax-flags 0 "apply PaX flags to rustc binaries (required for GRSecurity/PaX-patched kernels)"
 opt inject-std-version 1 "inject the current compiler version of libstd into programs"
+opt llvm-static-stdcpp 0 "statically link to libstdc++ for LLVM"
 opt rpath 1 "build rpaths into rustc itself"
 opt nightly 0 "build nightly packages"
 opt verify-install 1 "verify installed binaries work"
@@ -579,26 +580,32 @@ then
         CFG_ENABLE_CLANG=1
         putvar CFG_ENABLE_CLANG
     else
-        # on OS X, with xcode 5 and newer, certain developers may have
-        # cc, gcc and g++ point to a  mixture of clang and gcc
-        # if so, this will create very strange build errors
-        # this last stanza is to detect some such problems and save the future rust
-        # contributor some time solving that issue.
-        # this detection could be generalized to other OSes aside from OS X
-        # but the issue seems most likely to happen on OS X
-
-        chk_cc () {
-            $1 --version 2> /dev/null | grep -q $2
-        }
-        # check that gcc, cc and g++ all point to the same compiler.
-        # note that for xcode 5, g++ points to clang, not clang++
-        if !((chk_cc gcc clang  && chk_cc g++ clang) ||
-            (chk_cc gcc gcc  &&( chk_cc g++ g++ || chk g++ gcc))) then
-            err "the gcc and g++ in your path point to different compilers.
-Check which versions are in your path with  cc --version and g++ --version.
-To resolve this problem, either fix your PATH  or run configure with --enable-clang"
-        fi
+        if [ $("$CFG_GCC" --version 2>&1 | grep -c ' 4\.[0-6]') -ne 0 ]; then
+            step_msg "older GCC found, using clang instead"
+            CFG_ENABLE_CLANG=1
+            putvar CFG_ENABLE_CLANG
+        else
+            # on OS X, with xcode 5 and newer, certain developers may have
+            # cc, gcc and g++ point to a  mixture of clang and gcc
+            # if so, this will create very strange build errors
+            # this last stanza is to detect some such problems and save the future rust
+            # contributor some time solving that issue.
+            # this detection could be generalized to other OSes aside from OS X
+            # but the issue seems most likely to happen on OS X
+
+            chk_cc () {
+                $1 --version 2> /dev/null | grep -q $2
+            }
+            # check that gcc, cc and g++ all point to the same compiler.
+            # note that for xcode 5, g++ points to clang, not clang++
+            if !((chk_cc gcc clang  && chk_cc g++ clang) ||
+                (chk_cc gcc gcc  &&( chk_cc g++ g++ || chk g++ gcc))) then
+                err "the gcc and g++ in your path point to different compilers.
+    Check which versions are in your path with  cc --version and g++ --version.
+    To resolve this problem, either fix your PATH  or run configure with --enable-clang"
+            fi
 
+        fi
     fi
 fi
 
@@ -921,10 +928,6 @@ do
         LLVM_OPTS="$LLVM_OPTS --disable-terminfo"
         # Try to have LLVM pull in as few dependencies as possible (#9397)
         LLVM_OPTS="$LLVM_OPTS --disable-zlib --disable-libffi"
-        # LLVM says it needs a "new" clang/gcc, but we seem to get by ok with
-        # older versions on the bots. Get by for a little longer by asking it to
-        # not do version detection
-        LLVM_OPTS="$LLVM_OPTS --disable-compiler-version-checks"
 
         # Use win32 native thread/lock apis instead of pthread wrapper.
         # (llvm's configure tries to find pthread first, so we have to disable it explicitly.)
@@ -942,13 +945,15 @@ do
 
             LLVM_CXX_64="ccache clang++ -Qunused-arguments"
             LLVM_CC_64="ccache clang -Qunused-arguments"
+            LLVM_OPTS="$LLVM_OPTS --enable-libcpp"
             ;;
             ("clang")
             LLVM_CXX_32="clang++ -m32 -Qunused-arguments"
             LLVM_CC_32="clang -m32 -Qunused-arguments"
 
             LLVM_CXX_64="clang++ -Qunused-arguments"
             LLVM_CC_64="clang -Qunused-arguments"
+            LLVM_OPTS="$LLVM_OPTS --enable-libcpp"
             ;;
             ("ccache gcc")
             LLVM_CXX_32="ccache g++ -m32"

mk/llvm.mk

@@ -42,16 +42,25 @@ $$(LLVM_STAMP_$(1)): $(S)src/rustllvm/llvm-auto-clean-trigger
 	@$$(call E, make: done cleaning llvm)
 	touch $$@
 
+ifeq ($$(CFG_ENABLE_LLVM_STATIC_STDCPP),1)
+LLVM_STDCPP_LOCATION_$(1) = $$(shell $$(CC_$(1)) $$(CFG_GCCISH_CFLAGS_$(1)) \
+					-print-file-name=libstdc++.a)
+else
+LLVM_STDCPP_LOCATION_$(1) =
+endif
+
 endef
 
 $(foreach host,$(CFG_HOST), \
-    $(eval LLVM_CONFIGS := $(LLVM_CONFIGS) $(LLVM_CONFIG_$(host))))
+ $(eval $(call DEF_LLVM_RULES,$(host))))
 
 $(foreach host,$(CFG_HOST), \
- $(eval $(call DEF_LLVM_RULES,$(host))))
+ $(eval LLVM_CONFIGS := $(LLVM_CONFIGS) $(LLVM_CONFIG_$(host))))
 
 $(S)src/librustc/lib/llvmdeps.rs: \
 		    $(LLVM_CONFIGS) \
-		    $(S)src/etc/mklldeps.py
+		    $(S)src/etc/mklldeps.py \
+		    $(MKFILE_DEPS)
 	$(Q)$(CFG_PYTHON) $(S)src/etc/mklldeps.py \
-	    "$@" "$(LLVM_COMPONENTS)" $(LLVM_CONFIGS)
+		"$@" "$(LLVM_COMPONENTS)" "$(CFG_ENABLE_LLVM_STATIC_STDCPP)" \
+		$(LLVM_CONFIGS)

mk/target.mk

@@ -83,6 +83,7 @@ $$(TLIB$(1)_T_$(2)_H_$(3))/stamp.$(4):				    \
 		$$(WFLAGS_ST$(1)) \
 		-L "$$(RT_OUTPUT_DIR_$(2))" \
 		-L "$$(LLVM_LIBDIR_$(2))" \
+		-L "$$(dir $$(LLVM_STDCPP_LOCATION_$(2)))" \
 		--out-dir $$(@D) $$<
 	@touch $$@
 	$$(call LIST_ALL_OLD_GLOB_MATCHES,\

src/etc/mklldeps.py

@@ -11,11 +11,14 @@
 import os
 import sys
 import subprocess
+import itertools
+from os import path
 
 f = open(sys.argv[1], 'wb')
 
 components = sys.argv[2].split(' ')
 components = [i for i in components if i]  # ignore extra whitespaces
+enable_static = sys.argv[3]
 
 f.write("""// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
 // file at the top-level directory of this distribution and at
@@ -31,11 +34,20 @@
 //          take a look at src/etc/mklldeps.py if you're interested
 """)
 
-for llconfig in sys.argv[3:]:
+def run(args):
+    proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    out, err = proc.communicate()
+
+    if err:
+        print("failed to run llconfig: args = `{}`".format(args))
+        print(err)
+        sys.exit(1)
+    return out
+
+for llconfig in sys.argv[4:]:
     f.write("\n")
 
-    proc = subprocess.Popen([llconfig, '--host-target'], stdout = subprocess.PIPE)
-    out, err = proc.communicate()
+    out = run([llconfig, '--host-target'])
     arch, os = out.split('-', 1)
     arch = 'x86' if arch == 'i686' or arch == 'i386' else arch
     if 'darwin' in os:
@@ -55,35 +67,44 @@
 
     f.write("#[cfg(" + ', '.join(cfg) + ")]\n")
 
+    version = run([llconfig, '--version']).strip()
+
     # LLVM libs
-    args = [llconfig, '--libs']
+    if version < '3.5':
+      args = [llconfig, '--libs']
+    else:
+      args = [llconfig, '--libs', '--system-libs']
     args.extend(components)
-    proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-    out, err = proc.communicate()
-
-    if err:
-        print("failed to run llconfig: args = `{}`".format(args))
-        sys.exit(1)
+    out = run(args)
+    for lib in out.strip().replace("\n", ' ').split(' '):
+        lib = lib.strip()[2:] # chop of the leading '-l'
+        f.write("#[link(name = \"" + lib + "\"")
+        # LLVM libraries are all static libraries
+        if 'LLVM' in lib:
+            f.write(", kind = \"static\"")
+        f.write(")]\n")
 
-    for lib in out.strip().split(' '):
-        lib = lib[2:] # chop of the leading '-l'
-        f.write("#[link(name = \"" + lib + "\", kind = \"static\")]\n")
+    # llvm-config before 3.5 didn't have a system-libs flag
+    if version < '3.5':
+      if os == 'win32':
+        f.write("#[link(name = \"imagehlp\")]")
 
     # LLVM ldflags
-    args = [llconfig, '--ldflags']
-    proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-    out, err = proc.communicate()
-
-    if err:
-        print("failed to run llconfig: args = `{}`".format(args))
-        sys.exit(1)
-
+    out = run([llconfig, '--ldflags'])
     for lib in out.strip().split(' '):
         if lib[:2] == "-l":
             f.write("#[link(name = \"" + lib[2:] + "\")]\n")
 
-    #extra
-    f.write("#[link(name = \"stdc++\")]\n")
-    if os == 'win32':
-        f.write("#[link(name = \"imagehlp\")]\n")
+    # C++ runtime library
+    out = run([llconfig, '--cxxflags'])
+    if enable_static == '1':
+      assert('stdlib=libc++' not in out)
+      f.write("#[link(name = \"stdc++\", kind = \"static\")]\n")
+    else:
+      if 'stdlib=libc++' in out:
+        f.write("#[link(name = \"c++\")]\n")
+      else:
+        f.write("#[link(name = \"stdc++\")]\n")
+
+    # Attach everything to an extern block
     f.write("extern {}\n")

src/librustc/back/link.rs

@@ -1298,9 +1298,22 @@ fn add_local_native_libraries(args: &mut Vec<~str>, sess: &Session) {
         args.push("-L" + path.as_str().unwrap().to_owned());
     }
 
+    // Some platforms take hints about whether a library is static or dynamic.
+    // For those that support this, we ensure we pass the option if the library
+    // was flagged "static" (most defaults are dynamic) to ensure that if
+    // libfoo.a and libfoo.so both exist that the right one is chosen.
+    let takes_hints = sess.targ_cfg.os != abi::OsMacos;
+
     for &(ref l, kind) in sess.cstore.get_used_libraries().borrow().iter() {
         match kind {
             cstore::NativeUnknown | cstore::NativeStatic => {
+                if takes_hints {
+                    if kind == cstore::NativeStatic {
+                        args.push("-Wl,-Bstatic".to_owned());
+                    } else {
+                        args.push("-Wl,-Bdynamic".to_owned());
+                    }
+                }
                 args.push("-l" + *l);
             }
             cstore::NativeFramework => {
@@ -1309,6 +1322,9 @@ fn add_local_native_libraries(args: &mut Vec<~str>, sess: &Session) {
             }
         }
     }
+    if takes_hints {
+        args.push("-Wl,-Bdynamic".to_owned());
+    }
 }
 
 // # Rust Crate linking

src/librustc/lib/llvm.rs

@@ -1261,7 +1261,8 @@ pub mod llvm {
                                       LHS: ValueRef,
                                       CMP: ValueRef,
                                       RHS: ValueRef,
-                                      Order: AtomicOrdering)
+                                      Order: AtomicOrdering,
+                                      FailureOrder: AtomicOrdering)
                                       -> ValueRef;
         pub fn LLVMBuildAtomicRMW(B: BuilderRef,
                                   Op: AtomicBinOp,
@@ -1586,7 +1587,8 @@ pub mod llvm {
                                                Scope: DIDescriptor,
                                                File: DIFile,
                                                Line: c_uint,
-                                               Col: c_uint)
+                                               Col: c_uint,
+                                               Discriminator: c_uint)
                                                -> DILexicalBlock;
 
         pub fn LLVMDIBuilderCreateStaticVariable(Builder: DIBuilderRef,

src/librustc/middle/trans/base.rs

@@ -445,8 +445,8 @@ pub fn set_llvm_fn_attrs(attrs: &[ast::Attribute], llfn: ValueRef) {
     }
 
     // Add the no-split-stack attribute if requested
-    if contains_name(attrs, "no_split_stack") {
-        set_no_split_stack(llfn);
+    if !contains_name(attrs, "no_split_stack") {
+        set_split_stack(llfn);
     }
 
     if contains_name(attrs, "cold") {
@@ -458,8 +458,8 @@ pub fn set_always_inline(f: ValueRef) {
     lib::llvm::SetFunctionAttribute(f, lib::llvm::AlwaysInlineAttribute)
 }
 
-pub fn set_no_split_stack(f: ValueRef) {
-    "no-split-stack".with_c_str(|buf| {
+pub fn set_split_stack(f: ValueRef) {
+    "split-stack".with_c_str(|buf| {
         unsafe { llvm::LLVMAddFunctionAttrString(f, buf); }
     })
 }

src/librustc/middle/trans/build.rs

@@ -814,8 +814,9 @@ pub fn Resume(cx: &Block, exn: ValueRef) -> ValueRef {
 // Atomic Operations
 pub fn AtomicCmpXchg(cx: &Block, dst: ValueRef,
                      cmp: ValueRef, src: ValueRef,
-                     order: AtomicOrdering) -> ValueRef {
-    B(cx).atomic_cmpxchg(dst, cmp, src, order)
+                     order: AtomicOrdering,
+                     failure_order: AtomicOrdering) -> ValueRef {
+    B(cx).atomic_cmpxchg(dst, cmp, src, order, failure_order)
 }
 pub fn AtomicRMW(cx: &Block, op: AtomicBinOp,
                  dst: ValueRef, src: ValueRef,

src/librustc/middle/trans/builder.rs

@@ -949,9 +949,11 @@ impl<'a> Builder<'a> {
     // Atomic Operations
     pub fn atomic_cmpxchg(&self, dst: ValueRef,
                          cmp: ValueRef, src: ValueRef,
-                         order: AtomicOrdering) -> ValueRef {
+                         order: AtomicOrdering,
+                         failure_order: AtomicOrdering) -> ValueRef {
         unsafe {
-            llvm::LLVMBuildAtomicCmpXchg(self.llbuilder, dst, cmp, src, order)
+            llvm::LLVMBuildAtomicCmpXchg(self.llbuilder, dst, cmp, src,
+                                         order, failure_order)
         }
     }
     pub fn atomic_rmw(&self, op: AtomicBinOp,

src/librustc/middle/trans/debuginfo.rs

@@ -278,6 +278,12 @@ pub fn finalize(cx: &CrateContext) {
         if cx.sess().targ_cfg.os == abi::OsMacos {
             "Dwarf Version".with_c_str(
                 |s| llvm::LLVMRustAddModuleFlag(cx.llmod, s, 2));
+        } else {
+            // FIXME(#13611) this is a kludge fix because the linux bots have
+            //               gdb 7.4 which doesn't understand dwarf4, we should
+            //               do something more graceful here.
+            "Dwarf Version".with_c_str(
+                |s| llvm::LLVMRustAddModuleFlag(cx.llmod, s, 3));
         }
 
         // Prevent bitcode readers from deleting the debug info.
@@ -2421,7 +2427,8 @@ fn populate_scope_map(cx: &CrateContext,
                 parent_scope,
                 file_metadata,
                 loc.line as c_uint,
-                loc.col.to_uint() as c_uint)
+                loc.col.to_uint() as c_uint,
+                0)
         };
 
         scope_stack.push(ScopeStackEntry { scope_metadata: scope_metadata, ident: None });
@@ -2538,7 +2545,8 @@ fn populate_scope_map(cx: &CrateContext,
                                 parent_scope,
                                 file_metadata,
                                 loc.line as c_uint,
-                                loc.col.to_uint() as c_uint)
+                                loc.col.to_uint() as c_uint,
+                                0)
                         };
 
                         scope_stack.push(ScopeStackEntry {

src/librustc/middle/trans/intrinsic.rs

@@ -223,10 +223,23 @@ pub fn trans_intrinsic(ccx: &CrateContext,
 
         match *split.get(1) {
             "cxchg" => {
+                // See include/llvm/IR/Instructions.h for their implementation
+                // of this, I assume that it's good enough for us to use for
+                // now.
+                let strongest_failure_ordering = match order {
+                    lib::llvm::NotAtomic | lib::llvm::Unordered =>
+                        ccx.sess().fatal("cmpxchg must be atomic"),
+                    lib::llvm::Monotonic | lib::llvm::Release =>
+                        lib::llvm::Monotonic,
+                    lib::llvm::Acquire | lib::llvm::AcquireRelease =>
+                        lib::llvm::Acquire,
+                    lib::llvm::SequentiallyConsistent =>
+                        lib::llvm::SequentiallyConsistent,
+                };
                 let old = AtomicCmpXchg(bcx, get_param(decl, first_real_arg),
                                         get_param(decl, first_real_arg + 1u),
                                         get_param(decl, first_real_arg + 2u),
-                                        order);
+                                        order, strongest_failure_ordering);
                 Ret(bcx, old);
             }
             "load" => {