Skip to content

Commit

Permalink
Upgrade to LLVM's master branch (LLVM 7)
Browse files Browse the repository at this point in the history
This commit upgrades the main LLVM submodule to LLVM's current master branch.
The LLD submodule is updated in tandem as well as compiler-builtins.

Along the way support was also added for LLVM 7's new features. This primarily
includes the support for custom section concatenation natively in LLD so we now
add wasm custom sections in LLVM IR rather than having custom support in rustc
itself for doing so.

Some other miscellaneous changes are:

* We now pass `--gc-sections` to `wasm-ld`
* The optimization level is now passed to `wasm-ld`
* A `--stack-first` option is passed to LLD to have stack overflow always cause
  a trap instead of corrupting static data
* The wasm target for LLVM switched to `wasm32-unknown-unknown`.
* The syntax for aligned pointers has changed in LLVM IR and tests are updated
  to reflect this.
* The `thumbv6m-none-eabi` target is disabled due to an [LLVM bug][llbug]

Nowadays we've been mostly only upgrading whenever there's a major release of
LLVM but enough changes have been happening on the wasm target that there's been
growing motivation for quite some time now to upgrade out version of LLD. To
upgrade LLD, however, we need to upgrade LLVM to avoid needing to build yet
another version of LLVM on the builders.

The revision of LLVM in use here is arbitrarily chosen. We will likely need to
continue to update it over time if and when we discover bugs. Once LLVM 7 is
fully released we can switch to that channel as well.

[llbug]: https://bugs.llvm.org/show_bug.cgi?id=37382
  • Loading branch information
alexcrichton committed Jul 10, 2018
1 parent c30acc7 commit 42eb850
Show file tree
Hide file tree
Showing 39 changed files with 202 additions and 198 deletions.
2 changes: 1 addition & 1 deletion src/libcompiler_builtins
2 changes: 0 additions & 2 deletions src/librustc/dep_graph/dep_node.rs
Original file line number Diff line number Diff line change
Expand Up @@ -665,8 +665,6 @@ define_dep_nodes!( <'tcx>

[] InstanceDefSizeEstimate { instance_def: InstanceDef<'tcx> },

[] WasmCustomSections(CrateNum),

[input] Features,

[] ProgramClausesFor(DefId),
Expand Down
7 changes: 1 addition & 6 deletions src/librustc/hir/check_attr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ struct CheckAttrVisitor<'a, 'tcx: 'a> {
impl<'a, 'tcx> CheckAttrVisitor<'a, 'tcx> {
/// Check any attribute.
fn check_attributes(&self, item: &hir::Item, target: Target) {
if target == Target::Fn {
if target == Target::Fn || target == Target::Const {
self.tcx.codegen_fn_attrs(self.tcx.hir.local_def_id(item.id));
} else if let Some(a) = item.attrs.iter().find(|a| a.check_name("target_feature")) {
self.tcx.sess.struct_span_err(a.span, "attribute should be applied to a function")
Expand Down Expand Up @@ -85,11 +85,6 @@ impl<'a, 'tcx> CheckAttrVisitor<'a, 'tcx> {
if target != Target::Const {
self.tcx.sess.span_err(attr.span, "only allowed on consts");
}

if attr.value_str().is_none() {
self.tcx.sess.span_err(attr.span, "must be of the form \
#[wasm_custom_section = \"foo\"]");
}
}
}

Expand Down
2 changes: 2 additions & 0 deletions src/librustc/hir/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2259,6 +2259,7 @@ pub struct CodegenFnAttrs {
pub export_name: Option<Symbol>,
pub target_features: Vec<Symbol>,
pub linkage: Option<Linkage>,
pub wasm_custom_section: Option<Symbol>,
}

bitflags! {
Expand All @@ -2283,6 +2284,7 @@ impl CodegenFnAttrs {
export_name: None,
target_features: vec![],
linkage: None,
wasm_custom_section: None,
}
}

Expand Down
1 change: 1 addition & 0 deletions src/librustc/ich/impls_hir.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1120,6 +1120,7 @@ impl_stable_hash_for!(struct hir::CodegenFnAttrs {
export_name,
target_features,
linkage,
wasm_custom_section,
});

impl<'hir> HashStable<StableHashingContext<'hir>> for hir::CodegenFnAttrFlags
Expand Down
8 changes: 6 additions & 2 deletions src/librustc/mir/mono.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ pub enum MonoItem<'tcx> {
Fn(Instance<'tcx>),
Static(DefId),
GlobalAsm(NodeId),
CustomSection(DefId),
}

impl<'tcx> MonoItem<'tcx> {
Expand All @@ -36,7 +37,9 @@ impl<'tcx> MonoItem<'tcx> {
},
// Conservatively estimate the size of a static declaration
// or assembly to be 1.
MonoItem::Static(_) | MonoItem::GlobalAsm(_) => 1,
MonoItem::Static(_) |
MonoItem::GlobalAsm(_) |
MonoItem::CustomSection(_) => 1,
}
}
}
Expand All @@ -51,7 +54,8 @@ impl<'a, 'tcx> HashStable<StableHashingContext<'a>> for MonoItem<'tcx> {
MonoItem::Fn(ref instance) => {
instance.hash_stable(hcx, hasher);
}
MonoItem::Static(def_id) => {
MonoItem::Static(def_id) |
MonoItem::CustomSection(def_id) => {
def_id.hash_stable(hcx, hasher);
}
MonoItem::GlobalAsm(node_id) => {
Expand Down
6 changes: 0 additions & 6 deletions src/librustc/ty/query/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -776,12 +776,6 @@ impl<'tcx> QueryDescription<'tcx> for queries::instance_def_size_estimate<'tcx>
}
}

impl<'tcx> QueryDescription<'tcx> for queries::wasm_custom_sections<'tcx> {
fn describe(_tcx: TyCtxt, _: CrateNum) -> String {
format!("custom wasm sections for a crate")
}
}

impl<'tcx> QueryDescription<'tcx> for queries::generics_of<'tcx> {
#[inline]
fn cache_on_disk(def_id: Self::Key) -> bool {
Expand Down
1 change: 0 additions & 1 deletion src/librustc/ty/query/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -546,7 +546,6 @@ define_queries! { <'tcx>
ty::ParamEnv<'tcx>
) -> Clauses<'tcx>,

[] fn wasm_custom_sections: WasmCustomSections(CrateNum) -> Lrc<Vec<DefId>>,
[] fn wasm_import_module_map: WasmImportModuleMap(CrateNum)
-> Lrc<FxHashMap<DefId, String>>,
}
Expand Down
1 change: 0 additions & 1 deletion src/librustc/ty/query/plumbing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1208,7 +1208,6 @@ pub fn force_from_dep_node<'a, 'gcx, 'lcx>(tcx: TyCtxt<'a, 'gcx, 'lcx>,
DepKind::Features => { force!(features_query, LOCAL_CRATE); }

DepKind::ProgramClausesFor => { force!(program_clauses_for, def_id!()); }
DepKind::WasmCustomSections => { force!(wasm_custom_sections, krate!()); }
DepKind::WasmImportModuleMap => { force!(wasm_import_module_map, krate!()); }
DepKind::ForeignModules => { force!(foreign_modules, krate!()); }

Expand Down
31 changes: 1 addition & 30 deletions src/librustc_codegen_llvm/attributes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,8 @@
use std::ffi::{CStr, CString};

use rustc::hir::{self, CodegenFnAttrFlags};
use rustc::hir::CodegenFnAttrFlags;
use rustc::hir::def_id::{DefId, LOCAL_CRATE};
use rustc::hir::itemlikevisit::ItemLikeVisitor;
use rustc::session::Session;
use rustc::session::config::Sanitizer;
use rustc::ty::TyCtxt;
Expand Down Expand Up @@ -222,37 +221,9 @@ pub fn provide(providers: &mut Providers) {
}
};

providers.wasm_custom_sections = |tcx, cnum| {
assert_eq!(cnum, LOCAL_CRATE);
let mut finder = WasmSectionFinder { tcx, list: Vec::new() };
tcx.hir.krate().visit_all_item_likes(&mut finder);
Lrc::new(finder.list)
};

provide_extern(providers);
}

struct WasmSectionFinder<'a, 'tcx: 'a> {
tcx: TyCtxt<'a, 'tcx, 'tcx>,
list: Vec<DefId>,
}

impl<'a, 'tcx: 'a> ItemLikeVisitor<'tcx> for WasmSectionFinder<'a, 'tcx> {
fn visit_item(&mut self, i: &'tcx hir::Item) {
match i.node {
hir::ItemConst(..) => {}
_ => return,
}
if i.attrs.iter().any(|i| i.check_name("wasm_custom_section")) {
self.list.push(self.tcx.hir.local_def_id(i.id));
}
}

fn visit_trait_item(&mut self, _: &'tcx hir::TraitItem) {}

fn visit_impl_item(&mut self, _: &'tcx hir::ImplItem) {}
}

pub fn provide_extern(providers: &mut Providers) {
providers.wasm_import_module_map = |tcx, cnum| {
let mut ret = FxHashMap();
Expand Down
6 changes: 2 additions & 4 deletions src/librustc_codegen_llvm/back/link.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ use rustc::util::common::time;
use rustc::util::fs::fix_windows_verbatim_for_gcc;
use rustc::hir::def_id::CrateNum;
use tempfile::{Builder as TempFileBuilder, TempDir};
use rustc_target::spec::{PanicStrategy, RelroLevel, LinkerFlavor, TargetTriple};
use rustc_target::spec::{PanicStrategy, RelroLevel, LinkerFlavor};
use rustc_data_structures::fx::FxHashSet;
use context::get_reloc_model;
use llvm;
Expand Down Expand Up @@ -837,10 +837,8 @@ fn link_natively(sess: &Session,
}
}

if sess.opts.target_triple == TargetTriple::from_triple("wasm32-unknown-unknown") {
if sess.opts.target_triple.triple() == "wasm32-unknown-unknown" {
wasm::rewrite_imports(&out_filename, &codegen_results.crate_info.wasm_imports);
wasm::add_custom_sections(&out_filename,
&codegen_results.crate_info.wasm_custom_sections);
}
}

Expand Down
37 changes: 35 additions & 2 deletions src/librustc_codegen_llvm/back/linker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ impl LinkerInfo {
LinkerFlavor::Lld(LldFlavor::Wasm) => {
Box::new(WasmLd {
cmd,
sess,
}) as Box<Linker>
}
}
Expand Down Expand Up @@ -919,11 +920,12 @@ fn exported_symbols(tcx: TyCtxt, crate_type: CrateType) -> Vec<String> {
symbols
}

pub struct WasmLd {
pub struct WasmLd<'a> {
cmd: Command,
sess: &'a Session,
}

impl Linker for WasmLd {
impl<'a> Linker for WasmLd<'a> {
fn link_dylib(&mut self, lib: &str) {
self.cmd.arg("-l").arg(lib);
}
Expand Down Expand Up @@ -988,9 +990,20 @@ impl Linker for WasmLd {
}

fn gc_sections(&mut self, _keep_metadata: bool) {
self.cmd.arg("--gc-sections");
}

fn optimize(&mut self) {
self.cmd.arg(match self.sess.opts.optimize {
OptLevel::No => "-O0",
OptLevel::Less => "-O1",
OptLevel::Default => "-O2",
OptLevel::Aggressive => "-O3",
// Currently LLD doesn't support `Os` and `Oz`, so pass through `O2`
// instead.
OptLevel::Size => "-O2",
OptLevel::SizeMin => "-O2"
});
}

fn pgo_gen(&mut self) {
Expand Down Expand Up @@ -1020,8 +1033,28 @@ impl Linker for WasmLd {
// this isn't yet the bottleneck of compilation at all anyway.
self.cmd.arg("--no-threads");

// By default LLD only gives us one page of stack (64k) which is a
// little small. Default to a larger stack closer to other PC platforms
// (1MB) and users can always inject their own link-args to override this.
self.cmd.arg("-z").arg("stack-size=1048576");

// By default LLD's memory layout is:
//
// 1. First, a blank page
// 2. Next, all static data
// 3. Finally, the main stack (which grows down)
//
// This has the unfortunate consequence that on stack overflows you
// corrupt static data and can cause some exceedingly weird bugs. To
// help detect this a little sooner we instead request that the stack is
// placed before static data.
//
// This means that we'll generate slightly larger binaries as references
// to static data will take more bytes in the ULEB128 encoding, but
// stack overflow will be guaranteed to trap as it underflows instead of
// corrupting static data.
self.cmd.arg("--stack-first");

// FIXME we probably shouldn't pass this but instead pass an explicit
// whitelist of symbols we'll allow to be undefined. Unfortunately
// though we can't handle symbols like `log10` that LLVM injects at a
Expand Down
42 changes: 1 addition & 41 deletions src/librustc_codegen_llvm/back/wasm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.

use std::collections::BTreeMap;
use std::fs;
use std::path::Path;
use std::str;
Expand All @@ -24,45 +23,6 @@ const WASM_EXTERNAL_KIND_TABLE: u8 = 1;
const WASM_EXTERNAL_KIND_MEMORY: u8 = 2;
const WASM_EXTERNAL_KIND_GLOBAL: u8 = 3;

/// Append all the custom sections listed in `sections` to the wasm binary
/// specified at `path`.
///
/// LLVM 6 which we're using right now doesn't have the ability to create custom
/// sections in wasm files nor does LLD have the ability to merge these sections
/// into one larger section when linking. It's expected that this will
/// eventually get implemented, however!
///
/// Until that time though this is a custom implementation in rustc to append
/// all sections to a wasm file to the finished product that LLD produces.
///
/// Support for this is landing in LLVM in https://reviews.llvm.org/D43097,
/// although after that support will need to be in LLD as well.
pub fn add_custom_sections(path: &Path, sections: &BTreeMap<String, Vec<u8>>) {
if sections.len() == 0 {
return
}

let wasm = fs::read(path).expect("failed to read wasm output");

// see https://webassembly.github.io/spec/core/binary/modules.html#custom-section
let mut wasm = WasmEncoder { data: wasm };
for (section, bytes) in sections {
// write the `id` identifier, 0 for a custom section
wasm.byte(0);

// figure out how long our name descriptor will be
let mut name = WasmEncoder::new();
name.str(section);

// write the length of the payload followed by all its contents
wasm.u32((bytes.len() + name.data.len()) as u32);
wasm.data.extend_from_slice(&name.data);
wasm.data.extend_from_slice(bytes);
}

fs::write(path, &wasm.data).expect("failed to write wasm output");
}

/// Rewrite the module imports are listed from in a wasm module given the field
/// name to module name mapping in `import_map`.
///
Expand All @@ -80,7 +40,7 @@ pub fn add_custom_sections(path: &Path, sections: &BTreeMap<String, Vec<u8>>) {
///
/// Support for this was added to LLVM in
/// https://github.com/llvm-mirror/llvm/commit/0f32e1365, although support still
/// needs to be added (AFAIK at the time of this writing) to LLD
/// needs to be added, tracked at https://bugs.llvm.org/show_bug.cgi?id=37168
pub fn rewrite_imports(path: &Path, import_map: &FxHashMap<String, String>) {
if import_map.len() == 0 {
return
Expand Down
Loading

0 comments on commit 42eb850

Please sign in to comment.