rustc: Implement stack probes for x86 #42816

alexcrichton · 2017-06-22T01:38:52Z

This commit implements stack probes on x86/x86_64 using the freshly landed
support upstream in LLVM. The purpose of stack probes here are to guarantee a
segfault on stack overflow rather than having a chance of running over the guard
page already present on all threads by accident.

At this time there's no support for any other architecture because LLVM itself
does not have support for other architectures.

rust-highfive · 2017-06-22T01:39:02Z

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2017-06-22T01:40:06Z

This intends to address #16012 for x86/x86_64, and it also shouldn't land until the upstream diff has landed (although this seems likely to do so, so I figured I may as well go ahead and get this reviewed). Note that this is using a fork of our fork temporarily until the patches are upstreamed, this will not land pointing at a different fork of Rust.

bors · 2017-06-22T02:59:39Z

☔ The latest upstream changes (presumably #42682) made this pull request unmergeable. Please resolve the merge conflicts.

whitequark · 2017-06-22T15:50:45Z

src/libcompiler_builtins/probestack.rs

+    // our goal here is to take %eax and add it to %rsp, but we're also going to
+    // touch each page between %rsp+8 and %rsp+8-%rax
+    //
+    // The ABI here is that the stack frame size is located in `%eax`. Upon


%rax (also on the line below)

I think technically it's %eax because I'm seeing this before calls to probestack:

mov $0x2008,%eax

Although it's essentiall %rax as that zeroes out the top bits anyway.

whitequark · 2017-06-22T15:52:23Z

src/libcompiler_builtins/probestack.rs

+    //
+    // The ABI here is that the stack frame size is located in `%eax`. Upon
+    // return we're not supposed to modify `%esp` or `%eax`. It's unclear
+    // whether we can modify caller-saved registers, but to be on the safe side


According to SystemV x86_64 ABI, %rcx is the 4th function argument, so you'd better save it, no doubt here.

True, but I think the comment still applies. We could pick any caller-saved register for that scratch space (I think) but I'm not sure what LLVM guarantees here. Figured it'd be good to just stay conservative.

We could pick any caller-saved register for that scratch space (I think) but I'm not sure what LLVM guarantees here.

I don't think we're on the same page. The probestack function does not get invoked via the standard function call sequence; the prologue code inserted by LLVM consists of just a call instruction. So anything that's not defined by the ABI as a scratch register has to be saved in it.

Actually, I might be wrong, let me double-check it.

Yes sorry what I mean here is:

At the start of a function, all caller-saved registers are scratch space

Some unknown set of instructions happen

Then there's call __rust_probestack

From what I've seen the "unknown set of instructions" is the empty set, but I'm not sure if that's true 100% of the time with LLVM. If that set of instructions is always empty then all caller-saved registers should be scratch space here as well.

How so? %rcx is used for passing the 4th argument in SysV ABI. If it is not saved by the unknown set of instructions nor __rust_probestack then how will the callee know the value of its 4th argument?

To illustrate. Let's say we translate this LLVM IR:

declare void @use([40096 x i8]*) define i32 @test(i32 %a, i32 %b, i32 %c, i32 %d) "probe-stack"="__probestack" { %array = alloca [40096 x i8], align 16 call void @use([40096 x i8]* %array) ret i32 %d }

This results in the following assembly:

test: # @test pushq %rbx movl $40096, %eax # imm = 0x9CA0 callq __probestack subq %rax, %rsp movl %ecx, %ebx movq %rsp, %rdi callq use movl %ebx, %eax addq $40096, %rsp # imm = 0x9CA0 popq %rbx retq

As you can see %ecx is not saved around the __probestack call. You have to save it there.

This comment is clearly just confusing, I'm going to delete it.

alexcrichton · 2017-06-22T16:17:38Z

Ah also, I've updated our llvm fork now that patches are upstream and that's here, this should be ready to land.

pcwalton · 2017-06-22T17:53:02Z

src/libcompiler_builtins/probestack.rs

+#[cfg(target_arch = "x86_64")]
+pub unsafe extern fn __rust_probestack() {
+    // our goal here is to take %eax and add it to %rsp, but we're also going to
+    // touch each page between %rsp+8 and %rsp+8-%rax


I think this comment isn't accurate anymore — you don't actually modify rsp in this function.

pcwalton · 2017-06-22T17:54:35Z

src/libcompiler_builtins/probestack.rs

+#[no_mangle]
+#[cfg(all(target_arch = "x86", windows))]
+pub unsafe extern fn __rust_probestack() {
+    // This is similar to 32-bit unix but we're going to actually perform the


I'm pretty sure you don't need this function: just omit the probe-stack attribute on Windows and LLVM will automatically add a call to Microsoft's implementation. It's mandated by the Windows ABI, so LLVM has to do that.

pcwalton · 2017-06-22T17:55:10Z

src/librustc_back/target/x86_64_pc_windows_gnu.rs

@@ -16,6 +16,7 @@ pub fn target() -> TargetResult {
    base.cpu = "x86-64".to_string();
    base.pre_link_args.get_mut(&LinkerFlavor::Gcc).unwrap().push("-m64".to_string());
    base.max_atomic_width = Some(64);
+    base.stack_probes = true;


As above, I don't think you need this.

pcwalton · 2017-06-22T17:55:53Z

src/librustc_back/target/x86_64_pc_windows_msvc.rs

@@ -15,6 +15,7 @@ pub fn target() -> TargetResult {
    let mut base = super::windows_msvc_base::opts();
    base.cpu = "x86-64".to_string();
    base.max_atomic_width = Some(64);
+    base.stack_probes = true;


As above, I don't think you need this.

pcwalton · 2017-06-22T17:56:10Z

src/librustc_back/target/i686_pc_windows_msvc.rs

@@ -15,6 +15,7 @@ pub fn target() -> TargetResult {
    let mut base = super::windows_msvc_base::opts();
    base.cpu = "pentium4".to_string();
    base.max_atomic_width = Some(64);
+    base.stack_probes = true;


As above, I don't think you need this.

pcwalton · 2017-06-22T17:56:14Z

src/librustc_back/target/i686_pc_windows_gnu.rs

@@ -21,6 +21,7 @@ pub fn target() -> TargetResult {
    // space available to x86 Windows binaries on x86_64.
    base.pre_link_args
        .get_mut(&LinkerFlavor::Gcc).unwrap().push("-Wl,--large-address-aware".to_string());
+    base.stack_probes = true;


As above, I don't think you need this.

steveklabnik · 2017-06-22T18:41:42Z

src/libcompiler_builtins/probestack.rs

+//! guard page. If a function did not have a stack probe then there's a risk of
+//! having a stack frame *larger* than the guard page, so a function call could
+//! skip over the guard page entirely and then later hit maybe the heap,
+//! possibly leading to a security vulnerability.


worth maybe doing

//! possibly leading to security vulnerabilities such as [The Stack Clash], for example. //! //! [The Stack Clash]: https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash

pcwalton · 2017-06-22T19:10:49Z

src/libcompiler_builtins/probestack.rs

+    // The ABI here is that the stack frame size is located in `%eax`. Upon
+    // return we're not supposed to modify `%esp` or `%eax`.
+    asm!("
+        push   %rcx


nit: I think you can use r11 here and avoid this spill, since r11 is a designated scratch register.

whitequark · 2017-06-22T20:32:19Z

src/libcompiler_builtins/probestack.rs

+//! thread has a guard page then a stack overflow is guaranteed to hit that
+//! guard page. If a function did not have a stack probe then there's a risk of
+//! having a stack frame *larger* than the guard page, so a function call could
+//! skip over the guard page entirely and then later hit maybe the heap,


It's not just this issue. The Stack Clash vulnerability is not necessarily easy to trigger, especially on amd64, but if you have secondary threads, their stacks is just anonymous memory that is very likely to be close to heap in the first place. Even worse if the threads have small stacks.

whitequark · 2017-06-22T20:33:31Z

src/libcompiler_builtins/probestack.rs

+//! Note that `#[naked]` is typically used here for the stack probe because the
+//! ABI corresponds to no actual ABI. Additionally it means that on Windows we
+//! don't have to maintain assembly for MSVC and MinGW, we can just have one
+//! blob of assembly.


The Windows remark is no longer relevant.

pcwalton · 2017-06-23T01:20:57Z

src/libcompiler_builtins/probestack.rs

+        cmp    $$0x1000,%rax
+        ja     2b
+
+        // Finish up the last remainings stack space requested, getting the last


uber-nit: "remainings" -> "remaining"

bors · 2017-06-23T07:40:41Z

☔ The latest upstream changes (presumably #42828) made this pull request unmergeable. Please resolve the merge conflicts.

whitequark · 2017-06-23T19:00:46Z

Heads-up, you need to cherry-pick r306142.

@pcwalton Here goes a better part of my evening... that STI.isTargetWin32() definition is downright evil.

pftbest · 2017-06-26T11:15:42Z

Sorry for interrupting, I would like to submit a patch for MSP430, but it sits on top of the changes from this pull request. Do I need to wait until it's merged?

alexcrichton · 2017-07-06T02:41:05Z

@bors: r=nikomatsakis

bors · 2017-07-06T02:41:06Z

📌 Commit 51b54ca has been approved by nikomatsakis

bors · 2017-07-06T04:50:25Z

🔒 Merge conflict

bors · 2017-07-06T04:52:40Z

☔ The latest upstream changes (presumably #42899) made this pull request unmergeable. Please resolve the merge conflicts.

Will be required for rust-lang/rust#42816

Add `__rust_probestack` intrinsic Will be required for rust-lang/rust#42816

Will be required for rust-lang/rust#42816

Add `__rust_probestack` intrinsic Will be required for rust-lang/rust#42816

This commit implements stack probes on x86/x86_64 using the freshly landed support upstream in LLVM. The purpose of stack probes here are to guarantee a segfault on stack overflow rather than having a chance of running over the guard page already present on all threads by accident. At this time there's no support for any other architecture because LLVM itself does not have support for other architectures.

alexcrichton · 2017-07-06T15:58:36Z

@bors: r=nikomatsakis

bors · 2017-07-06T15:58:37Z

📌 Commit 5dbd97d has been approved by nikomatsakis

bors · 2017-07-06T17:15:20Z

⌛ Testing commit 5dbd97d with merge cd72f2e...

rustc: Implement stack probes for x86 This commit implements stack probes on x86/x86_64 using the freshly landed support upstream in LLVM. The purpose of stack probes here are to guarantee a segfault on stack overflow rather than having a chance of running over the guard page already present on all threads by accident. At this time there's no support for any other architecture because LLVM itself does not have support for other architectures.

bors · 2017-07-06T19:51:29Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing cd72f2e to master...

@alexcrichton

Retry downloading llvm commit tarball As promised on #42816 (comment) r? @alexcrichton

rust-highfive assigned nikomatsakis Jun 22, 2017

alexcrichton mentioned this pull request Jun 22, 2017

Replace stack overflow checking with stack probes #16012

Closed

alexcrichton force-pushed the probestack branch 3 times, most recently from 70a3f8f to 16098e7 Compare June 22, 2017 02:44

alexcrichton force-pushed the probestack branch 3 times, most recently from 9727e7a to 0c33a2e Compare June 22, 2017 04:48

alexcrichton added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 22, 2017

whitequark reviewed Jun 22, 2017

View reviewed changes

alexcrichton force-pushed the probestack branch from 0c33a2e to eada831 Compare June 22, 2017 15:55

pcwalton reviewed Jun 22, 2017

View reviewed changes

alexcrichton force-pushed the probestack branch from eada831 to ba021e5 Compare June 22, 2017 18:16

steveklabnik reviewed Jun 22, 2017

View reviewed changes

alexcrichton force-pushed the probestack branch from ba021e5 to 027321e Compare June 22, 2017 19:10

pcwalton reviewed Jun 22, 2017

View reviewed changes

alexcrichton force-pushed the probestack branch from 027321e to 4e5eb77 Compare June 22, 2017 19:14

whitequark reviewed Jun 22, 2017

View reviewed changes

pcwalton reviewed Jun 23, 2017

View reviewed changes

alexcrichton force-pushed the probestack branch from 4e5eb77 to 715eca5 Compare June 23, 2017 14:02

alexcrichton force-pushed the probestack branch from 715eca5 to baa3218 Compare June 24, 2017 17:30

alexcrichton added a commit to alexcrichton/compiler-builtins that referenced this pull request Jul 6, 2017

Add __rust_probestack intrinsic

f638229

Will be required for rust-lang/rust#42816

alexcrichton mentioned this pull request Jul 6, 2017

Add __rust_probestack intrinsic rust-lang/compiler-builtins#175

Merged

bors added a commit to rust-lang/compiler-builtins that referenced this pull request Jul 6, 2017

Auto merge of #175 - alexcrichton:probestack, r=alexcrichton

7e3aa90

Add `__rust_probestack` intrinsic Will be required for rust-lang/rust#42816

alexcrichton added a commit to alexcrichton/compiler-builtins that referenced this pull request Jul 6, 2017

Add __rust_probestack intrinsic

7ccf840

Will be required for rust-lang/rust#42816

bors added a commit to rust-lang/compiler-builtins that referenced this pull request Jul 6, 2017

Auto merge of #175 - alexcrichton:probestack, r=alexcrichton

e9b258b

Add `__rust_probestack` intrinsic Will be required for rust-lang/rust#42816

alexcrichton force-pushed the probestack branch from 51b54ca to 5dbd97d Compare July 6, 2017 15:58

aidanhs mentioned this pull request Jul 6, 2017

Retry downloading llvm commit tarball #43092

Merged

bors merged commit 5dbd97d into rust-lang:master Jul 6, 2017

alexcrichton deleted the probestack branch July 6, 2017 19:56

bors added a commit that referenced this pull request Jul 6, 2017

Auto merge of #43092 - aidanhs:aphs-retry-llvm-tarball, r=alexcrichton

696412d

Retry downloading llvm commit tarball As promised on #42816 (comment) r? @alexcrichton

est31 mentioned this pull request Jul 7, 2017

Since last nightly, winit doesn't create windows any more #43102

Closed

colin-kiegel mentioned this pull request Jul 12, 2017

August 2017 Rustaceans/rust-cologne#35

Closed

10 tasks

bstrie mentioned this pull request Jul 14, 2017

Extend stack probe support to non-tier-1 platforms, and clarify policy for mitigating LLVM-dependent unsafety #43241

Open

killercup mentioned this pull request Aug 11, 2017

September 2017 Rustaceans/rust-cologne#37

Closed

10 tasks

kennytm mentioned this pull request Aug 20, 2017

ThreadSanitizer broke between nightly build 07-06 and 07-07 #44002

Closed

alexcrichton mentioned this pull request Aug 30, 2017

Rust 1.20.0 release post rust-lang/blog.rust-lang.org#192

Merged

kennytm mentioned this pull request Nov 15, 2017

Enable TrapUnreachable in LLVM. #45920

Merged

kennytm mentioned this pull request Jan 22, 2018

Performance regressions of compiled code over the last year #47561

Open

goffrie mentioned this pull request Dec 22, 2019

Remove mem::uninitalized from tests #67507

Merged

kubo39 mentioned this pull request Jun 23, 2020

Implement stack probe for x86 crystal-lang/crystal#9535

Closed

3 tasks

rustc: Implement stack probes for x86 #42816

rustc: Implement stack probes for x86 #42816

Conversation

alexcrichton commented Jun 22, 2017

rust-highfive commented Jun 22, 2017

alexcrichton commented Jun 22, 2017

bors commented Jun 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Jun 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steveklabnik Jun 22, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitequark Jun 22, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Jun 23, 2017

whitequark commented Jun 23, 2017

pftbest commented Jun 26, 2017

alexcrichton commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 6, 2017

alexcrichton commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 6, 2017

bors commented Jul 6, 2017

steveklabnik Jun 22, 2017 •

edited

Loading

whitequark Jun 22, 2017 •

edited

Loading