Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustix "memory explosion" vulnerability #3338

Closed
brandonmarzolf opened this issue Jul 10, 2024 · 5 comments
Closed

rustix "memory explosion" vulnerability #3338

brandonmarzolf opened this issue Jul 10, 2024 · 5 comments
Labels

Comments

@brandonmarzolf
Copy link

brandonmarzolf commented Jul 10, 2024

Summary

tl;dr: rustix memory vulnerability affects rustix version < 0.38.19, please update to latest rustix version to resolve this vulnerability. What follows is directly from GitHub's Dependabot

When using rustix::fs::Dir using the linux_raw backend, it's possible for the iterator to "get stuck" when an IO error is encountered. Combined with a memory over-allocation issue in rustix::fs::Dir::read_more, this can cause quick and unbounded memory explosion (gigabytes in a few seconds if used on a hot path) and eventually lead to an OOM crash of the application.
Details
Discovery

The symptoms were initially discovered in imsnif/bandwhich#284. That post has lots of details of our investigation. See imsnif/bandwhich#284 (comment) and the Discord thread for details.
Diagnosis

This issue is caused by the combination of two independent bugs:

Stuck iterator

The rustix::fs::Dir iterator can fail to halt after encountering an IO error, causing the caller to be stuck in an infinite loop.

Memory over-allocation

Dir::read_more incorrectly grows the read buffer unconditionally each time it is called, regardless of necessity.

Since

::next calls Dir::read, which in turn calls Dir::read_more, this means an IO error encountered during reading a directory can lead to rapid and unbounded growth of memory use.
PoC

fn main() -> Result<(), Box> {
// create a directory, get a FD to it, then unlink the directory but keep the FD
std::fs::create_dir("tmp_dir")?;
let dir_fd = rustix::fs::openat(
rustix::fs::CWD,
rustix::cstr!("tmp_dir"),
rustix::fs::OFlags::RDONLY | rustix::fs::OFlags::CLOEXEC,
rustix::fs::Mode::empty(),
)?;
std::fs::remove_dir("tmp_dir")?;

// iterator gets stuck in infinite loop and memory explodes
rustix::fs::Dir::read_from(dir_fd)?
    // the iterator keeps returning `Some(Err(_))`, but never halts by returning `None`
    // therefore if the implementation ignores the error (or otherwise continues
    // after seeing the error instead of breaking), the loop will not halt
    .filter_map(|dirent_maybe_error| dirent_maybe_error.ok())
    .for_each(|dirent| {
        // your happy path
        println!("{dirent:?}");
    });

Ok(())

}

Impact

If a program tries to access a directory with its file descriptor after the file has been unlinked (or any other action that leaves the Dir iterator in the stuck state), and the implementation does not break after seeing an error, it can cause a memory explosion.

As an example, Linux's various virtual file systems (e.g. /proc, /sys) can contain directories that spontaneously pop in and out of existence. Attempting to iterate over them using rustix::fs::Dir directly or indirectly (e.g. with the procfs crate) can trigger this fault condition if the implementation decides to continue on errors.

An attacker knowledgeable about the implementation details of a vulnerable target can therefore try to trigger this fault condition via any one or a combination of several available APIs. If successful, the application host will quickly run out of memory, after which the application will likely be terminated by an OOM killer, leading to denial of service.

Info

  • SQLx version: v0.7.4
@dragonnn
Copy link
Contributor

dragonnn commented Jul 10, 2024

rustix 0.38.34 is semver compatible with 0.38.19. If you create a new project with sqlx it will use 0.38.34 instead .19 automatically picked by cargo. If you have an older project just run cargo update and it will fix it.

@brandonmarzolf
Copy link
Author

rustix in sqlx is at 0.37.xx so it will not update. SQLX is using sqlx-core v0.7.4, which is using async-io v1.13, instead of the latest v2.3.3 which contains the latest rustix. Because of it's dependency on the older async-io, it contains this vulnerability, and currently my project has the latest rustix in it, alongside the old 0.37.xx because the sqlx dependencies explicitly call out the older version.

@abonander
Copy link
Collaborator

The problem is async-std. We have to use the same async-io version as it to avoid spawning an extra I/O driver thread, and async-std still uses async-io ^1.

IMO, this is on the rustix authors to backport the fix.

@brandonmarzolf
Copy link
Author

The problem is async-std. We have to use the same async-io version as it to avoid spawning an extra I/O driver thread, and async-std still uses async-io ^1.

IMO, this is on the rustix authors to backport the fix.

That's valid. I'll close the issue. I've opened up this issue with async-std since it is fixable from their library by moving to the latest async-io

@CommanderStorm
Copy link
Contributor

I'll close the issue

@brandonmarzolf
I think you forgot to close the issue ^^
Discussion continuation in async-rs/async-std#1080

@abonander abonander closed this as not planned Won't fix, can't repro, duplicate, stale Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants