Use statx on Linux

[ariasuni]

Fix #59743

Using statx on Linux notably gives us access to stx_btime so we can return a value for Metadata::created().

I implemented the feature but need time to improve the code:

• Update the
• Find how to fallback in an efficient way when glibc version < 2.28
  • stat64 and statx returns different structs, and it seems that right now it’s not possible to know the glibc version at compile-time. I can make FileAttr an enum which can be either one.
  • Should I use sys::os::glibc_version() or something like

    rust/src/libstd/sys/unix/fast_thread_local.rs

    Line 24 in 9ebf478

    if !__cxa_thread_atexit_impl.is_null() {

• I added st_{r,}dev_{major,minor}, but I’m not sure it’s the proper way to do it.
• Being able to access all fields of statx as functions like the other information is probably something that you’d like to have.

[rust-highfive]

r? @KodrAus

(rust_highfive has picked a reviewer for you, use r? to override)

[xen0n]

If you get -ENOSYS when calling statx(2) first, that means the syscall is not implemented so you can fallback to stat(2). You probably don't want to detect for this every time stat is called, though.

[bors]

☔ The latest upstream changes (presumably #61408) made this pull request unmergeable. Please resolve the merge conflicts.

[ariasuni]

@xen0n Do you suggest using the syscall instead of the glibc wrapper?

You probably don't want to detect for this every time stat is called, though.

But I don’t see how to “save” the fact that statx is available or not. I would prefer being able to detect that at compile-time but it doesn’t seem possible as far as I know (which is not that much since I am new at contributing to Rust).

[oxalica]

For syscall to statx, I have already made a wrapper statx-sys which may helps. So we can keep dependency to the old version of glibc.

@ariasuni I think this can be simply achieved by setting an atomic static flag for the first ENOSYS.

But I don’t see how to “save” the fact that statx is available or not. I would prefer being able to detect that at compile-time but it doesn’t seem possible as far as I know (which is not that much since I am new at contributing to Rust).

[cuviper]

For tracking ENOSYS, look at how copy_file_range is handled. Note that it also calls manually through libc::syscall, avoiding the problem of libc not having the new symbol at all.

[ariasuni]

The problem I see is that if I use statx for the existing interface, I’ll need to convert stat64 struct (when statx is unavailable) to the statx struct or vice-versa.

Here are problematic extracts of the code I modified:

pub struct FileAttr {
    #[cfg(not(target_os = "linux"))]
    stat: stat64,
    #[cfg(target_os = "linux")]
    stat: statx,
}

[…]

// I don’t think I can do anything here, because I can only return one type
#[cfg(target_os = "linux")]
impl AsInner<statx> for FileAttr {
    fn as_inner(&self) -> &statx { &self.stat }
}

impl MetadataExt for Metadata {
    #[allow(deprecated)]
    fn as_raw_stat(&self) -> &raw::stat {
        unsafe {
            // this doesn’t work, and I’m not sure I can make it work
            // because I need to convert the struct from `statx` to `raw::stat`
            // but I can’t return a ref to something that I’ll create in the fn…
            &*(self.as_inner().as_inner() as *const libc::statx
                                          as *const raw::stat)
        }
    }
    […]
}

I don’t want to spend time on this if I can’t resolve the issues around compatibility.

[ariasuni]

My current understanding is that since we’re exposing the underlying type in the API and returning a reference to it, we just can’t change the implementation from stat64 to statx.

It’s maybe something we can fix with epoch but as such, it seems to be a lost cause (or it’s too complex for me to do).

[alexcrichton]

@ariasuni FWIW my personal take on how we could land this today would be:

• On Unix Metadata is an enum of stat and statx. We dynamically detect at runtime which is available (favoring statx via syscall if the kernel supports it)
• The as_raw_stat function just panics if the internal variant is statx
• All accessors will match on whether we got stat or statx to return the right file.

While it's "bad" to panic I don't think anyone's calling as_raw_stat anyway so I'd hate for that to block this feature.

[oxalica]

@alexcrichton

The as_raw_stat function just panics if the internal variant is statx

It's deprecated, but it's still a breaking change :(

One of my idea is to convert representation from statx to stat with extra fields (like btime) when the evil as_raw_stat is called. This seems acceptable, but make all of other access functions to perform an extra atomic load operation, since we have Metadata: Sync.

Note that struct stat is 144 bytes while struct statx is 256 bytes. It is zero-cost to append extra fields into stat representation. But having both of them will make the struct too big.

There is also a quite simple way that, just use struct FileAttr { stat: stat64, btime: Option<..>, }, and convert statx to stat64 to fill in the fields if the syscall to statx succeeds.
I think the main cost here is the syscall itself, so we can just copy (rearrange) 144+ bytes for every call.

[crlf0710]

@rustbot modify labels to -S-inactive +S-inactive-closed