Fallback to .init_array when no arguments are available on glibc Linux #66547

leo60228 · 2019-11-19T15:25:46Z

Linux is one of the only platforms where std::env::args doesn't work in a cdylib.

rust-highfive · 2019-11-19T15:25:58Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @joshtriplett (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

joshtriplett · 2019-11-19T20:44:47Z

You can't always assume that /proc is mounted (for instance, in a container environment or other types of embedded Linux uses). I don't think we should parse that to get arguments. It has the danger of working in the common case and breaking unexpectedly in unusual environments.

This is a known limitation on Linux, that you can't reliably get the program arguments except by saving them in the main entry point. I don't think we should try to work around that. (And honestly, libraries should ask the application to pass in the arguments the library should use, which might not be the original arguments of the program.)

joshtriplett · 2019-11-19T21:09:15Z

I would personally propose closing this, unless a method arises to get the arguments out of the C library or similar.

We really appreciate your contribution, however.

src/libstd/sys/unix/args.rs

leo60228 · 2019-11-20T13:08:43Z

You can put an extern "C" fn( libc::c_int, *const *const libc::c_char, *const *const libc::c_char, ) pointer in the .init_array section and it'll get called with argc, argv, and envp. This should be more portable, but I'm not sure the best way to implement it in std.

leo60228 · 2019-11-20T13:09:02Z

This is implemented in both glibc ~~and bionic~~ (glibc is the only libc that passes argc and argv), and likely other libcs.

leo60228 · 2019-11-20T13:56:56Z

There's also this approach that assumes __environ is a pointer to the stack (which is true on all major libcs), and uses the fact that arguments are always directly before the environment on the stack. I've confirmed this is the case on x86 and x86_64, and it's probably the case on other architectures. https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=732b382e6f91c3faa6e6a3a8d2a23c99

leo60228 · 2019-11-20T14:02:31Z

@joshtriplett What do you think about these options? Using .init_array only works on glibc, but walking the stack is both hacky and non-constant-time.

leo60228 · 2019-11-20T15:48:56Z

I've implemented these at https://docs.rs/libargs.

joshtriplett · 2019-11-20T23:51:29Z

On November 20, 2019 6:02:33 AM PST, leo60228 ***@***.***> wrote: @joshtriplett What do you think about these options? Using `.init_array` only works on glibc, but walking the stack is both hacky and non-constant-time.

Walking the stack seems like a bad idea, considering that it's legal to change the stack before loading a library, or for that matter call the library on a thread other than the initial thread. And you'd have no way to know if you got valid data. `.init_array` seems like a reasonable thing for the glibc target to try, given that you'll know if it got called or not. And if someone strips out the `.init_array` section, this fails gracefully. Only do this in the glibc target, though, and we need to document it so that library builders know about it.

leo60228 · 2019-11-21T13:21:28Z

Where should this be documented?

joshtriplett · 2019-11-21T16:06:11Z

On November 21, 2019 5:21:30 AM PST, leo60228 ***@***.***> wrote: Where should this be documented?

In the documentation of the args functions.

joshtriplett

Can you please add a platform-specific test for this behavior, so we know if we've done something that makes it stop working?

leo60228 · 2019-11-21T16:13:27Z

In the documentation of the args functions.
In std::env or std::sys::unix::args?

joshtriplett

One other thought: if we initialize arguments this way, we don't also need to initialize them in our main function in binaries. Can you compile out that code on linux-gnu?

joshtriplett · 2019-11-21T16:18:45Z

Can you confirm that this works in statically linked libraries and statically linked binaries (with static glibc)?

leo60228 · 2019-11-21T16:18:57Z

If the sys::args::init is disabled in rt::init, wouldn't that make a separate test unnecessary? If not, how can I write that test?

leo60228 · 2019-11-21T16:21:01Z

Can you confirm that this works in statically linked libraries and statically linked binaries (with static glibc)?

I've been having stability issues with my system, and compiling rustc causes a freeze, so I've been using ./x.py test --stage 0 --no-doc src/libstd. Is there a way to use this libstd to compile a program?

rust-highfive · 2019-11-21T16:33:30Z

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-11-21T16:22:29.1607324Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-11-21T16:22:29.1620984Z ##[command]git config gc.auto 0
2019-11-21T16:22:29.1624148Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-11-21T16:22:29.1627504Z ##[command]git config --get-all http.proxy
2019-11-21T16:22:29.1631478Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/66547/merge:refs/remotes/pull/66547/merge
---
2019-11-21T16:27:58.7502788Z tidy check
2019-11-21T16:27:59.7506603Z * 588 error codes
2019-11-21T16:27:59.7506725Z * highest error code: E0744
2019-11-21T16:28:00.0822242Z * 270 features
2019-11-21T16:28:00.3027299Z tidy error: /checkout/src/libstd/rt.rs:44: platform-specific cfg: cfg(not(all(target_os = "linux", target_env = "gnu")))
2019-11-21T16:28:00.9006434Z some tidy checks failed
2019-11-21T16:28:00.9006539Z Found 441 error codes
2019-11-21T16:28:00.9006648Z Found 0 error codes with no tests
2019-11-21T16:28:00.9006692Z Done!
2019-11-21T16:28:00.9006692Z Done!
2019-11-21T16:28:00.9006720Z 
2019-11-21T16:28:00.9006745Z 
2019-11-21T16:28:00.9007531Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-11-21T16:28:00.9007630Z 
2019-11-21T16:28:00.9007675Z 
2019-11-21T16:28:00.9014782Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-11-21T16:28:00.9015070Z Build completed unsuccessfully in 0:01:19
2019-11-21T16:28:00.9015070Z Build completed unsuccessfully in 0:01:19
2019-11-21T16:28:00.9063969Z == clock drift check ==
2019-11-21T16:28:00.9074767Z   local time: Thu Nov 21 16:28:00 UTC 2019
2019-11-21T16:28:01.1702061Z   network time: Thu, 21 Nov 2019 16:28:01 GMT
2019-11-21T16:28:01.1702207Z == end clock drift check ==
2019-11-21T16:28:02.5068956Z 
2019-11-21T16:28:02.5161810Z ##[error]Bash exited with code '1'.
2019-11-21T16:28:02.5187570Z ##[section]Starting: Checkout
2019-11-21T16:28:02.5189688Z ==============================================================================
2019-11-21T16:28:02.5189756Z Task         : Get sources
2019-11-21T16:28:02.5189803Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

leo60228 · 2019-11-21T16:34:30Z

2019-11-21T16:28:00.3027299Z tidy error: /checkout/src/libstd/rt.rs:44: platform-specific cfg: cfg(not(all(target_os = "linux", target_env = "gnu")))

Why is this an error?

leo60228 · 2019-11-21T16:36:04Z

Oh, it's because cfgs need to be in sys. Should I just keep executing args::init no matter what, then? That what it'd still work even if .init_array gets removed (like you mentioned).

src/libstd/sys/unix/args.rs

joshtriplett · 2019-11-21T18:24:13Z

@leo60228

In std::env or std::sys::unix::args?

In std::env::args and std::env::args_os where people will actually see it in the documentation.

Oh, it's because cfgs need to be in sys. Should I just keep executing args::init no matter what, then?

How about marking args::init as #[inline(always)], and making it a no-op on linux-gnu? Your .init_array code can call the "real" implementation. That way there's no cfg in the entry point code, and there's no duplicate initialization of arguments.

Also, investigating further, this does work in a binary, including a statically linked binary. Also worth noting, a glibc-based binary or library has content in .init_array by default, and will segfault in __libc_csu_init if you strip out .init_array, so please ignore my previous comments about the possibility of stripping out .init_array. Given that glibc uses this for its own purposes, using it in Rust doesn't seem unreasonable to me. It'll get left out when building for no_std.

So, at this point, I think we just need a test, and a bit of cfg logic to make rt::init's arg initialization a no-op.

joshtriplett · 2019-11-22T01:58:00Z

Thank you for all the work you've put into this!

@rust-lang/libs, does this seem reasonable to you? I think this seems worth doing.

alexcrichton · 2019-11-22T16:30:39Z

I would agree this seems reasonable to land, supporting argc/argv access in a cdylib is best effort on a per-platform basis, so adding platform specific code to support it somewhere seems fine by me.

joshtriplett · 2019-11-22T17:20:03Z

What's the libs process here? Should this go through rfcbot for libs, or since it doesn't add new API, is the current expressed support sufficient?

@leo60228 Could you please rebase -i into a set of logical commits (that don't include reverts or fixes to previous patches, for instance)?

I'm not entirely sure *why*, but this fixed a problem I was having.

JohnCSimon · 2019-11-29T03:15:16Z

Ping from triage
@leo60228 Could you please rebase this PR as per the change request?
Thank you.

Looks like it's been rebased.

dtolnay

This looks good to me, thanks @leo60228!

dtolnay · 2019-11-29T03:27:41Z

@bors r+

bors · 2019-11-29T03:27:43Z

📌 Commit c6bcea9 has been approved by dtolnay

bors · 2019-11-29T05:04:57Z

⌛ Testing commit c6bcea9 with merge 3907d59...

Fallback to .init_array when no arguments are available on glibc Linux Linux is one of the only platforms where `std::env::args` doesn't work in a cdylib.

bors · 2019-11-29T08:12:35Z

☀️ Test successful - checks-azure
Approved by: dtolnay
Pushing 3907d59 to master...

rust-highfive · 2019-11-29T08:14:01Z

📣 Toolstate changed by #66547!

Tested on commit 3907d59.
Direct link to PR: #66547

💔 miri on linux: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung, @rust-lang/infra).

@oli-obk

Tested on commit rust-lang/rust@3907d59. Direct link to PR: <rust-lang/rust#66547> 💔 miri on linux: test-pass → test-fail (cc @oli-obk @eddyb @RalfJung, @rust-lang/infra).

sdroege · 2022-12-21T13:10:38Z

FYI this is actually not safe like this, see #105999

rust-highfive assigned joshtriplett Nov 19, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 19, 2019

jonas-schievink added O-linux Operating system: Linux T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Nov 19, 2019

bjorn3 reviewed Nov 20, 2019

View reviewed changes

src/libstd/sys/unix/args.rs Outdated Show resolved Hide resolved

leo60228 changed the title ~~Fallback to procfs when no arguments are available on Linux~~ Fallback to .init_array when no arguments are available on glibc Linux Nov 21, 2019

joshtriplett requested changes Nov 21, 2019

View reviewed changes

joshtriplett previously requested changes Nov 21, 2019

View reviewed changes

joshtriplett reviewed Nov 21, 2019

View reviewed changes

src/libstd/sys/unix/args.rs Show resolved Hide resolved

bjorn3 mentioned this pull request Nov 21, 2019

Patch libstd arg initialization for rust#66547 rust-lang/rustc_codegen_cranelift#804

Closed

leo60228 added 6 commits November 22, 2019 12:27

Use .init_array section on glibc

d8b6be9

Set .init_array priority

1ff055d

I'm not entirely sure *why*, but this fixed a problem I was having.

Document ARGV_INIT_ARRAY

e282b22

Add documentation to std::env::args[_os]

55fe6d8

Make std::sys::unix::args::init a no-op on glibc Linux

d448ab0

Test std::env::args in a staticlib on glibc Linux

c6bcea9

leo60228 force-pushed the procfs-fallback branch from 06a3b2f to c6bcea9 Compare November 22, 2019 17:27

dtolnay approved these changes Nov 29, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 29, 2019

bors added a commit that referenced this pull request Nov 29, 2019

Auto merge of #66547 - leo60228:procfs-fallback, r=dtolnay

3907d59

Fallback to .init_array when no arguments are available on glibc Linux Linux is one of the only platforms where `std::env::args` doesn't work in a cdylib.

bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 29, 2019

bors merged commit c6bcea9 into rust-lang:master Nov 29, 2019

rust-highfive mentioned this pull request Nov 29, 2019

miri no longer builds after rust-lang/rust#66547 #66862

Closed

tesuji mentioned this pull request Mar 19, 2020

doc: Add quote to .init_array #70145

Merged

sdroege mentioned this pull request Dec 21, 2022

glibc .init_array usage for std::env::args is not safe #105999

Closed

lifthrasiir mentioned this pull request Jun 17, 2023

Robust bindgen support rust-cross/cargo-zigbuild#140

Merged

dtolnay self-assigned this Mar 24, 2024

ChrisDenton mentioned this pull request Apr 19, 2024

std::env::args_os always returns an empty iterator in a musl+gcompat environment #124126

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fallback to .init_array when no arguments are available on glibc Linux #66547

Fallback to .init_array when no arguments are available on glibc Linux #66547

leo60228 commented Nov 19, 2019

rust-highfive commented Nov 19, 2019

joshtriplett commented Nov 19, 2019 •

edited

Loading

joshtriplett commented Nov 19, 2019 •

edited

Loading

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019 •

edited

Loading

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019

joshtriplett commented Nov 20, 2019 via email

leo60228 commented Nov 21, 2019

joshtriplett commented Nov 21, 2019 via email

joshtriplett left a comment

leo60228 commented Nov 21, 2019

joshtriplett left a comment

joshtriplett commented Nov 21, 2019

leo60228 commented Nov 21, 2019

leo60228 commented Nov 21, 2019

rust-highfive commented Nov 21, 2019

leo60228 commented Nov 21, 2019

leo60228 commented Nov 21, 2019

joshtriplett commented Nov 21, 2019

joshtriplett commented Nov 22, 2019

alexcrichton commented Nov 22, 2019

joshtriplett commented Nov 22, 2019

JohnCSimon commented Nov 29, 2019

dtolnay left a comment

dtolnay commented Nov 29, 2019

bors commented Nov 29, 2019

bors commented Nov 29, 2019

bors commented Nov 29, 2019

rust-highfive commented Nov 29, 2019

sdroege commented Dec 21, 2022

Fallback to .init_array when no arguments are available on glibc Linux #66547

Fallback to .init_array when no arguments are available on glibc Linux #66547

Conversation

leo60228 commented Nov 19, 2019

rust-highfive commented Nov 19, 2019

joshtriplett commented Nov 19, 2019 • edited Loading

joshtriplett commented Nov 19, 2019 • edited Loading

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019 • edited Loading

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019

leo60228 commented Nov 20, 2019

joshtriplett commented Nov 20, 2019 via email

leo60228 commented Nov 21, 2019

joshtriplett commented Nov 21, 2019 via email

joshtriplett left a comment

Choose a reason for hiding this comment

leo60228 commented Nov 21, 2019

joshtriplett left a comment

Choose a reason for hiding this comment

joshtriplett commented Nov 21, 2019

leo60228 commented Nov 21, 2019

leo60228 commented Nov 21, 2019

rust-highfive commented Nov 21, 2019

leo60228 commented Nov 21, 2019

leo60228 commented Nov 21, 2019

joshtriplett commented Nov 21, 2019

joshtriplett commented Nov 22, 2019

alexcrichton commented Nov 22, 2019

joshtriplett commented Nov 22, 2019

JohnCSimon commented Nov 29, 2019

dtolnay left a comment

Choose a reason for hiding this comment

dtolnay commented Nov 29, 2019

bors commented Nov 29, 2019

bors commented Nov 29, 2019

bors commented Nov 29, 2019

rust-highfive commented Nov 29, 2019

sdroege commented Dec 21, 2022

joshtriplett commented Nov 19, 2019 •

edited

Loading

joshtriplett commented Nov 19, 2019 •

edited

Loading

leo60228 commented Nov 20, 2019 •

edited

Loading