Passing environment variables from down-stream to up-stream library

[ccll]

(Moved and re-phrased from #97)

Currently cargo can pass env vars from up-stream dependencies to down-stream, with user-defined metadata.

But what's still lacking is the reversed control, where up-stream library expects a env var to be set to build successfully, and down-stream must fill the env var for it.

This scenario is common in FFI libraries wrapping existing C lib, for example meh/rust-ffmpeg-sys expects the env var FFMPEG_DIR to be set to the path of pre-built C FFMpeg libs & binaries.

[norru]

I need this too!

My use case is to dynamically search for the most appropriate value of BOX2D_INCLUDE_PATH when using wrapped2d (https://github.com/Bastacyclop/rust_box2d). At the moment the options are:

• tell the user to set the variable manually, which is bad for automation and introduces much friction
• wrap the Cargo invocation in a batch file, which is not very IDE friendly (requires per-user setup, not platform agnostic, generally clunky)
• forcing the user to install the library at a very specific location, such as C:\ProgramData\Box2D

In all cases I need to provide Cargo with an absolute path, which is also very impractical. I have tried providing the full pre-built lib binaries in my project tree, together with all the .h files. Library links fine as it can be configured via build.rs but I could not find a reliable mechanism to reference the headers via a relative or dynamically discovered path.

The sequence would be:

• run a pre_build.rs (name TBD) script, configurable via Cargo.toml, which merges the current env with a set of custom, arbitrary env vars. This should replace the current kludge with batch files/shell scripts
• after pre_build.rs successfully completes, build all dependencies, in the scope of the environment above
• run the build.rs script as normal, in the scope of the environment above

This is so annoying that I am starting to look for Rust native physics libraries to replace Box2D altogether. Which is a shame as I'm otherwise super happy with both Box2D and the wrapped2d.

Cheers!

[axos88]

I need this too to be able to cross-compile with openssl dependency without having to invoke cargo like this every time: SYSROOT=/Volumes/ct-ng/arm-unknown-linux-gnueabihf/arm-unknown-linux-gnueabihf/sysroot OPENSSL_LIB_DIR=$SYSROOT/lib OPENSSL_INCLUDE_DIR=$SYSROOT/usr/include cargo build --target arm-unknown-linux-gnueabihf

[mberndt123]

+1 for the cross compilation scenario. When you depend on a crate that uses pkg-config to discover some C library, it would be nice to be able to set PKG_CONFIG_SYSROOT_DIR_mips-unknown-linux-musl in ~/.cargo/config once and be done with it

[joejune]

Me too, I want to set ARMV7_LINUX_ANDROIDEABI_OPENSSL_DIR to use a prebuilt openssl.

[ncalexan]

I also want Cargo to pass environment variables configuring the location of OpenSSL headers and libraries. In my case I'm cross compiling and the variables depend on the current target/toolchain.

[m-hilgendorf]

Any progress on this, or guidance on how to get started with a PR?

I have a use case where a crate needs to know the install location of a closed source SDK, which typically isn't placed in a user's $PATH since it can be useful to have multiple SDK versions installed at the same time.

Alternatively, adding an [env] section to the ./cargo/config file would work great.

[thedavidmeister]

same thing with libzmq here

[awakecoding]

I'm interested in a similar feature, where the major pain point is overriding OPENSSL_DIR with one that would be provided by a top-level crate that depends on openssl-sys. Since many crates depend on openssl, using a modified openssl-sys that won't cause conflicts with other dependencies is almost impossible.

[josephrocca]

This would be super useful! My current solution is to put the build command (e.g. env LIBRARY_PATH=/usr/local/cuda/lib64 cargo +nightly build --release) into the readme file, but this obviously isn't ideal. Does anyone know of any (hacky or otherwise) workarounds for this as of late 2019?

[hamaluik]

Also looking for this. Having to manually specify env vars on the command line before every build is not very ergonomic and caused more than a few of my builds to fail.

[guoxbin]

I need this too. One crate uses a wasm file, while another crate depends on the former crate and need to specify a custom wasm file.

[agentsim]

Has anyone tried using cargo-make as a potential workaround for this problem?

[blacktop]

🙏

[seunlanlege]

+1 🙏

[tom-bowles]

Same thing building stuff for Android, where some packages require env vars pointing to binaries in the NDK (eg CXX_aarch64_linux_android=path/to/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android26-clang++). I ended up putting a shell script in the project folder that exports the necessary variables then does cargo "$@", and using that in place of cargo.

I'm not convinced this is really a question of passing env vars from one crate to another. It feels more like local config. As others have suggested, I think an [env] section in config.toml would be great. (It might also be nice to be able to split out config.toml into more than one file so that you can choose to commit some parts and gitignore others, but that's a separate thing.)

[rib]

This issue is particularly awkward when it comes to things like rust-analyzer and RLS running within an editor like Visual Studio Code since it's particularly difficult to control the environment of cargo in this case.

Incidentally my use case is exactly the same as the OP; I need to be able to set the FFMPEG_DIR for rust-ffmpeg-sys

[rib]

Faced with this issue, I tried looking at modifying ffmpeg-sys to refer to an environment.json config file instead of relying on environment variables and made this pull request:

zmwangx/rust-ffmpeg-sys#7

My original plan had been that it would look for an environment.json file structured like:

{
    "FFMPEG_DIR": "/path/to/ffmpeg-build/"
}

at the very top of my project but I found that cargo doesn't expose enough information (as far as I could tell) to locate the top level Cargo.toml for a build.

My current solution looks for an environment.json that's adjacent to the ffmpeg-sys Cargo.toml which isn't really ideal because now I have to have a fork of ffmpeg-sys for the sake of configuring it. For me this is still preferable to dealing with environment variables but I think it would be much nicer if there was some way to locate the top level of a project within a build.rs.

I wonder if the Cargo maintainers would be open to supporting something like this directly in Cargo and avoid needing to modify projects like ffmpeg-sys?

I basically just added the following snippet to the top of their build.rs::main():

    // It's often impractical to use an environment variable to specify
    // FFMPEG_DIR so we also support parsing an environment.json file
    // Ref: rust-lang/cargo#4121
    let manifest_dir_env = env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR evironment variable unset");
    let env_path = Path::new(&manifest_dir_env).join("environment.json");
    if let Ok(env) = fs::read_to_string(env_path) {
        let env: Value = serde_json::from_str(&env).expect("Failed to parse environment.json");
        if let Some(env_map) = env.as_object() {
            for key in env_map.keys() {
                if let Some(env_value) = env[key].as_str() {
                    // Simply re-export as an actual environment variable for consistent handling below...
                    env::set_var(key, env_value);
                }
            }
        }
    }

And Cargo could essentially do exactly the same thing automatically except looking for an environment.json at the top of the project which would be much more friendly.

With this approach then VSCode diagnostics via rust-analyzer Just Works(tm) for me again so I definitely think it would be good for Cargo to have some kind of general solution like this that lets us eradicate the need to configure build environment variables. Environment variables are a nightmare to deal with when considering editor/IDE integrations if each different integration needs to provide some inconsistent mechanism for configuring the environment (not currently available for vscode/rust-analyzer).

Compared to the suggestions to add some way of configuring the environment within a Config.toml I think having a seperate environment.json like this could be better since these environment configurations probably don't belong in version control in most sitations and they relate more to the overall project context for a specific user than directly relating to a specific crate.

[rib]

Just to try and collate some of the related issues where people have been struggling with similar kinds of environment variable configurations:

#97
rust-lang/rls#915
rust-lang/vscode-rust#791
rust-lang/rust-analyzer#6099
intellij-rust/intellij-rust#1569
https://stackoverflow.com/questions/57017066/how-do-i-set-environment-variables-with-cargo

[rib]

I took a stab at making a pull request for addressing this issue: #8839

It's basically a first draft implementation of what I described above and works for my use case. Could be good to know if it could work for others here too.

[felipelalli]

How to set different env vars if --release is set? For example:

I'd like to set:

[env]
RUST_LOG = "trace"

if --release is not set and "info" otherwise.