Allow environment to be set when invoking Win32 process from WSL

[dra27]

When invoking a Win32 process from an LXSS process, it would be useful to have any way of passing some environment variables - for example, via execve(2). The new WSL->Win32 interop is game-changingly great, but having to dance via cmd /c set FOO=bar && some command is really awkward (and brittle, given the quoting nightmare).

For example, launch bash from a VS2015 Native Tools Command Prompt (ensure FOO unset or run set FOO= before invoking bash):

linux.c

#include <stdio.h>
#include <unistd.h>
int main(void) {
  char* newargv[] = { "win.exe", NULL };
  char* newenviron[] = { "FOO=bar", NULL };
  printf("Hello world! (LXSS)\n");
  execve("./win.exe", newargv, newenviron);
  return 1;
}

win.c

#include <stdio.h>
#include <stdlib.h>
int main(void) {
  printf("Hello world! (Win32)\nFOO=%s\n", getenv("FOO"));
  return 0;
}

Session (sudo apt-get install gcc if you don't have gcc!):

gcc -o linux linux.c
cl.exe /nologo /Fewin.exe win.c
./linux

output is:

Hello world! (LXSS)
Hello world! (Win32)
FOO=(null)

desired output would be FOO=bar at the end (naturally, if win.c is compiled with gcc and "./win.exe" changed to "./win" then this "works").

Build 14986.

[dra27]

Obviously related to #1363.

[fpqc]

Don't the Linux env variables get exported as CMD ones? When I started CMD from bash, it still had my TERM variable still set as %TERM%.

In Powershell, same thing shows with $env:term.

[dra27]

Not for me (but very happy to be shown what I'm doing wrong!):

Microsoft Windows [Version 10.0.14986]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\Users\DRA>echo %TERM%
%TERM%

C:\Users\DRA>bash
dra@Icelos:/mnt/c/Users/DRA$ echo $TERM
xterm
dra@Icelos:/mnt/c/Users/DRA$ cmd.exe
Microsoft Windows [Version 10.0.14986]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\Users\DRA>echo %TERM%
%TERM%

[fpqc]

Weird. It's working here, 14986. Try starting bash directly instead of from a cmd prompt (Win + R (run dialog), then bash.exe)

[dra27]

Same thing - still %TERM% having been xterm in bash.

[fpqc]

By Jove you're right!
It happens to work perfectly using wsltty, but it doesn't work at all in the native console for some reason.

[dra27]

Is it the same for any variable after you started WSL bash e.g.

export FOO=bar
cmd.exe
echo %FOO%

Either way - I'm trying to move away from using Cygwin components, so it'd be good to be able to do it using LXSS only!

[fpqc]

@dra27 I agree on that, but the native impementation isn't quite there for me yet (specifically shift+arrowkeys is still locked in the console to text selection, and no OTF support yet).

This does seem like a bug though (or maybe the WSLtty behavior is the bug!!). Anyway, I do like the idea!

[therealkenc]

@dra27 I hit the same thing, and set up a script to set the analogous VS 2015 environment variables in WSL, and then used clang -arch x86_64-pc-windows-msvc. This was inspired (laughably) by #1370. Believe it or not, it totally works. No Cygwin/MinGW-w64 in sight. And since cl.exe now has a clang front end anyway, if your source is clang-happy it works on both sides transparently; albeit with different codegen.

[dra27]

@therealkenc Thanks, but you seem to be solving a different problem! cl in this case is an irrelevant choice to generate a Win32 executable - the issue is the execve call from LXSS to Win32. My need is to be able to invoke arbitrary Win32 executables from bash but setting arbitrary environment variables for them. I'd be even happier if env FOO=bar ./win.exe worked as a result of fixing execve.

[aseering]

The following seems to work for me:

#!/bin/bash

WIN_ARG=""

until [[ "$1" != *"="* ]]; do
        WIN_ARG="$WIN_ARG SET $1 &&"
        shift
done

WIN_ARG="$WIN_ARG $@"

echo "Running: $WIN_ARG"
exec cmd.exe /c "$WIN_ARG"

Then run as, for example, ./launch_win.sh MYVAR="a b c d" env. The intention is to have similar semantics to Linux's env command.

bash's semantics seem to be adequate to handle the few simple cases of quoting that I've thrown at this. I've created a gist with this code if someone wants to improve the string-handling inside the inner loop, if there's some example that doesn't escape properly.

[dra27]

@aseering Indeed it does, but then my original issue notes that you can achieve it with cmd invocations. If you're going to wrap it in a general purpose script, then for a start the space needs to go from before the double ampersand. I'd question having string escaping as an afterthought - that script doesn't handle cmd escaping at all (trying using a caret symbol, any string with an ampersand) and it's easily open to code injection.

Anyway, while I appreciate the script-trickery and realise that others reading the thread may also too, what I'm really after is a Redmond-based "that's a great missing feature, we will [soon/at some point/maybe] add it" or "that's a great feature, but WSL/LXSS is never going to do that".

[fpqc]

@dra27 I'm sure they'll do that. It already does pass envvars in the other direction, right?

[dra27]

It doesn't, no - I believe only PATH is preserved. What is working usefully is that Win32 processes launched from within LXSS inherit the environment of the Win32 process which started LXSS which means, very usefully, that if you start bash from a VS2015 Tools command prompt then cl, etc. all work - but you can't in bash, for example, say echo $INCLUDE.

There are some serious complexities to doing this in general (beyond the need to translate directory names from LXSS to Win32), so I wouldn't share your default optimism that this'll make it!

[aseering]

@dra27 -- I'm sorry that you're having trouble understanding my script; could you perhaps provide an example where it does not work?

[aseering]

Regarding implementing this -- I personally see WSL's current behavior of stripping environment variables as the feature.

I would consider it to be a bug if my entire Linux environment showed up in Windows for a reason analogous to why it would be a bug if WSL used my Windows home directory as its home directory: Linux and Windows use the same names for different things; in general I don't want my Linux state stomping on my Windows state, nor vice versa.

There are various clever things that WSL could do. For example, capture environment variables that are set in the new Windows process but not in its parent. I am in general skeptical of this approach; too much magic. I'd rather be explicit.

[dra27]

@aseering I'm not having any trouble with it - the space before the double-ampersand means that every environment variable ends with a space. Try this (!): ./launch_win.sh FOO="bar && rd /s/q %HOMEPATH%\Documents" echo Hello

WSL is not stripping the environment, they're simply not the same - the technical details already released on Pico processes show that.

I agree that automatically exporting all variables would be bad - that's why I'm asking if an explicit syscall might be made available for it. I'd find that considerably more explicit and considerably less inelegant than trying to use a very dodgy shell invocation.

[aseering]

./launch_win.sh FOO="bar && rd /s/q %HOMEPATH%\Documents" echo Hello -- touché :-)

@dra27 -- apologies -- I read this thread much too quickly and misinterpreted it. I agree with you on all points.

[luser]

I'd also like to have some way to explicitly pass environment variables when spawning a Windows process. I'm working on getting the Firefox build working in WSL, which runs cl.exe and other tools. Most things work fine but there are some weird edge cases where I just really need to set PATH or some other environment variables for the Windows process. I'm currently resorting to spawning things via a batch file wrapper with cmd.exe, but it's kind of ugly and I wish there was a better way.

[nshtg]

Any update on this? This breaks many programs like e.g. VSCode since it needs to pass ENV variables to read cli options.

[AndrewPardoe]

I'd also like to be able to pass an environment from the node.js exec function.

[jstarks]

In insider build 17063, we introduced a new mechanism to allow op-in environment variable interop. Basically you can set WSLENV to a list of variables you want to interop, along with optional flags to automatically translate environment variables that contain paths.

Feedback on this is welcome.

[dra27]

@jstarks - that's awesome, thanks. I look forward to testing it soon for our build system!

[djensen47]

@jstarks For debugging purposes, what would you recommend?

For instance, if I export WSLENV=HOMEPATH/p and echo $HOMEPATH it shows up blank. However, if I export FOO and include it in WSLENV it works just fine.

[jstarks]

There's not a good way to see why the path failed to convert. You have to just reason through it. I'll think about whether this can be improved.

In your case, HOMEPATH is a relative path, not an absolute path (it's missing the drive letter), so it can't be converted. Try USERPROFILE instead.

[djensen47]

Relative paths can't be converted? 🤔 😕

[jstarks]

Currently no. Should we consider it? It seems a little weird since it would be converted relative to the working directory when you crossed the interop boundary. And we won't necessarily be able to preserve the relativity on the other side of the boundary; we would often have to convert to an absolute path.

Other than HOMEPATH (which I only found out about last week!), I'm not aware of any common environment variables that contain relative paths, so I currently don't think this is worth the potential complexity and confusion. But I'm happy to be educated on scenarios that I didn't anticipate.

[djensen47]

Heh, I think it's just %HOMEPATH% that I'd want available. 😉

[therealkenc]

%HOMEPATH% can't live in isolation without %HOMEDRIVE%. It's a longstanding weird convention. It isn't a relative directory thing. If you make HOMEDRIVE=C: and HOMEPATH="..\..\..\My Home Directory" I think your computer catches fire or something.

As for "scenarios", all of these environment variables are just strings. You can't make any assumptions about their meaning because they mean whatever their inventor meant. My personal take on the %HOMEPATH% scenario would be to drop the /p from your WSLENV and use sed on the WSL side. WSLENV is going to break when we get fstab and all the Cygwin and Docker people mount C: on /c instead of /mnt/c in their WSL environments anyway. [edit: that is incorrect] But that's just me.

[jstarks]

WSLENV does the right thing -- it uses your mount table to determine the path conversion. So if you remount on /c, the conversion will work fine.

Note that USERPROFILE is basically %HOMEPATH%%HOMEDRIVE%. Use that and you won't have any problems.

[samdenty]

You can alias the hyper command to run through cmd.exe by adding

alias hyper="cmd.exe /c hyper"

into your .bashrc or .zshrc

[dra27]

@samdenty99 - sure, that's fine for the prompt, but it's not great for scripts/build systems.