How executable action can locate runfiles in both Windows and sandboxed execution

[alexeagle]

Description of the problem / feature request / question:

Under windows, runfiles must be accessed via the runfiles_manifest since there are no symlinks.
However, this file is absent under the sandbox.

If possible, provide a minimal example to reproduce the problem:

https://gist.github.com/alexeagle/d7c57183f548c428469bec6ec4113419

$ bazel build :a
INFO: Found 1 target...
ERROR: /Users/alexeagle/Projects/repro_runfiles_manifest_missing/a/BUILD:9:1: SkylarkAction foo.txt failed (Exit 1).
hello world
writing to bazel-out/local-fastbuild/bin/foo.txt
/private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles
/private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles/__main__
/private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles/__main__/repro.sh
/private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles/__main__/sh
/private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles/__main__/some_data
ls: /private/var/tmp/_bazel_alexeagle/4c2cb9588752d9f05411073c701a9ebc/bazel-sandbox/2423683567443745504/execroot/__main__/bazel-out/host/bin/sh.runfiles_manifest: No such file or directory
Target //:a failed to build

The find command in repro.sh lists all the files visible when the sandboxed action executes; the manifest file isn't present at all.

/cc @philwo

Environment info

• Operating System:

• Bazel version (output of bazel info release):
  $ bazel version
  Build label: 0.5.4-homebrew
  Build target: bazel-out/darwin_x86_64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
  Build time: Fri Aug 25 16:55:29 2017 (1503680129)
  Build timestamp: 1503680129
  Build timestamp as int: 1503680129

• If bazel info release returns "development version" or "(@non-git)", please tell us what source tree you compiled Bazel from; git commit hash is appreciated (git rev-parse HEAD):

Have you found anything relevant by searching the web?

(e.g. StackOverflow answers,
GitHub issues,
email threads on the bazel-discuss Google group)

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

[alexeagle]

/cc @davidstanke this is a blocker for building typescript on windows

[philwo]

I will investigate and get back. Thank you very much for providing the repro case!

[philwo]

@ulfjack @dslomov It seems like the runfiles_manifest is not in the inputs of a Spawn, but is created (because when disabling sandboxing, the action in the repro example works).

Before I dive deeper into debugging this, do you have an idea out of your head why this is the case?

[dslomov]

runfiles manifest depends on local environment - it has absolute paths

…

On Sep 13, 2017 19:02, "Philipp Wollermann" ***@***.***> wrote: @ulfjack <https://github.com/ulfjack> @dslomov <https://github.com/dslomov> It seems like the runfiles_manifest is not in the inputs of a Spawn, but is created (because when disabling sandboxing, the action in the repro example works). Before I dive deeper into debugging this, do you have an idea out of your head why this is the case? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3726 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AExrX6yTqN0g5lUAFjWlyoNbnzq-ysSzks5siAqlgaJpZM4PUunK> .

[ulfjack]

The runfiles manifest is intentionally not in the inputs, like @dslomov says. This is WAI. We need to know what the actual problem is here.

[alexeagle]

The bigger problem is, how can an executable action locate its runtime dependencies? Symlinks are not present on Windows, and the manifest is not present in a sandbox. There seems to be no way remaining for the ctx.executable.compiler to find the some_data file in the repro above.

[ulfjack]

That run wasn't from a Windows run, and you didn't mention Windows above. We don't have sandboxing on Windows at this time, and we do have some tools to do runfiles lookups on Windows.

That said, I'm not sure we have adequately defined / designed a more permanent solution, but I'm sure it isn't making the current runfiles manifests (that contain absolute paths) available in the sandbox.

[alexeagle]

I mentioned at the beginning of the description, "Under windows, runfiles must be accessed via the runfiles_manifest since there are no symlinks" but it could have been more clear:

I would like to write the compiler once, and run on all platforms. I think you are saying this is not possible, even for a compiler that isn't platform-dependent (running in NodeJS) it needs to have separate file resolution logic for Windows than on Mac/Linux. That's also true for wrapper scripts that bootstrap the compiler (eg. find its entry point).

What is the recommended pattern then, to write rules that work on all platforms? Look for a runfiles manifest file, and if not found, assume that the runfiles symlink tree is populated?

[dslomov]

What is the recommended pattern then, to write rules that work on all platforms? Look for a runfiles manifest file, and if not found, assume that the runfiles symlink tree is populated?

This will work for now (I appreciate it is not awesome)

That said, I'm not sure we have adequately defined / designed a more permanent solution, but I'm sure it isn't making the current runfiles manifests (that contain absolute paths) available in the sandbox.

We need to provide good guidance to people willing to have rules that work cross-platform. One possible solution is make sure people can always rely on the manifest being present. That would mean that inside the sandbox, the manifest will point to the files inside the sandbox (sandbox will generate that file).

[ulfjack]

I don't think we want to guarantee the existence of a runfiles manifest for every single action.

[ulfjack]

Well, ok, I'll upgrade that to a maybe. Although I'm still hopeful that Microsoft will make symlinks usable.

I don't think we'll be able to resolve it in this issue thread. If you need a short term solution, I'm afraid you'll have to go with separate logic on Windows vs. everything else.

[ulfjack]

I apologize for missing the Windows reference in the first sentence.

[alexeagle]

Thanks, all. We do have a big challenge to make the whole ecosystem work on Windows. At least for Angular, I think this will be totally worth it :) If you want to do anything with this issue, I guess the best would be a mention in the docs somewhere of how an executable action should resolve its runtime files.

…

On Thu, Sep 14, 2017 at 7:58 AM Ulf Adams ***@***.***> wrote: I apologize for missing the Windows reference in the first sentence. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3726 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC5I_d_tNhCLRqX5yl68vCKCO8OvLCJks5siT8UgaJpZM4PUunK> .

[eustas]

I use $location substitution for (JVM) args for this. Works magically, until you try it on windows - bazel does not link data dependencies to .runfiles at all..

[alexeagle]

For NodeJS rules I wrapped the location expansion to give runfiles paths https://github.com/bazelbuild/rules_nodejs/blob/a9c12cebee0c267a32330533b47c22fb08aa81f0/internal/node.bzl#L64 Then you just need to read the manifest file to locate that key like https://github.com/bazelbuild/rules_nodejs/blob/a9c12cebee0c267a32330533b47c22fb08aa81f0/internal/check_bazel_version_test.py#L4

…

On Tue, Oct 10, 2017, 4:52 AM Eugene Kliuchnikov ***@***.***> wrote: I use $location substitution for (JVM) args for this. Works magically, until you try it on windows - bazel does not link data dependencies to .runfiles at all.. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3726 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC5IwqR568bri_zAWMHLTGBtyfy4U9zks5sqzA1gaJpZM4PUunK> .

[alexeagle]

I think this could be closed in favor of #3889