js_library: missing transitive dependencies for non-js files

[danielepolencic]

TL;DR: non-js files packaged in js_library are lost when nested (e.g. ts_project -> js_library -> ts_project -> js_library).

Disclaimer:

• I tried my best to look for answers in the existing issues, but I could not find any.
• Despite all the effort I'm putting into learning Bazel, I'm still a newbie. This might work as designed and I just misunderstood the intent.

Affected Rule

I think the issue is somewhere between ts_project and js_library. For some reason files such as images or text files are filtered when using nested js_librarys.

Is this a regression?

I don't know, to be honest.
I noticed this error because we migrated a monorepo from ts_library to ts_project + js_library. The expectation is that non-js files are passed down the dependency tree, but they are not.

🔬 Minimal Reproduction

Repo: https://github.com/danielepolencic/bazel-transitive-demo

The repository has three modules that depend on each other a -> b -> c.

1. Each module has a index.ts file and a {a,b,c}.txt file.
2. The output of the module is packaged in a js_library and passed on to the next dependent package.
3. The output from the root package A should be: A + B (b.txt) + C (c.txt).

You can execute bazel run //packages/a:run to require all packages and see their output concatenated.
Unfortunately, it does not work.
The script fails because c.txt is not available.

🔥 Exception or Error

 ➜ bazel run //packages/a:run 
INFO: Invocation ID: 4abf8cc2-5ea0-4317-b62a-070d4d5dafc9
INFO: Analyzed target //packages/a:run (1 packages loaded, 6 targets configured).
INFO: Found 1 target...
Target //packages/a:run up-to-date:
  dist/bin/packages/a/run.sh
  dist/bin/packages/a/run_loader.js
  dist/bin/packages/a/run_require_patch.js
INFO: Elapsed time: 6.712s, Critical Path: 6.35s
INFO: 4 processes: 3 internal, 1 darwin-sandbox.
INFO: Build completed successfully, 4 total actions
INFO: Build completed successfully, 4 total actions
Error: ENOENT: no such file or directory, open '/private/var/tmp/_bazel_pole/583905c96397c7450861d88c2de47f30/execroot/bazel_transitive/bazel-out/darwin-fastbuild/bin/packages/a/run.sh.runfiles/bazel_transitive/node_modules/@monorepo/packages/c/c.txt'
    at Object.openSync (fs.js:498:3)
    at readFileSync (fs.js:394:35)
    at Object. (private/var/tmp/_bazel_pole/583905c96397c7450861d88c2de47f30/execroot/bazel_transitive/bazel-out/darwin-fastbuild/bin/packages/a/run.sh.runfiles/packages/c/index.ts:4:36)
    at Module._compile (internal/modules/cjs/loader.js:1072:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
    at Module.load (internal/modules/cjs/loader.js:937:32)
    at Function.Module._load (internal/modules/cjs/loader.js:778:12)
    at Module.require (internal/modules/cjs/loader.js:961:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at Object. (private/var/tmp/_bazel_pole/583905c96397c7450861d88c2de47f30/execroot/bazel_transitive/bazel-out/darwin-fastbuild/bin/packages/a/run.sh.runfiles/packages/b/index.ts:1:1)

The fix

I found that I can fix it by changing the js_library in the b module to:

js_library(
    name = "b",
    srcs = ["b.txt"],
    package_name = "@monorepo/packages/b",
    deps = [
        "b_lib",
        "//packages/c" #<-- add this
    ],
)

The fix does not make sense to me.

• Should I add all the dependencies of the ts_project to js_library too?
• Why this is not required for ts files?

🌍 Your Environment

Operating System:

  
macOS 10.15.7
  

Output of bazel version:

  
Build label: 4.2.2-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800
  

[alexeagle]

Thanks for all the details.

The important bit here is that you use bazel run to execute some binary. The outputs from each rule in your chain are correct (ts_project only outputs the .js/.d.ts files for the sources it is given). However there's another Bazel provider called "runfiles" whose job is to pass along runtime dependencies through the graph so that some binary can find all the files needed by any transitive dependency library.

Looks like js_library has an edge case where the propagation misses these sources.