java_tools' ijar and singlejar don't build on windows

[iirina]

The ijar embedded in java_tools doesn't build from source on Windows.

Repro:

Build one of the java_tools zips in the bazel repo:

bazel build //src:java_tools_java10_zip
export JAVA_TOOLS_ZIP=~/java_tools.zip
cp bazel-bin/src/java_tools_java10.zip $JAVA_TOOLS_ZIP

mkdir java_tools_repro && cd java_tools_repro
cat >WORKSPACE <<EOF
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "local_java_tools",
    urls = ["$JAVA_TOOLS_ZIP"]
)
EOF
bazel build @local_java_tools//:lib-util

The log includes:

ERROR: D:/b/g4je5q24/execroot/io_bazel/_tmp/33823c8031b1529f3520a7c37487665f/root/ayzg7dvk/external/local_java_tools/BUILD:376:1: C++ compilation of rule '@local_java_tools//:lib-util' failed (Exit 2)
external/local_java_tools/java_tools/src/main/native/windows/util.cc(30): fatal error C1083: Cannot open include file: 'src/main/native/windows/file.h': No such file or directory
Target @local_java_tools//:ijar_cc_binary failed to build

The failing target is a cc_library which uses strip_include_prefix (from a naive point of view this looks like the culprit).

The BUILD file where the targets are defined is tools/jdk/BUILD.java_tools.

Example of log on Buildkite: https://storage.googleapis.com/bazel-untrusted-buildkite-artifacts/fdd75d7e-09da-44dc-8e2b-24c907722980/src%5Ctest%5Cshell%5Cbazel%5Cbazel_java_tools_javac10_test%5Cattempt_1.log

[iirina]

Not P1 because on Windows the java_tools use the prebuilt ijar binary.

[iirina]

cc: @laszlocsomor for windows expertise, @hlopko for C++

[iirina]

I also checked that the file java_tools/src/main/native/windows/file.h exists in the zip archive in #8621.

[laszlocsomor]

How can I help?

[laszlocsomor]

Found the culprit.

"lib-file" and "lib-util" were merged a few weeks ago. Let me update tools/jdk/BUILD.java_tools.

[iirina]

Thank you @laszlocsomor for looking into this! Can you explain why merging the two targets fixes this issue? In theory they're the same files and it shouldn't matter if they're one or two targets. I'm probably missing something.

[laszlocsomor]

Because src/main/native/windows/util.cc in "lib-util" includes src/main/native/windows/file.h, but the rule doesn't depend on that file.

I merged the two rules (in their "main" location) to avoid a circular dependency: util.cc includes file.h and file.cc includes util.h

[iirina]

I see, it makes sense. Thanks for the explanation!

[laszlocsomor]

I'm giving up, couldn't fix the errors in my PR. See #8742 (comment)

[iirina]

Thanks @laszlocsomor for looking into it! Now I understand what's causing the error. IIUC one fix would be to remove the ijar sources from @bazel_tools completely. I'll take a look to see if it's feasible.

[laszlocsomor]

@iirina : Sounds right. Did you try that?

[iirina]

@laszlocsomor Yes. Some ijar sources are still under @bazel_tools because they overalp with zipper's sources, which is a tool used by the Python rules for example. Because of this we don't have a good solution right now. Moving this tool in a remote java tools repository adds a Python -> java tools dependency which is unfortunate.

[laszlocsomor]

So python rules use zipper?

Do you know what dependencies of zipper we must cut?

For example, nothing of //src/main/native/windows except for the built windows-jni.dll should be in bazel_tools, I think.

[iirina]

All native Python rules have an implicit $zipper dependency on @bazel_tools//tools/zip:zipper.

LE: //tools/zip:zipper is an alias for //third_party/ijar:zipper

Do you know what dependencies of zipper we must cut?

Do you mean which sources overlap with ijar's sources? I don't know, but I can take a look and put together a list. From a quick glance they both depend on targets under //src/main/cpp/util (:errors, :logging, strings, etc)

[meteorcloudy]

I found out that if we depend on @bazel_tools//third_party/ijar:zip directly, ijar_cc_binary will build on Windows.
#8945

[meteorcloudy]

Fixed by #8954