Error when trying to build java_binary on Windows

[oferb]

Error when trying to build java_binary on Windows

Feature requests: what underlying problem are you trying to solve with this feature?

Compile :)

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Clone https://github.com/google/startup-os
Run bazel build //tools/reviewer/service

This results in the following (run with --verbose_failures):

bazel build //tools/reviewer/service --verbose_failures
ERROR: C:/root/storage/nodejs/startup-os/tools/reviewer/service/BUILD:27:1: Building tools/reviewer/service/libservice.jar (1 source file) and running annotation processors (AutoFactoryProcessor) failed (Exit 1): java.exe failed: error executing command
  cd C:/users/locked/_bazel_locked/zrv4ye2p/execroot/__main__
  SET LC_CTYPE=en_US.UTF-8
  external/local_jdk/bin/java.exe -Xbootclasspath/p:external/bazel_tools/third_party/java/jdk/langtools/javac-9+181-r4173-1.jar -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/x64_windows-fastbuild/bin/tools/reviewer/service/libservice.jar-2.params
tools\reviewer\service\CodeReviewService.java:17: error: cannot access com.google.startupos.tools.reviewer.service
package com.google.startupos.tools.reviewer.service;
^
  /com/google/startupos/common/Time.class
Target //tools/reviewer/service:service failed to build

What operating system are you running Bazel on?

Windows 10

What's the output of bazel info release?

Release 0.13.0

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

git remote get-url origin: git@github.com:google/startup-os.git
git rev-parse master: 2c3bcb49f8d1092241fd982874dce621bbcde91f
git rev-parse HEAD: 2c3bcb49f8d1092241fd982874dce621bbcde91f

Have you found anything relevant by searching the web?

Nope

Any other information, logs, or outputs that you want to share?

Seems to be an issue with java_binary, not java_library:
8 different java_library targets successfully compiled.
2 different java_binary targets resulted in the error.

This same error happened on 2 different Windows machines, and successfully compiled on 5 Ubuntu & macOS ones.

[laszlocsomor]

I distilled a minimal repro case, see laszlocsomor/projects@c0d5a9c.

bazel build //foo --verbose_failures succeeds on Linux, but fails on Windows the same way as it does for @oferb . Culprit is the java_plugin.generates_api = True BUILD rule attribute. That's all I know for now.

[laszlocsomor]

Found the culprit: the Java header compilation creates header jars with zip entries that use backslashes as path separators instead of forward slashes. This confuses javac or the annotation processor.

I verified this as the culprit by copying the bazel-bin/foo/libbar-hjar.jar from Linux to Windows and retried the failing command, and it now works. Also, unzip -l shows the zip entries as different, but the files in the jar are the same:

# Bad header jar, generated on Windows:
$ unzip -l ../../../bazel-bin/foo/libbar-hjar.jar
Archive:  ../../../bazel-bin/foo/libbar-hjar.jar
  Length      Date    Time    Name
---------  ---------- -----   ----
       93  2010-01-01 00:00   com\google\bar\Bar.class
        0  2010-01-01 00:00   META-INF/
       69  2010-01-01 00:00   META-INF/MANIFEST.MF
---------                     -------
      162                     3 files

# Good header jar, generated on Linux
$ unzip -l ../libbar-hjar.jar
Archive:  ../libbar-hjar.jar
  Length      Date    Time    Name
---------  ---------- -----   ----
       93  2010-01-01 00:00   com/google/bar/Bar.class
        0  2010-01-01 00:00   META-INF/
       69  2010-01-01 00:00   META-INF/MANIFEST.MF
---------                     -------
      162                     3 files

# Diff the extracted files
$ diff com/google/bar/Bar.class ../good/com/google/bar/Bar.class

$ diff META-INF/MANIFEST.MF ../good/META-INF/MANIFEST.MF

[laszlocsomor]

Summary: Turbine writes zip entries into the hjar using backslashes on Windows:

C:\_bazel\c39O6PEq\execroot\__main__>external\local_jdk\bin\java.exe -Xverify:none -Xbootclasspath/p:external/bazel_tools/third_party/java/jdk/langtools/javac-9+181-r4173-1.jar -jar external/bazel_tools/tools/jdk/turbine_deploy.jar --output c:/tmp/foo-hjar.jar --temp_dir c:/tmp --bootclasspath external/local_jdk/jre/lib/rt.jar external/local_jdk/jre/lib/resources.jar external/local_jdk/jre/lib/jsse.jar external/local_jdk/jre/lib/jce.jar external/local_jdk/jre/lib/charsets.jar external/local_jdk/jre/lib/ext/access-bridge-64.jar external/local_jdk/jre/lib/ext/cldrdata.jar external/local_jdk/jre/lib/ext/dnsns.jar external/local_jdk/jre/lib/ext/jaccess.jar external/local_jdk/jre/lib/ext/localedata.jar external/local_jdk/jre/lib/ext/nashorn.jar external/local_jdk/jre/lib/ext/sunec.jar external/local_jdk/jre/lib/ext/sunjce_provider.jar external/local_jdk/jre/lib/ext/sunmscapi.jar external/local_jdk/jre/lib/ext/sunpkcs11.jar external/local_jdk/jre/lib/ext/zipfs.jar --sources foo/Bar.java --javacopts -source 8 -target 8 -encoding UTF-8 -XDskipDuplicateBridges=true -g -parameters -- --target_label //foo:bar --processors common.AnnotProc --processorpath bazel-out/host/bin/foo/libannotproc.jar

C:\_bazel\c39O6PEq\execroot\__main__>c:\tools\msys64\usr\bin\unzip.exe -l c:\tmp\foo-hjar.jar
Archive:  c:\tmp\foo-hjar.jar
  Length      Date    Time    Name
---------  ---------- -----   ----
       93  2010-01-01 00:00   com\google\bar\Bar.class
        0  2010-01-01 00:00   META-INF/
       69  2010-01-01 00:00   META-INF/MANIFEST.MF
---------                     -------
      162                     3 files

[laszlocsomor]

/cc @cushon @iirina

[laszlocsomor]

@cushon , @iirina : See my previous comment for a summary.

[dslomov]

Sounds like a release blocker to me?

[laszlocsomor]

Yes.

[laszlocsomor]

However it's not a regression, so it shouldn't block any ongoing release.

[laszlocsomor]

Actually, this isn't a release blocker as such, because it's been broken in every release.

[oferb]

Thanks for the prompt investigation! It's mind-boggling how many eng-hours the / vs \ incompatibility has probably cost over the years.

I guess we have an interesting combination of technologies in StartupOS that surfaced this bug.
I think what we'll do once this is fixed, is build bazel from head and try it out again. This way we can have faster cycles.

[oferb]

Thanks! Perhaps you have a nightly build or something like that we could use, instead of building it ourselves?

[laszlocsomor]

Opening again, as it's a legitimate multiplatform build bug that we must fix.

@oferb : unfortunately we don't have one. But if your system is already set up to build with Bazel, you can easily build a Bazel binary from HEAD, just clone https://github.com/bazelbuild/bazel and bazel --output_user_root=c:\tmp build //src:bazel.exe

[cushon]

Opening again, as it's a legitimate multiplatform build bug that we must fix.

Is it not fixed by 10141a3?

[laszlocsomor]

Oh sorry, I didn't see that! Yes it is. Thanks!

[oferb]

Hey,
Trying to build from head on Windows. Running:
bazel --output_user_root=c:\tmp build //src:bazel.exe
Getting:
ERROR: C:/root/storage/nodejs/bazel/src/BUILD:229:2: Executing genrule //src:package-zip failed (Exit 127) src/package-bazel.sh: line 49: zip: command not found

Trying:
bazel build //src:bazel
Getting:

ERROR: C:/root/storage/nodejs/bazel/src/main/java/com/google/devtools/build/lib/skylarkbuildapi/platform/BUILD:18:1: Building src/main/java/com/google/devtools/build/lib/skylarkbuildapi/platform/libplatform.jar (4 source files) and running annotation processors (SkylarkCallableProcessor) failed (Exit 1)
java.io.IOException: Cannot clean 'bazel-out\x64_windows-fastbuild\bin\src\main\java\com\google\devtools\build\lib\skylarkbuildapi\platform\_javac\platform\libplatform_classes'
Caused by: java.nio.file.DirectoryNotEmptyException: bazel-out\x64_windows-fastbuild\bin\src\main\java\com\google\devtools\build\lib\skylarkbuildapi\platform\_javac\platform\libplatform_classes\com\google\devtools\build\lib

Perhaps the command is not the correct one? If it's correct, I can open another issue for this.

[laszlocsomor]

Ah, that's annoying. Culprit: #5265.
You can solve this by installing zip: open an MSYS shell and run pacman -Syu zip unzip. Follow the instructions MSYS gives you -- it may ask you to close its window and restart it.

[laszlocsomor]

I confirmed that startup-os builds successfully at google/startup-os@c712366 using Bazel built at a5ecd81.

It does NOT build at current startup-os HEAD (google/startup-os@d58cd6f) but the reason looks like an error with startup-os' WORKSPACE file.

[oferb]

Is the fix in 0.14.0?

[laszlocsomor]

@oferb : if you mean 10141a3, no it's not, because 0.14.0's baseline (5c3f5c9) was merged on 2018/05/11 and the fix was merged on 2018/05/25, and was not cherry-picked into the release.