Distribution build not working on Linux

[juhalindfors]

The current HEAD (currently at commit 5038016) is not able to complete the full distribution build on a Ubuntu Linux system.

Following the steps described here: https://bazel.build/contributing.html#compiling-bazel

> bazel build //:bazel-distfile
INFO: Found 1 target...
Target //:bazel-distfile up-to-date:
  bazel-genfiles/bazel-distfile.zip
...
> cd bazel-genfiles/bazel-dist
> jar xf bazel-distfile.zip 
> bash compile.sh all
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    compile.sh all /path/to/bazel
🍃  Building Bazel from scratch.......
🍃  Building Bazel with Bazel.
.WARNING: /tmp/bazel_rqWgcLir/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_rqWgcLir/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
INFO: Found 1 target...
INFO: From Compiling src/main/cpp/blaze_util_posix.cc:
src/main/cpp/blaze_util_posix.cc: In function 'void blaze::Daemonize(const string&)':
src/main/cpp/blaze_util_posix.cc:235:28: warning: ignoring return value of 'int dup(int)', declared with attribute warn_unused_result [-Wunused-result]
   (void) dup(STDOUT_FILENO);  // stderr (2>&1)
                            ^
src/main/cpp/blaze_util_posix.cc: In function 'uint64_t blaze::AcquireLock(const string&, bool, bool, blaze::BlazeLock*)':
src/main/cpp/blaze_util_posix.cc:520:30: warning: ignoring return value of 'int ftruncate(int, __off_t)', declared with attribute warn_unused_result [-Wunused-result]
   (void) ftruncate(lockfd, 0);
                              ^
src/main/cpp/blaze_util_posix.cc:525:47: warning: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Wunused-result]
   (void) write(lockfd, msg.data(), msg.size());
                                               ^
ERROR: /home/u/bazel/bazel-genfiles/src/main/native/BUILD:36:1: Linking of rule '//src/main/native:libunix.so' failed: link_dynamic_library.sh failed: error executing command 
  (cd /tmp/bazel_rqWgcLir/out/execroot/bazel-dist && \
  exec env - \
    LD_LIBRARY_PATH=/home/u/Applications/torch/install/lib: \
    PATH=/home/u/Applications/torch/install/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin \
    PWD=/proc/self/cwd \
  external/bazel_tools/tools/cpp/link_dynamic_library.sh no ignored ignored ignored /usr/bin/gcc -shared -o bazel-out/local-opt/bin/src/main/native/libunix.so '-fuse-ld=gold' -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/usr/bin -B/usr/bin -pass-exit-codes -Wl,--gc-sections -Wl,@bazel-out/local-opt/bin/src/main/native/libunix.so-2.params): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 126.
/tmp/bazel_rqWgcLir/out/execroot/bazel-dist/_bin/process-wrapper: 12: /tmp/bazel_rqWgcLir/out/execroot/bazel-dist/_bin/process-wrapper: external/bazel_tools/tools/cpp/link_dynamic_library.sh: Permission denied
Target //src:bazel failed to build
INFO: Elapsed time: 224.878s, Critical Path: 19.03s

ERROR: Could not build Bazel

Environment info

• Operating System: Ubuntu Linux LTS 14.04

Have you found anything relevant by searching the web?

Permission denied errors (may not be related): #2891, #2482

[juhalindfors]

Fixed title. It was originally for another issue ;-)

[damienmg]

The point that was used was also broken on our CI, CI bootstrap test is passing now so it should be back online. Closing feel free to ping back if you are still experiencing the issue.

[juhalindfors]

There was a secondary issue in using jar to extract the distribution zip archive which doesn't preserve the executable bit on *.sh files. Unzip works better.

The compile process passes now without errors (against HEAD revision of this morning), however the whole process completes with:

@@ -2 +2 @@
-8fc1517c777e3b3a94210a9b507ad220  bazel-bin/src/bazel
+177ef65edadf517a6c8a448f26e59074  bazel-bin/src/bazel
@@ -8130,3 +8130,3 @@
-c9939d577654715a7293d81f871ae9c3  bazel-genfiles/src/embedded_tools.zip
-56418735b2536112d56e6ecb3181a71d  bazel-genfiles/src/install_base_key
-e519a04413325eafa8e9e79d5562ff0f  bazel-genfiles/src/java_tools/junitrunner/java/com/google/testing/coverage/JacocoCoverage_jarjar_deploy.jar
+ae7dcb31bb1c5cca55fc8a2d5da36b5b  bazel-genfiles/src/embedded_tools.zip
+fad5b337f259a356be7a1631e8a2d1b3  bazel-genfiles/src/install_base_key
+6d9e33de68e9f8d7d51ff7d543455709  bazel-genfiles/src/java_tools/junitrunner/java/com/google/testing/coverage/JacocoCoverage_jarjar_deploy.jar
@@ -8142 +8142 @@
-71d3e99dcccb5fedd27bafb8b1d05b65  bazel-genfiles/src/package.zip
+46c912c2cb233a76f5ce22c6029123de  bazel-genfiles/src/package.zip

ERROR: Differences detected in outputs!

Not sure if that is the expected result currently, or something that needs attention. It's the result of the determinism tests.

[damienmg]

Is it on macos?

…

On Wed, May 3, 2017, 9:20 PM Juha Lindfors ***@***.***> wrote: There was a secondary issue in using jar to extract the distribution zip archive which doesn't preserve the executable bit on *.sh files. Unzip works better. The compile process passes now without errors (against HEAD revision of this morning), however the whole process completes with: @@ -2 +2 @@ -8fc1517c777e3b3a94210a9b507ad220 bazel-bin/src/bazel +177ef65edadf517a6c8a448f26e59074 bazel-bin/src/bazel @@ -8130,3 +8130,3 @@ -c9939d577654715a7293d81f871ae9c3 bazel-genfiles/src/embedded_tools.zip -56418735b2536112d56e6ecb3181a71d bazel-genfiles/src/install_base_key -e519a04413325eafa8e9e79d5562ff0f bazel-genfiles/src/java_tools/junitrunner/java/com/google/testing/coverage/JacocoCoverage_jarjar_deploy.jar +ae7dcb31bb1c5cca55fc8a2d5da36b5b bazel-genfiles/src/embedded_tools.zip +fad5b337f259a356be7a1631e8a2d1b3 bazel-genfiles/src/install_base_key +6d9e33de68e9f8d7d51ff7d543455709 bazel-genfiles/src/java_tools/junitrunner/java/com/google/testing/coverage/JacocoCoverage_jarjar_deploy.jar @@ -8142 +8142 @@ -71d3e99dcccb5fedd27bafb8b1d05b65 bazel-genfiles/src/package.zip +46c912c2cb233a76f5ce22c6029123de bazel-genfiles/src/package.zip ERROR: Differences detected in outputs! Not sure if that is the expected result currently, or something that needs attention. It's the result of the determinism tests. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2912 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADjHf84XHSO4g5VX1j0C1mnQmsoaeQ3tks5r2NNrgaJpZM4NMQWF> .

[juhalindfors]

This is still on Ubuntu Linux, against checkout of 8598824 (May 3 10:22:08 2017 +0200)

[damienmg]

Humm I have a determinism test that was failing, seems like you hit it. So Jarjar seems to not be deterministic. Renaming the bug.

[damienmg]

(side-note: not an issue for you)

[damienmg]

https://bazel-review.googlesource.com/c/10452/ is the new test that will replace determinism that cannot run on our ci anymore.