test-setup.sh: outputs.zip creation may fail silently

[ghost]

Description of the problem / feature request:

If the zip command from the following blurb fails for any reason, outputs.zip is not created and the user has no idea why.

bazel/tools/test/test-setup.sh

Lines 355 to 361 in c797411

	# Zip up undeclared outputs.
	if [[ -n "$TEST_UNDECLARED_OUTPUTS_ZIP" ]] && cd "$TEST_UNDECLARED_OUTPUTS_DIR"; then
	(
	shopt -s dotglob failglob
	zip -qr "$TEST_UNDECLARED_OUTPUTS_ZIP" -- *
	) 2> /dev/null
	fi

This can happen if --

• zip isn't installed on the test system;
• the version of zip is too old and chokes on -- (stop laughing);
• any other error occurs.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ touch WORKSPACE
$ cat > BUILD.bazel <<EOF
sh_test(name = "foo_test", srcs = ["foo_test.sh"])
EOF
$ cat > foo_test.sh <<"EOF"
#!/bin/bash
touch $TEST_UNDECLARED_OUTPUTS_DIR/foo
EOF
$ cat > zip <<EOF
#!/bin/bash
exit 1
EOF
$ chmod +x foo_test.sh zip
$ bazel test :foo_test --test_env=PATH=$PWD:/bin:/usr/bin

Observe that the test succeeds and bazel-testlogs/foo_test/test.outputs_manifest exists, but .../test.outputs/outputs.zip is missing.

What operating system are you running Bazel on?

• CentOS 6.6
• Debian 7.1

What's the output of bazel info release?

release 0.24.1+vmware

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

n/a

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

n/a

Have you found anything relevant by searching the web?

No.

Any other information, logs, or outputs that you want to share?

No.

[hlopko]

@laszlocsomor is the person who's spent a lot of time in this area, maybe he could review the PR? :)

@laszlocsomor do you have plans to work on the test-setup.sh in the near term, maybe in the context of #5265?

[laszlocsomor]

Thanks for the report and repro! Gentle facepalm at the culprit.

Currently I'm not planning to work on test-setup.sh, but I could quick-fix this (e.g. display warning message, ignore error). That'd let the test succeed -- failing to zip up the undeclared outputs is arguably a nuisance but no fatal flaw. How does that sound?

[ghost]

Currently I'm not planning to work on test-setup.sh, but I could quick-fix this (e.g. display warning message, ignore error). That'd let the test succeed -- failing to zip up the undeclared outputs is arguably a nuisance but no fatal flaw. How does that sound?

Sounds good to me. :)

[laszlocsomor]

I can't print a warning to the console because Bazel prints no output for passing tests. So I'll print the warning to the test log.