Tests don't run to completion on macOS

[winterqt]

On macOS, some acceptance tests fail with the following:

Command failed with ENOENT: ./setup.sh
spawn ./setup.sh ENOENT

However, others execute the setup script just fine, which Objective-See's Process Monitor shows.

---

When setting shell to true and piping the stdout and stderr of the setup script, I at least saw these:

+ set -o pipefail
+ SUBJECT=setup-test-repo
+ LOCK_FILE=/tmp/setup-test-repo.lock
+ '[' -f /tmp/setup-test-repo.lock ']'
+ echo 'Script is already running'
+ exit

Wouldn't this then cause the subsequent test to fail, since the repo may or may not be created at this point?

This is a very strange issue that I can't reproduce on Linux, and may be an execa bug...?

[korthout]

On macOS, some acceptance tests fail

That's interesting, I personally work on macOS as well and "it works on my machine" 😄

Can you tell me a bit more about how you've tried to run the tests.

+ set -o pipefail
+ SUBJECT=setup-test-repo
+ LOCK_FILE=/tmp/setup-test-repo.lock
+ '[' -f /tmp/setup-test-repo.lock ']'
+ echo 'Script is already running'
+ exit

You've run into the safety mechanism in the test setup script that makes sure it's not run multiple times simultaneously. It does so using a lock file in your tmp folder. It seems that file exists on your machine. Generally, the cleanup should delete the associated lock file, but it might have failed somehow. Perhaps due to permissions. You could try deleting the file yourself: rm /tmp/setup-test-repo.lock and retry running the tests.

[winterqt]

You've run into the safety mechanism in the test setup script that makes sure it's not run multiple times simultaneously.

Since tests can be run simultaneously, and this script exits with a success error code, isn't a race condition possible?

1. Setup script runs in test A
2. Setup script tries to run in test B, but exits successfully since test A's is currently running
3. Test A completes, running the cleanup script
4. Test B fails since the repo is deleted

I set shell to false (the default), and now in some tests the setup script is running fine, but spawning the cleanup script is (sometimes?) throwing ENOENT. No clue what could possibly be happening here 🙁, since clearly the scripts are successfully executing, just not all the time.

I used Process Monitor to see that the setup script was only executed once, and the cleanup script was only executed once, during the entire test run (the other attempts that are throwing ENONET just... not happening). This is so strange 🙃

Logs