Test framework seems broken in certain circumstances

[jorgensigvardsson]

I have found that finit fails unless it's been configured with --without-random-seed. For some reason, when finit tries to copy 32 KB of random data from /dev/urandom to /var/lib/misc/random-seed it only manages to copy 512 bytes. Not really sure why this is: maybe my machine is a very poor source of entropy? Maybe the test README should mention this?

Recompiling/reconfiguring finit with --without-random-seed makes test succeed on my machine, but the teardown phase causes the tests to superficially fail. It is this block of code that causes the failure:

    if [ -d "$TENV_ROOT/var/lock" ]; then
        chmod +r "$TENV_ROOT/var/lock"
    fi

This code generates the error message:

chmod: changing permissions of '/home/jorsi/Projects/finit/./test/tenv-root/var/lock': Operation not permitted

Upon examination, this is what the directory listing of the test environments var directory looks like:

$ ls -l test/tenv-root/var
total 40
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 cache
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 db
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 empty
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 games
drwxr-xr-x 5 jorsi jorsi 4096 feb 14 08:29 lib
lrwxrwxrwx 1 jorsi jorsi    9 feb 14 08:29 lock -> /run/lock
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 log
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 mail
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 opt
lrwxrwxrwx 1 jorsi jorsi    4 feb 14 08:29 run -> /run
drwxr-xr-x 3 jorsi jorsi 4096 feb 14 08:29 spool
drwxr-xr-x 2 jorsi jorsi 4096 feb 14 08:29 tmp

lock is a symlink pointing to /run/lock. I think the teardown phase is running outside the test context, the symlink is resolved to my machine's /run/lock, and that's where the operation not permitted comes from. I may be wrong about the code not being run in the test context - but a quick reading of the test frame work code, it sure looks like it.

I think, we need to reparse all/any path inside the test environment. If the path being touched is behind any symlink, we have to evaluate them against test/tenv-root.

Not sure why I am the one finding this issue, but it could be that my Linux setup differs from everybody else's. After all, this is only a problem if the symlink points to something that exists on the host machine.

[troglobit]

Hmm, I see the contours of at least two distinct problems here.

1. The way we run Finit in the tests, in an unshare(1), can compared to the container use-case. For which many of the plugins are not really applicable; modprobe/modules-load, urandom, rtc, tty, hotplug, etc. So in the case of saving random-seed it could also be that the unshare doesn't have the privileges to read -- could be prevented by AppArmor or similar. I think we should remove all these plugins and have a separate test suite that runs Finit in qemu instances for various targets to verify these.
2. The reason you're the only one who sees the /var/lock warning is probably that you've checked the test log 😏 I've just basically run make check since Jacques and the others handed it off ... but yeah, I've seen it and we should definitely fix it. The simplest fix is probably to change to a relative symlink.

[jorgensigvardsson]

Also, to be clear: I have not checked the test log. I get this issue from a clean build followed by make check.

[jorgensigvardsson]

I think this is an issue with the test environment. The PR I'm about to issue will not change finit itself, but rather the setup of tenv-root IMO.

edit: I have seen the setup() function in finit, which is why I'm hesitant to change the test subject.

[troglobit]

Also, to be clear: I have not checked the test log. I get this issue from a clean build followed by make check.

Interesting, I don't see that from a make check at the top directory. Maybe there's a difference in output from a newer autoconf, I guess you're on a modern installation? I'm on Linux Mint 20.3, based on Ubuntu 20.04 ... a lot has happened since, I hear.

Good, I always found it annoying some errors were hidden in the log files.

[troglobit]

Looking forward to the PR for the symlinks. I'll have a look myself at installing only the necessary plugins in the tenv-root.

[jorgensigvardsson]

Sorry, didn't have time to investigate further today. Got wound up in meetings, release work on other products, and the family Valentine's dinner.

I think there's an easy way through this, but I'm not convinced we should do the workaround. What puzzles me is: why is lib.sh trying to make the lock file readable in the teardown? If I just comment that piece out, all tests pass. If all tests pass with that code eliminated, it begs the question: is it really needed?

[troglobit]

To anyone arriving here, we've now done a lot of cleanup and simplification of the test system. We're still running in an unshare, so more actual test targets are needed to be automated. As it looks now, adding that to the existing test framework is not likely possible, or something we'd want, but a "farm" of test servers (qemu machines) that can build and deploys test images on push to Finit would be nice. Currently I'm running all this manually using a combination of myLinux and Westermo NetBox.