contest: stop the VM after a crash?

[matttbe]

When quickly looking at the recent modifications, I noticed 290b9b0 reduced the timeout in case of crash.

To deal with that, in our CI scripts for MPTCP, we add CONFIG_PANIC_TIMEOUT=-1 in the kernel config. To avoid the VM to reboot and mess-up with automation, we launch virtme with --qemu-opts -no-reboot, but I don't know what's the equivalent with virtme-ng.

While at it, we also enabled CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC and CONFIG_BOOTPARAM_HUNG_TASK_PANIC.

Maybe it would be easier to maintain a ci.config file in the selftests? Or always set that in lib/vm.py?

[kuba-moo]

Interesting. I was trying to stay away from CONFIG_PANIC_TIMEOUT=-1 because when I do testing on real HW before PRs I see unrelated stack traces very often :( Maybe QEMU is much better tested. Let's keep this as an option if things start acting up again!

[arighi]

Interesting point, I'm wondering if with CONFIG_PANIC_TIMEOUT=-1 always gives enough time to flush the kernel log and channel it to the stderr of the guest via the virtio console. Maybe CONFIG_PANIC_TIMEOUT=1 can be more reliable (need to do some tests...)?

Usually I manage the timeout externally via timeout (see for example https://github.com/sched-ext/sched_ext/blob/sched_ext/.github/workflows/run-schedulers#L35), but having a panic timeout in the guest seems more elegant.

[arighi]

And I'd rather change the default boot options in vng, adding panic=N, instead of changing the default CONFIG_PANITC_TIMEOUT in the kernel config. In this way we can have the same behavior independently on the specific .config.

And the more I think about it, the more I'm convinced that we should always use -no-reboot with vng, even for interactive sessions and not just with with script mode.

Again, in this way we will have exactly the same behavior (by default) independently of the particular .config, or if we run a script vs an interactive session.

[kuba-moo]

panic=1 on command line and -no-reboot regardless of script vs interactive sounds perfect to me :)
Do you have the cycles to code it up, or should one of us take a stab?

[arighi]

Well... I've already implemented it :D arighi/virtme-ng#70

With this applied, if a panic happens, vng immediately exits, returning the special errcode 255. About interactive mode, I take it back... in interactive mode maybe it's more reasonable to just hang by default, because the user may want to attach a debugger, enter the qemu monitor, trigger a memory dump, etc. So for now I've changed only the behavior of script mode. Let me know if this seems reasonable to you.

[kuba-moo]

Hah :D
It's alright, there may indeed be use cases for not shutting down in interactive mode. And if we add a knob to enable it we can as well just use -a "panic=1" -o "-no-reboot" on our end. Most important thing we don't have to inject custom kernel options from the CI, I worry those will make repro harder for contributors :)