CLI: interface for manully clean stale PID file

[unkcpz]

In 8d963c9 orphaned PID files can now only be removed by starting the daemon again. In order to provide a way to clean the stale pid file manually. In AiiDAlab docker stack, we run verdi daemon stop before database migration. But if the container is shut down ungracefully, the daemon is not properly stopped and leaves a stale PID file which causes the exception when running verdi daemon stop next time.
With the CLI interface in this PR, it allows running the verdi daemon clean-stale-pid-file first, to make sure verdi daemon stop will not block the starting of the container.

EDIT: the argument clean-stale-pid-file seems nasty, maybe call it verdi daemon clean?

[sphuber]

Thanks @unkcpz . When I refactored the daemon implementation, it was a request of @ltalirz to only have stale PID files be cleaned when starting the daemon. I agree though, that when this message is shown when asking the daemon status, it makes sense to me as well to resolve it by stopping the daemon. I believe the main problem @ltalirz had with the original implementation was that the PID was cleaned when asking the status as well. Which would be an unexpected side-effect for a status command. But I would argue that it makes sense for both starting and stopping the daemon.

So how about, instead of adding a new dedicated command, we just have stopping the daemon also clean stale PID files? Would that work for you @unkcpz and @ltalirz ?

[ltalirz]

That sounds reasonable to me as well, thanks Seb

[unkcpz]

Thanks @sphuber, it is even better for me. Then we just need to bump the aiida-core version in aiidalab-docker-stack. I update this PR.

[sphuber]

@unkcpz Please make sure to make the change in the DaemonClient.stop_daemon function and not in the verdi command. We want all functionality in the Python API so that its behavior and verdi are identical

[unkcpz]

Yes it is in DaemonClient.stop_daemon 😉

[sphuber]

Thanks @unkcpz

need additional changes

[sphuber]

Actually, sorry. I think we should also update the warning message if the stale PID is found. Now it suggest to start the daemon. I think suggesting to stop it makes more sense, or at least to suggest running either. As either will solve the problem now.

[unkcpz]

@sphuber thanks!

[danielhollas]

@unkcpz thanks for taking this on. @sphuber is the plan here to release this as part of 2.3.1? Or do we need to still have a workaround in AiiDAlab for now?

[sphuber]

I am indeed planning to release this with v2.3.1, but that is blocked by #6000 . As soon as that is merged, I can make the release. It has already been reviewed by @chrisjsewell but I needed to add one more commit, because it turns out that the refactoring of v2.0 broke the mechanism for any new migrations. They would work but not actually be committed.