Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bisq does not handle SIGTERM properly #3884

Closed
wiz opened this issue Jan 9, 2020 · 5 comments · Fixed by #4047
Closed

Bisq does not handle SIGTERM properly #3884

wiz opened this issue Jan 9, 2020 · 5 comments · Fixed by #4047
Labels
is:critical https://bisq.wiki/Critical_bug is:priority PR or issue marked with this label is up for compensation re:Tor

Comments

@wiz
Copy link
Contributor

wiz commented Jan 9, 2020

[Background]
On posix/unix systems, when you send a SIGTERM to a process, it should save all data and gracefully shut down.

[How to Reproduce]

  1. Prepare a Bisq Seednode using installer script
  2. Start the Bisq Seednode service
  3. Wait a few minutes
  4. Stop the Bisq Seednode service

[Expected Result]
After step 4, Bisq should save all data and gracefully shut down within a few seconds.

[Actual Result]
Bisq's child Tor process gracefully shuts down, but Bisq does not. For the next 60 seconds, Bisq floods the logs with Tor connection failure errors, until systemd forcefully kill -9's the Java process

[Log Snippet]

Jan 10 08:15:05 wizvm bisq-seednode[68826]: #011message=GetPeersRequest
Jan 10 08:15:05 wizvm bisq-seednode[68826]: #011throwable=java.io.IOException: Cannot connect to hidden service
Jan 10 08:15:05 wizvm bisq-seednode[68826]: #033[0;39m#033[34mJan-10 08:15:05.939 [SeedNodeMain] INFO  b.n.p.n.NetworkNode: onFailure at sendMessage: peersNodeAddress=extj4rlmchb3zzzn.onion:9999
Jan 10 08:15:05 wizvm bisq-seednode[68826]: #011message=GetPeersRequest
Jan 10 08:15:05 wizvm bisq-seednode[68826]: #011throwable=java.io.IOException: Cannot connect to hidden service
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #033[0;39m#033[34mJan-10 08:15:06.339 [NetworkNode:SendMessage-to-bcrhutzsmvgvp6lh.onion:9999] INFO  b.n.p.n.NetworkNode: onFailure at sendMessage: peersNodeAddress=bcrhutzsmvgvp6lh.onion:9999
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #011message=GetPeersRequest
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #011throwable=java.io.IOException: Cannot connect to hidden service
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #033[0;39m#033[34mJan-10 08:15:06.540 [SeedNodeMain] INFO  b.n.p.n.NetworkNode: onFailure at sendMessage: peersNodeAddress=ikicl65q45twvmyu.onion:9999
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #011message=GetPeersRequest
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #011throwable=java.io.IOException: Cannot connect to hidden service
Jan 10 08:15:06 wizvm systemd[1]: bisq-seednode.service: State 'stop-sigterm' timed out. Killing.
Jan 10 08:15:06 wizvm bisq-seednode[68826]: #033[0;39m
Jan 10 08:15:06 wizvm systemd[1]: bisq-seednode.service: Main process exited, code=killed, status=9/KILL
wiz added a commit to wiz/bisq that referenced this issue Jan 25, 2020
@wiz
Seednode must be shutdown using kill -9 until bisq-network#3884 is …
2b6f988 
…fixed
@wiz wiz mentioned this issue Jan 25, 2020
Merged
wiz added a commit to wiz/bisq that referenced this issue Jan 25, 2020
@wiz
Seednode must be shutdown using kill -9 until bisq-network#3884 is …
c6401b5 
…fixed
@ripcurlx
Copy link
Contributor

@freimair Could you have a look at this? Thanks!

@wiz
Copy link
Contributor Author

wiz commented Jan 28, 2020

To be clear, this issue is not related to Tor, that is merely a symptom of the Tor process properly handling SIGTERM and Bisq ignoring it

ripcurlx pushed a commit that referenced this issue Jan 29, 2020
@wiz
Various improvements to seednode service / install scripts (#3882)
02e7448 
* Split seednode systemd service ExecStart command into multiple lines

* Add setting in seednode configuration to specify btcnode host/port

* Tweak seednode torrc configuration options to improve P2P connectivity

* Require bitcoin.service from bisq-seednode.service via systemd binding

* Make seednode installer run from master and build bisq from release tag

* Seednode must be shutdown using `kill -9` until #3884 is fixed

* Fix seednode uninstall script to use correct service names

* Disable CircuitBuildTimeout in seednode Tor configuration

* Disable seednode torrc advanced configuration options for now
@freimair freimair mentioned this issue Mar 2, 2020
Merged
@wiz
Copy link
Contributor Author

wiz commented Mar 3, 2020

@ripcurlx @freimair can you guys prioritize this issue? it really affects running bisq on linux because if you reboot a server you have a ~10% (guessing) chance of corrupting the bisq node from not getting shut down properly. one of my nodes got corrupted again today.

@ripcurlx ripcurlx added is:critical https://bisq.wiki/Critical_bug is:priority PR or issue marked with this label is up for compensation labels Mar 10, 2020
@freimair
Copy link
Contributor

I will look into it

@freimair
Copy link
Contributor

freimair commented Mar 11, 2020 •
edited
Loading

I have not seen any graceful shutdown attempts when shutting down any Bisq app product (except for the monitor) when not using the GUI. I strongly believe that adding such a thing will affect #4026.

@freimair freimair mentioned this issue Mar 11, 2020
Merged
@wiz wiz mentioned this issue Mar 14, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
is:critical https://bisq.wiki/Critical_bug is:priority PR or issue marked with this label is up for compensation re:Tor
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Added graceful shutdown hook freimair/bisq
3 participants
@ripcurlx @wiz @freimair