-
Notifications
You must be signed in to change notification settings - Fork 737
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Tool to Dump Runtime State #36284
Comments
Another option would be to use JVM signal handling mechanism to dump runtime state. As an example java use SGQUIT to generate thread dump. Go also use similar mechanism to generate stack-traces for go routines. https://kostenko.org/blog/2019/08/java-catch-os-signals.html |
Since Jballerina runtime use standard error stream to output logs We can dump the strands information to standard out. |
We have updated the proposal with latest design. @sameerajayasoma @manuranga Let us know if you have any feedback. |
What is the name of the strand created by |
Currently we are not giving any information other than the above. |
The proposal is updated again with the latest design. Example Strand Dump Outputs1. Service which handles a high load of http requestsBallerina code import ballerina/http;
listener http:Listener httpListener = new (8080);
service / on httpListener {
resource function get greeting() returns string {
return "Hello, World!";
}
} Using jmeter During the execution, strand dump is obtained multiple times. The obtained output is given below. (click to expand)
2. Ballerina code with large running loopsDuring the execution, strand dump is obtained multiple times.The obtained output is given below. (click to expand)
|
Fixed with #36767. Also updated the proposal with latest design details and example outputs. |
Summary
This tool will provide the user, the ability to dump runtime state information in human readable format during the execution of a Ballerina program. This can be used to do the following.
Goals
Tool should be able to dump the relevant runtime information in human readable format. Those information include the status of the schedulers, status of the strands and their stack trace, strand groups, etc.
Motivation
Ballerina runtime can have unexpected behaviours due to user code errors, bugs or issues with the running environment. These will result in memory leaks, cpu spinning, runtime hangs, performance degradation or crashes with various exceptions. These will be critical when they happen in a production environment.
Description
Strand Dump Tool
Runtime will implement APIs to extract required information from strands. The user will send a POSIX signal possibly using the
kill
CLI command, or a command line tool will be provided, to extract that information to plain text, which will be outputted to the standard out (System.out
) stream.In other words, to get the strand dump when a Ballerina program is running, the user needs to get the PID of the Ballerina program (possibly using
jps
) and then send theSIGTRAP
signal to that process.Eg:
kill -5 $PID
or
kill -s TRAP $PID
(Note: Though the above can be done in both MacOS and Linux, in Windows
SIGTRAP
signal is not available. This option was taken as mostly the environment the containers are running is Linux.)Example Strand Dump Output
Consider the following Ballerina program. (click to expand)
Obtained strand dump would be as follows.
Strand Dump Output Format
Available Strand Group States
Available Strand States
Remarks
Since we need to provide the ability to the runtime state dump for the Windows users, one of the following options will also be provided.
Option 1 - Add to Ballerina CLI
Following CLI command will be used to dump the runtime state of an executing Ballerina program whose PID will be provided by the user.
bal dump-state $PID
Cons: Might not work seamlessly inside Docker, since Ballerina needs to be installed to make the CLI work.
Option 2 - Add to executable jar
Dumping the runtime state can be invoked by the following command.
The text was updated successfully, but these errors were encountered: