Add tracing tools and documentation #898
Conversation
These tools utilize the tracepoints added in mmtk#883 Co-authored-by: Claire Huang <claire.x.huang@gmail.com>
|
Directory structure: I think this tracing tool will not be the only tool we provide. I recommend adding one level of sub-directory, such as |
tools/run.py
Outdated
| with tempfile.NamedTemporaryFile(mode="w+t") as tmp: | ||
| content = template.safe_substitute( | ||
| MMTK=mmtk_bin, TMP_FILE=tmp.name) | ||
| if args.print_script: | ||
| print(content) | ||
| tmp.write(content) | ||
| tmp.flush() | ||
| os.execvp("sudo", ["sudo", args.bpftrace, | ||
| "--unsafe", "-f", args.format, tmp.name]) |
There was a problem hiding this comment.
Python will automatically delete the NamedTemporaryFile when exiting the scope of with. However, execvp never returns. You can replace this with subprocess.run or subprocess.Popen.
An alternative to using temp file is using pipes. I tried piping the script into sudo bpftrace and it works. For example, I am currently using erb (from Ruby) for templating, and my command line looks like this:
erb record_every=50 so_path=/home/wks/projects/mmtk-github/openjdk/build/linux-x86_64-normal-server-release/jdk/lib/server/libmmtk_openjdk.so gcvis.erb | sudo bpftrace - > openjdk-packet-size3-nopoll.logIn Python, the subprocess.run function (or popen.communicate) allows you to specify the stdin as a pipe, so you can directly pipe the Python string prologue + content + epilogue into sudo bpftrace.
There was a problem hiding this comment.
That's a deliberate choice. I will add comments to explain this.
tools/epilogue.bt.fragment
Outdated
| END { | ||
| system("rm $TMP_FILE"); | ||
| } |
There was a problem hiding this comment.
We only need to delete the temp file if we use exec. See the comment on run.py.
tools/README.md
Outdated
| ### Measuring the time spend in different GC stages (`gc_stages`) | ||
| This tool measures the time spent in different stages of GC: before `Closure`, during `Closure`, and after `Closure`. | ||
| The time unit is ns. |
There was a problem hiding this comment.
A general style problem: I find that you never uses newline characters within one paragraph. Markdown allows this.
But here the text spans over three lines. I think you intend to introduce one title and two paragraphs. However, Markdown will interpret consecutive non-blank lines as one paragraph (except for titles (### blah blah blah)). If you intend to put "The time unit is ns." in a single paragraph, you need to add an empty line before it.
I also recommend formatting the text so that the line width is no more than 100 characters long (or 80 or 72 if you prefer). The fmt command line utility may come in handy. If you are a Vim user, you can set tw=100, select one or more paragraphs and press gq to format them. In this way, this text file will still be easy to read using text editors.
tools/README.md
Outdated
| In the above output, we can see that the lock instance (140637228007056, or 0x7fe8a8047e90) roughly has a bimodal distribution in terms of the time spent in lock contended code path. The first peak is around 512ns\~1024ns, and the second peak is around 66us\~131us. | ||
| If you can't tell which lock instance is for which lock in MMTk, you can trace the allocation of the Mutex and record the stack trace (note that you might want to compile MMTk with `force-frame-pointers` to obtain better stack traces). |
There was a problem hiding this comment.
If they are two paragraphs, there needs to be an empty line in between.
Yeah, that makes sense. Fixed. |
caizixian
added
A-utils
|
I am confused about the time "units". Do you really mean units or do you mean resolution? The text seems to use seconds (or us / ns) as the units. Other than that, this looks excellent. Nicely documented and packaged. |
@steveblackburn Yes I meant units. Just to clarify, it is about how much one time unit is in the tracing output. We sometimes scale the numbers to have better resolution on the log2-based histogram, where the numbers on the histogram bins are in the time unit. This is different from the measurement resolution. |
Co-authored-by: Kunshan Wang <wks1986@gmail.com>
Co-authored-by: Kunshan Wang <wks1986@gmail.com>
Co-authored-by: Kunshan Wang <wks1986@gmail.com>
These tools utilize the tracepoints added in mmtk#883. The GC visualization tools requires some post-processing and is just a bit more complicated in general. That will be added in a separate PR. --------- Co-authored-by: Claire Huang <claire.x.huang@gmail.com>
These tools utilize the tracepoints added in #883. The GC visualization tools requires some post-processing and is just a bit more complicated in general. That will be added in a separate PR.