Write .debug_frame information

[yurydelendik]

Requires bytecodealliance/cranelift#679 and #50

There is a problem when lldb is using wasmtime's frame information -- the JIT's debug frame information a) is not present, and b) debugger confused about which info to used.

This patch generates .debug_frame data for each function wasmtime generates. It can be added to the object file.

See also the patch for lldb to force using JIT's .debug_frame can be found at https://gist.github.com/yurydelendik/12e95a8eb474cc72b6f57a5e23091416

[yurydelendik]

/cc @philipc

This PR will be needed .debug_frame writing support -- currently using its own writer.

[jlb6740]

@yurydelendik Hi ... I am getting started reviewing the work you've done here and any related patch for producing debug information in WASM binaries. My two goals are to (1) be able to step through and look up values in WASM code being executed by WASMTIME and (2) to understand the .debug_frame enough to be able to add support for performance profile tools that rely on and take advantage of the debug_frame for mapping codes. For (1) I've simply compiled a fibonnaci example from C to WASM using (Clang-8/LLVM-8) and launched wasmtime in lldb-8. Granted, I am familiar with gdb more so than lldb, but from here I am unsure how to properly set breakpoints with lldb in the WASM code before execution and moreover I am also unsure if this patch is the only patch needed. There are 3 other patches you are working on that may be relevant:

• Updates to wasm2obj to produce stubs for C code #46
• Generate debug info for LLDB/GDB #50
• Transform ranges and simple expressions #63

You can state the state of completeness here as perhaps there is much more work needed, but if it is possible can you give me some steps and pointers on how to get started with (1) ... stepping through a WASM file in wasmtime with help from your patches. Also .. is there a way to see and dump out the .debug_frame you are writing?

Thanks

[yurydelendik]

be able to step through

The breaking and stepping through functionality is landed in the wasmtime. Try to follow the https://gist.github.com/yurydelendik/4103d782dfc62634bc9f4de98de20835 example for debugging

... and look up values in WASM code being executed by WASMTIME

This work is still at #63 and https://reviews.llvm.org/D52634

is there a way to see and dump out the .debug_frame you are writing?

Via wasm2obj (but it can be broken atm) or uncomment https://github.com/CraneStation/wasmtime/blob/master/wasmtime-debug/src/lib.rs#L86-L87. Please notice that the .debug_frame is not relevant to any of the tasks you are trying to perform: stepping and looking up values.

[yurydelendik]

(2) to understand the .debug_frame enough to be able to add support for performance profile tools that rely on and take advantage of the debug_frame for mapping codes.

@jlb6740 I'm trying to collect more information about this -- it is really hard to find any information how the .debug_frame is used in performance tools for JIT'ed code. The GDB JIT interface is only used by debuggers as far as I know.

At this moment, AOT/wasm2obj is the way to go, but it is really hard to use produced artifacts -- maybe #46 will give you some ideas but it is not a complete solution.

[jlb6740]

Thanks ... I've been able to step through code following your gist and also dump the debug output by uncommenting those lines: https://github.com/CraneStation/wasmtime/blob/master/wasmtime-debug/src/lib.rs#L86-L87 . I don't quite have an understanding though .. This patch here to generate a debug frame for each jitted function .. how is it intended to be used? It is not needed for things like stepping through code, looking up values, but is it used for unwinding stack frames? Are other related dwarf sections already emitted (looks like they are from the dump)? How is this section mapped into memory and used at runtime? There are performance tools that I believe can take advantage of this and other dwarf sections and I am trying to wrap my head around it all to understand gaps and help enable that functionality.

[yurydelendik]

is it used for unwinding stack frames?

Correct. It is used for unwinding stack frames, e.g. during debugging or crash. Debuggers have heuristics to walk stack without this information -- they rely on standard calling methods. As you noticed in the gist, this section could be useful when cranelift was generating bad prologue, though now it is not important. This sections is still useful for crash reports (and profilers).

How is this section mapped into memory and used at runtime?

For wasmtime (JIT), it directly refer jitted functions, and the runtime registers an ELF image (with this section),the dump you see by uncommenting the lines above, using GDB JIT interface. For wasm2obj (AOT), it creates relocatable records, so linker properly will point that to the right places.

[jlb6740]

@yurydelendik Hi .. Maybe I should start a new issue for this since this patch doesn't really target profiling. I am looking at profiling jitted wasmtime code in perf and vtune. Perf requires map files. Does wasmtime generate map files supported by perf .. or something similar that can be modified?

[bjorn3]

I dont know if wasmtime supports perf map files, but if it doesnt, you can take a look at bytecodealliance/cranelift#571 for an example how to implement it. (that one is for cranelift-simplejit)

[jlb6740]

I dont know if wasmtime supports perf map files, but if it doesnt, you can take a look at CraneStation/cranelift#571 for an example how to implement it. (that one is for cranelift-simplejit)

Cool! Thanks. I will take a look.

[yurydelendik]

Maybe I should start a new issue for this since this patch doesn't really target profiling

Yes.

Perf requires map files.

I have a feeling that amount of information /tmp/perf-PID.map file provides is not enough to do efficient profiling. I noticed something about JITDUMP in the perf source code -- this format contains eh_frame information and am planning to look at.

bytecodealliance/cranelift#571

👍

[tschneidereit]

@yurydelendik what's the status of this PR, are there parts that we should still land, or is this superseded by things that landed in the meantime?

[yurydelendik]

what's the status of this PR, are there parts that we should still land, or is this superseded by things that landed in the meantime?

There is no direct need in writing .debug_frame information -- debuggers now can successfully guess frame backtrace. Though I would like to convert this PR into "write .eh_frame information" issue -- there is a change we will need this information for cranelift's wasm exception handling.

[tschneidereit]

Ok, makes sense. Should we perhaps close this PR and you open a new one with that new purpose, though?

[bjorn3]

debuggers now can successfully guess frame backtrace

Last time I checked, gdb couldn't guess it when opt_level=none is used, because it didn't understand redundant rex prefixes for push rbp.

[yurydelendik]

gdb couldn't guess it when opt_level=none is used, because it didn't understand redundant rex prefixes for push rbp.

Soon I'm about to dive into compatibility with GDB, and I would like to track that independently in separate bug(s). AFAIK rex issue was addressed, at least for LLDB. I will rebase this PR anyway, e.g. to check the theory that it will help with GDB.

[bjorn3]

For reference: https://github.com/bjorn3/rustc_codegen_cranelift/issues/146#issuecomment-449474527

[yurydelendik]

@bjorn3 I rebased the PR. At this momennt I know there is plenty of code duplication, e.g. with writing fde or map_reg, but this PR can be used to produce object file with the proper .debug_frame.

Maybe after its landing, we can refactor it for .eh_frame support and move to the cranelift ?