Performance profiling for WebAssembly

[SteveSandersonMS]

Tentative goals

• annotate Mono Interp with event pipe events
• annotate Mono AOT with event pipe events
  • FRAME_TYPE_IL_STATE
• annotate AOT/Interp transitions
• annotate JS interop with event pipe events
• explore how event pipe output could be used for PGO AOT in Mono
  • [mono] Profiled AOT - PGO tracking  #69512
• use stats about Interp of runtime libraries to bring more intrinsic methods to Interp
• propagate event pipe events as performance.mark() and performance.measure()
  • optional, configurable
  • HTTP header Timing-Allow-Origin
• propagate browser events as event pipe events
• implement convertors: dotnet trace convert --help

Related #69268

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[Stamo-Gochev]

@SteveSandersonMS Can you share more information where this profiler can be found? I see that it is internal, but I can try to rebuild the source code on my side to expose it in some way as I want to debug a wasm app for performance reasons. I have no expectations for it to be working as you mentioned it is targeting 6.0 or later, but I can give it a try for testing purposes.

[Stamo-Gochev]

@SteveSandersonMS Any updates?

[SteveSandersonMS]

The experimental code we used is in https://github.com/dotnet/runtime/tree/stevesa/hierarchical-profile, however it's not intended for external use, hence there's no information about how to build or use it, or how to understand any data it collects, and it most likely no longer works with current builds of the runtime.

[szalapski]

Would like to see a way to profile Blazor on the client, with an eye toward optimizing performance. Main scenario: I just clicked on an item and it worked 20 seconds before I can see the new UI, and I am quite certain it is not due to download time/network latency; what's taking so long that I should try to optimize?

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[jirisykora83]

@SteveSandersonMS Any updates?

[jirisykora83]

Would like to see a way to profile Blazor on the client, with an eye toward optimizing performance. Main scenario: I just clicked on an item and it worked 20 seconds before I could see the new UI, and I am quite certain it is not due to download time/network latency; what's taking so long that I should try to optimize?

I face similar issue and one workround is create BaseComponent add it to _Imports.razor as base class for every component @inherits XXX.BaseComponent and then use component live cycle with https://developer.mozilla.org/en-US/docs/Web/API/console/time this give me at least some insight.

This is inspired by https://remibou.github.io/Profiling-in-Blazor/

console.time produce some sort of flamegraph in chrome. This is nowhere near the dev experience with full .NET profiler of course.

[MertUssakliMSFT]

If there was at least a way to relate compiled wasm-functionss you see in the Edge profiler back to .NET code symbols, that would be a huge step forward! I can profile and see CPU-heavy workload causing UI delays, but it is impossible to debug it back to what part of the code would need optimization.

[Chris-D-Turk]

@MertUssakliMSFT you can actually see the real method-names if you enable AOT compilation in your csproj:

    <RunAOTCompilation>true</RunAOTCompilation>
    <RunAOTCompilationAfterBuild>true</RunAOTCompilationAfterBuild>
    <WasmNativeStrip>false</WasmNativeStrip>

But the build takes a very long time with this config - ~10min for a new blazor wasm solution on my machine.
Maybe you can speed up the build, if your strip unused assemblies - but I don't know how to do this.

[MertUssakliMSFT]

@Chris-D-Turk, cool tip -- will keep this in mind the next time I need to do perf profiling.

Does it work in Release/Ship as well as Debug?

[Chris-D-Turk]

@MertUssakliMSFT yes, it works for both debug and release builds

[zHaytam]

Any news about this?

[ghost]

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In 5.0 we implemented an internal prototype of profile capture for dotnet.wasm. Let's look at making this into part of the product for 6.0.

Author:	SteveSandersonMS
Assignees:	-
Labels:	area-Diagnostics-coreclr
Milestone:	-

[mkArtakMSFT]

@lewing let's include this in an agenda for our weekly sync meetings. We'd like to have something for customers here in .NET 8.

[pavelsavara]

Notes to self
#70851

dotnet-trace collect --providers Microsoft-Windows-DotNETRuntime:0x1F000080018:5

runtime/src/coreclr/vm/ClrEtwAll.man

Lines 14 to 92 in fb104e3

	<!--Keywords-->
	<keywords>
	<keyword name="GCKeyword" mask="0x1"
	message="$(string.RuntimePublisher.GCKeywordMessage)" symbol="CLR_GC_KEYWORD"/>
	<keyword name="GCHandleKeyword" mask="0x2"
	message="$(string.RuntimePublisher.GCHandleKeywordMessage)" symbol="CLR_GCHANDLE_KEYWORD"/>
	<keyword name="AssemblyLoaderKeyword" mask="0x4"
	message="$(string.RuntimePublisher.AssemblyLoaderKeywordMessage)" symbol="CLR_ASSEMBLY_LOADER_KEYWORD"/>
	<keyword name="LoaderKeyword" mask="0x8"
	message="$(string.RuntimePublisher.LoaderKeywordMessage)" symbol="CLR_LOADER_KEYWORD"/>
	<keyword name="JitKeyword" mask="0x10"
	message="$(string.RuntimePublisher.JitKeywordMessage)" symbol="CLR_JIT_KEYWORD"/>
	<keyword name="NGenKeyword" mask="0x20"
	message="$(string.RuntimePublisher.NGenKeywordMessage)" symbol="CLR_NGEN_KEYWORD"/>
	<keyword name="StartEnumerationKeyword" mask="0x40"
	message="$(string.RuntimePublisher.StartEnumerationKeywordMessage)" symbol="CLR_STARTENUMERATION_KEYWORD"/>
	<keyword name="EndEnumerationKeyword" mask="0x80"
	message="$(string.RuntimePublisher.EndEnumerationKeywordMessage)" symbol="CLR_ENDENUMERATION_KEYWORD"/>
	<!-- Keyword mask 0x100 is now defunct -->
	<!-- Keyword mask 0x200 is now defunct -->
	<keyword name="SecurityKeyword" mask="0x400"
	message="$(string.RuntimePublisher.SecurityKeywordMessage)" symbol="CLR_SECURITY_KEYWORD"/>
	<keyword name="AppDomainResourceManagementKeyword" mask="0x800"
	message="$(string.RuntimePublisher.AppDomainResourceManagementKeywordMessage)" symbol="CLR_APPDOMAINRESOURCEMANAGEMENT_KEYWORD"/>
	<keyword name="JitTracingKeyword" mask="0x1000"
	message="$(string.RuntimePublisher.JitTracingKeywordMessage)" symbol="CLR_JITTRACING_KEYWORD"/>
	<keyword name="InteropKeyword" mask="0x2000"
	message="$(string.RuntimePublisher.InteropKeywordMessage)" symbol="CLR_INTEROP_KEYWORD"/>
	<keyword name="ContentionKeyword" mask="0x4000"
	message="$(string.RuntimePublisher.ContentionKeywordMessage)" symbol="CLR_CONTENTION_KEYWORD"/>
	<keyword name="ExceptionKeyword" mask="0x8000"
	message="$(string.RuntimePublisher.ExceptionKeywordMessage)" symbol="CLR_EXCEPTION_KEYWORD"/>
	<keyword name="ThreadingKeyword" mask="0x10000"
	message="$(string.RuntimePublisher.ThreadingKeywordMessage)" symbol="CLR_THREADING_KEYWORD"/>
	<keyword name="JittedMethodILToNativeMapKeyword" mask="0x20000"
	message="$(string.RuntimePublisher.JittedMethodILToNativeMapKeywordMessage)" symbol="CLR_JITTEDMETHODILTONATIVEMAP_KEYWORD"/>
	<keyword name="OverrideAndSuppressNGenEventsKeyword" mask="0x40000"
	message="$(string.RuntimePublisher.OverrideAndSuppressNGenEventsKeywordMessage)" symbol="CLR_OVERRIDEANDSUPPRESSNGENEVENTS_KEYWORD"/>
	<keyword name="TypeKeyword" mask="0x80000"
	message="$(string.RuntimePublisher.TypeKeywordMessage)" symbol="CLR_TYPE_KEYWORD"/>
	<keyword name="GCHeapDumpKeyword" mask="0x100000"
	message="$(string.RuntimePublisher.GCHeapDumpKeywordMessage)" symbol="CLR_GCHEAPDUMP_KEYWORD"/>
	<keyword name="GCSampledObjectAllocationHighKeyword" mask="0x200000"
	message="$(string.RuntimePublisher.GCSampledObjectAllocationHighKeywordMessage)" symbol="CLR_GCHEAPALLOCHIGH_KEYWORD"/>
	<keyword name="GCHeapSurvivalAndMovementKeyword" mask="0x400000"
	message="$(string.RuntimePublisher.GCHeapSurvivalAndMovementKeywordMessage)" symbol="CLR_GCHEAPSURVIVALANDMOVEMENT_KEYWORD"/>
	<keyword name="GCHeapCollectKeyword" mask="0x800000"
	message="$(string.RuntimePublisher.GCHeapCollectKeyword)" symbol="CLR_GCHEAPCOLLECT_KEYWORD"/>
	<keyword name="GCHeapAndTypeNamesKeyword" mask="0x1000000"
	message="$(string.RuntimePublisher.GCHeapAndTypeNamesKeyword)" symbol="CLR_GCHEAPANDTYPENAMES_KEYWORD"/>
	<keyword name="GCSampledObjectAllocationLowKeyword" mask="0x2000000"
	message="$(string.RuntimePublisher.GCSampledObjectAllocationLowKeywordMessage)" symbol="CLR_GCHEAPALLOCLOW_KEYWORD"/>
	<keyword name="PerfTrackKeyword" mask="0x20000000"
	message="$(string.RuntimePublisher.PerfTrackKeywordMessage)" symbol="CLR_PERFTRACK_KEYWORD"/>
	<keyword name="StackKeyword" mask="0x40000000"
	message="$(string.RuntimePublisher.StackKeywordMessage)" symbol="CLR_STACK_KEYWORD"/>
	<keyword name="ThreadTransferKeyword" mask="0x80000000"
	message="$(string.RuntimePublisher.ThreadTransferKeywordMessage)" symbol="CLR_THREADTRANSFER_KEYWORD"/>
	<keyword name="DebuggerKeyword" mask="0x100000000"
	message="$(string.RuntimePublisher.DebuggerKeywordMessage)" symbol="CLR_DEBUGGER_KEYWORD" />
	<keyword name="MonitoringKeyword" mask="0x200000000"
	message="$(string.RuntimePublisher.MonitoringKeywordMessage)" symbol="CLR_MONITORING_KEYWORD" />
	<keyword name="CodeSymbolsKeyword" mask="0x400000000"
	message="$(string.RuntimePublisher.CodeSymbolsKeywordMessage)" symbol="CLR_CODESYMBOLS_KEYWORD" />
	<keyword name="EventSourceKeyword" mask="0x800000000"
	message="$(string.RuntimePublisher.EventSourceKeywordMessage)" symbol="CLR_EVENTSOURCE_KEYWORD" />
	<keyword name="CompilationKeyword" mask="0x1000000000"
	message="$(string.RuntimePublisher.CompilationKeywordMessage)" symbol="CLR_COMPILATION_KEYWORD" />
	<keyword name="CompilationDiagnosticKeyword" mask="0x2000000000"
	message="$(string.RuntimePublisher.CompilationDiagnosticKeywordMessage)" symbol="CLR_COMPILATIONDIAGNOSTIC_KEYWORD" />
	<keyword name="MethodDiagnosticKeyword" mask="0x4000000000"
	message="$(string.RuntimePublisher.MethodDiagnosticKeywordMessage)" symbol="CLR_METHODDIAGNOSTIC_KEYWORD" />
	<keyword name="TypeDiagnosticKeyword" mask="0x8000000000"
	message="$(string.RuntimePublisher.TypeDiagnosticKeywordMessage)" symbol="CLR_TYPEDIAGNOSTIC_KEYWORD" />
	<keyword name="JitInstrumentationDataKeyword" mask="0x10000000000"
	message="$(string.RuntimePublisher.JitInstrumentationDataKeywordMessage)" symbol="CLR_JITINSTRUMENTEDDATA_KEYWORD" />
	<keyword name="ProfilerKeyword" mask="0x20000000000"
	message="$(string.RuntimePublisher.ProfilerKeywordMessage)" symbol="CLR_PROFILER_KEYWORD" />
	</keywords>

runtime/src/mono/mono/eventpipe/ep-rt-mono.c

Lines 220 to 240 in fb104e3

	#define GC_KEYWORD 0x1
	#define GC_HANDLE_KEYWORD 0x2
	#define LOADER_KEYWORD 0x8
	#define JIT_KEYWORD 0x10
	#define APP_DOMAIN_RESOURCE_MANAGEMENT_KEYWORD 0x800
	#define CONTENTION_KEYWORD 0x4000
	#define EXCEPTION_KEYWORD 0x8000
	#define THREADING_KEYWORD 0x10000
	#define GC_HEAP_DUMP_KEYWORD 0x100000
	#define GC_ALLOCATION_KEYWORD 0x200000
	#define GC_MOVES_KEYWORD 0x400000
	#define GC_HEAP_COLLECT_KEYWORD 0x800000
	#define GC_FINALIZATION_KEYWORD 0x1000000
	#define GC_RESIZE_KEYWORD 0x2000000
	#define GC_ROOT_KEYWORD 0x4000000
	#define GC_HEAP_DUMP_VTABLE_CLASS_REF_KEYWORD 0x8000000
	#define METHOD_TRACING_KEYWORD 0x20000000
	#define TYPE_DIAGNOSTIC_KEYWORD 0x8000000000
	#define TYPE_LOADING_KEYWORD 0x8000000000
	#define MONITOR_KEYWORD 0x10000000000
	#define METHOD_INSTRUMENTATION_KEYWORD 0x40000000000

[jjzhang12]

@pavelsavara : How can I enable the browserProfilerOptions of #77449 in Blazor WebAssembly?

[pavelsavara]

@jjzhang12 with latest previews you can use new API configureRuntime which allows you to configure runtime.
I tested with 8.0.0-rc.1.23402.3 wasm-tools workload installed.

In your project file <WasmProfilers>browser;</WasmProfilers><WasmBuildNative>true</WasmBuildNative>
And in your index.html

<script src="_framework/blazor.webassembly.js" autostart="false"></script>
<script>
Blazor.start({
    configureRuntime: function (builder) {
        builder.withConfig({
            browserProfilerOptions: {}
        });
    }
});
</script>

[jjzhang12]

@pavelsavara Thanks, I installed the latest .net 8 preview 7 and implemented the above change to the template sample app, I still see "wasm-functions[]" in performance profile. I tried debug, release, published and published with AOT, all same result.
I saw you are using rc version (8.0.0-rc.1.23402.3), how can I get that wasm-tools?

[pavelsavara]

I still see "wasm-functions[]" in performance profile.

Those are C symbols of the mono VM. You can enable them by
<WasmNativeDebugSymbols>true</WasmNativeDebugSymbols><WasmNativeStrip>false</WasmNativeStrip>

Note that is being improved right now #89754

I saw you are using rc version (8.0.0-rc.1.23402.3), how can I get that wasm-tools?

Wait for next preview to be released or use nightly build https://aka.ms/dotnet/8.0.1xx/daily/dotnet-sdk-win-x64.zip

[jjzhang12]

@pavelsavara Thank you very much. Adding those two flags works perfectly.
Now waiting for the full performance porfiler to be released:-)

[ts-indikaf]

@jjzhang12 with latest previews you can use new API configureRuntime which allows you to configure runtime. I tested with 8.0.0-rc.1.23402.3 wasm-tools workload installed.

In your project file <WasmProfilers>browser;</WasmProfilers><WasmBuildNative>true</WasmBuildNative> And in your index.html

<script src="_framework/blazor.webassembly.js" autostart="false"></script>
<script>
Blazor.start({
    configureRuntime: function (builder) {
        builder.withConfig({
            browserProfilerOptions: {}
        });
    }
});
</script>

@pavelsavara Is there a solution for pure WebAssembly projects?

[pavelsavara]

<WasmProfilers>browser;</WasmProfilers><WasmBuildNative>true</WasmBuildNative>

import { dotnet } from './dotnet.js'
const runtime  = dotnet
        .withConfig({
            browserProfilerOptions: {}
        })
        .create()
;

[ts-indikaf]

@pavelsavara thanks - it works.

[jjzhang12]

@pavelsavara : I can profile my app in Preview 7. But for the same Blazor app in rc2, if I turn on those flags, the page goes out of memory quickly. I did look at the task bar and saw the Edge used more than 4GB memory before the page crashes. Without the profile flags, the memory is about 350MB.

I still can profile a simple app though.

Do we have option to reduce the memory consumption of this profile feature? I mean, back to preview 7 level for example.

[pavelsavara]

This is logging, not sampling profiler and so it collects everything. (besides memory it also consumes CPU and affects timing).
I'm not aware of any recent changes to profiler which would cause more calls to be reported.

Maybe some you have some build flags which make this to generate more methods or more calls in the profiled code ?

[ts-indikaf]

@pavelsavara thanks - it works.

@pavelsavara Unfortunately, this only works with the sample DotNet project not with my real Wasm app. Chrome Profile loading goes forever with the real app. I'm using .DotNet8 RC1.

[MattParkerDev]

@pavelsavara How much effort is involved in adding method filtering to #77449?

[pavelsavara]

@pavelsavara How much effort is involved in adding method filtering to #77449?

It would be good to filter in early on Mono side, probably here

runtime/src/mono/mono/profiler/browser.c

Line 70 in f39dae8

// TODO filter by namespace ?

Similar to

runtime/src/mono/mono/profiler/log.c

Lines 1974 to 1976 in f39dae8

	if (log_config.callspec.len > 0 &&
	!mono_callspec_eval (method, &log_config.callspec))
	return MONO_PROFILER_CALL_INSTRUMENTATION_NONE;

And you need to pass the configuration flags
this is how other mono profilers do it

runtime/src/mono/mono/profiler/coverage.c

Line 1053 in f39dae8

parse_args (const char *desc)

I'm not sure this is best way how to do it together with JavaScript. I would prefer to make it work with browserProfilerOptions

[ocoka]

Hello for community !

I am interested in profiling my project and have read the whole discussion, but I still don't understand clearly:

1. What exactly are the minimum parameters I need in a .csproj file ?
2. Can this be used with dotnet run ?
3. Does there have to be a Release configuration ?
4. and what is the purpose of the expression in Blazor index.html:

   Blazor.start({
            configureRuntime: function (builder) {
                builder.withConfig({
                    browserProfilerOptions: {}
                });
            }
        });

I tested the solution on .NET8, done with dotnet publish -c Debug, and served from Nginx, and having in the .csproj configuration:

 <RunAOTCompilation>true</RunAOTCompilation>.
 <RunAOTCompilationAfterBuild>true</RunAOTCompilationAfterBuild>
 <WasmNativeStrip>false</WasmNativeStrip>
 <WasmProfilers>browser;</WasmProfilers>
 <WasmBuildNative>true</WasmBuildNative>

This does crash in Google Chrome with a SIGILL signal, but Firefox survives and profiling works. Maybe I don't need the whole list of above parameters anymore since I have .NET8 ?

[tibuprophen]

Hi! I am also having some trouble enabling profiling on WASM. May someone clear it up how to set up profiling properly? I achived chrome profiling on a fresh web project with changes @ocoka posted. Is there any option without AOT complilation and publishing the project?

[EliseoDiegoViola]

Any news about this? AOT needs to be a release build only, making debugging annoying.
What are exactly the combination of flags and code that you need to use for it to work on NET 9?

Edit : I partially made it work without AOT as @pavelsavara recommended. Sadly in my Web App, is unusable. Memory usage goes up like crazy for no good reason and chrome crashes. Firefox can go up indefinitely until the app becomes so slow that it cant be of any usage anymore. In an attempt to find the culprit I tried to deactivate different plugins to see what happens, Serilog seems to be doing something, and Fluxor as well. Still not sure what they are doing that makes the app behave like that.

[MikiraSora]

Any news about this? AOT needs to be a release build only, making debugging annoying. What are exactly the combination of flags and code that you need to use for it to work on NET 9?

Edit : I partially made it work without AOT as @pavelsavara recommended. Sadly in my Web App, is unusable. Memory usage goes up like crazy for no good reason and chrome crashes. Firefox can go up indefinitely until the app becomes so slow that it cant be of any usage anymore. In an attempt to find the culprit I tried to deactivate different plugins to see what happens, Serilog seems to be doing something, and Fluxor as well. Still not sure what they are doing that makes the app behave like that.

Same problem in my Avalonia browser.