Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a benchmark for measuring call overhead #3883

Merged
merged 1 commit into from
Mar 4, 2022

Conversation

alexcrichton
Copy link
Member

The goal of this new benchmark, call, is to help us measure the
overhead of both calling into WebAssembly from the host as well as
calling the host from WebAssembly. There's lots of various ways to
measure this so this benchmark is a bit large but should hopefully be
pretty thorough. It's expected that this benchmark will rarely be run in
its entirety but rather only a subset of the benchmarks will be run at
any one given time.

Some metrics measured here are:

  • Typed vs Untyped vs Unchecked - testing the cost of both calling wasm
    with these various methods as well as having wasm call the host where
    the host function is defined with these various methods.

  • With and without call_hook - helps to measure the overhead of the
    Store::call_hook API.

  • Synchronous and Asynchronous - measures the overhead of calling into
    WebAssembly asynchronously (with and without the pooling allocator) in
    addition to defining host APIs in various methods when wasm is called
    asynchronously.

Currently all the numbers are as expected, notably:

  • Host calling WebAssembly is ~25ns of overhead
  • WebAssembly calling the host is ~3ns of overhead
  • "Unchecked" is a bit slower than "typed", and "Untyped" is slower than
    unchecked.
  • Asynchronous wasm calling a synchronous host function has ~3ns of
    overhead (nothing more than usual).
  • Asynchronous calls are much slower, on the order of 2-3us due to
    madvise.

Lots of other fiddly bits that can be measured here, but this will
hopefully help establish a benchmark through which we can measure in the
future in addition to measuring changes such as #3876

The goal of this new benchmark, `call`, is to help us measure the
overhead of both calling into WebAssembly from the host as well as
calling the host from WebAssembly. There's lots of various ways to
measure this so this benchmark is a bit large but should hopefully be
pretty thorough. It's expected that this benchmark will rarely be run in
its entirety but rather only a subset of the benchmarks will be run at
any one given time.

Some metrics measured here are:

* Typed vs Untyped vs Unchecked - testing the cost of both calling wasm
  with these various methods as well as having wasm call the host where
  the host function is defined with these various methods.

* With and without `call_hook` - helps to measure the overhead of the
  `Store::call_hook` API.

* Synchronous and Asynchronous - measures the overhead of calling into
  WebAssembly asynchronously (with and without the pooling allocator) in
  addition to defining host APIs in various methods when wasm is called
  asynchronously.

Currently all the numbers are as expected, notably:

* Host calling WebAssembly is ~25ns of overhead
* WebAssembly calling the host is ~3ns of overhead
* "Unchecked" is a bit slower than "typed", and "Untyped" is slower than
  unchecked.
* Asynchronous wasm calling a synchronous host function has ~3ns of
  overhead (nothing more than usual).
* Asynchronous calls are much slower, on the order of 2-3us due to
  `madvise`.

Lots of other fiddly bits that can be measured here, but this will
hopefully help establish a benchmark through which we can measure in the
future in addition to measuring changes such as bytecodealliance#3876
@alexcrichton alexcrichton merged commit 8c9c72c into bytecodealliance:main Mar 4, 2022
@alexcrichton alexcrichton deleted the call-benchmark branch March 4, 2022 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants