This commit is the start of a refactoring which will move from each code
generator having one monolithic test into many separate sub-tests. This
should enable smaller and more focused tests which benefits bringing new
bindings generators online (can get some tests working before others)
and also allows writing tests for some generators but not others.

The way that this is currently architected is that there's a new
`tests/runtime/*` directory in which a test-per-directory lives. For
each tests `tests/runtime/foo` there will be a few conventions:

* `tests/runtime/foo/imports.witx` - these are what hosts must
  implement and what the wasm can import from the host. This file is
  required to exist.
* `tests/runtime/foo/exports.witx` - this is what wasms must implement
  and what hosts can call. This file must exist.
* `tests/runtime/foo/host.*` - these are separate host implementations
  to execute the wasm code. The extension of the file indicates which
  generator it's used for (e.g. right now `host.{ts,rs,py}` are
  supported). If `host.py` doesn't exist for the test `foo` then testing
  the Python code generator will skip the test `foo`. Otherwise it's
  expected to work. At least one `host.*` file is required to work, but
  there's no need to have a file-per-host.
* `tests/runtime/foo/wasm.*` - these are separate guest compiled-to-wasm
  implementations. Currently only `wasm.{c,rs}` are supported, but the
  intent is to support `wasm.js` for #71 as well. Not all languages need
  to have an implementation here either, they'll be skipped if omitted.

The general goal is that adding a test for some witx functionality
should be "drop a few files in `tests/runtime/your_new_test` and you're
good to go. Over time more generators can add more tests if it becomes
necessary, and generator-specific tests can also be added to this
directory such as the one test exercising JS instantiation added here
too.

Overall I think this turned out quite well. The biggest downside is that
Rust-compiled-to-wasm needs to list explicitly each test that's being
compiled to wasm, which is a bit of a bummer.

Starts the extraction process from the monolithic tests that already
exist into smaller tests.

There was an accidentaly `.to_snake_case()` when it should have been
using the raw name from the witx.

Closes #76

This commit is the initial steps towards supporting and implementing
`async` function as-specified in `*.witx` itself. The goal of this is to
implement the callback-based ABI of async functions and streams as
presented in WASI subgroup meetings.

This commit itself only implements async functions for the
compiled-to-Rust guest and a JS host. The next PR is to implement async
functions for a Wasmtime host, followed by an implementation of streams
afterwards.

Included here is:

* Updates to the `*.witx` syntax and parsing.
* Assertions in code generators that don't support async that async
  isn't used.
* An implementation of the callback ABI for `async` functions for
  compiled-to-wasm Rust modules. This includes a simple "executor" to
  track the state of async imports/exports. Everything is only exposed
  with `async` functions in Rust, no callbacks are part of the interface
  of the generated bindings.
* An implementation of the callback ABI for JS hosts. This is
  exposed with `async` functions and only uses callbacks under-the-hood.

This change had some significant ramifications on the generated bindings
for JS hosts. Namely both imports and exports needed to access the same
state of callbacks so they are no longer generated separately in
isolation but rather together in the same manner as the spidermonkey
bindings. I expect that most generators will migrate to this pattern in
the future. Work was done internally to deduplicate intrinsics across
bindings modules, namely the promise-related intrinsics.

The testing so far isn't super strenuous, but a major thing I made sure
to implement is that when an asynchronous wasm function is called and
later some asynchronous import throws an exception that should carry the
exception to the original wasm export's promise in JS. This is intended
to map to basically what `async` and `await` feel like in JS normally.
Otherwise though there's codegen tests for flavorful types being used
in parameters/results for async functions (and this test is ignored in
most generators at the moment) and runtime tests for some concurrency.

* feat: initial work to make nullable fields use option `?`

* fix: unwrap nullable type and update test to use `?`

* fix: missing semicolon

* fix: refactor to return option when unwrapping nullables

* feat: refactor nullable methods to iface struct

* fix: fmt forgot to run it in root

* chore: refactor nullable methods back to js impl

I today started looking at updating this project to the new draft of the
canonical ABI specified at WebAssembly/component-model#23 and while
there's quite a few changes that need to happen the first thing that
struck me was that maintaining all of the support for old-style witx
files is pretty onerous. Especially the ABI-calculating code has lots of
special cases for various witx constructs and the Preview1 ABI, and I
wasn't really sure how to update this in many situations.

Overall the original purpose of the witx support was to prove out that
it's possible to support both the old witx abi and the new canonical ABI
in the same generator. The canonical ABI has changed significantly in
the meantime however and this doesn't necessarily make sense to do any
more. I think it would be best now to reevaluate at the point when WASI
is ready to switch to the component model what to do with the old witx
support. I no longer think that "build it in here" is the obvious
option. As this diff shows there's quite a bit of weight to carry the
old witx abis as this commit clocks in at nearly 7000 lines removed.

The specifics being dropped here are:

* Parsing support for `*.witx`
* Support for `Pointer` and `ConstPointer`
* Support for `WitxInstruction`
* Support for push/pull buffers

The push/pull buffer feature was never actually fully implemented, even
for Rust hosts they only kind-of worked. Most other generators never
even implemented support for them. Additionally support for other
`*.witx` constructs was somewhat spotty at best with very few tests.

My hope is that there are no existing users of this support. If there
are then I think it's best to re-evaluate how best to solve the scenario
on-hand.

* Rename f32/f64 to float32/float64

This commit renames the previously-known `f32` and `f64` interface types
to `float32` and `float64` to match the upstream specification.

* Fix a test

* Add support for params passed indirectly

This commit adds support for the `MAX_FLAT_PARAMS` variable in the
current canonical ABI draft. Argument lists which exceed a static number
of parameters are now always passed indirectly through memory. In export
this is dynamically allocated/freed and in imports this is statically
passed.

* Review comments

* Split out the `Option` and `Expected` types from `Variant`

This commit is like prior PRs to split out specializations of types into
their own AST type to avoid conflicting with the main type (in this case
`variant`). I originally thought these two types would be relatively
simple but this is probably one of the more complicated transitions, as
evidenced by the lines changed here. The main churn was that variants
already have a significant amount of code to support them and this is in
some places "duplicating" code for option/expected and in other cases
splitting what was already an if/else.

Overall I think that the generated code gets a little better since it's
clear when something is and `option` vs `expected` now rather than
trying to have everything shoehorned into one. Notably the C code
generator now generates descriptive fields like `bool is_some` or `bool
is_err` instead of a bland `uint8_t tag` with some comments about how to
use it.

* Remove `Variant::as_{option,expected}`

... as these are separate variants now.

* Review comments

* Split out a `Union` type from `Variant`

This commit splits out the union specialization from `variant`,
finalizing the transition to the new AST of types available in the
current component model draft. This is mostly like the prior
splittings-out where there's some duplication but it's relatively
localized and hopefully not too hard to follow.

* Fill out TODO

* Update enum representation

* Generate variant docs

* Fix demo

* wit-bindgen-rust: fix lowering and lifting of zero-length lists.

This commit fixes the Rust bindings generated for lifting and lowering of lists
to account for lists of zero-length.

It is undefined behavior to call `std::alloc::alloc` with a zero-sized layout.

Additionally, as the behavior of the `canonical_abi_realloc` implemented in
`wit-bindgen` is to return a non-null dummy pointer for a zero-sized alloc, the
bindings were calling `std::alloc::dealloc` with a pointer that was not
allocated with `std::alloc::alloc` (also undefined behavior).

To fix this, the bindings now account for zero-length lists and skip allocation
and deallocation where appropriate.

* Update `list` runtime tests to pass empty lists and strings.

* initial work

* Fix formatting

* Fix references to Rust library for guest code
Improve documentation
Other minor fixes

* Fix wastime -> host_wasmtime_rust references
Fixed feature flags

* Fix demo deploy
update readme
improve help text

* Update readme

Co-authored-by: Kyle Brown <kbrown@singlestore.com>
Co-authored-by: Kyle Brown <kyleb@liquidrocketry.com>

* Update JS integer lowering

Updates JS integer lowering to match the component model explainer (assuming that WebAssembly/component-model#52 is merged).

The main differences are that out-of-range numbers now wrap instead of throwing, and that non-integers are now rounded down instead of throwing.

* Remove `async func` support.

The `async` keyword has been removed from the [async proposal], in favor
of using plain functions that with `future` or `stream` types. See also
the [component-model PR to remove the syntax].

This removes the (now) old `async func` suppport from wit-bindgen, to make
it easier to develop the current Canonical ABI proposal.

[async proposal]: https://docs.google.com/presentation/d/1MNVOZ8hdofO3tI0szg_i-Yoy0N2QPU2C--LzVuoGSlE/edit#slide=id.g1270ef7d5b6_0_111
[component-model PR to remove the syntax]: WebAssembly/component-model#98

* Remove support for the `async` checkbox in wit-bindgen-demo.

* Update flag representation

* Use `flags.repr().count()` instead of manually matching

* Remove flag validation intrinsics

* Validate that flags don't have extraneous bits set when lifting

* grammar nit

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* Rename `canonical_abi_realloc` to `cabi_realloc`.

This follows the current Canonical ABI. This doesn't rename
`canonical_abi_free`, as that's expected to be removed when post-return
functions are implemented.

* Rename the "expected" type to "result".

This follows the current Canonical ABI.

* Implement function and value type name mangling.

This implements the name-mangling scheme in the current canonical ABI,
with the modification proposed in WebAssembly/component-model#104,
though that can be easily removed if the proposal is declined.

* Use name mangling in the bindings generators.

* Use the export base name rather than the mangled name for python identifiers.

Signed-off-by: GitHub <noreply@github.com>

Signed-off-by: GitHub <noreply@github.com>

* WIP: Update for component model multireturn and removal of unit

This PR updates wit-bindgen for the latest changes to the component model:

- the return of multireturn
- removal of the `unit` type
- a new syntax for optional types in results, streams, futures and variants

all of which go hand in hand.

This also pulls in the latest versions of wasm-encoder, wasmprinter,
and wasmparser crates to get their updates for these component model changes.

* Get all wit-bindgen tests working

* Implement tests for multi-return

Exercise this throughout the runtime tests and additionally add a few
variants in codegen tests to ensure that this is exercised.

* Update markdown generator

Co-authored-by: George Kulakowski <gkulakowski@fastly.com>

This adds a few new pseudo-instructions plus a new method on `Interface`
to generate a `post-return` function. For now these are simply named
`{name}_post_return` and the integration point there will probably
change as the component model shapes up.

The integration here is intended to still be relatively primitive in
that the actual component model integration will likely look different
in the future. Given how `wit-bindgen` works today, though, this should
at least get things part of the way there.

* remove Canonical ABI name mangling

this reverts a big chunk of the functionality introduced in bytecodealliance/wit-bindgen#309

* multi-return.wit: namespace these export functions

the tests already export functions named a,b,c in strings.wit. We can't
define more functions with that same name.

* remove mangled names from tests/runtime/invalid/wasm.rs

Try to reduce the number of places versions are mentioned and encourage
shared dependencies between crates.

This migrates everything from structopt and clap 3.* to 4.0.9, while
preserving the (sub)command syntax of all the binaries in all the
crates in this repo.

* Remove support for handles and resources

This commit removes all support for the `resource` and `Handle` types
from the AST of `wit-parser` and all related support in all code
generators. The motivation for this commit is that `wit-bindgen` is on
the cusp of actually being able to work with components: producing a
component from guest output and consuming components in host generators.
More detail about this is in #314. With components as an intermediate
format, however, there is no way to encode resources since they are not
part of the component model proposal yet.

All is not lost for handles and resources, though. The official design
for handles and resources is being worked on upstream in the component
model repository itself at this time and once added all of this support
will be re-added to `wit-bindgen`. In the meantime though I personally
think that the best way forward is to remove the interim support for a
few reasons:

* Primarily it unblocks progress at this time towards fully integrating
  components and the `wit-bindgen` generators. The requirement to run
  existing tests that use handles would mean that no host generator
  could actually switch to components and/or modes for today's
  core-wasm-lookalike would need to be preserved.

* Otherwise though the semantics of the current handles are basically
  invented out of thin air by myself and were never really formally
  specified, debated, or designed deliberately. I grafted `witx`-style
  handles into `wit-component` and added features as necessary over
  time, but it seems highly unlikely that the handles designed as part
  of the component model will be the ones that `wit-bindgen` currently
  supports. This inevitably means that a new system would need new code
  anyway and would likely result in removal regardless.

As usual git always has the history of handles and this all may come
back in one shape or another if only slightly tweaked. I'm confident in
our history spelunking abilities, though, so I don't feel that keeping
support in the repository is necessary for this purpose.

* Remove resources from the demo

* Fix rebase conflict

* Remove support for handles and resources

This commit removes all support for the `resource` and `Handle` types
from the AST of `wit-parser` and all related support in all code
generators. The motivation for this commit is that `wit-bindgen` is on
the cusp of actually being able to work with components: producing a
component from guest output and consuming components in host generators.
More detail about this is in #314. With components as an intermediate
format, however, there is no way to encode resources since they are not
part of the component model proposal yet.

All is not lost for handles and resources, though. The official design
for handles and resources is being worked on upstream in the component
model repository itself at this time and once added all of this support
will be re-added to `wit-bindgen`. In the meantime though I personally
think that the best way forward is to remove the interim support for a
few reasons:

* Primarily it unblocks progress at this time towards fully integrating
  components and the `wit-bindgen` generators. The requirement to run
  existing tests that use handles would mean that no host generator
  could actually switch to components and/or modes for today's
  core-wasm-lookalike would need to be preserved.

* Otherwise though the semantics of the current handles are basically
  invented out of thin air by myself and were never really formally
  specified, debated, or designed deliberately. I grafted `witx`-style
  handles into `wit-component` and added features as necessary over
  time, but it seems highly unlikely that the handles designed as part
  of the component model will be the ones that `wit-bindgen` currently
  supports. This inevitably means that a new system would need new code
  anyway and would likely result in removal regardless.

As usual git always has the history of handles and this all may come
back in one shape or another if only slightly tweaked. I'm confident in
our history spelunking abilities, though, so I don't feel that keeping
support in the repository is necessary for this purpose.

* Remove resources from the demo

* smuggle wit information in custom sections

* move transcoder to the crate, and make it available in the cli

* gen-guest-rust can emit custom component-type section

* custom section takes a pub static, not a const

* ComponentEncoder needs to own its Interfaces

so that I can use the Interfaces decoded from the module's custom
section

* make ComponentEncoder always transcode component-type info from custom sections

* flavorful tests: types and functions are actually the same namespace

theyre not in wit, but thats a bug we need to fix, because they are in
component types

* test-helpers: build rust guests with wasm32-unknown-unknown and assert they encode as components

except for "invalid" and "handles" which are not gonna work

* refactor

* gen-guest-rust: now infallible

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

async-functions.wit, host.wit, wasi-next.wit, and resource.wit are no longer
in the tree, so we no longer need to explicitly disable testing them.

And add the comment about disabling tests to all the codegen tests, so
that it's easy to find by people debugging them.

* Update all crates to heck 0.4

resolves #342

* rustfmt

This commit is a large refactor of how testing works in this repository
for wit-bindgen and hosts/guests to simplify the flow of tests and make
it easier to modify and less monolithic. Previously tons of logic was
located in a `test_helpers` macro crate but now that large crate has
been split up to delegate more work to each individual test. Some common
functionality is still present in a `test_helpers` crate but it's more
clear that the `test_helpers::codegen_test` macro is purely responsible
for simply expanding a glob at compile time and delegating to a
macro-per-test.

The tests of Rust & other languages are now also more uniform where
everything "just invokes a macro" where Rust's macros further invoke
proc-macros for the respective generators and other languages delegate
to shared functionality for "generate code" then "run verification
program".

Overall I'm hoping this makes the testing easier to follow and
understand, and in upcoming changes for components it will make it
easier to change each generator, individually, over to components
ideally.

)

* point at wasmtime in git, and turn on component-model

we need a git wasmtime version to have an up-to-date wasmparser inside.
the 1.0.1 release (latest at this moment) wasmparser chokes on basic
stuff

* enough fixes to get smoke import to work

* wasi mock: return ERRNO_SUCCESS in {environ|args}_{get|sizes_get}

so that wasi-libc can initialize args and env as empty in rust guest

* java component encoding: provide flags after module

so that the exported interface isnt overwritten when one
is not found inside the module.

* wasi mock: demonstrate java memory incompatibility by importing a memory

* wasi adapter build: set stack size to 0

otherwise llvm requires 16 pages of imported memory, which will conflict
when linking against modules created by teavm which have a smaller
minimum memory size

* gen-host-wasmtime-rust: slash and burn all adapter funcs out

* ok, now smoke tests actually work

* start working on getting the `records` test to pass

still some WIP on the flags macro over in wasmtime required

* gen-host-wasmtime: fix return values on import funcs

* call the post return function

* records tests pass for host-wasmtime-rust

* flags fixes landed in wasmtime main

* bugfix in return types, and numbers works

* cargo.lock bump

* many-arguments test: 16 arguments is enough

Wasmtime decided that 16 is the limit of the ComponentNamedList tuple
length. This limit is arbitrary, so I shrunk the test rather than raise
wasmtime's limit.

* gen-host-wasmtime-rust: fix some ownership stuff in imports/exports for lists test

* lists runtime test: imports get called with owned values

* move unaligned pointers test from `lists` to `invalid`

* correctly adding a Lift derive was pretty invasive

* fix

* unions test: add rust host

* unions tests: fix bugs in wasm.rs and host.py

i am skeptical about the python embedding, these tests shouldnt have
passed before

* params in imports are owned, and push record derive lift logic down into gen-rust-lib

* test fixes

* rebase fixes

* many arguments codegen test: reduce to 16 arguments

* runtime_tests_wasmtime: pass components in

* made a branch for catching invalid lifts

* found a bug with post_return calling

* invalid: here are some more error messages

the one for enum discriminant is worse than before, i guess

* back out "fix" to oob trapping semantics

* invalid test: cant re-enter instance after a trap, so make a new store

* host-wasmtime-rust is just a shim to export anyhow, wasmtime, & the proc macros

* invalid/host.py: create a new store to reset

* code review

* code review: remove custom_error config, and add tracing impl for imports

* enable tracing in host-wasmtime-rust tests

the flavorful/host.rs shows you can enable it explicitly, but its
actually enabled for all tests because the cargo feature is turned on.

[phickey@pch-tower:src/wit-bindgen]% RUST_LOG=trace RUST_LOG_SPAN_EVENTS=full cargo test -p wit-bindgen-gen-host-wasmtime-rust  -- records_rust --nocapture
    Finished test [unoptimized + debuginfo] target(s) in 0.10s
     Running tests/codegen.rs (target/debug/deps/codegen-a87f1b59880af895)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 68 filtered out; finished in 0.00s

     Running tests/runtime.rs (target/debug/deps/runtime-e18eabec1fbb6c9b)

running 1 test
2022-10-12T17:50:48.399919Z TRACE wit-bindgen import{module="exports" function="test-imports"}: runtime::records_rust::exports: new
2022-10-12T17:50:48.400036Z TRACE wit-bindgen import{module="exports" function="test-imports"}: runtime::records_rust::exports: enter
2022-10-12T17:50:48.400380Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="multiple-results"}: runtime::records_rust::imports: new
2022-10-12T17:50:48.400449Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="multiple-results"}: runtime::records_rust::imports: enter
2022-10-12T17:50:48.400522Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="multiple-results"}: runtime::records_rust::imports: exit
2022-10-12T17:50:48.400575Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="multiple-results"}: runtime::records_rust::imports: close time.busy=73.0µs time.idle=140µs
2022-10-12T17:50:48.400772Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="swap-tuple"}: runtime::records_rust::imports: new
2022-10-12T17:50:48.400834Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="swap-tuple"}: runtime::records_rust::imports: enter
2022-10-12T17:50:48.400890Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="swap-tuple"}: runtime::records_rust::imports: exit
2022-10-12T17:50:48.400939Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="swap-tuple"}: runtime::records_rust::imports: close time.busy=56.0µs time.idle=114µs
2022-10-12T17:50:48.401089Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="roundtrip-flags1"}: runtime::records_rust::imports: new
2022-10-12T17:50:48.401150Z TRACE wit-bindgen import{module="exports" function="test-imports"}:wit-bindgen export{module="imports" function="roundtrip-flags1"}: runtime::records_rust::imports: enter
...snip

* invalid test: disable (by commenting out) the unaligned memory test for now

* to fully test this behavior, we need to split out the two roundtrip
functions into one per argument-type: 10 total.
* js & wasmtime-py hosts don't actually check alignment of arguments,
so this test expects different behavior on different host architectures.
this doesn't work with our testing strategy, so this should be fixed up
once js & wasmtime-py check alignment.

* many arguments host.ts: i have no idea why this fixes broken CI, but it does

* latest main has fix required for wasmtime's treatment of export types

* wit-bindgen-demo: remove support for wasmtime custom error

* js: Take a component as input, not interfaces

This commit is the implementation of #314 for the JS host generator.
The commit here covers basically everything in that issue for JS except
it generates a slightly different structure of the JS output. Otherwise
the main highlights are:

* The JS host generator no longer implements the core `Generator` trait
  since it doesn't really fit in the component-as-input world. For now I
  created an `InterfaceGenerator` trait since similar functionality is
  used just not precisely the same. I expect that this will get iterated
  on over time as worlds and other host generators take shape.

* The `wasmtime-environ` crate from Wasmtime itself, typically a "private"
  dependency of Wasmtime, is used to parse the input component and
  generate an `instantiate` function in JS. Wasmtime does all the heavy
  lifting of creating a linear list of initializers for the component
  and this empowers the generator to simply generate JS that is the list
  of initializers.

* The `wit-component` crate is used to "extract" the `Interface`
  descriptions from the input component. This is used to generate type
  information for TypeScript as well as types for lifting/lowering.
  Internally a correlation is done from a lowering/lifting to an
  `Interface` function to connect the dots when instantiating a component.

Lots of pieces needed updating here such as the runtime tests, the
`wit-bindgen` CLI tool, and the demo web page. These are all updated for
the new structure of the JS host generator.

Overall this surprisingly went much more smoothly than I expected. With
`wit-bindgen` being able to extract `Interface` representations from a
component most of the prior JS host code generation was able to be
reused. Along the way I also felt that the addition of "worlds" would
have relatively obvious insertion points and would be relatively easily
handled in all the places.

The demo and runtime tests are proof enough to me at least that this all
works internally, and this feels like a solid foundation to iterate from
with the addition of worlds and continued support for JS hosts.

* Rebase and fix tests

* Pull in latest `wasmtime` where `main` now works
* Add a `testwasi` implementation for JS and use it in all tests
* Add a dummy `printf` to a C test to ensure it imports `testwasi` like
  the other languages.
* Add a `fd_fdstat_get` stub to make C happy
* Update `fd_write` to only work for fd 1

* Review comments

These tests can't run in JS/Python hosts yet, however. I've also filed
issues for other differences between JS/Python and the official spec.

This was my mistake in merging #377 a bit too hastily.

* Extract common code from JS host generation

This commit extracts a few traits and utilities to the
`wit-bindgen-core` crate and the `wit-bindgen` CLI. The intention here
is to make it simple to implement this same generation scheme for other
languages, such as Python, and additionally provide a path forward to
centering the CLI interface around world files.

* Migrate the host-wasmtime-rust generator to `WorldGenerator`

This is intended to be the first step down the path of reorienting
the Wasmtime generator around worlds instead of single interfaces
through the new `WorldGenerator` trait rather than the old `Generator`
trait.

* Initial support for worlds in Rust guests

* Get all tests working again for Rust guests/hosts

* Fix demo build

* Minor nits

* Reimplement the host Python generator with worlds

This commit is the equivalent of #381 but for Wasmtime Python host
bindings. This is a large-scale refactor of the internals of the Python
host generator in which I took quite a few liberties in internal
restructuring as well as output restructuring. The tests are probably
the best to review and better reflect what changed, but there are some
aspects of worlds that the tests are not currently exercising which
we'll want to add with the introduction of worlds in the future.

This means that all host generators are now working with worlds as input
rather than individual `*.wit` files, and the only two remaining
generators are the C and Java generators for guests (which I hope are
easier).

The high level summary of the new output is:

    out_dir/
      __init__.py             # top-level component exports/definitions
      types.py                # shared type information, currently just `Result`
      imports/
        __init__.py           # only here if something is imported
        foo.py                # type and protocol definition per interface
      exports/
        __init__.py           # only here if an instance is exported
        bar.py                # one per exported instance

"Default exports" will show up on the generated structure in
`out_dir/__init__.py` so all runtime tests, for example, do not generate
`exports` at this time. Lots of fiddly stuff went into structuring this
all right to get past `mypy` and additionally try to avoid name
conflicts. It's still somewhat easy to have name conflicts but ideally
they're of the more esoteric category rather than "really easy to run
into".

* Try to fix windows

* gen-host-js: utf16 support

* whitespace

* use all camelcasing

* test integration

* remove unused file

* formatting

* pr review

* fixup wasi test reference

* c internal string tests, js encoding defaults

* fixup unicode string literal

* fix string assertion

* actually fix c string assertion

* le decoding only

* remove utf16 decoder intrinsic use

This commit removes the `Generator` trait impl for the C generator and
adds the `WorldGenerator` trait impl to replace it. This is the
equivalent of #386 for the guest C generator.

The generated C code is largely the same, except that the entire world
is represented in one header file instead of per-interface header files
as before. Additionally a tiny amount of "type sharing" is now done
where all interfaces share the same string type (namespaced by the
world name). More type sharing should come with a more first-class
implementation of worlds.

There was quite a lot of code movement within the generator as I got it
working again, but at the surface level very little has changed and it
should largely be the same as before.

* gen-host-js: whitespace & formatting

* collect utf8EncodedLen into Utf8Encode

* single line without braces

* pr feedback

* ws fixup

#402)

* gen-host-js: support undefined values in lowering for nullable options

* add a test

* only explicit undefined test

* tsignore

* gen-host-js: instance ESM output by default

Outputs a direct ES module from the JS host generator by default
that can be directly imported in JS environments.

For the custom instantiation API that was the previous output, a new
`--instantiation` flag is provided.

Instance-level customization flags are also added including:
* `--compat`: Enables compatibility mode, based on the assumption
  that a new --tla-compat flag will be added in a future PR.
* `--nodejs-compat`: Handle Node.js instantiation when no fetch global
  is present

The above flags are only compatible with instance output and will throw
otherwise.

This PR also parallelizes the core Wasm instantiations, and converts the 
instantiation API from `instantiateCore(path, imports)` to
`compileCore(path)` to enable this parallelization.

In addition a new `--map` is added with support for custom import
specifier mappings that can apply to both output modes.

…409)

* gen-host-js: Error wrapping and unwrapping for singular results

When an exported component function has a singular return of type
result, treat this as a throwable in the JS enviroment, by throwing
the error payload.

Similarly, when an imported function has a singular return of type
result, add a catch handler and wrap the function in a result object
when handling.

Further optimization work should be done to avoid unnecessary wrapping
in future.

* better payload check, return fixup

* fixup demo

* improved error payload extraction

* fixup tests, intrinsic dependence

* remove unnecessary tostring

* pr feedback

* remove unnecessary include

* simpler throws check

* review feedback

* explicit result flattening

* throws signature

* fixup throws signature

* Add empty `imports.wit` for `exports_only` test

Otherwise the build process thinks this is always missing so tests are
always slowly rebuilt.

* Fix compiling the `runtime` test for JS in isolation

This requires the `clap` feature to be enabled now which works with
`cargo test --workspace` but isn't working with individual crate
testing.

* Don't test wasm size in exports_only test

Different versions of `wasi-libc` will produce differently sized
binaries, so only test the generated JS which this repo controls.

…ining (#425)

In addition, --instantiation now supports a third instantiateCore optional argument to
replace the --load=custom wrapping use case

)

* Update all codegen tests to world syntax

This enables simplifying codegen tests as well by having each test
exercise imports/exports instead of having all code generators have one
case for imports and one for exports.

* Update codegen tests for the C guest generator

* Update teavm codegen tests for worlds

* Remove the `--name` argument from the CLI

No longer needed as the name is inferred from the `*.wit` world.

* Get all js host generator tests working

Update all `runtime/*` tests with new naming conventions, world files,
etc. Minor updates were made to names as I ended up using different
conventions for the `*.wit` world files than were previously exercised.

This also fixes a few minor issues with the output for the JS generator
in the default ESM mode.

* Get all host-wasmtime tests working with worlds

Various small updates here and there to namings and such.

* Get all Python tests working with worlds

Fix support for exported instances by using correct import paths for
various types/intrinsics/etc. Additionally update the codegen tests to
specify a mypy cache dir to avoid racing between tests.

* Update demo build for worlds

Also fix an issue with the Rust host generator where it used an
interface's name instead of the name of the import for import module names.

* Update demo with world files

* Bump version of dependency crates

This commit updates this repository to published versions of the
`wasm-tools` family of crates. Additionally Wasmtime is updated with the
same upstream versions of the `wasm-tools` crates. Put together this
updates everything to the most recent version of the component model
with optional URLs encoded and an updated version of Wasmtime.

Part of this transition is the removal of
`wit_component::ComponentInterfaces` which is plumbed through as part of
this commit.

* Update to wasmtime main

* Fix demo build

* Update wasm-tools binary

* Fix cli build

* Install wasm-tools from crates.io, not git