This commit adds support to `wasm-bindgen` to emit a WebAssembly module
that contains a WebAssembly Interface Types section. As of today there are no
native consumers of these WebAssembly modules, and the actual binary format
here is basically arbitrary (chosen by the `wasm-webidl-bindings` crate). The
intention is that we'll be following the [WebAssembly Interface
Types proposal][proposal] very closely and update here as necessary.
The main feature added in this PR is that a new experimental environment
variable, `WASM_INTERFACE_TYPES=1`, is recognized by the `wasm-bindgen`
CLI tool. When present the CLI tool will act differently than it does
today:
* The `anyref` feature will be implicitly enabled
* A WebAssembly interface types section will be emitted in the
WebAssembly module
* For now, the WebAssembly module is strictly validated to require zero
JS glue. This means that `wasm-bindgen` is producing a fully
standalone WebAssembly module.
The last point here is one that will change before this functionality is
stabilized in `wasm-bindgen`. For now it reflects the major use case of
this feature which is to produce a standalone WebAssembly module with no
support JS glue, and to do that we need to verify properties like it's
not using JS global names, nonstandard binding expressions, etc. The
error messages here aren't the best but they at least fail compilation
at some point instead of silently producing weird wasm modules.
Eventually it's envisioned that a WebAssembly module will contain an
interface types section but *also* have JS glue so binding expressions
can be used when available but otherwise we'd still generate JS glue for
things like nonstandard expressions and accessing JS global values.
It should be noted that a major feature not implemented in
`wasm-bindgen` yet is the multi-value proposal for WebAssembly. This is
coming soon (as soon as we can) in `walrus` and later for a pass here,
but for now this means that returning multiple values (like a string
which has a pointer/length) is a bit of a hack. To enable this use case
a `wasm-bindgen`-specific-convention which will never be stabilized is
invented here by using binding expression to indicate "this return value
is actually returned through an out-ptr as the first argument list".
This is a gross hack and is guaranteed to be removed. Eventually we will
support multi-value and the wasm module emitted will simply use
multi-value and contain internal polyfills for Rust's ABI which returns
values through out-ptrs.
Overall this should make `wasm-bindgen` usable for playing around with
the WebIDL bindings proposal and helping us get a taste of what it looks
like to have entirely standalone WebAssembly modules running in multiple
environments, no extra fluff necessary!
[proposal]: https://github.com/webassembly/webidl-bindings
This commit improves our `instantiateStreaming` fallback to only
actually trigger the fallback if the headers look wrong. If the headers
look right then we let through the original error which should help
avoid accidentally papering over bugs with different bugs in
misconfigured situations.
Closes#1696
This commit fixes an issue previously introduced around handling the
anyref table, gracefully handling the case where the source module
doesn't actually use the anyref table at all, meaning that the logic
around initializing it can be entirely skipped.
This is currently required by our ABI for wasm-bindgen where `None` js
values going out have an index of 0 and are intended to be `undefined`.
This also refactors initialization a bit to be slightly more generic
over the constants we already have defined in this module.
This functionality got lost in recent refactorings for WebIDL bindings
unfortunately, so this commit touches things up to ensure that the
anyref table initialization in anyref-mode is hooked up correctly, even
when tests are enabled. This invovled moving injection of the start
function to the webidl processing pass and ensuring its intrinsic is
registered in the internal maps of wasm-bindgen.
Support was previously (re-)added in #1654 for importing direct JS
values into a WebAssembly module by completely skipping JS shim
generation. This commit takes that PR one step further by *also*
embedding a direct import in the wasm file, where supported. The wasm
file currently largely just imports from the JS shim file that we
generate, but this allows it to directly improt from ES modules where
supported and where possible. Note that like #1654 this only happens
when the function signature doesn't actually require any conversions to
happen in JS (such as handling closures).
For imports from ES modules, local snippets, or inline JS they'll all
have their import directives directly embedded into the final
WebAssembly binary without any shims necessary to hook it all up. For
imports from the global namespace or possibly vendor-prefixed items
these still unconditionally require an import shim to be generated
because there's no way to describe that import in an ES-friendly way
(yet).
There's a few consequences of this commit which are also worth noting:
* The logic in `wasm-bindgen` where it gracefully handles (to some
degree) not-defined items now only is guaranteed to be applied to the
global namespace. If you import from a module, it'll be an
instantiation time error rather than today's runtime error when the
import is called.
* Handling imports in the wasm module not registered with
`#[wasm_bindgen]` has become more strict. Previously these imports
were basically ignored, leaving them up for interpretation depending
on the output format. The changes for each output target are:
* `bundler` - not much has changed here. Previously these ignored
imports would have been treated as ES module imports, and after this
commit there might just be some more of these imports for bundlers
to resolve.
* `web` - previously the ignored imports would likely cause
instantiation failures because the import object never actually
included a binding for other imports. After this commit though the
JS glue which instantiates the module now interprets all
unrecognized wasm module imports as ES module imports, emitting an
`import` directive. This matches what we want for the direct import
functionality, and is also largely what we want for modules in
general.
* `nodejs` - previously ignored imports were handled in the
translation shim for Node to generate `require` statements, so they
were actually "correctly handled" sort of with module imports. The
handling of this hasn't changed, and reflects what we want for
direct imports of values where loading a wasm module in Node ends up
translating the module field of each import to a `require`.
* `no-modules` - this is very similar to the `web` target where
previously this didn't really work one way or the other because we'd
never fill in more fields of the import object when instantiating
the module. After this PR though this is a hard-error to have
unrecognized imports from `#[wasm_bindgen]` with the `no-modules`
output type, because we don't know how to handle the imports.
Note that this touches on #1584 and will likely break the current use
case being mentioned there. I think though that this tightening up of
how we handle imports is what we'll want in the long run where
everything is interpreted as modules, and we'll need to figure out
best how wasi fits into this.
This commit is unlikely to have any real major immediate effects. The
goal here is to continue to inch us towards a world where there's less
and less JS glue necessary and `wasm-bindgen` is just a polyfill for web
standards that otherwise all already exist.
Also note that there's no explicitly added tests for this since this is
largely just a refactoring of an internal implementation detail of
`wasm-bindgen`, but the main `wasm` test suite has many instances of
this path being taken, for example having imports like:
(import "tests/wasm/duplicates_a.js" "foo" (func $__wbg_foo_969c253238f136f0 (type 1)))
(import "tests/wasm/duplicates_b.js" "foo" (func $__wbg_foo_027958cb2e320a94 (type 0)))
(import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "trivial" (func $__wbg_trivial_75e27c84882af23b (type 1)))
(import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "incoming_bool" (func $__wbg_incomingbool_0f2d9f55f73a256f (type 0)))
Instead of assuming names like `URL` and `Request` are defined, instead
check to see if they exist first and otherwise skip the checks that
reference them.
After this change, any import that only takes and returns ABI-safe numbers (signed
integers less than 64 bits and unrestricted floating point numbers) will be a
direct import, and will not have a little JS shim in the middle.
We don't have a great mechanism for testing the generated bindings' contents --
as opposed to its behavior -- but I manually verified that everything here does
the Right Thing and doesn't have a JS shim:
```rust
\#[wasm_bindgen]
extern "C" {
fn trivial();
fn incoming_i32() -> i32;
fn incoming_f32() -> f32;
fn incoming_f64() -> f64;
fn outgoing_i32(x: i32);
fn outgoing_f32(y: f32);
fn outgoing_f64(z: f64);
fn many(x: i32, y: f32, z: f64) -> i32;
}
```
Furthermore, I verified that when our support for emitting native `anyref` is
enabled, then we do not have a JS shim for the following import, but if it is
disabled, then we do have a JS shim:
```rust
\#[wasm_bindgen]
extern "C" {
fn works_when_anyref_support_is_enabled(v: JsValue) -> JsValue;
}
```
Fixes#1636.
We don't actually need this since we can simply pass in a number like 8
for the return pointer all the time. There's no need to allocate more
space in static data for a return pointer tha may not even get used!
After a module goes through its primary GC pass we need to look over the
set of remaining imports and use that to prune the set of imports that
we're binding.
Closes#1613
This commit is the second, and hopefully last massive, refactor for
using WebIDL bindings internally in `wasm-bindgen`. This commit actually
fully executes on the task at hand, moving `wasm-bindgen` to internally
using WebIDL bindings throughout its code generation, anyref passes,
etc. This actually fixes a number of issues that have existed in the
anyref pass for some time now!
The main changes here are to basically remove the usage of `Descriptor`
from generating JS bindings. Instead two new types are introduced:
`NonstandardIncoming` and `NonstandardOutgoing` which are bindings lists
used for incoming/outgoing bindings. These mirror the standard
terminology and literally have variants which are the standard values.
All `Descriptor` types are now mapped into lists of incoming/outgoing
bindings and used for process in wasm-bindgen. All JS generation has
been refactored and updated to now process these lists of bindings
instead of the previous `Descriptor`.
In other words this commit takes `js2rust.rs` and `rust2js.rs` and first
splits them in two. Interpretation of `Descriptor` and what to do for
conversions is in the binding selection modules. The actual generation
of JS from the binding selection is now performed by `incoming.rs` and
`outgoing.rs`. To boot this also deduplicates all the code between the
argument handling of `js2rust.rs` and return value handling of
`rust2js.rs`. This means that to implement a new binding you only need
to implement it one place and it's implemented for free in the other!
This commit is not the end of the story though. I would like to add a
mdoe to `wasm-bindgen` that literally emits a WebIDL bindings section.
That's left for a third (and hopefully final) refactoring which is also
intended to optimize generated JS for bindings.
This commit currently loses the optimization where an imported is hooked
up by value directly whenever a shim isn't needed. It's planned that
the next refactoring to emit a webidl binding section that can be added
back in. It shouldn't be too too hard hopefully since all the
scaffolding is in place now.
cc #1524
Commit 8ace8287ff75214fe955bb1819df9e8aa216d325 made the argument to the
generated init() function optional (when the target is "web"), but it is still
marked as required in the generated .d.ts file.
Fix the generated declaration to match the function definition again.
Instead of allocating space on the stack and returning a pointer we
should be able to use a single global memory location to communicate
this error payload information. This shouldn't run into any reentrancy
issues since it's only stored just before returning to wasm and it's
always read just after returning from wasm.
This was once required due to flavorful management of the `WeakRef`
proposal but nowadays it's simple enough that we don't need to refactor
it out here.
Iteration order of hash maps is nondeterministic, so add a `sorted_iter`
function and then use that throughout whenever iteration order of a hash
map would affect the generated JS.
This commit reimplements the `anyref` transformation pass tasked with
taking raw rustc output and enhancing the module to use `anyref`. This
was disabled in the previous commits during refactoring, and now the
pass is re-enabled in the manner originally intended.
Instead of being tangled up in the `js/mod.rs` pass, the anyref
transformation now happens locally within one module,
`cli-support/src/anyref.rs`, which exclusively uses the output of the
`webidl` module which produces a WebIDL bindings section as well as an
auxiliary wasm-bindgen specific section. This makes the anyref transform
much more straightforward and local, ensuring that it doesn't propagate
elsewhere and can be a largely local concern during the transformation.
The main addition needed to support this pass was detailed knowledge of
the ABI of a `Descriptor`. This knowledge is already implicitly
hardcoded in `js2rust.rs` and `rust2js.rs` through the ABI shims
generated. This was previously used for the anyref transformation to
piggy-back what was already there, but as a separate pass we are unable
to reuse the knowledge in the binding generator.
Instead `Descriptor` now has two dedicated methods describing the
various ABI properties of a type. This is then asserted to be correct
(all the time) when processing bindings, ensuring that the two are kept
in sync.
This is no longe rneeded now that we precisely track what needs to be
exported for an imported item, so all the imports are hooked up
correctly elsewhere without the need for the `__exports` map.
This commit starts the `wasm-bindgen` CLI tool down the road to being a
true polyfill for WebIDL bindings. This refactor is probably the first
of a few, but is hopefully the largest and most sprawling and everything
will be a bit more targeted from here on out.
The goal of this refactoring is to separate out the massive
`crates/cli-support/src/js/mod.rs` into a number of separate pieces of
functionality. It currently takes care of basically everything
including:
* Binding intrinsics
* Handling anyref transformations
* Generating all JS for imports/exports
* All the logic for how to import and how to name imports
* Execution and management of wasm-bindgen closures
Many of these are separable concerns and most overlap with WebIDL
bindings. The internal refactoring here is intended to make it more
clear who's responsible for what as well as making some existing
operations much more straightforward. At a high-level, the following
changes are done:
1. A `src/webidl.rs` module is introduced. The purpose of this module is
to take all of the raw wasm-bindgen custom sections from the module
and transform them into a WebIDL bindings section.
This module has a placeholder `WebidlCustomSection` which is nowhere
near the actual custom section but if you squint is in theory very
similar. It's hoped that this will eventually become the true WebIDL
custom section, currently being developed in an external crate.
Currently, however, the WebIDL bindings custom section only covers a
subset of the functionality we export to wasm-bindgen users. To avoid
leaving them high and dry this module also contains an auxiliary
custom section named `WasmBindgenAux`. This custom section isn't
intended to have a binary format, but is intended to represent a
theoretical custom section necessary to couple with WebIDL bindings to
achieve all our desired functionality in `wasm-bindgen`. It'll never
be standardized, but it'll also never be serialized :)
2. The `src/webidl.rs` module now takes over quite a bit of
functionality from `src/js/mod.rs`. Namely it handles synthesis of an
`export_map` and an `import_map` mapping export/import IDs to exactly
what's expected to be hooked up there. This does not include type
information (as that's in the bindings section) but rather includes
things like "this is the method of class A" or "this import is from
module `foo`" and things like that. These could arguably be subsumed
by future JS features as well, but that's for another time!
3. All handling of wasm-bindgen "descriptor functions" now happens in a
dedicated `src/descriptors.rs` module. The output of this module is
its own custom section (intended to be immediately consumed by the
WebIDL module) which is in theory what we want to ourselves emit one
day but rustc isn't capable of doing so right now.
4. Invocations and generations of imports are completely overhauled.
Using the `import_map` generated in the WebIDL step all imports are
now handled much more precisely in one location rather than
haphazardly throughout the module. This means we have precise
information about each import of the module and we only modify
exactly what we're looking at. This also vastly simplifies intrinsic
generation since it's all simply a codegen part of the `rust2js.rs`
module now.
5. Handling of direct imports which don't have a JS shim generated is
slightly different from before and is intended to be
future-compatible with WebIDL bindings in its full glory, but we'll
need to update it to handle cases for constructors and method calls
eventually as well.
6. Intrinsic definitions now live in their own file (`src/intrinsic.rs`)
and have a separated definition for their symbol name and signature.
The actual implementation of each intrinsic lives in `rust2js.rs`
There's a number of TODO items to finish before this merges. This
includes reimplementing the anyref pass and actually implementing import
maps for other targets. Those will come soon in follow-up commits, but
the entire `tests/wasm/main.rs` suite is currently passing and this
seems like a good checkpoint.
