* Add tests for the interface types output of wasm-bindgen
This commit expands the test suite with assertions about the output of
the interface types pass in wasm-bindgen. The goal here is to actually
assert that we produce the right output and have a suite of reference
files to show how the interface types output is changing over time.
The `reference` test suite added in the previous PR has been updated to
work for interface types as well, generating `*.wit` file assertions
which are printed via the `wit-printer` crate on crates.io.
Along the way a number of bugs were fixed with the interface types
output, such as:
* Non-determinism in output caused by iteration of a `HashMap`
* Avoiding JS generation entirely in interface types mode, ensuring that
we don't export extraneous intrinsics that aren't otherwise needed.
* Fixing location of the stack pointer for modules where it's GC'd out.
It's now rooted in the aux section of wasm-bindgen so it's available
to later passes, like the multi-value pass.
* Interface types emission now works in debug mode, meaning the
`--release` flag is no longer required. This previously did not work
because the `__wbindgen_throw` intrinsic was required in debug mode.
This comes about because of the `malloc_failure` and `internal_error`
functions in the anyref pass. The purpose of these functions is to
signal fatal runtime errors, if any, in a way that's usable to the
user. For wasm interface types though we can replace calls to these
functions with `unreachable` to avoid needing to import the
intrinsic. This has the accidental side effect of making
`wasm_bindgen::throw_str` "just work" with wasm interface types by
aborting the program, but that's not actually entirely intended. It's
hoped that a split of a `wasm-bindgen-core` crate would solve this
issue for the future.
* Run the wasm interface types validator in tests
* Add more gc roots for adapter gc
* Improve stack pointer detection
The stack pointer is never initialized to zero, but some other mutable
globals are (TLS, thread ID, etc), so let's filter those out.
* Add reference output tests for JS operations
This commit starts adding a test suite which checks in, to the
repository, test assertions for both the JS and wasm file outputs of a
Rust crate compiled with `#[wasm_bindgen]`. These aren't intended to be
exhaustive or large scale tests, but rather micro-tests to help observe
the changes in `wasm-bindgen`'s output over time.
The motivation for this commit is basically overhauling how all the GC
passes work in `wasm-bindgen` today. The reorganization is also included
in this commit as well.
Previously `wasm-bindgen` would, in an ad-hoc fashion, run the GC passes
of `walrus` in a bunch of places to ensure that less "garbage" was seen
by future passes. This not only was a source of slowdown but it also was
pretty brittle since `wasm-bindgen` kept breaking if extra iteams leaked
through.
The strategy taken in this commit is to have one precise location for a
GC pass, and everything goes through there. This is achieved by:
* All internal exports are removed immediately when generating the
nonstandard wasm interface types section. Internal exports,
intrinsics, and runtime support are all referenced by the various
instructions and/or sections that use them. This means that we now
have precise tracking of what an adapter uses.
* This in turn enables us to implement the `add_gc_roots` function for
`walrus` custom sections, which in turn allows walrus GC passes to do
what `unexport_unused_intrinsics` did before. That function is now no
longer necessary, but effectively works the same way. All intrinsics
are unexported at the beginning and then they're selectively
re-imported and re-exported through the JS glue generation pass as
necessary and defined by the bindings.
* Passes like the `anyref` pass are now much more precise about the
intrinsics that they work with. The `anyref` pass also deletes any
internal intrinsics found and also does some rewriting of the adapters
aftewards now to hook up calls to the heap count import to the heap
count intrinsic in the wasm module.
* Fix handling of __wbindgen_realloc
The final user of the `require_internal_export` function was
`__wbindgen_realloc`. This usage has now been removed by updating how we
handle usage of the `realloc` function.
The wasm interface types standard doesn't have a `realloc` function
slot, nor do I think it ever will. This means that as a polyfill for
wasm interface types we'll always have to support the lack of `realloc`.
For direct Rust to JS, however, we can still optionally handle
`realloc`. This is all handled with a few internal changes.
* Custom `StringToMemory` instructions now exist. These have an extra
`realloc` slot to store an intrinsic, if found.
* Our custom instructions are lowered to the standard instructions when
generating an interface types section.
* The `realloc` function, if present, is passed as an argument like the
malloc function when passing strings to wasm. If it's not present we
use a slower fallback, but if it's present we use the faster
implementation.
This should mean that there's little-to-no impact on existing users of
`wasm-bindgen`, but this should continue to still work for wasm
interface types polyfills and such. Additionally the GC passes now work
in that they don't delete `__wbindgen_realloc` which we later try to
reference.
* Add an empty test for the anyref pass
* Precisely track I32FromOptionAnyref's dependencies
This depends on the anyref table and a function to allocate an index if
the anyref pass is running, so be sure to track that in the instruction
itself for GC rooting.
* Trim extraneous exports from nop anyref module
Or if you're otherwise not using anyref slices, don't force some
intrinsics to exist.
* Remove globals from reference tests
Looks like these values adjust in slight but insignificant ways over
time
* Update the anyref xform tests
This commit is a pretty large scale rewrite of the internals of wasm-bindgen. No user-facing changes are expected as a result of this PR, but due to the scale of changes here it's likely inevitable that at least something will break. I'm hoping to get more testing in though before landing!
The purpose of this PR is to update wasm-bindgen to the current state of the interface types proposal. The wasm-bindgen tool was last updated when it was still called "WebIDL bindings" so it's been awhile! All support is now based on https://github.com/bytecodealliance/wasm-interface-types which defines parsers/binary format/writers/etc for wasm-interface types.
This is a pretty massive PR and unfortunately can't really be split up any more afaik. I don't really expect realistic review of all the code here (or commits), but some high-level changes are:
* Interface types now consists of a set of "adapter functions". The IR in wasm-bindgen is modeled the same way not.
* Each adapter function has a list of instructions, and these instructions work at a higher level than wasm itself, for example with strings.
* The wasm-bindgen tool has a suite of instructions which are specific to it and not present in the standard. (like before with webidl bindings)
* The anyref/multi-value transformations are now greatly simplified. They're simply "optimization passes" over adapter functions, removing instructions that are otherwise present. This way we don't have to juggle so much all over the place, and instructions always have the same meaning.
* Add support for #[wasm_bindgen(inspectable)]
This annotation generates a `toJSON` and `toString` implementation for
generated JavaScript classes which display all readable properties
available via the class or its getters
This is useful because wasm-bindgen classes currently serialize to
display one value named `ptr`, which does not model the properties of
the struct in Rust
This annotation addresses rustwasm/wasm-bindgen#1857
* Support console.log for inspectable attr in Nodejs
`#[wasm_bindgen(inspectable)]` now generates an implementation of
`[util.inspect.custom]` for the Node.js target only. This implementation
causes `console.log` and friends to yield the same class-style output,
but with all readable fields of the Rust struct displayed
* Reduce duplication in generated methods
Generated `toString` and `[util.inspect.custom]` methods now call
`toJSON` to reduce duplication
* Store module name in variable
Using Typescript I have this warning:
`./pkg/index.js
Line 52:22: Expected '!==' and instead saw '!=' eqeqeq
Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.
`
I guess this should solve the warning.
Thank you for all the work.
This commit switches all of `wasm-bindgen` from the `failure` crate to
`anyhow`. The `anyhow` crate should serve all the purposes that we
previously used `failure` for but has a few advantages:
* It's based on the standard `Error` trait rather than a custom `Fail`
trait, improving ecosystem compatibility.
* We don't need a `#[derive(Fail)]`, which means that's less code to
compile for `wasm-bindgen`. This notably helps the compile time of
`web-sys` itself.
* Using `Result<()>` in `fn main` with `anyhow::Error` produces
human-readable output, so we can use that natively.
This came up during #1760 where `Promise.resolve` must be invoked with
`this` as the `Promise` object, but we were erroneously importing it in
such a way that it didn't have a shim and `this` was `undefined`.
Turns out that `JSON.stringify(undefined)` doesn't actually return a
string, it returns `undefined`! If we're requested to serialize
`undefined` into JSON instead just interpret it as `null` which should
have the expected semantics of serving as a placeholder for `None`.
Closes#1778
To benefit users in debug mode we log any unexpected exceptions to help
diagnose any issues that might arise. It turns out, though, we log this
for *every* exception happening for *every* import, including imports
like `__wbindgen_throw` which are explicitly intended to throw an
exception. This can cause distracting debug logs to get emitted to the
console, so let's squelch the debug logging for known imports that we
shouldn't log for, such as intrinsics.
Closes#1785
* Adding ignoreBOM and fatal to TextDecoder
* Minor tweak to expose_text_processor
* Adding in unit tests for BOM
* Adding in comment for expose_text_decoder
* Attempting to fix build failure
* Temporarily disabling unit tests
This commit adds support to `wasm-bindgen` to emit a WebAssembly module
that contains a WebAssembly Interface Types section. As of today there are no
native consumers of these WebAssembly modules, and the actual binary format
here is basically arbitrary (chosen by the `wasm-webidl-bindings` crate). The
intention is that we'll be following the [WebAssembly Interface
Types proposal][proposal] very closely and update here as necessary.
The main feature added in this PR is that a new experimental environment
variable, `WASM_INTERFACE_TYPES=1`, is recognized by the `wasm-bindgen`
CLI tool. When present the CLI tool will act differently than it does
today:
* The `anyref` feature will be implicitly enabled
* A WebAssembly interface types section will be emitted in the
WebAssembly module
* For now, the WebAssembly module is strictly validated to require zero
JS glue. This means that `wasm-bindgen` is producing a fully
standalone WebAssembly module.
The last point here is one that will change before this functionality is
stabilized in `wasm-bindgen`. For now it reflects the major use case of
this feature which is to produce a standalone WebAssembly module with no
support JS glue, and to do that we need to verify properties like it's
not using JS global names, nonstandard binding expressions, etc. The
error messages here aren't the best but they at least fail compilation
at some point instead of silently producing weird wasm modules.
Eventually it's envisioned that a WebAssembly module will contain an
interface types section but *also* have JS glue so binding expressions
can be used when available but otherwise we'd still generate JS glue for
things like nonstandard expressions and accessing JS global values.
It should be noted that a major feature not implemented in
`wasm-bindgen` yet is the multi-value proposal for WebAssembly. This is
coming soon (as soon as we can) in `walrus` and later for a pass here,
but for now this means that returning multiple values (like a string
which has a pointer/length) is a bit of a hack. To enable this use case
a `wasm-bindgen`-specific-convention which will never be stabilized is
invented here by using binding expression to indicate "this return value
is actually returned through an out-ptr as the first argument list".
This is a gross hack and is guaranteed to be removed. Eventually we will
support multi-value and the wasm module emitted will simply use
multi-value and contain internal polyfills for Rust's ABI which returns
values through out-ptrs.
Overall this should make `wasm-bindgen` usable for playing around with
the WebIDL bindings proposal and helping us get a taste of what it looks
like to have entirely standalone WebAssembly modules running in multiple
environments, no extra fluff necessary!
[proposal]: https://github.com/webassembly/webidl-bindings
This commit improves our `instantiateStreaming` fallback to only
actually trigger the fallback if the headers look wrong. If the headers
look right then we let through the original error which should help
avoid accidentally papering over bugs with different bugs in
misconfigured situations.
Closes#1696
This commit fixes an issue previously introduced around handling the
anyref table, gracefully handling the case where the source module
doesn't actually use the anyref table at all, meaning that the logic
around initializing it can be entirely skipped.
This is currently required by our ABI for wasm-bindgen where `None` js
values going out have an index of 0 and are intended to be `undefined`.
This also refactors initialization a bit to be slightly more generic
over the constants we already have defined in this module.
This functionality got lost in recent refactorings for WebIDL bindings
unfortunately, so this commit touches things up to ensure that the
anyref table initialization in anyref-mode is hooked up correctly, even
when tests are enabled. This invovled moving injection of the start
function to the webidl processing pass and ensuring its intrinsic is
registered in the internal maps of wasm-bindgen.
Support was previously (re-)added in #1654 for importing direct JS
values into a WebAssembly module by completely skipping JS shim
generation. This commit takes that PR one step further by *also*
embedding a direct import in the wasm file, where supported. The wasm
file currently largely just imports from the JS shim file that we
generate, but this allows it to directly improt from ES modules where
supported and where possible. Note that like #1654 this only happens
when the function signature doesn't actually require any conversions to
happen in JS (such as handling closures).
For imports from ES modules, local snippets, or inline JS they'll all
have their import directives directly embedded into the final
WebAssembly binary without any shims necessary to hook it all up. For
imports from the global namespace or possibly vendor-prefixed items
these still unconditionally require an import shim to be generated
because there's no way to describe that import in an ES-friendly way
(yet).
There's a few consequences of this commit which are also worth noting:
* The logic in `wasm-bindgen` where it gracefully handles (to some
degree) not-defined items now only is guaranteed to be applied to the
global namespace. If you import from a module, it'll be an
instantiation time error rather than today's runtime error when the
import is called.
* Handling imports in the wasm module not registered with
`#[wasm_bindgen]` has become more strict. Previously these imports
were basically ignored, leaving them up for interpretation depending
on the output format. The changes for each output target are:
* `bundler` - not much has changed here. Previously these ignored
imports would have been treated as ES module imports, and after this
commit there might just be some more of these imports for bundlers
to resolve.
* `web` - previously the ignored imports would likely cause
instantiation failures because the import object never actually
included a binding for other imports. After this commit though the
JS glue which instantiates the module now interprets all
unrecognized wasm module imports as ES module imports, emitting an
`import` directive. This matches what we want for the direct import
functionality, and is also largely what we want for modules in
general.
* `nodejs` - previously ignored imports were handled in the
translation shim for Node to generate `require` statements, so they
were actually "correctly handled" sort of with module imports. The
handling of this hasn't changed, and reflects what we want for
direct imports of values where loading a wasm module in Node ends up
translating the module field of each import to a `require`.
* `no-modules` - this is very similar to the `web` target where
previously this didn't really work one way or the other because we'd
never fill in more fields of the import object when instantiating
the module. After this PR though this is a hard-error to have
unrecognized imports from `#[wasm_bindgen]` with the `no-modules`
output type, because we don't know how to handle the imports.
Note that this touches on #1584 and will likely break the current use
case being mentioned there. I think though that this tightening up of
how we handle imports is what we'll want in the long run where
everything is interpreted as modules, and we'll need to figure out
best how wasi fits into this.
This commit is unlikely to have any real major immediate effects. The
goal here is to continue to inch us towards a world where there's less
and less JS glue necessary and `wasm-bindgen` is just a polyfill for web
standards that otherwise all already exist.
Also note that there's no explicitly added tests for this since this is
largely just a refactoring of an internal implementation detail of
`wasm-bindgen`, but the main `wasm` test suite has many instances of
this path being taken, for example having imports like:
(import "tests/wasm/duplicates_a.js" "foo" (func $__wbg_foo_969c253238f136f0 (type 1)))
(import "tests/wasm/duplicates_b.js" "foo" (func $__wbg_foo_027958cb2e320a94 (type 0)))
(import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "trivial" (func $__wbg_trivial_75e27c84882af23b (type 1)))
(import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "incoming_bool" (func $__wbg_incomingbool_0f2d9f55f73a256f (type 0)))
Instead of assuming names like `URL` and `Request` are defined, instead
check to see if they exist first and otherwise skip the checks that
reference them.
After this change, any import that only takes and returns ABI-safe numbers (signed
integers less than 64 bits and unrestricted floating point numbers) will be a
direct import, and will not have a little JS shim in the middle.
We don't have a great mechanism for testing the generated bindings' contents --
as opposed to its behavior -- but I manually verified that everything here does
the Right Thing and doesn't have a JS shim:
```rust
\#[wasm_bindgen]
extern "C" {
fn trivial();
fn incoming_i32() -> i32;
fn incoming_f32() -> f32;
fn incoming_f64() -> f64;
fn outgoing_i32(x: i32);
fn outgoing_f32(y: f32);
fn outgoing_f64(z: f64);
fn many(x: i32, y: f32, z: f64) -> i32;
}
```
Furthermore, I verified that when our support for emitting native `anyref` is
enabled, then we do not have a JS shim for the following import, but if it is
disabled, then we do have a JS shim:
```rust
\#[wasm_bindgen]
extern "C" {
fn works_when_anyref_support_is_enabled(v: JsValue) -> JsValue;
}
```
Fixes#1636.
We don't actually need this since we can simply pass in a number like 8
for the return pointer all the time. There's no need to allocate more
space in static data for a return pointer tha may not even get used!
After a module goes through its primary GC pass we need to look over the
set of remaining imports and use that to prune the set of imports that
we're binding.
Closes#1613
This commit is the second, and hopefully last massive, refactor for
using WebIDL bindings internally in `wasm-bindgen`. This commit actually
fully executes on the task at hand, moving `wasm-bindgen` to internally
using WebIDL bindings throughout its code generation, anyref passes,
etc. This actually fixes a number of issues that have existed in the
anyref pass for some time now!
The main changes here are to basically remove the usage of `Descriptor`
from generating JS bindings. Instead two new types are introduced:
`NonstandardIncoming` and `NonstandardOutgoing` which are bindings lists
used for incoming/outgoing bindings. These mirror the standard
terminology and literally have variants which are the standard values.
All `Descriptor` types are now mapped into lists of incoming/outgoing
bindings and used for process in wasm-bindgen. All JS generation has
been refactored and updated to now process these lists of bindings
instead of the previous `Descriptor`.
In other words this commit takes `js2rust.rs` and `rust2js.rs` and first
splits them in two. Interpretation of `Descriptor` and what to do for
conversions is in the binding selection modules. The actual generation
of JS from the binding selection is now performed by `incoming.rs` and
`outgoing.rs`. To boot this also deduplicates all the code between the
argument handling of `js2rust.rs` and return value handling of
`rust2js.rs`. This means that to implement a new binding you only need
to implement it one place and it's implemented for free in the other!
This commit is not the end of the story though. I would like to add a
mdoe to `wasm-bindgen` that literally emits a WebIDL bindings section.
That's left for a third (and hopefully final) refactoring which is also
intended to optimize generated JS for bindings.
This commit currently loses the optimization where an imported is hooked
up by value directly whenever a shim isn't needed. It's planned that
the next refactoring to emit a webidl binding section that can be added
back in. It shouldn't be too too hard hopefully since all the
scaffolding is in place now.
cc #1524
Commit 8ace8287ff75214fe955bb1819df9e8aa216d325 made the argument to the
generated init() function optional (when the target is "web"), but it is still
marked as required in the generated .d.ts file.
Fix the generated declaration to match the function definition again.
Instead of allocating space on the stack and returning a pointer we
should be able to use a single global memory location to communicate
this error payload information. This shouldn't run into any reentrancy
issues since it's only stored just before returning to wasm and it's
always read just after returning from wasm.
