* Create JavaScript array without using `new` keyword.
At present [this line of code](https://github.com/rustwasm/wasm-bindgen/blob/master/crates/cli-support/src/js/mod.rs#L747) creates the heap using JavaScript's new keyword.
```
//Line 747
self.global(&format!("const heap = new Array({});", INITIAL_HEAP_OFFSET));
self.global("heap.fill(undefined);");
```
Assuming that the `INITIAL_HEAP_OFFSET` is always 32 (because it is set as a constant in the Rust code), below is the equivalent of what this code will produce; an Array Object with 32 items which are all undefined.
```
const heap = new Array(32);
//(32) [empty × 32]
//Where
var zero_element = heap[0];
//undefined
var one_element = heap[1];
//undefined
```
I believe that this is the desired outcome for the program. All good.
### Suggestion to consider
I am always reminded **not** to use the `new` keyword. Mainly by reading or listening to JavaScript ["The Good Parts"](https://youtu.be/XFTOG895C7c?t=1654).
For example if the `INITIAL_HEAP_OFFSET` was ever anything but one number, the heap would be created in a different way. For example if two numbers are passed in, then an array of size 2 would be created; where both items in the array are individual numbers.
```
const heap = new Array(32, 32);
var zero_element = heap[0];
var one_element = heap[1];
//32
//32
```
I know that this is highly unlikely, due to the fact that the `INITIAL_HEAP_OFFSET` is set as a `const` in the Rust. But thought that I would put out the following suggestion for consideration anyway. This comes from a place of just wanting to contribute in a way that could make this already awesome program a little better. :)
### Suggested update
The heap array could be created using the following code
```
const heap = [];
heap.length = INITIAL_HEAP_OFFSET;
heap[0]
heap[1]
//undefined
//undefined
```
This would create a JavaScript Array of length `INITIAL_HEAP_OFFSET`, where are items are `undefined`
The new code generates (in raw JavaScript)
```
const heap = [];
heap.length = 32;
```
Which produces
```
(32) [empty × 32]
```
In the same way that the original code does.
* Add closing parenthesis to close out self.global
* Adding files which were altered by the BLESS=1 system variable. Essentially updating generated files that are used for testing.
* Adding code generated wat file, by way of running tests using BLESS=1
* Adding table.wat that was generated by running the tests with BLESS=1 set
* Update code that creates heap array line 747 mod.rs
* Updating files that are automatically generated when using BLESS=1
* Add reference output tests for JS operations
This commit starts adding a test suite which checks in, to the
repository, test assertions for both the JS and wasm file outputs of a
Rust crate compiled with `#[wasm_bindgen]`. These aren't intended to be
exhaustive or large scale tests, but rather micro-tests to help observe
the changes in `wasm-bindgen`'s output over time.
The motivation for this commit is basically overhauling how all the GC
passes work in `wasm-bindgen` today. The reorganization is also included
in this commit as well.
Previously `wasm-bindgen` would, in an ad-hoc fashion, run the GC passes
of `walrus` in a bunch of places to ensure that less "garbage" was seen
by future passes. This not only was a source of slowdown but it also was
pretty brittle since `wasm-bindgen` kept breaking if extra iteams leaked
through.
The strategy taken in this commit is to have one precise location for a
GC pass, and everything goes through there. This is achieved by:
* All internal exports are removed immediately when generating the
nonstandard wasm interface types section. Internal exports,
intrinsics, and runtime support are all referenced by the various
instructions and/or sections that use them. This means that we now
have precise tracking of what an adapter uses.
* This in turn enables us to implement the `add_gc_roots` function for
`walrus` custom sections, which in turn allows walrus GC passes to do
what `unexport_unused_intrinsics` did before. That function is now no
longer necessary, but effectively works the same way. All intrinsics
are unexported at the beginning and then they're selectively
re-imported and re-exported through the JS glue generation pass as
necessary and defined by the bindings.
* Passes like the `anyref` pass are now much more precise about the
intrinsics that they work with. The `anyref` pass also deletes any
internal intrinsics found and also does some rewriting of the adapters
aftewards now to hook up calls to the heap count import to the heap
count intrinsic in the wasm module.
* Fix handling of __wbindgen_realloc
The final user of the `require_internal_export` function was
`__wbindgen_realloc`. This usage has now been removed by updating how we
handle usage of the `realloc` function.
The wasm interface types standard doesn't have a `realloc` function
slot, nor do I think it ever will. This means that as a polyfill for
wasm interface types we'll always have to support the lack of `realloc`.
For direct Rust to JS, however, we can still optionally handle
`realloc`. This is all handled with a few internal changes.
* Custom `StringToMemory` instructions now exist. These have an extra
`realloc` slot to store an intrinsic, if found.
* Our custom instructions are lowered to the standard instructions when
generating an interface types section.
* The `realloc` function, if present, is passed as an argument like the
malloc function when passing strings to wasm. If it's not present we
use a slower fallback, but if it's present we use the faster
implementation.
This should mean that there's little-to-no impact on existing users of
`wasm-bindgen`, but this should continue to still work for wasm
interface types polyfills and such. Additionally the GC passes now work
in that they don't delete `__wbindgen_realloc` which we later try to
reference.
* Add an empty test for the anyref pass
* Precisely track I32FromOptionAnyref's dependencies
This depends on the anyref table and a function to allocate an index if
the anyref pass is running, so be sure to track that in the instruction
itself for GC rooting.
* Trim extraneous exports from nop anyref module
Or if you're otherwise not using anyref slices, don't force some
intrinsics to exist.
* Remove globals from reference tests
Looks like these values adjust in slight but insignificant ways over
time
* Update the anyref xform tests
This commit is a pretty large scale rewrite of the internals of wasm-bindgen. No user-facing changes are expected as a result of this PR, but due to the scale of changes here it's likely inevitable that at least something will break. I'm hoping to get more testing in though before landing!
The purpose of this PR is to update wasm-bindgen to the current state of the interface types proposal. The wasm-bindgen tool was last updated when it was still called "WebIDL bindings" so it's been awhile! All support is now based on https://github.com/bytecodealliance/wasm-interface-types which defines parsers/binary format/writers/etc for wasm-interface types.
This is a pretty massive PR and unfortunately can't really be split up any more afaik. I don't really expect realistic review of all the code here (or commits), but some high-level changes are:
* Interface types now consists of a set of "adapter functions". The IR in wasm-bindgen is modeled the same way not.
* Each adapter function has a list of instructions, and these instructions work at a higher level than wasm itself, for example with strings.
* The wasm-bindgen tool has a suite of instructions which are specific to it and not present in the standard. (like before with webidl bindings)
* The anyref/multi-value transformations are now greatly simplified. They're simply "optimization passes" over adapter functions, removing instructions that are otherwise present. This way we don't have to juggle so much all over the place, and instructions always have the same meaning.
If there's no need for a transformation then there's no need to inject
anything, so make sure that wasm-bindgen with anyref passes enabled
works on non-wasm-bindgen blobs as well.
Closesbytecodealliance/cargo-wasi#16
This commit switches all of `wasm-bindgen` from the `failure` crate to
`anyhow`. The `anyhow` crate should serve all the purposes that we
previously used `failure` for but has a few advantages:
* It's based on the standard `Error` trait rather than a custom `Fail`
trait, improving ecosystem compatibility.
* We don't need a `#[derive(Fail)]`, which means that's less code to
compile for `wasm-bindgen`. This notably helps the compile time of
`web-sys` itself.
* Using `Result<()>` in `fn main` with `anyhow::Error` produces
human-readable output, so we can use that natively.
Turns out #1704 was buggy and ended up never injecting initialization
because the anyref table was never present! This fixes that issue and
this should now be tested on CI to ensure this doesn't regress and
future changes preserve correctness
This commit updates `wasm-bindgen` to the latest version of `walrus`
which transforms all internal IR representations to a list-based IR
instead of a tree-based IR. This isn't a major change other than
cosmetic for `wasm-bindgen` itself, but involves a lot of changes to the
threads/anyref passes.
This commit also updates our CI configuration to actually run all the
anyref tests on CI. This is done by downloading a nightly build of
node.js which is theorized to continue to be there for awhile until the
full support makes its way into releases.
With more than two anyref stack arguments we were accidentally storing
the anyref values one higher in the stack than intended, so fix this
off-by-one by switching up some addition logic.
This functionality got lost in recent refactorings for WebIDL bindings
unfortunately, so this commit touches things up to ensure that the
anyref table initialization in anyref-mode is hooked up correctly, even
when tests are enabled. This invovled moving injection of the start
function to the webidl processing pass and ensuring its intrinsic is
registered in the internal maps of wasm-bindgen.
Ensure that we enable the new `parallel` feature in the CLI so our tools all use
parallelized parsing, but none of our specific crates need it for usage.
This commit updates the `walrus` dependency with recent upstream API
changes in `walrus` itself, namely updates to passive segements and how
memory data segments are handled
This commit reimplements the `anyref` transformation pass tasked with
taking raw rustc output and enhancing the module to use `anyref`. This
was disabled in the previous commits during refactoring, and now the
pass is re-enabled in the manner originally intended.
Instead of being tangled up in the `js/mod.rs` pass, the anyref
transformation now happens locally within one module,
`cli-support/src/anyref.rs`, which exclusively uses the output of the
`webidl` module which produces a WebIDL bindings section as well as an
auxiliary wasm-bindgen specific section. This makes the anyref transform
much more straightforward and local, ensuring that it doesn't propagate
elsewhere and can be a largely local concern during the transformation.
The main addition needed to support this pass was detailed knowledge of
the ABI of a `Descriptor`. This knowledge is already implicitly
hardcoded in `js2rust.rs` and `rust2js.rs` through the ABI shims
generated. This was previously used for the anyref transformation to
piggy-back what was already there, but as a separate pass we are unable
to reuse the knowledge in the binding generator.
Instead `Descriptor` now has two dedicated methods describing the
various ABI properties of a type. This is then asserted to be correct
(all the time) when processing bindings, ensuring that the two are kept
in sync.
Most of the CLI crates were already in the 2018 edition, and it turns
out that one of the macro crates was already in the 2018 edition so we
may as well move everything to the 2018 edition!
Always nice to remove those `extern crate` statements nowadays!
This commit also does a `cargo fmt --all` to make sure we're conforming
with style again.
This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.
This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.
The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.
This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.
The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.
The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.
