... and add a parallel raytracing demo!
This commit adds enough support to `wasm-bindgen` to produce a workable
wasm binary *today* with the experimental WebAssembly threads support
implemented in Firefox Nightly. I've tried to comment what's going on in
the commits and such, but at a high level the changes made here are:
* A new transformation, living in a new `wasm-bindgen-threads-xform`
crate, prepares a wasm module for parallel execution. This performs a
number of mundane tasks which I hope to detail in a blog post later on.
* The `--no-modules` output is enhanced with more support for when
shared memory is enabled, allowing passing in the module/memory to
initialize the wasm instance on multiple threads (sharing both module
and memory).
* The `wasm-bindgen` crate now offers the ability, in `--no-modules`
mode, to get a handle on the `WebAssembly.Module` instance.
* The example itself requires Xargo to recompile the standard library
with atomics and an experimental feature enabled. Afterwards it
experimentally also enables threading support in wasm-bindgen.
I've also added hopefully enough CI support to compile this example in a
builder so we can upload it and poke around live online. I hope to
detail more about the technical details here in a blog post soon as
well!
This commit migrates away from using Serde for the custom section in
wasm executables. This is a refactoring of a purely-internal data
structure to `wasm-bindgen` and should have no visible functional change
on users.
The motivation for this commit is two fold:
* First, the compile times using `serde_json` and `serde_derive` for the
syntax extension isn't the most fun.
* Second, eventually we're going to want to stablize the layout of the
custom section, and it's highly unlikely to be json!
Primarily, though, the intention of this commit is to improve the
cold-cache compile time of `wasm-bindgen` by ensuring that for new users
this project builds as quickly as possible. By removing some heavyweight
dependencies from the procedural macro, `serde`, `serde_derive`, and
`serde_json`, we're able to get a pretty nice build time improvement for
the `wasm-bindgen` crate itself:
| | single-core build | parallel build |
|-------------|-------------------|----------------|
| master | 36.5s | 17.3s |
| this commit | 20.5s | 11.8s |
These are't really end-all-be-all wins but they're much better
especially on the spectrum of weaker CPUs (in theory modeled by the
single-core case showing we have 42% less CPU work in theory).
The `wasm-bindgen` crate is effectively the only user of this crate now
that the `wasm-gc` tool has been deprecated. It's also much easier to
keep it in this repository as it's easier to sync changes to
`parity-wasm`. I'd also like to start refactoring out utilities for
managing a `parity_wasm::Module` to share between this crate and the
other CLI support code.
This resulted in trailing whitespace in the generated file. In addition
to wasting space in a file that gets served over the wire, this also
gets highlighted as a problem when reviewing the generated file in an
editor that highlights trailing whitespace.
The default of Rust wasm binaries is to export the memory that they contain, but
LLD also supports an `--import-memory` option where memory is imported into a
module instead. It's looking like importing memory is along the lines of how
shared memory wasm modules will work (they'll all import the same memory).
This commit adds support to wasm-bindgen to support modules which import memory.
Memory accessors are tweaked to no longer always assume that the wasm module
exports its memory. Additionally JS bindings will create a `memory` option
automatically because LLD always imports memory from an `env` module which won't
actually exist.
This is a pretty heavyweight dependency which accounts for a surprising amount
of runtime for larger modules in `wasm-bindgen`. We don't need 90% of the crate
and so this commit bundles a small interpreter for instructions we know are only
going to appear in describe-related functions.
This commit adds experimental support for `WeakRef` to be used to automatically
free wasm objects instead of having to always call the `free` function manually.
Note that when enabled the `free` function for all exported objects is still
generated, it's just optionally invoked by the application.
Support isn't exposed through a CLI flag right now due to the early stages of
the `WeakRef` proposal, but the env var `WASM_BINDGEN_WEAKREF` can be used to
enable this generation. Upon doing so the output can then be edited slightly as
well to work in the SpiderMonkey shell and it looks like this is working!
Closes#704
First added in #161 this never ended up panning out, so let's remove the
experimental suport which isn't actually used by anything today and hold off on
any other changes until an RFC happens.
* Fix importing the same identifier from two modules
This needed a fix in two locations:
* First the generated descriptor function needed its hash to include the module
that the import came from in order to generate unique descriptor functions.
* Second the generation of the JS shim needed to handle duplicate identifiers in
a more uniform fashion, ensuring that imported names didn't clash.
* Fix importing the same name in two modules
Previously two descriptor functions with duplicate symbols were emitted, and now
only one function is emitted by using a global table to keep track of state
across macro invocations.
In addition to being more ergonomic these are much more efficient at reading
large files as they preallocate internally. This provides a nice speed boost
locally, reducing the overhead of `wasm-bindgen-test-runner` from 0.23s to
0.19s, yay!
This commit updates the test runner to only deserialize a `Module` once and then
directly pass it to the `wasm-bindgen` config, avoiding pulling in a public
dependency with the same strategy as the `wasm-gc-api` crate for now.
This reduces the runtime of this step for `wasm-bindgen-test-runner` from ~0.23s
to ~0.19s on my machine.
Since `wasmi` already has a public dependency on `parity_wasm` let's just use
it! A `clone` is much faster than a serialize + parse, reducing a `wasm-bindgen`
invocation on my machine from 0.2s to 0.18s.
This is a bit of a refinement of the solution from #548 to make sure that these
statics are only present on the `wasm32-*` targets, as otherwise these
descriptors are completely inert on other platforms!
* Bump to 0.2.12
* Update all version numbers and deps
* Update all listed authors to `["The wasm-bindgen Developers"]`
* Update `repository` links to specific paths for each crate
* Update `homepage` links to the online book
* Update all links away from `alexcrichton/wasm-bindgen`
* Add `#[doc]` directives for HTML URLs
* Update more version requirements
* Fill out CHANGELOG
This commit adds a hack to the `wasm-bindgen` CLI tool to work around #483 which
is present on nightly Rust with the recent LLVM upgrade. Hopefully this'll carry
us forward until the [upstream bug][1] is fixed.
Closes#483
[1]: https://bugs.llvm.org/show_bug.cgi?id=38184
The changes on master Rust insert debug sections now (yay!) but this means that
wasm binaries by default pick up debug sections from the standard library, so
let's remove them by default in wasm-bindgen unless `--debug` is passed
* Reorganize Travis configuration
* Add a `JOB` env var descriptor to all matrix entries. Not used anywhere but is
useful when viewing the whole build on Travis's web interface.
* Reorganize where builds are located, moving slow builds first and fast ones
last.
* Change checking the CLI builds from `cargo build` to `cargo check`
* Use YAML references to reduce some duplication
* Print some more timing statistics for each test
* Extract `Project` helper in tests to a module
This'll help make it a bit more extensible over time. At the same time the
methods are also slightly reorganized to read more clearly from top to bottom.
* Migrate all tests away from Webpack
Wepback can take a significant amount of time to execute and when it's
multiplied by hundreds of tests that adds up really quickly! After investigating
Node's `--experimental-modules` option it looks like it's suitable for our use
so this switches all tests to using JS files (moving away from TypeScript as
well) with `--experimental-modules` with Node.
Tests will be selectively re-enabled with webpack and node.js specific output
(that doesn't require `--experimental-modules`), coming in later commits.
* Restore the node test for node.js output
Ensures it's workable as-is
* Only generate typescript with webpack
* Only read wasm files for webpack
* Skip package.json/node_modules for now
* Only generate webpack config if needed
* Start a dedicated test module for typescript
Will hopefully verify the generated Typescript compiles OK.
* Remove unneeded `node` method
* Fixup some rebase conflicts
* Don't run asmjs example on travis
* Fixup generator tests
* Attempt to fix windows
* Comment windows fix
* More test fixes
* More exclusions
* More test fixes
* Relax eslint regex
Catch mjs modules as well
* Fix eslint
* Speed up travis on examples slightly
This can happen when a nested dependency crate exports things but the root crate
doesn't use them. In these cases, it is fine to ignore the missing descriptor,
because the thing it describes was removed as dead code.
This commit adds a `#[wasm_bindgen(version = "...")]` attribute support. This
information is eventually written into a `__wasm_pack_unstable` section.
Currently this is a strawman for the proposal in ashleygwilliams/wasm-pack#101
Currently errors are reported via Rust panics but there's lots more errors being
added over time so this commit starts the movement towards the `failure` crate
to more idiomatically report errors as well as provide better error messages
over time.
Currently the entire `Program` is deserialized to match schema versions but this
is likely to fail when the schema changes. Instead just deserialize the
schema/version fields, compare those, and if successful go ahead and deserialize
everything.
This commit is a complete overhaul of how the `#[wasm_bindgen]` macro
communicates type information to the CLI tool, and it's done in a somewhat...
unconventional fashion.
Today we've got a problem where the generated JS needs to understand the types
of each function exported or imported. This understanding is what enables it to
generate the appropriate JS wrappers and such. We want to, however, be quite
flexible and extensible in types that are supported across the boundary, which
means that internally we rely on the trait system to resolve what's what.
Communicating the type information historically was done by creating a four byte
"descriptor" and using associated type projections to communicate that to the
CLI tool. Unfortunately four bytes isn't a lot of space to cram information like
arguments to a generic function, tuple types, etc. In general this just wasn't
flexible enough and the way custom references were treated was also already a
bit of a hack.
This commit takes a radical step of creating a **descriptor function** for each
function imported/exported. The really crazy part is that the `wasm-bindgen` CLI
tool now embeds a wasm interpreter and executes these functions when the CLI
tool is invoked. By allowing arbitrary functions to get executed it's now *much*
easier to inform `wasm-bindgen` about complicated structures of types. Rest
assured though that all these descriptor functions are automatically unexported
and gc'd away, so this should not have any impact on binary sizes
A new internal trait, `WasmDescribe`, is added to represent a description of all
types, sort of like a serialization of the structure of a type that
`wasm-bindgen` can understand. This works by calling a special exported function
with a `u32` value a bunch of times. This means that when we run a descriptor we
effectively get a `Vec<u32>` in the `wasm-bindgen` CLI tool. This list of
integers can then be parsed into a rich `enum` for the JS generation to work
with.
This commit currently only retains feature parity with the previous
implementation. I hope to soon solve issues like #123, #104, and #111 with this
support.
Node's fs APIs resolve relative paths relative to the current working directory:
https://nodejs.org/api/fs.html#fs_file_paths
This creates a problem if you try to require the wasm-bindgen-generated
JavaScript from a different directory. For example, if you have
build/foo.js
build/foo_bg.js
build/foo_bg.wasm
and another script, script/index.js, that requires build/foo.js. We can instead
use __dirname to get the correct path to the file.
As soon as we've removed unneeded exports immediately run a gc pass to ensure
that we don't bind functions in JS that don't actually end up getting needed.
This commit starts wasm-bindgen down the path of supporting closures. We
discussed this at the recent Rust All-Hands but I ended up needing to pretty
significantly scale back the ambitions of what closures are supported. This
commit is just the initial support and provides only a small amount of support
but will hopefully provide a good basis for future implementations.
Specifically this commit adds support for passing `&Fn(...)` to an *imported
function*, but nothing elese. The `&Fn` type can have any lifetime and the JS
object is invalidated as soon as the import returns. The arguments and return
value of `Fn` must currently implement the `WasmAbi` trait, aka they can't
require any conversions like strings/types/etc.
I'd like to soon expand this to `&mut FnMut` as well as `'static` closures that
can be passed around for a long time in JS, but for now I'm putting that off
until later. I'm not currently sure how to implement richer argument types, but
hopefully that can be figured out at some point!
This commit starts wasm-bindgen down a path of removing the special
casing it currently has around vectors, slices, and strings. This has
long been a thorn in wasm-bindgen's side as it doesn't handle other
kinds of vectors and otherwise is very inflexible with future additions.
Additionally it leads to a lot of duplicated-ish code throughout various
portions of codegen.
The fundamental reason for this was that two arguments were required to
be passed back to wasm, and I couldn't figure out a way to shove both
those arguments into a function argument. The new strategy here is that
there is one global stack well known to both JS and Rust which arguments
*may* also be transferred between.
By default all ABI arguments pass as literal function arguments, but if
two or more arguments need to be passed then the extra ones are all
passed through this global stack. The stack is effectively temporary
scratch space when crossing the JS/Rust boundary (both ways). No long
term storage is intended here.
The `simple` test is passing as a result of this commit, using strings
internally. The `Vector` type in the AST has been removed (yay!) and the
bulk of the implementation of slices and vectors now resides in the
`wasm-bindgen` crate itself, defining how to pass all these arguments
around. The JS generator, however, still needs to know about all the
sorts of vectors so it can generate appropriate code for JS.
Future commits will continue cleanup and get the rest of the tests
working.
