99 Commits

Author SHA1 Message Date
Nick Fitzgerald
5f90951441 Use the multi-value xform when targeting interface types 2019-09-10 17:32:30 -07:00
Nick Fitzgerald
582b733967 Update to walrus 0.12.0 2019-09-10 17:32:30 -07:00
Alex Crichton
c5dd572d9e Add support for emitting a Wasm Interface Types section
This commit adds support to `wasm-bindgen` to emit a WebAssembly module
that contains a WebAssembly Interface Types section. As of today there are no
native consumers of these WebAssembly modules, and the actual binary format
here is basically arbitrary (chosen by the `wasm-webidl-bindings` crate). The
intention is that we'll be following the [WebAssembly Interface
Types proposal][proposal] very closely and update here as necessary.

The main feature added in this PR is that a new experimental environment
variable, `WASM_INTERFACE_TYPES=1`, is recognized by the `wasm-bindgen`
CLI tool. When present the CLI tool will act differently than it does
today:

* The `anyref` feature will be implicitly enabled
* A WebAssembly interface types section will be emitted in the
  WebAssembly module
* For now, the WebAssembly module is strictly validated to require zero
  JS glue. This means that `wasm-bindgen` is producing a fully
  standalone WebAssembly module.

The last point here is one that will change before this functionality is
stabilized in `wasm-bindgen`. For now it reflects the major use case of
this feature which is to produce a standalone WebAssembly module with no
support JS glue, and to do that we need to verify properties like it's
not using JS global names, nonstandard binding expressions, etc. The
error messages here aren't the best but they at least fail compilation
at some point instead of silently producing weird wasm modules.

Eventually it's envisioned that a WebAssembly module will contain an
interface types section but *also* have JS glue so binding expressions
can be used when available but otherwise we'd still generate JS glue for
things like nonstandard expressions and accessing JS global values.

It should be noted that a major feature not implemented in
`wasm-bindgen` yet is the multi-value proposal for WebAssembly. This is
coming soon (as soon as we can) in `walrus` and later for a pass here,
but for now this means that returning multiple values (like a string
which has a pointer/length) is a bit of a hack. To enable this use case
a `wasm-bindgen`-specific-convention which will never be stabilized is
invented here by using binding expression to indicate "this return value
is actually returned through an out-ptr as the first argument list".
This is a gross hack and is guaranteed to be removed. Eventually we will
support multi-value and the wasm module emitted will simply use
multi-value and contain internal polyfills for Rust's ABI which returns
values through out-ptrs.

Overall this should make `wasm-bindgen` usable for playing around with
the WebIDL bindings proposal and helping us get a taste of what it looks
like to have entirely standalone WebAssembly modules running in multiple
environments, no extra fluff necessary!

[proposal]: https://github.com/webassembly/webidl-bindings
2019-08-16 12:02:01 -07:00
Alex Crichton
1d0c333a2b Run cargo fmt over all code 2019-08-12 11:28:37 -07:00
Alex Crichton
732b691bb8
Merge pull request #1690 from alexcrichton/webidl-polyfill
Add support as a vanilla polyfill of WebIDL bindings
2019-08-02 09:47:57 -05:00
Alex Crichton
6e3e9d2dae Correctly hook up the anyref table initialization
This functionality got lost in recent refactorings for WebIDL bindings
unfortunately, so this commit touches things up to ensure that the
anyref table initialization in anyref-mode is hooked up correctly, even
when tests are enabled. This invovled moving injection of the start
function to the webidl processing pass and ensuring its intrinsic is
registered in the internal maps of wasm-bindgen.
2019-08-01 11:56:57 -07:00
Alex Crichton
2aac606d3f Support emitting direct imports in wasm files
Support was previously (re-)added in #1654 for importing direct JS
values into a WebAssembly module by completely skipping JS shim
generation. This commit takes that PR one step further by *also*
embedding a direct import in the wasm file, where supported. The wasm
file currently largely just imports from the JS shim file that we
generate, but this allows it to directly improt from ES modules where
supported and where possible. Note that like #1654 this only happens
when the function signature doesn't actually require any conversions to
happen in JS (such as handling closures).

For imports from ES modules, local snippets, or inline JS they'll all
have their import directives directly embedded into the final
WebAssembly binary without any shims necessary to hook it all up. For
imports from the global namespace or possibly vendor-prefixed items
these still unconditionally require an import shim to be generated
because there's no way to describe that import in an ES-friendly way
(yet).

There's a few consequences of this commit which are also worth noting:

* The logic in `wasm-bindgen` where it gracefully handles (to some
  degree) not-defined items now only is guaranteed to be applied to the
  global namespace. If you import from a module, it'll be an
  instantiation time error rather than today's runtime error when the
  import is called.

* Handling imports in the wasm module not registered with
  `#[wasm_bindgen]` has become more strict. Previously these imports
  were basically ignored, leaving them up for interpretation depending
  on the output format. The changes for each output target are:

  * `bundler` - not much has changed here. Previously these ignored
    imports would have been treated as ES module imports, and after this
    commit there might just be some more of these imports for bundlers
    to resolve.

  * `web` - previously the ignored imports would likely cause
    instantiation failures because the import object never actually
    included a binding for other imports. After this commit though the
    JS glue which instantiates the module now interprets all
    unrecognized wasm module imports as ES module imports, emitting an
    `import` directive. This matches what we want for the direct import
    functionality, and is also largely what we want for modules in
    general.

  * `nodejs` - previously ignored imports were handled in the
    translation shim for Node to generate `require` statements, so they
    were actually "correctly handled" sort of with module imports. The
    handling of this hasn't changed, and reflects what we want for
    direct imports of values where loading a wasm module in Node ends up
    translating the module field of each import to a `require`.

  * `no-modules` - this is very similar to the `web` target where
    previously this didn't really work one way or the other because we'd
    never fill in more fields of the import object when instantiating
    the module. After this PR though this is a hard-error to have
    unrecognized imports from `#[wasm_bindgen]` with the `no-modules`
    output type, because we don't know how to handle the imports.

  Note that this touches on #1584 and will likely break the current use
  case being mentioned there. I think though that this tightening up of
  how we handle imports is what we'll want in the long run where
  everything is interpreted as modules, and we'll need to figure out
  best how wasi fits into this.

This commit is unlikely to have any real major immediate effects. The
goal here is to continue to inch us towards a world where there's less
and less JS glue necessary and `wasm-bindgen` is just a polyfill for web
standards that otherwise all already exist.

Also note that there's no explicitly added tests for this since this is
largely just a refactoring of an internal implementation detail of
`wasm-bindgen`, but the main `wasm` test suite has many instances of
this path being taken, for example having imports like:

    (import "tests/wasm/duplicates_a.js" "foo" (func $__wbg_foo_969c253238f136f0 (type 1)))
    (import "tests/wasm/duplicates_b.js" "foo" (func $__wbg_foo_027958cb2e320a94 (type 0)))
    (import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "trivial" (func $__wbg_trivial_75e27c84882af23b (type 1)))
    (import "./snippets/wasm-bindgen-3dff2bc911f0a20c/inline0.js" "incoming_bool" (func $__wbg_incomingbool_0f2d9f55f73a256f (type 0)))
2019-07-31 12:28:24 -07:00
Alex Crichton
452ce2916e Add support as a vanilla polyfill of WebIDL bindings
This commit adds support to `wasm-bindgen` to be a drop-in polyfill for
the WebIDL bindings proposal. Lots of internal refactoring has happened
previously to `wasm-bindgen` to make this possible, so this actually
ends up being a very small PR!

Most of `wasm-bindgen` is geared towards Rust-specific types and
Rust-specific support, but with the advent of WebIDL bindings this is a
standard way for a WebAssembly module to communicate its intended
interface in terms of higher level types. This PR allows `wasm-bindgen`
to be a polyfill for any WebAssembly module that has a valid WebIDL
bindings section, regardless of its producer. A standard WebIDL bindings
section is recognized in any input wasm module and that is slurped up
into wasm-bindgen's own internal data structures to get processed in the
same way that all Rust imports/exports are already processed.

The workflow for `wasm-bindgen` looks the same way that it does in Rust
today. You'd execute `wasm-bindgen path/to/foo.wasm --out-dir .` which
would output a new wasm file and a JS shim with the desired interface,
and the new wasm file would be suitable for loading in MVP
implementations of WebAssembly.

Note that this isn't super thoroughly tested, so there's likely still
some lingering assumptions that `wasm-bindgen` makes (such as
`__wbindgen_malloc` and others) which will need to be patched in the
future, but the intention of this commit is to start us down a road of
becoming a drop-in polyfill for WebIDL bindings, regardless of the
source. Also note that there's not actually any producer (AFAIK) of a
WebIDL bindings custom section, so it'd be that much harder to write
tests to do so!
2019-07-31 12:12:38 -07:00
Alex Crichton
8cb792474c Fully update threading support for LLVM 9
Support has landed in rust-lang/rust for full support for LLVM 9's
interpretation of WebAssembly threads. This commit updates our thread
transformation pass to take all this into account, namely:

* The threadign pass now runs by default and is keyed on whether memory
  is shared, not off an env var.
* TLS is initialized in addition to memory on each thread.
* Stack pointer finding is tweaked to account for the TLS base also
  being a mutable global.
* The build of the parallel raytrace example was updated to use today's
  nightly.
2019-07-30 09:17:23 -07:00
Alex Crichton
10e400bee4 Add support to emit output to memory
Don't necessarily require a filesystem to execute `wasm-bindgen`,
allowing the `wasm-bindgen-cli-support` crate to be compiled to
WebAssembly, for example, and possibly run `wasm-bindgen` in your
browser! For now this is largely just an internal refactoring and won't
result in many use cases, but it felt like a good refactoring to have
regardless.
2019-07-30 08:08:17 -07:00
Alex Crichton
3cc30843e3 Second large refactor for WebIDL bindings
This commit is the second, and hopefully last massive, refactor for
using WebIDL bindings internally in `wasm-bindgen`. This commit actually
fully executes on the task at hand, moving `wasm-bindgen` to internally
using WebIDL bindings throughout its code generation, anyref passes,
etc. This actually fixes a number of issues that have existed in the
anyref pass for some time now!

The main changes here are to basically remove the usage of `Descriptor`
from generating JS bindings. Instead two new types are introduced:
`NonstandardIncoming` and `NonstandardOutgoing` which are bindings lists
used for incoming/outgoing bindings. These mirror the standard
terminology and literally have variants which are the standard values.
All `Descriptor` types are now mapped into lists of incoming/outgoing
bindings and used for process in wasm-bindgen. All JS generation has
been refactored and updated to now process these lists of bindings
instead of the previous `Descriptor`.

In other words this commit takes `js2rust.rs` and `rust2js.rs` and first
splits them in two. Interpretation of `Descriptor` and what to do for
conversions is in the binding selection modules. The actual generation
of JS from the binding selection is now performed by `incoming.rs` and
`outgoing.rs`. To boot this also deduplicates all the code between the
argument handling of `js2rust.rs` and return value handling of
`rust2js.rs`. This means that to implement a new binding you only need
to implement it one place and it's implemented for free in the other!

This commit is not the end of the story though. I would like to add a
mdoe to `wasm-bindgen` that literally emits a WebIDL bindings section.
That's left for a third (and hopefully final) refactoring which is also
intended to optimize generated JS for bindings.

This commit currently loses the optimization where an imported is hooked
up by value directly whenever a shim isn't needed. It's planned that
the next refactoring to emit a webidl binding section that can be added
back in. It shouldn't be too too hard hopefully since all the
scaffolding is in place now.

cc #1524
2019-06-20 19:16:10 -07:00
Alex Crichton
83a3f5d44a Run cargo fmt --all 2019-06-13 08:30:06 -07:00
Alex Crichton
c22b907e7f Touch up some comments 2019-06-05 07:52:14 -07:00
Alex Crichton
b51df39bc9 Reimplement anyref processing and passes
This commit reimplements the `anyref` transformation pass tasked with
taking raw rustc output and enhancing the module to use `anyref`. This
was disabled in the previous commits during refactoring, and now the
pass is re-enabled in the manner originally intended.

Instead of being tangled up in the `js/mod.rs` pass, the anyref
transformation now happens locally within one module,
`cli-support/src/anyref.rs`, which exclusively uses the output of the
`webidl` module which produces a WebIDL bindings section as well as an
auxiliary wasm-bindgen specific section. This makes the anyref transform
much more straightforward and local, ensuring that it doesn't propagate
elsewhere and can be a largely local concern during the transformation.

The main addition needed to support this pass was detailed knowledge of
the ABI of a `Descriptor`. This knowledge is already implicitly
hardcoded in `js2rust.rs` and `rust2js.rs` through the ABI shims
generated. This was previously used for the anyref transformation to
piggy-back what was already there, but as a separate pass we are unable
to reuse the knowledge in the binding generator.

Instead `Descriptor` now has two dedicated methods describing the
various ABI properties of a type. This is then asserted to be correct
(all the time) when processing bindings, ensuring that the two are kept
in sync.
2019-06-05 07:52:14 -07:00
Alex Crichton
68c5233f80 First refactor for WebIDL bindings
This commit starts the `wasm-bindgen` CLI tool down the road to being a
true polyfill for WebIDL bindings. This refactor is probably the first
of a few, but is hopefully the largest and most sprawling and everything
will be a bit more targeted from here on out.

The goal of this refactoring is to separate out the massive
`crates/cli-support/src/js/mod.rs` into a number of separate pieces of
functionality. It currently takes care of basically everything
including:

* Binding intrinsics
* Handling anyref transformations
* Generating all JS for imports/exports
* All the logic for how to import and how to name imports
* Execution and management of wasm-bindgen closures

Many of these are separable concerns and most overlap with WebIDL
bindings. The internal refactoring here is intended to make it more
clear who's responsible for what as well as making some existing
operations much more straightforward. At a high-level, the following
changes are done:

1. A `src/webidl.rs` module is introduced. The purpose of this module is
   to take all of the raw wasm-bindgen custom sections from the module
   and transform them into a WebIDL bindings section.

  This module has a placeholder `WebidlCustomSection` which is nowhere
  near the actual custom section but if you squint is in theory very
  similar. It's hoped that this will eventually become the true WebIDL
  custom section, currently being developed in an external crate.

  Currently, however, the WebIDL bindings custom section only covers a
  subset of the functionality we export to wasm-bindgen users. To avoid
  leaving them high and dry this module also contains an auxiliary
  custom section named `WasmBindgenAux`. This custom section isn't
  intended to have a binary format, but is intended to represent a
  theoretical custom section necessary to couple with WebIDL bindings to
  achieve all our desired functionality in `wasm-bindgen`. It'll never
  be standardized, but it'll also never be serialized :)

2. The `src/webidl.rs` module now takes over quite a bit of
   functionality from `src/js/mod.rs`. Namely it handles synthesis of an
   `export_map` and an `import_map` mapping export/import IDs to exactly
   what's expected to be hooked up there. This does not include type
   information (as that's in the bindings section) but rather includes
   things like "this is the method of class A" or "this import is from
   module `foo`" and things like that. These could arguably be subsumed
   by future JS features as well, but that's for another time!

3. All handling of wasm-bindgen "descriptor functions" now happens in a
   dedicated `src/descriptors.rs` module. The output of this module is
   its own custom section (intended to be immediately consumed by the
   WebIDL module) which is in theory what we want to ourselves emit one
   day but rustc isn't capable of doing so right now.

4. Invocations and generations of imports are completely overhauled.
   Using the `import_map` generated in the WebIDL step all imports are
   now handled much more precisely in one location rather than
   haphazardly throughout the module. This means we have precise
   information about each import of the module and we only modify
   exactly what we're looking at. This also vastly simplifies intrinsic
   generation since it's all simply a codegen part of the `rust2js.rs`
   module now.

5. Handling of direct imports which don't have a JS shim generated is
   slightly different from before and is intended to be
   future-compatible with WebIDL bindings in its full glory, but we'll
   need to update it to handle cases for constructors and method calls
   eventually as well.

6. Intrinsic definitions now live in their own file (`src/intrinsic.rs`)
   and have a separated definition for their symbol name and signature.
   The actual implementation of each intrinsic lives in `rust2js.rs`

There's a number of TODO items to finish before this merges. This
includes reimplementing the anyref pass and actually implementing import
maps for other targets. Those will come soon in follow-up commits, but
the entire `tests/wasm/main.rs` suite is currently passing and this
seems like a good checkpoint.
2019-06-05 07:52:14 -07:00
Gus Caplan
66ade77720
Rewrite weakrefs to use current proposal 2019-05-26 09:30:33 -05:00
Nick Fitzgerald
d422436487 Update dependencies and use new walrus custom sections APIs 2019-05-17 14:58:50 -07:00
Alex Crichton
3d43d6e5e8 Fix importing and exporting the same name
Run exports through the same identifier generation as imports to ensure
that everything gets a unique identifier and then just make sure all the
appropriate wires are hooked up when dealing with exports and imports.

Closes #1496
2019-05-03 07:15:20 -07:00
Nick Fitzgerald
dff9b9b1e3 Don't put anything before ES module imports
Because of some incorrect use of `js.push_str(..)`, we could sometimes emit code
before the ES modules imports, which is syntactically invalid:

    const __exports = {};
    import { Thing } from '...'; // Syntax error!

This has been fixed by making sure that the correct `imports` or `imports_post`
string is built up. We now also assert that the `js` string is empty at the
location where we add imports if we're using ES modules.
2019-04-04 15:53:43 -07:00
Alex Crichton
a6fe0cefa8 Migrate all crates to the 2018 edition
Most of the CLI crates were already in the 2018 edition, and it turns
out that one of the macro crates was already in the 2018 edition so we
may as well move everything to the 2018 edition!

Always nice to remove those `extern crate` statements nowadays!

This commit also does a `cargo fmt --all` to make sure we're conforming
with style again.
2019-03-26 08:10:53 -07:00
Alex Crichton
362777fc75 Start implementing a test suite for the CLI
We have very few tests today so this starts to add the basics of a test
suite which compiles Cargo projects on-the-fly which will hopefully help
us bolster the amount of assertions we can make about the output.
2019-03-22 11:32:03 -07:00
Alex Crichton
6edb40a807 Implement transitive support for NPM dependencies
This commit implements [RFC 8], which enables transitive and transparent
dependencies on NPM. The `module` attribute, when seen and not part of a
local JS snippet, triggers detection of a `package.json` next to
`Cargo.toml`. If found it will cause the `wasm-bindgen` CLI tool to load
and parse the `package.json` within each crate and then create a merged
`package.json` at the end.

[RFC 8]: https://github.com/rustwasm/rfcs/pull/8
2019-03-22 09:42:40 -07:00
Alex Crichton
c9a972ac55
Merge pull request #1376 from alexcrichton/more-debugging
Add some logging useful in debugging #1373
2019-03-21 16:55:04 -05:00
Alex Crichton
995be7c027 Replace target flags with --target
This commit deprecates the `--web`, `--no-modules`, and `--nodejs` flags
in favor of one `--target` flag. The motivation for this commit is to be
consistent between `wasm-bindgen` and `wasm-pack` so documentation for
one is applicable for the other (so we don't have to document everywhere
what the translation is between flags). Additionally this should make it
a bit easier to add new targets (if necessary) in the future as it won't
add to the proliferation of flags.

For now the old flags (like `--web`) continue to be accepted, but
they'll be removed during the next set of breaking changes for
`wasm-bindgen`.
2019-03-21 14:00:33 -07:00
Alex Crichton
12355ce81c Add some logging useful in debugging #1373
Ended up helping diagnose the problem in the end!
2019-03-21 08:30:30 -07:00
Alex Crichton
a7e7f8b5e1
Merge pull request #1328 from alexcrichton/switch-to-web
Switch the `--browser` argument to `--web`
2019-03-07 15:25:26 -06:00
Vlad Shcherbina
2a2581188a Create out_dir (usually "pkg") if it does not exist 2019-03-07 19:56:57 +03:00
Alex Crichton
cd3781cb73 Switch the --browser argument to --web
This commit reverts part of the implementation of [RFC 6]. That RFC
specified that the `--browser` flag was going to be repurposed for the
new "natively loadable as ES module output", but unfortunately the
breakage is far broader than initially expected. It turns out that
`wasm-pack` passes `--browser` by default which means that a change to
break `--browser` would break all historical versions of `wasm-pack`
which is a bit much for now.

To solve this the `--browser` flag is going back to what it represents
on the current released version of `wasm-bindgen` (optimize away some
node.js checks in a few places for bundler-style output) and a new
`--web` flag is being introduced as the new deployment strategy.

[RFC 6]: https://github.com/rustwasm/rfcs/pull/6

Closes #1318
2019-03-07 08:26:36 -08:00
Alex Crichton
d6e3770350 Scope snippets within a crate
Use the same crate identifier for manually included snippets as well as
inline snippets to help with debugging.
2019-03-05 14:53:14 -08:00
Alex Crichton
b762948456 Implement the local JS snippets RFC
This commit is an implementation of [RFC 6] which enables crates to
inline local JS snippets into the final output artifact of
`wasm-bindgen`. This is accompanied with a few minor breaking changes
which are intended to be relatively minor in practice:

* The `module` attribute disallows paths starting with `./` and `../`.
  It requires paths starting with `/` to actually exist on the filesystem.
* The `--browser` flag no longer emits bundler-compatible code, but
  rather emits an ES module that can be natively loaded into a browser.

Otherwise be sure to check out [the RFC][RFC 6] for more details, and
otherwise this should implement at least the MVP version of the RFC!
Notably at this time JS snippets with `--nodejs` or `--no-modules` are
not supported and will unconditionally generate an error.

[RFC 6]: https://github.com/rustwasm/rfcs/pull/6

Closes #1311
2019-03-05 08:00:47 -08:00
Alex Crichton
745b16e3d2 Add support for TextEncoder#encodeInto
This commit adds support for the recently implemented standard of
[`TextEncoder#encodeInto`][standard]. This new function is a "bring your
own buffer" style function where we can avoid an intermediate allocation
and copy by encoding strings directly into wasm's memory.

Currently we feature-detect whether `encodeInto` exists as it is only
implemented in recent browsers and not in all browsers. Additionally
this commit emits the binding using `encodeInto` by default, but this
requires `realloc` functionality to be exposed by the wasm module.
Measured locally an empty binary which takes `&str` previously took
7.6k, but after this commit takes 8.7k due to the extra code needed for
`realloc`.

[standard]: https://encoding.spec.whatwg.org/#dom-textencoder-encodeinto

Closes #1172
2019-02-26 10:29:39 -08:00
Alex Crichton
4181fb311a Add experimental support for the anyref type
This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.

This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.

The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.

This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.

The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.

The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.
2019-02-20 07:28:54 -08:00
Alex Crichton
8fb705a1ef Upgrade to walrus 0.4
Also be sure to have an explicit GC pass!
2019-02-19 14:14:01 -08:00
Alex Crichton
a3c03b5265 Revert "Temporarily unconditionally remove producers section"
This reverts commit f0cd51a56642599a89daee8330ca4bd20fc97c87.
2019-02-15 11:22:46 -08:00
Alex Crichton
f0cd51a566 Temporarily unconditionally remove producers section
We've had a lot of bug reports with upstream webpack currently and while
webpack has a fix it may take a moment to deploy. Let's try and fix
wasm-bindgen in the meantime!

Once webpack is updated we can go back to emitting a producers section
by default and publish a new version of wasm-bindgen.
2019-02-15 07:39:42 -08:00
Alex Crichton
5e3cedfaf2 Add a flag to remove producers section
This should help handle instances like the recent Webpack bug and is
also a useful flag in its own right. For now it's set to `false`, but if
the Webpack bug persists through to tomorrow we likely want to publish a
version of `wasm-bindgen` with it default set to `true`.
2019-02-14 10:08:24 -08:00
Nick Fitzgerald
17dc79e4c5 Configure walrus to generate DWARF/names at the right times
Fixes ##1254
2019-02-14 07:20:43 -08:00
Alex Crichton
894b479213 Migrate wasm-bindgen to using walrus
This commit moves `wasm-bindgen` the CLI tool from internally using
`parity-wasm` for wasm parsing/serialization to instead use `walrus`.
The `walrus` crate is something we've been working on recently with an
aim to replace the usage of `parity-wasm` in `wasm-bindgen` to make the
current CLI tool more maintainable as well as more future-proof.

The `walrus` crate provides a much nicer AST to work with as well as a
structured `Module`, whereas `parity-wasm` provides a very raw interface
to the wasm module which isn't really appropriate for our use case. The
many transformations and tweaks that wasm-bindgen does have a huge
amount of ad-hoc index management to carefully craft a final wasm
binary, but this is all entirely taken care for us with the `walrus`
crate.

Additionally, `wasm-bindgen` will ingest and rewrite the wasm file,
often changing the binary offsets of functions. Eventually with DWARF
debug information we'll need to be sure to preserve the debug
information throughout the transformations that `wasm-bindgen` does
today. This is practically impossible to do with the `parity-wasm`
architecture, but `walrus` was designed from the get-go to solve this
problem transparently in the `walrus` crate itself. (it doesn't today,
but this is planned work)

It is the intention that this does not end up regressing any
`wasm-bindgen` use cases, neither in functionality or in speed. As a
large change and refactoring, however, it's likely that at least
something will arise! We'll want to continue to remain vigilant to any
issues that come up with this commit.

Note that the `gc` crate has been deleted as part of this change, as the
`gc` crate is no longer necessary since `walrus` does it automatically.
Additionally the `gc` crate was one of the main problems with preserving
debug information as it often deletes wasm items!

Finally, this also starts moving crates to the 2018 edition where
necessary since `walrus` requires the 2018 edition, and in general it's
more pleasant to work within the 2018 edition!
2019-02-12 07:25:53 -08:00
Alex Crichton
b21489368c Ensure internal exported_classes map is consistent
Throw it in an `Option` and then `take()` it when we consume it to
ensure that future calls to insert data into it panic instead of
producing inconsistent JS.
2019-01-14 15:53:29 -08:00
Alex Crichton
c0ddabcca1 Sanity check exposed_globals map
Make sure that we don't actually try to expose something when they've
already been written, causing an internal tool panic rather than wrong
JS.
2019-01-14 13:09:05 -08:00
DavidOConnor
a4bc5049c6 Added an --out-name param to the CLI, to allow custom output file names 2018-12-04 21:35:05 -05:00
Alex Crichton
29531c0abf Run rustfmt 2018-11-30 13:04:27 -08:00
Alex Crichton
a2aa28e4d3 Add a #[wasm_bindgen(start)] attribute
This commit adds a new attribute to `#[wasm_bindgen]`: `start`. The
`start` attribute can be used to indicate that a function should be
executed when the module is loaded, configuring the `start` function of
the wasm executable. While this doesn't necessarily literally configure
the `start` section, it does its best!

Only one crate in a crate graph may indicate `#[wasm_bindgen(start)]`,
so it's not recommended to be used in libraries but only end-user
applications. Currently this still must be used with the `crate-type =
["cdylib"]` annotation in `Cargo.toml`.

The implementation here is somewhat tricky because of the circular
dependency between our generated JS and the wasm file that we emit. This
circular dependency makes running initialization routines (like the
`start` shim) particularly fraught with complications because one may
need to run before the other but bundlers may not necessarily respect
it. Workarounds have been implemented for various emission strategies,
for example calling the start function directly after exports are wired
up with `--no-modules` and otherwise working around what appears to be
a Webpack bug with initializers running in a different order than we'd
like. In any case, this in theory doesn't show up to the end user!

Closes #74
2018-11-28 22:11:15 -08:00
Alex Crichton
e3b628689f
Merge pull request #1053 from alexcrichton/dts-wasm
Generate a `*.d.ts` file for wasm files
2018-11-27 16:55:25 -06:00
Alex Crichton
047c41c1ec Generate a *.d.ts file for wasm files
This generates a `*.d.ts` file for the wasm file that wasm-bindgen emits
whenever typescript is enable *in addition* to the `*.d.ts` file that
already exists for the JS shim.

Closes #1040
2018-11-27 12:36:55 -08:00
Alex Crichton
48f4adfa8c Run rustfmt over everything 2018-11-27 12:07:59 -08:00
Alex Crichton
68537b9649 Add an optimization to directly wire up imported functions
This commit adds an optimization to `wasm-bindgen` to directly import
and invoke other modules' functions from the wasm module, rather than
going through a shim in the imported bindings. This will be an important
optimization in the future for the host bindings proposal, but for now
it's largely just a proof-of-concept to show that we can do it and is
unlikely to bring about many performance benefits.

The implementation in this commit is largely refactoring to reorganize a
bit how functions are imported, but the implementation happens in
`generate_import_function`.

With this commit, 71/287 imports in the `tests/wasm/main.rs` suite get
hooked up directly to the ES modules, no shims needed!
2018-11-13 13:16:38 -08:00
Alex Crichton
12fc09a124 Add a flag to remove the wasm name section
This commit adds a `--remove-name-section` flag to the `wasm-bindgen`
command which will remove the `name` section of the wasm file, used to
indicate the names of functions typically used in debugging. This flag
is off-by-default and will primarily be controlled by wasm-pack,
typically being passed by default with `wasm-pack build --release`.

Closes #1021
2018-11-09 07:45:19 -08:00
Alex Crichton
25b26f41e7 Implement support for WebAssembly threads
... and add a parallel raytracing demo!

This commit adds enough support to `wasm-bindgen` to produce a workable
wasm binary *today* with the experimental WebAssembly threads support
implemented in Firefox Nightly. I've tried to comment what's going on in
the commits and such, but at a high level the changes made here are:

* A new transformation, living in a new `wasm-bindgen-threads-xform`
  crate, prepares a wasm module for parallel execution. This performs a
  number of mundane tasks which I hope to detail in a blog post later on.
* The `--no-modules` output is enhanced with more support for when
  shared memory is enabled, allowing passing in the module/memory to
  initialize the wasm instance on multiple threads (sharing both module
  and memory).
* The `wasm-bindgen` crate now offers the ability, in `--no-modules`
  mode, to get a handle on the `WebAssembly.Module` instance.
* The example itself requires Xargo to recompile the standard library
  with atomics and an experimental feature enabled. Afterwards it
  experimentally also enables threading support in wasm-bindgen.

I've also added hopefully enough CI support to compile this example in a
builder so we can upload it and poke around live online. I hope to
detail more about the technical details here in a blog post soon as
well!
2018-10-23 01:20:18 -07:00
Alex Crichton
f749c7cf95 Don't use JSON for custom section format
This commit migrates away from using Serde for the custom section in
wasm executables. This is a refactoring of a purely-internal data
structure to `wasm-bindgen` and should have no visible functional change
on users.

The motivation for this commit is two fold:

* First, the compile times using `serde_json` and `serde_derive` for the
  syntax extension isn't the most fun.
* Second, eventually we're going to want to stablize the layout of the
  custom section, and it's highly unlikely to be json!

Primarily, though, the intention of this commit is to improve the
cold-cache compile time of `wasm-bindgen` by ensuring that for new users
this project builds as quickly as possible. By removing some heavyweight
dependencies from the procedural macro, `serde`, `serde_derive`, and
`serde_json`, we're able to get a pretty nice build time improvement for
the `wasm-bindgen` crate itself:

|             | single-core build | parallel build |
|-------------|-------------------|----------------|
| master      |             36.5s |          17.3s |
| this commit |             20.5s |          11.8s |

These are't really end-all-be-all wins but they're much better
especially on the spectrum of weaker CPUs (in theory modeled by the
single-core case showing we have 42% less CPU work in theory).
2018-10-12 11:23:00 -07:00