fluencelabs/asmble
1
0
Fork 0
mirror of https://github.com/fluencelabs/asmble synced 2025-03-18 12:10:51 +00:00
Code Issues Projects Releases Wiki Activity

merge asmble-master to master

Browse Source
This commit is contained in:
Constantine Solovev 2018-07-26 10:49:28 +04:00
commit 1990f46743
26 changed files with 612 additions and 129 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
LICENSE
View File

@ -1,6 +1,6 @@
MIT License
Copyright (c) 2017 Chad Retz
Copyright (c) 2018 Chad Retz
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

35
README.md
View File

@ -183,9 +183,16 @@ JVM languages.
### Getting
The latest tag can be added to your build script via [JitPack](https://jitpack.io). For example,
[here](https://jitpack.io/#cretz/asmble/0.1.0) are instructions for using the 0.1.0 release and
[here](https://jitpack.io/#cretz/asmble/master-SNAPSHOT) are instructions for the latest master.
The compiler and annotations are deployed to Maven Central. The compiler is written in Kotlin and can be added as a
Gradle dependency with:
compile 'com.github.cretz.asmble:asmble-compiler:0.3.0'
This is only needed to compile of course, the compiled code has no runtime requirement. The compiled code does include
some annotations (but in Java its ok to have annotations that are not found). If you do want to reflect the annotations,
the annotation library can be added as a Gradle dependency with:
compile 'com.github.cretz.asmble:asmble-annotations:0.3.0'
### Building and Testing
@ -256,15 +263,16 @@ In the WebAssembly MVP a table is just a set of function pointers. This is store
#### Globals
Globals are stored as fields on the class. A non-import global is simply a field, but an import global is a
`MethodHandle` to the getter (and would be a `MethodHandle` to the setter if mutable globals were supported). Any values
for the globals are set in the constructor.
Globals are stored as fields on the class. A non-import global is simply a field that is final if not mutable. An import
global is a `MethodHandle` to the getter and a `MethodHandle` to the setter if mutable. Any values for the globals are
set in the constructor.
#### Imports
The constructor accepts all imports as params. Memory is imported via a `ByteBuffer` param, then function
imports as `MethodHandle` params, then global imports as `MethodHandle` params, then a `MethodHandle` array param for an
imported table. All of these values are set as fields in the constructor.
imports as `MethodHandle` params, then global imports as `MethodHandle` params (one for getter and another for setter if
mutable), then a `MethodHandle` array param for an imported table. All of these values are set as fields in the
constructor.
#### Exports
@ -363,9 +371,12 @@ stack (e.g. some places where we do a swap).
Below are some performance and implementation quirks where there is a bit of an impedance mismatch between WebAssembly
and the JVM:
* WebAssembly has a nice data section for byte arrays whereas the JVM does not. Right now we build a byte array from
a bunch of consts at runtime which is multiple operations per byte. This can bloat the class file size, but is quite
fast compared to alternatives such as string constants.
* WebAssembly has a nice data section for byte arrays whereas the JVM does not. Right now we use a single-byte-char
string constant (i.e. ISO-8859 charset). This saves class file size, but this means we call `String::getBytes` on
init to load bytes from the string constant. Due to the JVM using an unsigned 16-bit int as the string constant
length, the maximum byte length is 65536. Since the string constants are stored as UTF-8 constants, they can be up to
four bytes a character. Therefore, we populate memory in data chunks no larger than 16300 (nice round number to make
sure that even in the worse case of 4 bytes per char in UTF-8 view, we're still under the max).
* The JVM makes no guarantees about trailing bits being preserved on NaN floating point representations like WebAssembly
does. This causes some mismatch on WebAssembly tests depending on how the JVM "feels" (I haven't dug into why some
bit patterns stay and some don't when NaNs are passed through methods).
@ -417,6 +428,8 @@ WASM compiled from Rust, C, Java, etc if e.g. they all have their own way of han
definition of an importable set of modules that does all of these things, even if it's in WebIDL. I dunno, maybe the
effort is already there, I haven't really looked.
There is https://github.com/konsoletyper/teavm
**So I can compile something in C via Emscripten and have it run on the JVM with this?**
Yes, but work is required. WebAssembly is lacking any kind of standard library. So Emscripten will either embed it or

1
annotations/src/main/java/asmble/annotation/WasmImport.java
View File

@ -13,4 +13,5 @@ public @interface WasmImport {
WasmExternalKind kind();
int resizableLimitInitial() default -1;
int resizableLimitMaximum() default -1;
boolean globalSetter() default false;
}

89
build.gradle
View File

@ -2,7 +2,7 @@ group 'asmble'
version '0.2.0'
buildscript {
ext.kotlin_version = '1.2.41'
ext.kotlin_version = '1.2.51'
ext.asm_version = '5.2'
repositories {
@ -19,12 +19,24 @@ buildscript {
allprojects {
apply plugin: 'java'
group 'com.github.cretz.asmble'
version '0.4.0-SNAPSHOT'
repositories {
mavenCentral()
}
}
project(':annotations') {
javadoc {
options.links 'https://docs.oracle.com/javase/8/docs/api/'
// TODO: change when https://github.com/gradle/gradle/issues/2354 is fixed
options.addStringOption 'Xdoclint:all', '-Xdoclint:-missing'
}
publishSettings(project, 'asmble-annotations', 'Asmble WASM Annotations', true)
}
project(':compiler') {
apply plugin: 'kotlin'
apply plugin: 'application'
@ -45,6 +57,8 @@ project(':compiler') {
testCompile "org.jetbrains.kotlin:kotlin-test-junit:$kotlin_version"
testCompile "org.ow2.asm:asm-debug-all:$asm_version"
}
publishSettings(project, 'asmble-compiler', 'Asmble WASM Compiler', false)
}
project(':examples') {
@ -149,4 +163,77 @@ project(':examples:rust-string') {
dependsOn compileRustWasm
}
mainClassName = 'asmble.examples.ruststring.Main'
}
def publishSettings(project, projectName, projectDescription, includeJavadoc) {
project.with {
if (!project.hasProperty('ossrhUsername')) return
apply plugin: 'maven'
apply plugin: 'signing'
archivesBaseName = projectName
task packageSources(type: Jar) {
classifier = 'sources'
from sourceSets.main.allSource
}
if (includeJavadoc) {
task packageJavadoc(type: Jar, dependsOn: 'javadoc') {
from javadoc.destinationDir
classifier = 'javadoc'
}
} else {
task packageJavadoc(type: Jar) {
// Empty to satisfy Sonatype's javadoc.jar requirement
classifier 'javadoc'
}
}
artifacts {
archives packageSources, packageJavadoc
}
signing {
sign configurations.archives
}
uploadArchives {
repositories {
mavenDeployer {
beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }
repository(url: 'https://oss.sonatype.org/service/local/staging/deploy/maven2/') {
authentication(userName: ossrhUsername, password: ossrhPassword)
}
snapshotRepository(url: 'https://oss.sonatype.org/content/repositories/snapshots/') {
authentication(userName: ossrhUsername, password: ossrhPassword)
}
pom.project {
name projectName
packaging 'jar'
description projectDescription
url 'https://github.com/cretz/asmble'
scm {
connection 'scm:git:git@github.com:cretz/asmble.git'
developerConnection 'scm:git:git@github.com:cretz/asmble.git'
url 'git@github.com:cretz/asmble.git'
}
licenses {
license {
name 'MIT License'
url 'https://opensource.org/licenses/MIT'
}
}
developers {
developer {
id 'cretz'
name 'Chad Retz'
url 'https://github.com/cretz'
}
}
}
}
}
}
}
}

7
compiler/src/main/kotlin/asmble/ast/Node.kt
View File

@ -27,6 +27,7 @@ sealed class Node {
val elems: List<Elem> = emptyList(),
val funcs: List<Func> = emptyList(),
val data: List<Data> = emptyList(),
val names: NameSection? = null,
val customSections: List<CustomSection> = emptyList()
) : Node()
@ -160,6 +161,12 @@ sealed class Node {
}
}
data class NameSection(
val moduleName: String?,
val funcNames: Map<Int, String>,
val localNames: Map<Int, Map<Int, String>>
) : Node()
sealed class Instr : Node() {
fun op() = InstrOp.classToOpMap[this::class] ?: throw Exception("No op found for ${this::class}")

2
compiler/src/main/kotlin/asmble/cli/Compile.kt
View File

@ -51,7 +51,7 @@ open class Compile : Command<Compile.Args>() {
val inFormat =
if (args.inFormat != "<use file extension>") args.inFormat
else args.inFile.substringAfterLast('.', "<unknown>")
val script = Translate.inToAst(args.inFile, inFormat)
val script = Translate().also { it.logger = logger }.inToAst(args.inFile, inFormat)
val mod = (script.commands.firstOrNull() as? Script.Cmd.Module) ?:
error("Only a single sexpr for (module) allowed")
val outStream = when (args.outFile) {

2
compiler/src/main/kotlin/asmble/cli/Translate.kt
View File

@ -87,7 +87,7 @@ open class Translate : Command<Translate.Args>() {
}
}
"wasm" ->
Script(listOf(Script.Cmd.Module(BinaryToAst.toModule(
Script(listOf(Script.Cmd.Module(BinaryToAst(logger = logger).toModule(
ByteReader.InputStream(inBytes.inputStream())), null)))
else -> error("Unknown in format '$inFormat'")
}

4
compiler/src/main/kotlin/asmble/compile/jvm/AsmExt.kt
View File

@ -210,3 +210,7 @@ fun ByteArray.asClassNode(): ClassNode {
ClassReader(this).accept(newNode, 0)
return newNode
}
fun ByteArray.chunked(v: Int) = (0 until size step v).asSequence().map {
copyOfRange(it, (it + v).takeIf { it < size } ?: size)
}

113
compiler/src/main/kotlin/asmble/compile/jvm/AstToAsm.kt
View File

@ -48,10 +48,13 @@ open class AstToAsm {
})
// Now all import globals as getter (and maybe setter) method handles
ctx.cls.fields.addAll(ctx.importGlobals.mapIndexed { index, import ->
if ((import.kind as Node.Import.Kind.Global).type.mutable) throw CompileErr.MutableGlobalImport(index)
FieldNode(Opcodes.ACC_PRIVATE + Opcodes.ACC_FINAL, ctx.importGlobalGetterFieldName(index),
val getter = FieldNode(Opcodes.ACC_PRIVATE + Opcodes.ACC_FINAL, ctx.importGlobalGetterFieldName(index),
MethodHandle::class.ref.asmDesc, null, null)
})
if (!(import.kind as Node.Import.Kind.Global).type.mutable) listOf(getter)
else listOf(getter, FieldNode(
Opcodes.ACC_PRIVATE + Opcodes.ACC_FINAL, ctx.importGlobalSetterFieldName(index),
MethodHandle::class.ref.asmDesc, null, null))
}.flatten())
// Now all non-import globals
ctx.cls.fields.addAll(ctx.mod.globals.mapIndexed { index, global ->
val access = Opcodes.ACC_PRIVATE + if (!global.type.mutable) Opcodes.ACC_FINAL else 0
@ -181,9 +184,11 @@ open class AstToAsm {
fun constructorImportTypes(ctx: ClsContext) =
ctx.importFuncs.map { MethodHandle::class.ref } +
// We know it's only getters
ctx.importGlobals.map { MethodHandle::class.ref } +
ctx.mod.imports.filter { it.kind is Node.Import.Kind.Table }.map { Array<MethodHandle>::class.ref }
ctx.importGlobals.flatMap {
// If it's mutable, it also comes with a setter
if ((it.kind as? Node.Import.Kind.Global)?.type?.mutable == false) listOf(MethodHandle::class.ref)
else listOf(MethodHandle::class.ref, MethodHandle::class.ref)
} + ctx.mod.imports.filter { it.kind is Node.Import.Kind.Table }.map { Array<MethodHandle>::class.ref }
fun toConstructorNode(ctx: ClsContext, func: Func) = mutableListOf<List<AnnotationNode>>().let { paramAnns ->
// If the first param is a mem class and imported, add annotation
@ -200,7 +205,15 @@ open class AstToAsm {
}
// All non-mem imports one after another
ctx.importFuncs.forEach { paramAnns.add(listOf(importAnnotation(ctx, it))) }
ctx.importGlobals.forEach { paramAnns.add(listOf(importAnnotation(ctx, it))) }
ctx.importGlobals.forEach {
paramAnns.add(listOf(importAnnotation(ctx, it)))
// There are two annotations here if it's mutable
if ((it.kind as? Node.Import.Kind.Global)?.type?.mutable == true)
paramAnns.add(listOf(importAnnotation(ctx, it).also {
it.values.add("globalSetter")
it.values.add(true)
}))
}
ctx.mod.imports.forEach {
if (it.kind is Node.Import.Kind.Table) paramAnns.add(listOf(importAnnotation(ctx, it)))
}
@ -241,14 +254,25 @@ open class AstToAsm {
}
fun setConstructorGlobalImports(ctx: ClsContext, func: Func, paramsBeforeImports: Int) =
ctx.importGlobals.indices.fold(func) { func, importIndex ->
ctx.importGlobals.foldIndexed(func to ctx.importFuncs.size + paramsBeforeImports) {
importIndex, (func, importParamOffset), import ->
// Always a getter handle
func.addInsns(
VarInsnNode(Opcodes.ALOAD, 0),
VarInsnNode(Opcodes.ALOAD, ctx.importFuncs.size + importIndex + paramsBeforeImports + 1),
VarInsnNode(Opcodes.ALOAD, importParamOffset + 1),
FieldInsnNode(Opcodes.PUTFIELD, ctx.thisRef.asmName,
ctx.importGlobalGetterFieldName(importIndex), MethodHandle::class.ref.asmDesc)
)
}
).let { func ->
// If it's mutable, it has a second setter handle
if ((import.kind as? Node.Import.Kind.Global)?.type?.mutable == false) func to importParamOffset + 1
else func.addInsns(
VarInsnNode(Opcodes.ALOAD, 0),
VarInsnNode(Opcodes.ALOAD, importParamOffset + 2),
FieldInsnNode(Opcodes.PUTFIELD, ctx.thisRef.asmName,
ctx.importGlobalSetterFieldName(importIndex), MethodHandle::class.ref.asmDesc)
) to importParamOffset + 2
}
}.first
fun setConstructorFunctionImports(ctx: ClsContext, func: Func, paramsBeforeImports: Int) =
ctx.importFuncs.indices.fold(func) { func, importIndex ->
@ -262,7 +286,10 @@ open class AstToAsm {
fun setConstructorTableImports(ctx: ClsContext, func: Func, paramsBeforeImports: Int) =
if (ctx.mod.imports.none { it.kind is Node.Import.Kind.Table }) func else {
val importIndex = ctx.importFuncs.size + ctx.importGlobals.size + paramsBeforeImports + 1
val importIndex = ctx.importFuncs.size +
// Mutable global imports have setters and take up two spots
ctx.importGlobals.sumBy { if ((it.kind as? Node.Import.Kind.Global)?.type?.mutable == true) 2 else 1 } +
paramsBeforeImports + 1
func.addInsns(
VarInsnNode(Opcodes.ALOAD, 0),
VarInsnNode(Opcodes.ALOAD, importIndex),
@ -300,11 +327,14 @@ open class AstToAsm {
global.type.contentType.typeRef,
refGlobalKind.type.contentType.typeRef
)
val paramOffset = ctx.importFuncs.size + paramsBeforeImports + 1 +
ctx.importGlobals.take(it.index).sumBy {
// Immutable jumps 1, mutable jumps 2
if ((it.kind as? Node.Import.Kind.Global)?.type?.mutable == false) 1
else 2
}
listOf(
VarInsnNode(
Opcodes.ALOAD,
ctx.importFuncs.size + it.index + paramsBeforeImports + 1
),
VarInsnNode(Opcodes.ALOAD, paramOffset),
MethodInsnNode(
Opcodes.INVOKEVIRTUAL,
MethodHandle::class.ref.asmName,
@ -357,7 +387,10 @@ open class AstToAsm {
// Otherwise, it was imported and we can set the elems on the imported one
// from the parameter
// TODO: I think this is a security concern and bad practice, may revisit
val importIndex = ctx.importFuncs.size + ctx.importGlobals.size + paramsBeforeImports + 1
val importIndex = ctx.importFuncs.size + ctx.importGlobals.sumBy {
// Immutable is 1, mutable is 2
if ((it.kind as? Node.Import.Kind.Global)?.type?.mutable == false) 1 else 2
} + paramsBeforeImports + 1
return func.addInsns(VarInsnNode(Opcodes.ALOAD, importIndex)).
let { func -> addElemsToTable(ctx, func, paramsBeforeImports) }.
// Remove the array that's still there
@ -533,28 +566,58 @@ open class AstToAsm {
is Either.Left -> (global.v.kind as Node.Import.Kind.Global).type
is Either.Right -> global.v.type
}
if (type.mutable) throw CompileErr.MutableGlobalExport(export.index)
// Create a simple getter
val method = MethodNode(Opcodes.ACC_PUBLIC, "get" + export.field.javaIdent.capitalize(),
val getter = MethodNode(Opcodes.ACC_PUBLIC, "get" + export.field.javaIdent.capitalize(),
"()" + type.contentType.typeRef.asmDesc, null, null)
method.addInsns(VarInsnNode(Opcodes.ALOAD, 0))
if (global is Either.Left) method.addInsns(
getter.addInsns(VarInsnNode(Opcodes.ALOAD, 0))
if (global is Either.Left) getter.addInsns(
FieldInsnNode(Opcodes.GETFIELD, ctx.thisRef.asmName,
ctx.importGlobalGetterFieldName(export.index), MethodHandle::class.ref.asmDesc),
MethodInsnNode(Opcodes.INVOKEVIRTUAL, MethodHandle::class.ref.asmName, "invokeExact",
"()" + type.contentType.typeRef.asmDesc, false)
) else method.addInsns(
) else getter.addInsns(
FieldInsnNode(Opcodes.GETFIELD, ctx.thisRef.asmName, ctx.globalName(export.index),
type.contentType.typeRef.asmDesc)
)
method.addInsns(InsnNode(when (type.contentType) {
getter.addInsns(InsnNode(when (type.contentType) {
Node.Type.Value.I32 -> Opcodes.IRETURN
Node.Type.Value.I64 -> Opcodes.LRETURN
Node.Type.Value.F32 -> Opcodes.FRETURN
Node.Type.Value.F64 -> Opcodes.DRETURN
}))
method.visibleAnnotations = listOf(exportAnnotation(export))
ctx.cls.methods.plusAssign(method)
getter.visibleAnnotations = listOf(exportAnnotation(export))
ctx.cls.methods.plusAssign(getter)
// If mutable, create simple setter
if (type.mutable) {
val setter = MethodNode(Opcodes.ACC_PUBLIC, "set" + export.field.javaIdent.capitalize(),
"(${type.contentType.typeRef.asmDesc})V", null, null)
setter.addInsns(VarInsnNode(Opcodes.ALOAD, 0))
if (global is Either.Left) setter.addInsns(
FieldInsnNode(Opcodes.GETFIELD, ctx.thisRef.asmName,
ctx.importGlobalSetterFieldName(export.index), MethodHandle::class.ref.asmDesc),
VarInsnNode(when (type.contentType) {
Node.Type.Value.I32 -> Opcodes.ILOAD
Node.Type.Value.I64 -> Opcodes.LLOAD
Node.Type.Value.F32 -> Opcodes.FLOAD
Node.Type.Value.F64 -> Opcodes.DLOAD
}, 1),
MethodInsnNode(Opcodes.INVOKEVIRTUAL, MethodHandle::class.ref.asmName, "invokeExact",
"(${type.contentType.typeRef.asmDesc})V", false),
InsnNode(Opcodes.RETURN)
) else setter.addInsns(
VarInsnNode(when (type.contentType) {
Node.Type.Value.I32 -> Opcodes.ILOAD
Node.Type.Value.I64 -> Opcodes.LLOAD
Node.Type.Value.F32 -> Opcodes.FLOAD
Node.Type.Value.F64 -> Opcodes.DLOAD
}, 1),
FieldInsnNode(Opcodes.PUTFIELD, ctx.thisRef.asmName, ctx.globalName(export.index),
type.contentType.typeRef.asmDesc),
InsnNode(Opcodes.RETURN)
)
setter.visibleAnnotations = listOf(exportAnnotation(export))
ctx.cls.methods.plusAssign(setter)
}
}
fun addExportMemory(ctx: ClsContext, export: Node.Export) {

29
compiler/src/main/kotlin/asmble/compile/jvm/ByteBufferMem.kt
View File

@ -46,16 +46,25 @@ open class ByteBufferMem(val direct: Boolean = true) : Mem {
let(buildOffset).popExpecting(Int::class.ref).
addInsns(
forceFnType<ByteBuffer.(Int) -> Buffer>(ByteBuffer::position).invokeVirtual(),
TypeInsnNode(Opcodes.CHECKCAST, memType.asmName),
// We're going to do this as an LDC string in ISO-8859 and read it back at runtime
LdcInsnNode(bytes.toString(Charsets.ISO_8859_1)),
LdcInsnNode("ISO-8859-1"),
// Ug, can't do func refs on native types here...
MethodInsnNode(Opcodes.INVOKEVIRTUAL, String::class.ref.asmName,
"getBytes", "(Ljava/lang/String;)[B", false),
0.const,
bytes.size.const,
forceFnType<ByteBuffer.(ByteArray, Int, Int) -> ByteBuffer>(ByteBuffer::put).invokeVirtual(),
TypeInsnNode(Opcodes.CHECKCAST, memType.asmName)
).addInsns(
// We're going to do this as an LDC string in ISO-8859 and read it back at runtime. However,
// due to JVM limits, we can't have a string > 65536 chars. We chunk into 16300 because when
// converting to UTF8 const it can be up to 4 bytes per char, so this makes sure it doesn't
// overflow.
bytes.chunked(16300).flatMap { bytes ->
sequenceOf(
LdcInsnNode(bytes.toString(Charsets.ISO_8859_1)),
LdcInsnNode("ISO-8859-1"),
// Ug, can't do func refs on native types here...
MethodInsnNode(Opcodes.INVOKEVIRTUAL, String::class.ref.asmName,
"getBytes", "(Ljava/lang/String;)[B", false),
0.const,
bytes.size.const,
forceFnType<ByteBuffer.(ByteArray, Int, Int) -> ByteBuffer>(ByteBuffer::put).invokeVirtual()
)
}.toList()
).addInsns(
InsnNode(Opcodes.POP)
)

20
compiler/src/main/kotlin/asmble/compile/jvm/ClsContext.kt
View File

@ -39,6 +39,24 @@ data class ClsContext(
val hasTable: Boolean by lazy {
mod.tables.isNotEmpty() || mod.imports.any { it.kind is Node.Import.Kind.Table }
}
val dedupedFuncNames: Map<Int, String>? by lazy {
// Consider all exports as seen
val seen = mod.exports.flatMap { export ->
when {
export.kind == Node.ExternalKind.FUNCTION -> listOf(export.field.javaIdent)
// Just to make it easy, consider all globals as having setters
export.kind == Node.ExternalKind.GLOBAL ->
export.field.javaIdent.capitalize().let { listOf("get$it", "set$it") }
else -> listOf("get" + export.field.javaIdent.capitalize())
}
}.toMutableSet()
mod.names?.funcNames?.toList()?.sortedBy { it.first }?.map { (index, origName) ->
var name = origName.javaIdent
var nameIndex = 0
while (!seen.add(name)) name = origName.javaIdent + (nameIndex++)
index to name
}?.toMap()
}
fun assertHasMemory() { if (!hasMemory) throw CompileErr.UnknownMemory(0) }
@ -71,7 +89,7 @@ data class ClsContext(
fun importGlobalGetterFieldName(index: Int) = "import\$get" + globalName(index)
fun importGlobalSetterFieldName(index: Int) = "import\$set" + globalName(index)
fun globalName(index: Int) = "\$global$index"
fun funcName(index: Int) = "\$func$index"
fun funcName(index: Int) = dedupedFuncNames?.get(index) ?: "\$func$index"
private fun syntheticFunc(
nameSuffix: String,

12
compiler/src/main/kotlin/asmble/compile/jvm/CompileErr.kt
View File

@ -102,18 +102,6 @@ sealed class CompileErr(message: String, cause: Throwable? = null) : RuntimeExce
override val asmErrString get() = "global is immutable"
}
class MutableGlobalImport(
val index: Int
) : CompileErr("Attempted to import mutable global at index $index") {
override val asmErrString get() = "mutable globals cannot be imported"
}
class MutableGlobalExport(
val index: Int
) : CompileErr("Attempted to export global $index which is mutable") {
override val asmErrString get() = "mutable globals cannot be exported"
}
class GlobalInitNotConstant(
val index: Int
) : CompileErr("Expected init for global $index to be single constant value") {

12
compiler/src/main/kotlin/asmble/compile/jvm/InsnReworker.kt
View File

@ -183,6 +183,7 @@ open class InsnReworker {
if (!foundUnconditionalJump) throw CompileErr.StackInjectionMismatch(count, insn)
}
var traceStackSize = 0 // Used only for trace
// Go over each insn, determining where to inject
insns.forEachIndexed { index, insn ->
// Handle special injection cases
@ -221,8 +222,15 @@ open class InsnReworker {
else -> { }
}
// Log some trace output
ctx.trace {
insnStackDiff(ctx, insn).let {
traceStackSize += it
"Stack diff is $it for insn #$index $insn, stack size now: $traceStackSize"
}
}
// Add the current diff
ctx.trace { "Stack diff is ${insnStackDiff(ctx, insn)} for insn #$index $insn" }
stackManips += insnStackDiff(ctx, insn) to index
}
@ -266,7 +274,7 @@ open class InsnReworker {
is Node.Instr.I64Load32S, is Node.Instr.I64Load32U -> POP_PARAM + PUSH_RESULT
is Node.Instr.I32Store, is Node.Instr.I64Store, is Node.Instr.F32Store, is Node.Instr.F64Store,
is Node.Instr.I32Store8, is Node.Instr.I32Store16, is Node.Instr.I64Store8, is Node.Instr.I64Store16,
is Node.Instr.I64Store32 -> POP_PARAM
is Node.Instr.I64Store32 -> POP_PARAM + POP_PARAM
is Node.Instr.MemorySize -> PUSH_RESULT
is Node.Instr.MemoryGrow -> POP_PARAM + PUSH_RESULT
is Node.Instr.I32Const, is Node.Instr.I64Const,

27
compiler/src/main/kotlin/asmble/io/AstToBinary.kt
View File

@ -5,6 +5,7 @@ import asmble.util.toRawIntBits
import asmble.util.toRawLongBits
import asmble.util.toUnsignedBigInt
import asmble.util.toUnsignedLong
import java.io.ByteArrayOutputStream
open class AstToBinary(val version: Long = 1L) {
@ -140,6 +141,9 @@ open class AstToBinary(val version: Long = 1L) {
fromResizableLimits(b, n.limits)
}
fun fromModule(n: Node.Module) =
ByteArrayOutputStream().also { fromModule(ByteWriter.OutputStream(it), n) }.toByteArray()
fun fromModule(b: ByteWriter, n: Node.Module) {
b.writeUInt32(0x6d736100)
b.writeUInt32(version)
@ -160,10 +164,33 @@ open class AstToBinary(val version: Long = 1L) {
wrapListSection(b, n, 9, n.elems, this::fromElem)
wrapListSection(b, n, 10, n.funcs, this::fromFuncBody)
wrapListSection(b, n, 11, n.data, this::fromData)
n.names?.also { fromNames(b, it) }
// All other custom sections after the previous
n.customSections.filter { it.afterSectionId > 11 }.forEach { fromCustomSection(b, it) }
}
fun fromNames(b: ByteWriter, n: Node.NameSection) {
fun <T> indexMap(b: ByteWriter, map: Map<Int, T>, fn: (T) -> Unit) {
b.writeVarUInt32(map.size)
map.forEach { index, v -> b.writeVarUInt32(index).also { fn(v) } }
}
fun nameMap(b: ByteWriter, map: Map<Int, String>) = indexMap(b, map) { b.writeString(it) }
b.writeVarUInt7(0)
b.withVarUInt32PayloadSizePrepended { b ->
b.writeString("name")
n.moduleName?.also { moduleName ->
b.writeVarUInt7(0)
b.withVarUInt32PayloadSizePrepended { b -> b.writeString(moduleName) }
}
if (n.funcNames.isNotEmpty()) b.writeVarUInt7(1).also {
b.withVarUInt32PayloadSizePrepended { b -> nameMap(b, n.funcNames) }
}
if (n.localNames.isNotEmpty()) b.writeVarUInt7(2).also {
b.withVarUInt32PayloadSizePrepended { b -> indexMap(b, n.localNames) { nameMap(b, it) } }
}
}
}
fun fromResizableLimits(b: ByteWriter, n: Node.ResizableLimits) {
b.writeVarUInt1(n.maximum != null)
b.writeVarUInt32(n.initial)

63
compiler/src/main/kotlin/asmble/io/AstToSExpr.kt
View File

@ -62,13 +62,21 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
Node.ExternalKind.GLOBAL -> newMulti("global") + v.index
}
fun fromFunc(v: Node.Func, name: String? = null, impExp: ImportOrExport? = null) =
newMulti("func", name) + impExp?.let(this::fromImportOrExport) + fromFuncSig(v.type) +
fromLocals(v.locals) + fromInstrs(v.instructions).unwrapInstrs()
fun fromFunc(
v: Node.Func,
name: String? = null,
impExp: ImportOrExport? = null,
localNames: Map<Int, String> = emptyMap()
) =
newMulti("func", name) + impExp?.let(this::fromImportOrExport) + fromFuncSig(v.type, localNames) +
fromLocals(v.locals, v.type.params.size, localNames) + fromInstrs(v.instructions).unwrapInstrs()
fun fromFuncSig(v: Node.Type.Func): List<SExpr> {
fun fromFuncSig(v: Node.Type.Func, localNames: Map<Int, String> = emptyMap()): List<SExpr> {
var ret = emptyList<SExpr>()
if (v.params.isNotEmpty()) ret += newMulti("param") + v.params.map(this::fromType)
if (v.params.isNotEmpty()) {
if (localNames.isEmpty()) ret += newMulti("param") + v.params.map(this::fromType)
else ret += v.params.mapIndexed { index, param -> newMulti("param", localNames[index]) + fromType(param) }
}
v.ret?.also { ret += newMulti("result") + fromType(it) }
return ret
}
@ -80,8 +88,8 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
fun fromGlobalSig(v: Node.Type.Global) =
if (v.mutable) newMulti("mut") + fromType(v.contentType) else fromType(v.contentType)
fun fromImport(v: Node.Import, types: List<Node.Type.Func>) =
(newMulti("import") + v.module.quoted) + v.field.quoted + fromImportKind(v.kind, types)
fun fromImport(v: Node.Import, types: List<Node.Type.Func>, name: String? = null) =
(newMulti("import") + v.module.quoted) + v.field.quoted + fromImportKind(v.kind, types, name)
fun fromImportFunc(v: Node.Import.Kind.Func, types: List<Node.Type.Func>, name: String? = null) =
fromImportFunc(types.getOrElse(v.typeIndex) { throw Exception("No type at ${v.typeIndex}") }, name)
@ -91,11 +99,11 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
fun fromImportGlobal(v: Node.Import.Kind.Global, name: String? = null) =
newMulti("global", name) + fromGlobalSig(v.type)
fun fromImportKind(v: Node.Import.Kind, types: List<Node.Type.Func>) = when(v) {
is Node.Import.Kind.Func -> fromImportFunc(v, types)
is Node.Import.Kind.Table -> fromImportTable(v)
is Node.Import.Kind.Memory -> fromImportMemory(v)
is Node.Import.Kind.Global -> fromImportGlobal(v)
fun fromImportKind(v: Node.Import.Kind, types: List<Node.Type.Func>, name: String? = null) = when(v) {
is Node.Import.Kind.Func -> fromImportFunc(v, types, name)
is Node.Import.Kind.Table -> fromImportTable(v, name)
is Node.Import.Kind.Memory -> fromImportMemory(v, name)
is Node.Import.Kind.Global -> fromImportGlobal(v, name)
}
fun fromImportMemory(v: Node.Import.Kind.Memory, name: String? = null) =
@ -161,8 +169,10 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
return listOf(SExpr.Multi(untilNext().first))
}
fun fromLocals(v: List<Node.Type.Value>) =
if (v.isEmpty()) null else newMulti("local") + v.map(this::fromType)
fun fromLocals(v: List<Node.Type.Value>, paramOffset: Int, localNames: Map<Int, String> = emptyMap()) =
if (v.isEmpty()) emptyList()
else if (localNames.isEmpty()) listOf(newMulti("local") + v.map(this::fromType))
else v.mapIndexed { index, v -> newMulti("local", localNames[paramOffset + index]) + fromType(v) }
fun fromMemory(v: Node.Type.Memory, name: String? = null, impExp: ImportOrExport? = null) =
newMulti("memory", name) + impExp?.let(this::fromImportOrExport) + fromMemorySig(v)
@ -175,7 +185,7 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
is Script.Cmd.Meta.Output -> newMulti("output", v.name) + v.str
}
fun fromModule(v: Node.Module, name: String? = null): SExpr.Multi {
fun fromModule(v: Node.Module, name: String? = v.names?.moduleName): SExpr.Multi {
var ret = newMulti("module", name)
// If there is a call_indirect, then we need to output all types in exact order.
@ -187,8 +197,14 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
v.types.filterIndexed { i, _ -> importIndices.contains(i) } - v.funcs.map { it.type }
}
// Keep track of the current function index for names
var funcIndex = -1
ret += types.map { fromTypeDef(it) }
ret += v.imports.map { fromImport(it, v.types) }
ret += v.imports.map {
if (it.kind is Node.Import.Kind.Func) funcIndex++
fromImport(it, v.types, v.names?.funcNames?.get(funcIndex))
}
ret += v.exports.map(this::fromExport)
ret += v.tables.map { fromTable(it) }
ret += v.memories.map { fromMemory(it) }
@ -196,7 +212,14 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
ret += v.elems.map(this::fromElem)
ret += v.data.map(this::fromData)
ret += v.startFuncIndex?.let(this::fromStart)
ret += v.funcs.map { fromFunc(it) }
ret += v.funcs.map {
funcIndex++
fromFunc(
v = it,
name = v.names?.funcNames?.get(funcIndex),
localNames = v.names?.localNames?.get(funcIndex) ?: emptyMap()
)
}
return ret
}
@ -235,10 +258,8 @@ open class AstToSExpr(val parensInstrs: Boolean = true) {
if (exp == null) this else this.copy(vals = this.vals + exp)
private operator fun SExpr.Multi.plus(exps: List<SExpr>?) =
if (exps == null || exps.isEmpty()) this else this.copy(vals = this.vals + exps)
private fun newMulti(initSymb: String? = null, initName: String? = null): SExpr.Multi {
initName?.also { require(it.startsWith("$")) }
return SExpr.Multi() + initSymb + initName
}
private fun newMulti(initSymb: String? = null, initName: String? = null) =
SExpr.Multi() + initSymb + initName?.let { "$$it" }
private fun List<SExpr.Multi>.unwrapInstrs() =
if (parensInstrs) this else this.single().vals
private val String.quoted get() = fromString(this, true)

55
compiler/src/main/kotlin/asmble/io/BinaryToAst.kt
View File

@ -2,11 +2,13 @@ package asmble.io
import asmble.ast.Node
import asmble.util.*
import java.io.ByteArrayInputStream
import java.nio.ByteBuffer
open class BinaryToAst(
val version: Long = 1L,
val logger: Logger = Logger.Print(Logger.Level.OFF)
val logger: Logger = Logger.Print(Logger.Level.OFF),
val includeNameSection: Boolean = true
) : Logger by logger {
fun toBlockType(b: ByteReader) = b.readVarInt7().toInt().let {
@ -19,6 +21,23 @@ open class BinaryToAst(
payload = b.readBytes()
)
fun toNameSection(b: ByteReader) = generateSequence {
if (b.isEof) null
else b.readVarUInt7().toInt() to b.read(b.readVarUInt32AsInt())
}.fold(Node.NameSection(null, emptyMap(), emptyMap())) { sect, (type, b) ->
fun <T> indexMap(b: ByteReader, fn: (ByteReader) -> T) =
b.readList { it.readVarUInt32AsInt() to fn(it) }.let { pairs ->
pairs.toMap().also { require(it.size == pairs.size) { "Malformed names: duplicate indices" } }
}
fun nameMap(b: ByteReader) = indexMap(b) { it.readString() }
when (type) {
0 -> sect.copy(moduleName = b.readString())
1 -> sect.copy(funcNames = nameMap(b))
2 -> sect.copy(localNames = indexMap(b, ::nameMap))
else -> error("Malformed names: unrecognized type: $type")
}.also { require(b.isEof) }
}
fun toData(b: ByteReader) = Node.Data(
index = b.readVarUInt32AsInt(),
offset = toInitExpr(b),
@ -144,28 +163,31 @@ open class BinaryToAst(
}
}
fun toLocals(b: ByteReader) = b.readVarUInt32AsInt().let { size ->
toValueType(b).let { type -> List(size) { type } }
fun toLocals(b: ByteReader): List<Node.Type.Value> {
val size = try { b.readVarUInt32AsInt() } catch (e: NumberFormatException) { throw IoErr.InvalidLocalSize(e) }
return toValueType(b).let { type -> List(size) { type } }
}
fun toMemoryType(b: ByteReader) = Node.Type.Memory(toResizableLimits(b))
fun toModule(bytes: ByteReader): Node.Module {
if (bytes.readUInt32() != 0x6d736100L) throw IoErr.InvalidMagicNumber()
bytes.readUInt32().let { if (it != version) throw IoErr.InvalidVersion(it, listOf(version)) }
fun toModule(b: ByteArray) = toModule(ByteReader.InputStream(b.inputStream()))
fun toModule(b: ByteReader): Node.Module {
if (b.readUInt32() != 0x6d736100L) throw IoErr.InvalidMagicNumber()
b.readUInt32().let { if (it != version) throw IoErr.InvalidVersion(it, listOf(version)) }
// Slice up all the sections
var maxSectionId = 0
var sections = emptyList<Pair<Int, ByteReader>>()
while (!bytes.isEof) {
val sectionId = bytes.readVarUInt7().toInt()
while (!b.isEof) {
val sectionId = b.readVarUInt7().toInt()
if (sectionId > 11) throw IoErr.InvalidSectionId(sectionId)
if (sectionId != 0)
require(sectionId > maxSectionId) { "Section ID $sectionId came after $maxSectionId" }.
also { maxSectionId = sectionId }
val sectionLen = bytes.readVarUInt32AsInt()
val sectionLen = b.readVarUInt32AsInt()
// each 'read' invocation creates new InputStream and don't closes it
sections += sectionId to bytes.read(sectionLen)
sections += sectionId to b.read(sectionLen)
}
// Now build the module
@ -174,6 +196,7 @@ open class BinaryToAst(
val types = readSectionList(1, this::toFuncType)
val funcIndices = readSectionList(3) { it.readVarUInt32AsInt() }
var nameSection: Node.NameSection? = null
return Node.Module(
types = types,
imports = readSectionList(2, this::toImport),
@ -194,10 +217,18 @@ open class BinaryToAst(
val afterSectionId = if (index == 0) 0 else sections[index - 1].let { (prevSectionId, _) ->
if (prevSectionId == 0) customSections.last().afterSectionId else prevSectionId
}
customSections + toCustomSection(b, afterSectionId)
// Try to parse the name section
val section = toCustomSection(b, afterSectionId).takeIf { section ->
val shouldParseNames = includeNameSection && nameSection == null && section.name == "name"
!shouldParseNames || try {
nameSection = toNameSection(ByteReader.InputStream(section.payload.inputStream()))
false
} catch (e: Exception) { warn { "Failed parsing name section: $e" }; true }
}
if (section == null) customSections else customSections + section
}
}
)
).copy(names = nameSection)
}
fun toResizableLimits(b: ByteReader) = b.readVarUInt1().let {

4
compiler/src/main/kotlin/asmble/io/IoErr.kt
View File

@ -123,4 +123,8 @@ sealed class IoErr(message: String, cause: Throwable? = null) : RuntimeException
override val asmErrString get() = "integer representation too long"
override val asmErrStrings get() = listOf(asmErrString, "integer too large")
}
class InvalidLocalSize(cause: NumberFormatException) : IoErr("Invalid local size", cause) {
override val asmErrString get() = "too many locals"
}
}

89
compiler/src/main/kotlin/asmble/io/SExprToAst.kt
View File

@ -9,22 +9,27 @@ import asmble.util.*
import java.io.ByteArrayInputStream
import java.math.BigInteger
typealias NameMap = Map<String, Int>
open class SExprToAst {
open class SExprToAst(
val includeNames: Boolean = true
) {
data class ExprContext(
val nameMap: NameMap,
val blockDepth: Int = 0,
val types: List<Node.Type.Func> = emptyList(),
val callIndirectNeverBeforeSeenFuncTypes: MutableList<Node.Type.Func> = mutableListOf()
)
) {
companion object {
val empty = ExprContext(NameMap(emptyMap(), null, null))
}
}
data class FuncResult(
val name: String?,
val func: Node.Func,
val importOrExport: ImportOrExport?,
// These come from call_indirect insns
val additionalFuncTypesToAdd: List<Node.Type.Func>
val additionalFuncTypesToAdd: List<Node.Type.Func>,
val nameMap: NameMap
)
fun toAction(exp: SExpr.Multi): Script.Cmd.Action {
@ -36,7 +41,7 @@ open class SExprToAst {
return when(exp.vals.first().symbolStr()) {
"invoke" ->
Script.Cmd.Action.Invoke(name, str, exp.vals.drop(index).map {
toExprMaybe(it as SExpr.Multi, ExprContext(emptyMap()))
toExprMaybe(it as SExpr.Multi, ExprContext.empty)
})
"get" ->
Script.Cmd.Action.Get(name, str)
@ -49,7 +54,7 @@ open class SExprToAst {
return when(exp.vals.first().symbolStr()) {
"assert_return" ->
Script.Cmd.Assertion.Return(toAction(mult),
exp.vals.drop(2).map { toExprMaybe(it as SExpr.Multi, ExprContext(emptyMap())) })
exp.vals.drop(2).map { toExprMaybe(it as SExpr.Multi, ExprContext.empty) })
"assert_return_canonical_nan" ->
Script.Cmd.Assertion.ReturnNan(toAction(mult), canonical = true)
"assert_return_arithmetic_nan" ->
@ -176,7 +181,7 @@ open class SExprToAst {
var innerCtx = ctx.copy(blockDepth = ctx.blockDepth + 1)
exp.maybeName(opOffset)?.also {
opOffset++
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap + ("block:$it" to innerCtx.blockDepth))
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap.add("block", it, innerCtx.blockDepth))
}
val sigs = toBlockSigMaybe(exp, opOffset)
@ -233,7 +238,7 @@ open class SExprToAst {
var (nameMap, exprsUsed, sig) = toFuncSig(exp, currentIndex, origNameMap, types)
currentIndex += exprsUsed
val locals = exp.repeated("local", currentIndex, { toLocals(it) }).mapIndexed { index, (nameMaybe, vals) ->
nameMaybe?.also { require(vals.size == 1); nameMap += "local:$it" to (index + sig.params.size) }
nameMaybe?.also { require(vals.size == 1); nameMap = nameMap.add("local", it, index + sig.params.size) }
vals
}
currentIndex += locals.size
@ -250,7 +255,8 @@ open class SExprToAst {
name = name,
func = Node.Func(sig, locals.flatten(), instrs),
importOrExport = maybeImpExp,
additionalFuncTypesToAdd = ctx.callIndirectNeverBeforeSeenFuncTypes
additionalFuncTypesToAdd = ctx.callIndirectNeverBeforeSeenFuncTypes,
nameMap = nameMap
)
}
@ -268,7 +274,7 @@ open class SExprToAst {
} else null to offset
var nameMap = origNameMap
val params = exp.repeated("param", offset, { toParams(it) }).mapIndexed { index, (nameMaybe, vals) ->
nameMaybe?.also { require(vals.size == 1); nameMap += "local:$it" to index }
nameMaybe?.also { require(vals.size == 1); nameMap = nameMap.add("local", it, index) }
vals
}
val resultExps = exp.repeated("result", offset + params.size, this::toResult)
@ -395,7 +401,7 @@ open class SExprToAst {
val maybeName = exp.maybeName(offset + opOffset)
if (maybeName != null) {
opOffset++
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap + ("block:$maybeName" to innerCtx.blockDepth))
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap.add("block", maybeName, innerCtx.blockDepth))
}
val sigs = toBlockSigMaybe(exp, offset + opOffset)
opOffset += sigs.size
@ -428,7 +434,7 @@ open class SExprToAst {
opOffset++
exp.maybeName(offset + opOffset)?.also {
opOffset++
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap + ("block:$it" to ctx.blockDepth))
innerCtx = innerCtx.copy(nameMap = innerCtx.nameMap.add("block", it, ctx.blockDepth))
}
toInstrs(exp, offset + opOffset, innerCtx, false).also {
ret += it.first
@ -522,7 +528,7 @@ open class SExprToAst {
val exps = exp.vals.mapNotNull { it as? SExpr.Multi }
// Eagerly build the names (for forward decls)
val (nameMap, eagerTypes) = toModuleForwardNameMapAndTypes(exps)
var (nameMap, eagerTypes) = toModuleForwardNameMapAndTypes(exps)
mod = mod.copy(types = eagerTypes)
fun Node.Module.addTypeIfNotPresent(type: Node.Type.Func): Pair<Node.Module, Int> {
@ -555,6 +561,7 @@ open class SExprToAst {
Node.Import.Kind.Memory(kind) to (memoryCount++ to Node.ExternalKind.MEMORY)
else -> throw Exception("Unrecognized import kind: $kind")
}
mod = mod.copy(
imports = mod.imports + Node.Import(module, field, importKind),
exports = mod.exports + exportFields.map {
@ -579,11 +586,14 @@ open class SExprToAst {
"elem" -> mod = mod.copy(elems = mod.elems + toElem(exp, nameMap))
"data" -> mod = mod.copy(data = mod.data + toData(exp, nameMap))
"start" -> mod = mod.copy(startFuncIndex = toStart(exp, nameMap))
"func" -> toFunc(exp, nameMap, mod.types).also { (_, fn, impExp, additionalFuncTypes) ->
"func" -> toFunc(exp, nameMap, mod.types).also { (_, fn, impExp, additionalFuncTypes, localNameMap) ->
if (impExp is ImportOrExport.Import) {
handleImport(impExp.module, impExp.name, fn.type, impExp.exportFields)
} else {
if (impExp is ImportOrExport.Export) addExport(impExp, Node.ExternalKind.FUNCTION, funcCount)
if (includeNames) nameMap = nameMap.copy(
localNames = nameMap.localNames!! + (funcCount to localNameMap.getAllNamesByIndex("local"))
)
funcCount++
mod = mod.copy(funcs = mod.funcs + fn).addTypeIfNotPresent(fn.type).first
mod = additionalFuncTypes.fold(mod) { mod, typ -> mod.addTypeIfNotPresent(typ).first }
@ -644,6 +654,15 @@ open class SExprToAst {
if (mod.tables.size + mod.imports.count { it.kind is Node.Import.Kind.Table } > 1)
throw IoErr.MultipleTables()
// Set the name map pieces if we're including them
if (includeNames) mod = mod.copy(
names = Node.NameSection(
moduleName = name,
funcNames = nameMap.funcNames!!,
localNames = nameMap.localNames!!
)
)
return name to mod
}
@ -680,10 +699,14 @@ open class SExprToAst {
var globalCount = 0
var tableCount = 0
var memoryCount = 0
var namesToIndices = emptyMap<String, Int>()
var nameMap = NameMap(
names = emptyMap(),
funcNames = if (includeNames) emptyMap() else null,
localNames = if (includeNames) emptyMap() else null
)
var types = emptyList<Node.Type.Func>()
fun maybeAddName(name: String?, index: Int, type: String) {
name?.let { namesToIndices += "$type:$it" to index }
name?.also { nameMap = nameMap.add(type, it, index) }
}
// All imports first
@ -711,12 +734,12 @@ open class SExprToAst {
"memory" -> maybeAddName(kindName, memoryCount++, "memory")
// We go ahead and do the full type def build here eagerly
"type" -> maybeAddName(kindName, types.size, "type").also { _ ->
toTypeDef(it, namesToIndices).also { (_, type) -> types += type }
toTypeDef(it, nameMap).also { (_, type) -> types += type }
}
else -> {}
}
}
return namesToIndices to types
return nameMap to types
}
fun toOpMaybe(exp: SExpr.Multi, offset: Int, ctx: ExprContext): Pair<Node.Instr, Int>? {
@ -753,7 +776,8 @@ open class SExprToAst {
// First lookup the func sig
val (updatedNameMap, expsUsed, funcType) = toFuncSig(exp, offset + 1, ctx.nameMap, ctx.types)
// Make sure there are no changes to the name map
if (ctx.nameMap.size != updatedNameMap.size) throw IoErr.IndirectCallSetParamNames()
if (ctx.nameMap.size != updatedNameMap.size)
throw IoErr.IndirectCallSetParamNames()
// Obtain the func index from the types table, the indirects table, or just add it
var funcTypeIndex = ctx.types.indexOf(funcType)
// If it's not in the type list, check the call indirect list
@ -910,7 +934,7 @@ open class SExprToAst {
fun toVarMaybe(exp: SExpr, nameMap: NameMap, nameType: String): Int? {
return exp.symbolStr()?.let { it ->
if (it.startsWith("$"))
nameMap["$nameType:$it"] ?:
nameMap.get(nameType, it.drop(1)) ?:
throw Exception("Unable to find index for name $it of type $nameType in $nameMap")
else if (it.startsWith("0x")) it.substring(2).toIntOrNull(16)
else it.toIntOrNull()
@ -1005,7 +1029,7 @@ open class SExprToAst {
private fun SExpr.Multi.maybeName(index: Int): String? {
if (this.vals.size > index && this.vals[index] is SExpr.Symbol) {
val sym = this.vals[index] as SExpr.Symbol
if (!sym.quoted && sym.contents[0] == '$') return sym.contents
if (!sym.quoted && sym.contents[0] == '$') return sym.contents.drop(1)
}
return null
}
@ -1028,5 +1052,26 @@ open class SExprToAst {
return this.vals.first().requireSymbol(contents, quotedCheck)
}
data class NameMap(
// Key prefixed with type then colon before actual name
val names: Map<String, Int>,
// Null if not including names
val funcNames: Map<Int, String>?,
val localNames: Map<Int, Map<Int, String>>?
) {
val size get() = names.size
fun add(type: String, name: String, index: Int) = copy(
names = names + ("$type:$name" to index),
funcNames = funcNames?.let { if (type == "func") it + (index to name) else it }
)
fun get(type: String, name: String) = names["$type:$name"]
fun getAllNamesByIndex(type: String) = names.mapNotNull { (k, v) ->
k.takeIf { k.startsWith("$type:") }?.let { v to k.substring(type.length + 1) }
}.toMap()
}
companion object : SExprToAst()
}

7
compiler/src/main/kotlin/asmble/io/StrToSExpr.kt
View File

@ -18,6 +18,13 @@ open class StrToSExpr {
data class Error(val pos: Pos, val msg: String) : ParseResult()
}
fun parseSingleMulti(str: CharSequence) = parse(str).let {
when (it) {
is ParseResult.Success -> (it.vals.singleOrNull() as? SExpr.Multi) ?: error("Not a single multi-expr")
is ParseResult.Error -> error("Failed parsing at ${it.pos.line}:${it.pos.char} - ${it.msg}")
}
}
fun parse(str: CharSequence): ParseResult {
val state = ParseState(str)
val ret = mutableListOf<SExpr>()

6
compiler/src/main/kotlin/asmble/run/jvm/Module.kt
View File

@ -116,9 +116,9 @@ interface Module {
}
// Global imports
val globalImports = mod.imports.mapNotNull {
if (it.kind is Node.Import.Kind.Global) ctx.resolveImportGlobal(it, it.kind.type)
else null
val globalImports = mod.imports.flatMap {
if (it.kind is Node.Import.Kind.Global) ctx.resolveImportGlobals(it, it.kind.type)
else emptyList()
}
constructorParams += globalImports

8
compiler/src/main/kotlin/asmble/run/jvm/RunErr.kt
View File

@ -55,4 +55,12 @@ sealed class RunErr(message: String, cause: Throwable? = null) : RuntimeExceptio
override val asmErrString get() = "unknown import"
override val asmErrStrings get() = listOf(asmErrString, "incompatible import type")
}
class ImportGlobalInvalidMutability(
val module: String,
val field: String,
val expected: Boolean
) : RunErr("Expected imported global $module::$field to have mutability as ${!expected}") {
override val asmErrString get() = "incompatible import type"
}
}

20
compiler/src/main/kotlin/asmble/run/jvm/ScriptContext.kt
View File

@ -277,10 +277,12 @@ data class ScriptContext(
return Module.Compiled(mod, classLoader.fromBuiltContext(ctx), name, ctx.mem)
}
fun bindImport(import: Node.Import, getter: Boolean, methodType: MethodType): MethodHandle {
fun bindImport(import: Node.Import, getter: Boolean, methodType: MethodType) = bindImport(
import, if (getter) "get" + import.field.javaIdent.capitalize() else import.field.javaIdent, methodType)
fun bindImport(import: Node.Import, javaName: String, methodType: MethodType): MethodHandle {
// Find a method that matches our expectations
val module = registrations[import.module] ?: throw RunErr.ImportNotFound(import.module, import.field)
val javaName = if (getter) "get" + import.field.javaIdent.capitalize() else import.field.javaIdent
val kind = when (import.kind) {
is Node.Import.Kind.Func -> WasmExternalKind.FUNCTION
is Node.Import.Kind.Table -> WasmExternalKind.TABLE
@ -295,8 +297,18 @@ data class ScriptContext(
bindImport(import, false,
MethodType.methodType(funcType.ret?.jclass ?: Void.TYPE, funcType.params.map { it.jclass }))
fun resolveImportGlobal(import: Node.Import, globalType: Node.Type.Global) =
bindImport(import, true, MethodType.methodType(globalType.contentType.jclass))
fun resolveImportGlobals(import: Node.Import, globalType: Node.Type.Global): List<MethodHandle> {
val getter = bindImport(import, true, MethodType.methodType(globalType.contentType.jclass))
// Whether the setter is present or not defines whether it is mutable
val setter = try {
bindImport(import, "set" + import.field.javaIdent.capitalize(),
MethodType.methodType(Void.TYPE, globalType.contentType.jclass))
} catch (e: RunErr.ImportNotFound) { null }
// Mutability must match
if (globalType.mutable == (setter == null))
throw RunErr.ImportGlobalInvalidMutability(import.module, import.field, globalType.mutable)
return if (setter == null) listOf(getter) else listOf(getter, setter)
}
fun resolveImportMemory(import: Node.Import, memoryType: Node.Type.Memory, mem: Mem) =
bindImport(import, true, MethodType.methodType(Class.forName(mem.memType.asm.className))).

4
compiler/src/test/kotlin/asmble/SpecTestUnit.kt
View File

@ -13,8 +13,8 @@ class SpecTestUnit(name: String, wast: String, expectedOutput: String?) : BaseTe
override val shouldFail get() = name.endsWith(".fail")
override val defaultMaxMemPages get() = when (name) {
"nop"-> 20
"resizing" -> 830
"nop" -> 20
"memory_grow" -> 830
"imports" -> 5
else -> 1
}

45
compiler/src/test/kotlin/asmble/compile/jvm/LargeDataTest.kt Normal file
View File

@ -0,0 +1,45 @@
package asmble.compile.jvm
import asmble.TestBase
import asmble.ast.Node
import asmble.run.jvm.ScriptContext
import asmble.util.get
import org.junit.Test
import java.nio.ByteBuffer
import java.util.*
import kotlin.test.assertEquals
class LargeDataTest : TestBase() {
@Test
fun testLargeData() {
// This previously failed because string constants can't be longer than 65536 chars.
// We create a byte array across the whole gambit of bytes to test UTF8 encoding.
val bytesExpected = ByteArray(70000) { ((it % 255) - Byte.MIN_VALUE).toByte() }
val mod = Node.Module(
memories = listOf(Node.Type.Memory(
limits = Node.ResizableLimits(initial = 2, maximum = 2)
)),
data = listOf(Node.Data(
index = 0,
offset = listOf(Node.Instr.I32Const(0)),
data = bytesExpected
))
)
val ctx = ClsContext(
packageName = "test",
className = "Temp" + UUID.randomUUID().toString().replace("-", ""),
mod = mod,
logger = logger
)
AstToAsm.fromModule(ctx)
val cls = ScriptContext.SimpleClassLoader(javaClass.classLoader, logger).fromBuiltContext(ctx)
// Instantiate it, get the memory out, and check it
val field = cls.getDeclaredField("memory").apply { isAccessible = true }
val buf = field[cls.newInstance()] as ByteBuffer
// Grab all + 1 and check values
val bytesActual = ByteArray(70001).also { buf.get(0, it) }
bytesActual.forEachIndexed { index, byte ->
assertEquals(if (index == 70000) 0.toByte() else bytesExpected[index], byte)
}
}
}

38
compiler/src/test/kotlin/asmble/compile/jvm/NamesTest.kt Normal file
View File

@ -0,0 +1,38 @@
package asmble.compile.jvm
import asmble.TestBase
import asmble.io.SExprToAst
import asmble.io.StrToSExpr
import asmble.run.jvm.ScriptContext
import org.junit.Test
import java.util.*
class NamesTest : TestBase() {
@Test
fun testNames() {
// Compile and make sure the names are set right
val (_, mod) = SExprToAst.toModule(StrToSExpr.parseSingleMulti("""
(module ${'$'}mod_name
(import "foo" "bar" (func ${'$'}import_func (param i32)))
(type ${'$'}some_sig (func (param ${'$'}type_param i32)))
(func ${'$'}some_func
(type ${'$'}some_sig)
(param ${'$'}func_param i32)
(local ${'$'}func_local0 i32)
(local ${'$'}func_local1 f64)
)
)
""".trimIndent()))
val ctx = ClsContext(
packageName = "test",
className = "Temp" + UUID.randomUUID().toString().replace("-", ""),
mod = mod,
logger = logger
)
AstToAsm.fromModule(ctx)
val cls = ScriptContext.SimpleClassLoader(javaClass.classLoader, logger).fromBuiltContext(ctx)
// Make sure the import field and the func are present named
cls.getDeclaredField("import_func")
cls.getDeclaredMethod("some_func", Integer.TYPE)
}
}

47
compiler/src/test/kotlin/asmble/io/NamesTest.kt Normal file
View File

@ -0,0 +1,47 @@
package asmble.io
import asmble.ast.Node
import org.junit.Test
import kotlin.test.assertEquals
class NamesTest {
@Test
fun testNames() {
// First, make sure it can parse from sexpr
val (_, mod1) = SExprToAst.toModule(StrToSExpr.parseSingleMulti("""
(module ${'$'}mod_name
(import "foo" "bar" (func ${'$'}import_func (param i32)))
(type ${'$'}some_sig (func (param ${'$'}type_param i32)))
(func ${'$'}some_func
(type ${'$'}some_sig)
(param ${'$'}func_param i32)
(local ${'$'}func_local0 i32)
(local ${'$'}func_local1 f64)
)
)
""".trimIndent()))
val expected = Node.NameSection(
moduleName = "mod_name",
funcNames = mapOf(
0 to "import_func",
1 to "some_func"
),
localNames = mapOf(
1 to mapOf(
0 to "func_param",
1 to "func_local0",
2 to "func_local1"
)
)
)
assertEquals(expected, mod1.names)
// Now back to binary and then back and make sure it's still there
val bytes = AstToBinary.fromModule(mod1)
val mod2 = BinaryToAst.toModule(bytes)
assertEquals(expected, mod2.names)
// Now back to sexpr and then back to make sure the sexpr writer works
val sexpr = AstToSExpr.fromModule(mod2)
val (_, mod3) = SExprToAst.toModule(sexpr)
assertEquals(expected, mod3.names)
}
}