You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Discussion: Strategy to remove shell-exec from compilation path #202
Should TinyGo / Go generate core wasm modules and leave metadata/component encoding to 3rd party tools; or should they be part of a wasi target ( or goos/goarch combination )?
Assuming we implement wit parsing / wasm patching in Go:
TinyGo, by design and convention, has depended on external tools to handle parts of compilation (LLVM, asyncify, wasm-ld, among others). Based on statements made by the Go team and by historical precedent, I (and others, I believe) assume that this strategy would not apply to mainline ("big") Go. The go command (and associated toolchain) contains the necessary capability to compile (and cross-compile) binaries for a variety of platforms, including wasm.
Is the issue that shell exec exists at all in the current Component Model tooling for Go?
Do we (the authors and maintainers of this tooling, TinyGo, and Go) agree that this is an issue worth addressing?
Wasm in Go has so far taken an approach of "prototype in TinyGo, learn and implement later in big Go."
With respect to tooling and compilation, what if TinyGo skipped the wasm-tools component steps, and generated a wasm32-wasip2-module form suitable for consumption by wasm-tools or conforming runtimes?
we chatted about it during BA's meeting and no decisive actions other than researching what wasm-tools is actually doing under the hood and follow up next meeting.
We identified a few options to pass down wit:
Continue with wit directory so wasm-tools (or something else) can perform embedding
Nuke the wit directory and rely on information from generated code in downstream tooling. Options:
go embed per package: Attaching each interface wit to its corresponding go package
go embed per project: Attach the whole wit to the entrypoint package under compilation
A combination of go:wasm{import,export} and markers to carry type information, so the wit can be discarded after codegen
With respect to tooling and compilation, what if TinyGo skipped the wasm-tools component steps, and generated a WebAssembly/component-model#378 form suitable for consumption by wasm-tools or conforming runtimes?
This is a 🔑 point. We touched briefly on the point wrt making TinyGo / BigGo emit wasm32 module and leave wasm-tools steps to other tools. It would allow us to defer implementations based on upstream stability ( and yes, I'm looking at you WIT spec! 😂 )
The interesting bit here is wasm-tools-go is already a requirement, so something like this is not far from reality:
# generate bindings
go run wasm-tools/cmd/wit-bindgen-go
# compile module ( core wasm )
tinygo build ... -o module.wasm
# componetize it
go run wasm-tools/cmd/assemble-component -in module.wasm -out component.wasm
wasm-tools-go as pre & post compilation steps, having wit-bindgen-go and assemble-component defining their own api / data passing mechanism.
The first incarnation of assemble-component could be a full shell exec into wasm-tools, and feels this is a pattern acceptable in Big Go.
again, no decisions made but wanted to relay the discussion.
I have WIT generation working with tree shaking in a branch, so we can emit minimal WIT files containing a single world to a world's Go package directory.
wasm-tools-go is only a prerequisite for code generation, so it's not really required to compile and build a Go program. This is similar to other tools used by big Go projects like protoc or avo.
Activity
ydnar commentedon Oct 8, 2024
TinyGo, by design and convention, has depended on external tools to handle parts of compilation (LLVM, asyncify, wasm-ld, among others). Based on statements made by the Go team and by historical precedent, I (and others, I believe) assume that this strategy would not apply to mainline ("big") Go. The
gocommand (and associated toolchain) contains the necessary capability to compile (and cross-compile) binaries for a variety of platforms, including wasm.wasm-tools componentsteps, and generated awasm32-wasip2-moduleform suitable for consumption bywasm-toolsor conforming runtimes?lxfontes commentedon Oct 8, 2024
we chatted about it during BA's meeting and no decisive actions other than researching what wasm-tools is actually doing under the hood and follow up next meeting.
We identified a few options to pass down wit:
witdirectory so wasm-tools (or something else) can perform embeddingwitdirectory and rely on information from generated code in downstream tooling. Options:go embedper package: Attaching each interface wit to its corresponding go packagego embedper project: Attach the whole wit to the entrypoint package under compilationgo:wasm{import,export}and markers to carry type information, so the wit can be discarded after codegenThis is a 🔑 point. We touched briefly on the point wrt making TinyGo / BigGo emit wasm32 module and leave
wasm-toolssteps to other tools. It would allow us to defer implementations based on upstream stability ( and yes, I'm looking at you WIT spec! 😂 )The interesting bit here is
wasm-tools-gois already a requirement, so something like this is not far from reality:wasm-tools-goas pre & post compilation steps, havingwit-bindgen-goandassemble-componentdefining their own api / data passing mechanism.The first incarnation of
assemble-componentcould be a full shell exec intowasm-tools, and feels this is a pattern acceptable in Big Go.again, no decisions made but wanted to relay the discussion.
ydnar commentedon Oct 9, 2024
I have WIT generation working with tree shaking in a branch, so we can emit minimal WIT files containing a single world to a world's Go package directory.
wasm-tools-go is only a prerequisite for code generation, so it's not really required to compile and build a Go program. This is similar to other tools used by big Go projects like protoc or avo.
ydnar commentedon Dec 8, 2024
Work is progressing here:
wasm-toolsas a WASI 0.1 module inside ofwit-bindgen-go: internal/wasmtools: run wasm-tools.wasm in embedded Wazero runtime #252ydnar commentedon Dec 24, 2024
Merged #272.
wasm-toolsis now vendored as a WebAssembly module into packageinternal/wasmtools. Thanks @Mossaka!