Skip to content

Add external-id attribute - #672

Merged
lukewagner merged 1 commit into
mainfrom
external-id
Jul 8, 2026
Merged

lukewagner merged 1 commit into
mainfrom
external-id

Conversation

@lukewagner

@lukewagner lukewagner commented Jun 29, 2026 •

Copy link
Copy Markdown
Member

This PR extends #613, which added an (implements <interfacename>) attribute to imports and exports, with an (external-id <string>) attribute. The external-id attribute is useful when a component import or export name wants to match some external identifier, but the kebab-case syntax is too restrictive. One example is ESM-integration, where Module Specifiers can (now) be arbitrary Unicode strings (including URLs). Another is when importing stateful stores, e.g.:

world w {
  import redis: wasi:keyvalue/store;
  import memcached: wasi:keyvalue/store;
}

(as enabled by #613) and the platform-defined identifier of these stores isn't a valid <plainname>, so you'd either have to add a whole separate name-mapping scheme (e.g. mapping redis and memcached to their host IDs) or mangle the host IDs into the <plainname> (say, via URL-encoding; ew). Instead, with this PR, you can use the @external-id attribute in WIT:

world w {
  @external-id("$whatever:host_id")
  import redis: wasi:keyvalue/store;

  @external-id("$something:else")
  import memcached: wasi:keyvalue/store;
}

and the WAT (external-id <string>) is plumbed into the host, alongside (implements <interfacename>), so that the host can reflect on it and act accordingly.

As mentioned above, @external-id can subsume the use cases for url=<...> names for ESM-integration. Moreover, the reasoning in #613 for moving implements out of the name and into a separate attribute I think similarly suggests that the <urlname>, <depname> and <hashname> cases of <importname> aren't really the right place to solve their associated use cases. IIUC, these 3 name cases aren't currently being produced since there's no WIT syntax to emit them. Thus, this PR tentatively proposes removing these 3 name cases. The use cases are still important, but I think when we're ready to prioritize adding them to WIT/tooling, we should consider adding them as attributes instead. In the meantime, it lets us merge importname and exportname into externname, which simplifies nicely things.

The PR also tidies up some sloppiness in the text-format grammar around string literals (e.g., "foo-bar") vs. the contents inside the quotes (e.g. foo-bar), using <Xstr> for the former and <X> for the latter.

@lukewagner
lukewagner force-pushed the external-id branch 2 times, most recently from 9e2219f to eeb2ed0 Compare June 30, 2026 15:47

@ricochet ricochet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I'd use it right away.

You may want to update Linking.md as well:

mutually known to later stages in the deployment pipeline (specifically with
the [depname] case of importname in the text and binary format).

To something like:

mutually known to later stages in the deployment pipeline. (How an import
name is resolved to a particular registry artifact is left to the host or
build tooling; the optional [external-id] attribute can carry a
tooling-specific identifier such as a registry path or URL.)

@lukewagner

Copy link
Copy Markdown
Member Author

@ricochet Thanks, good catch! Fixed

Comment thread design/mvp/WIT.md Resolved
@alexcrichton

Copy link
Copy Markdown
Collaborator

I've got an initial implementation of this over at bytecodealliance/wasm-tools#2555 if you wouldn't mind skimming over the test I've added

@lukewagner

Copy link
Copy Markdown
Member Author

Awesome! Test looks good.

Comment thread design/mvp/Binary.md Resolved
Comment thread design/mvp/Explainer.md Resolved
@ricochet
ricochet requested review from alexcrichton and ricochet July 2, 2026 18:46
PhoebeSzmucer pushed a commit to PhoebeSzmucer/wasm-tools that referenced this pull request Jul 6, 2026
…2555)

* Initial work for `versionsuffix` and `external-id`

This is intended to at least start laying groundwork and implementation
for two features in the component model that this repository does not
yet implement:

* `(versionsuffix "...")` for components -- introduced in
  WebAssembly/component-model#536 this is intended to pave a path
  forward to canonicalizing versions in component imports/exports in the
  future. For example where today you'd import `a:b/c@1.0.0` tomorrow
  you'd import `a:b/c@1`. This is not yet fully integrated into
  `wit-component`, however, and further work will be necessary for
  bindings generators as well to take advantage of this. It's all
  encoded in the same place as `external-id`, though, so I figured I'd
  at least start to lay the groundwork.

* `(external-id "...")` for components -- being introduced in
  WebAssembly/component-model#672 this is intended to attach arbitrary
  metadata to imports/exports in a component. This is surfaced in WIT as
  an `@external-id("...")` attribute. This should be hooked up
  throughout this implementation to WIT and such such that the
  implementation is intended to be complete. There's probably something
  I forgot, but all the major pieces should be there.

The `versionsuffix` parsing/validation is gated by a new
`cm-canon-names` feature, and the `external-id` parsing/validation is
gated behind the preexisting `cm-implements` feature to match the
specification's classification.

* Fix msrv

* Update expected stdout

* Reduce debug mode stack size usage
@alexcrichton

Copy link
Copy Markdown
Collaborator

Ok with bytecodealliance/wasm-tools#2558 that shoudl, I believe, implement all of this and have everything round-tripped in all the places

@lukewagner

Copy link
Copy Markdown
Member Author

That's great! I guess then I'll wait a bit to see if anyone else has comments and then merge, noting that the feature isn't yet released, so we can continue to iterate in subsequent issues/PRs.

@lukewagner
lukewagner merged commit 609567d into main Jul 8, 2026
2 checks passed
@lukewagner
lukewagner deleted the external-id branch July 8, 2026 17:40
q82419 added a commit to WasmEdge/WasmEdge that referenced this pull request Sep 23, 2026
- task.return takes the functype resultlist, `0x00 t` or `0x01 0x00`; the
  AST keeps one optional result for functions and for task.return.
- Accept only plain names and interface names as extern names: the
  dependency, url and integrity forms left the specification with the
  `external-id` attribute (WebAssembly/component-model#672), and their
  diagnostics go with them.
- Name the canonical built-ins in the diagnostics as the specification
  spells them (thread.yield, stream.cancel-read, error-context.debug-message,
  thread.spawn-ref, thread.spawn-indirect, thread.available-parallelism).

Assisted-by: Claude (Anthropic)
Signed-off-by: YiYing He <yiying@secondstate.io>
q82419 added a commit to WasmEdge/WasmEdge that referenced this pull request Sep 24, 2026
The dependency, url and integrity forms left the extern-name grammar with
the `external-id` attribute (WebAssembly/component-model#672): drop their
parsing, their diagnostics and the tests that covered them.

The wasm-core-20260922 wasm-tools suites still exercise those forms, so
this change stays on the series, which pins wasm-dev/component.

Assisted-by: Claude (Anthropic)
Signed-off-by: YiYing He <yiying@secondstate.io>
q82419 added a commit to WasmEdge/WasmEdge that referenced this pull request Sep 29, 2026
The dependency, url and integrity forms left the extern-name grammar with
the `external-id` attribute (WebAssembly/component-model#672): drop their
parsing, their diagnostics and the tests that covered them.

The wasm-core-20260922 wasm-tools suites still exercise those forms, so
this change stays on the series, which pins wasm-dev/component.

Assisted-by: Claude (Anthropic)
Signed-off-by: YiYing He <yiying@secondstate.io>
q82419 added a commit to WasmEdge/WasmEdge that referenced this pull request Sep 30, 2026
The dependency, url and integrity forms left the extern-name grammar with
the `external-id` attribute (WebAssembly/component-model#672): drop their
parsing, their diagnostics and the tests that covered them.

The wasm-core-20260922 wasm-tools suites still exercise those forms, so
this change stays on the series, which pins wasm-dev/component.

Assisted-by: Claude (Anthropic)
Signed-off-by: YiYing He <yiying@secondstate.io>
q82419 added a commit to WasmEdge/WasmEdge that referenced this pull request Oct 1, 2026
The dependency, url and integrity forms left the extern-name grammar with
the `external-id` attribute (WebAssembly/component-model#672): drop their
parsing, their diagnostics and the tests that covered them.

The wasm-core-20260922 wasm-tools suites still exercise those forms, so
this change stays on the series, which pins wasm-dev/component.

Assisted-by: Claude (Anthropic)
Signed-off-by: YiYing He <yiying@secondstate.io>
chenyan2002 pushed a commit to chenyan2002/wasm-tools that referenced this pull request Oct 6, 2026
This commit removes support for component names of the form
`unlocked-dep=<...>`, `locked-dep=<...>`, `url=<...>` and
`integrity=<...>`. These were removed upstream in
WebAssembly/component-model#672 in favor of the `external-id`
attribute/annotation. I'm not aware of anyone actually using these forms
of imports anywhere in the ecosystem so the expected impact of this
change in theory is negligible. If there are users, however, these can
always be re-added with a more phased deletion plan.
bitwalker added a commit to 0xMiden/compiler that referenced this pull request Oct 13, 2026
…nal-id` (#1414)

Close #1378

This PR makes Miden paths the only naming of component procedures below
the Wasm frontend parser. The SDK macros write the full Miden path of
every exported and imported function into the WIT `@external-id`
attribute (WebAssembly/component-model#672), and
the compiler reads only that attribute. The component-model names
(`miden:counter-contract/counter-contract@0.1.0`, `get-count`) are used
only to pair `canon lift`/`canon lower` with the core functions and in
diagnostics.

Until now the compiler did not convert names at all. The CM interface id
became one quoted MASM path segment and the kebab function name stayed
as the leaf, so a package exported
`::"miden:counter-contract/counter-contract@0.1.0"::"get-count"`, a note
called the wallet via `call
::"miden:basic-wallet/basic-wallet@0.1.0"::"receive-asset"`, and the tx
script entrypoint was `miden:base/transaction-script@1.0.0::run`. These
are now `::miden::counter_contract::counter_contract::get_count`, `call
::miden::basic_wallet::basic_wallet::receive_asset` and
`::miden::basic_wallet_tx_script::basic_wallet_tx_script::run`.

This PR:
- bumps `wasmparser`, `wasmprinter`, `wat`, `wit-component`,
`wit-parser` to `0.259` and `wit-bindgen` to `0.62`, and enables the
`cm-implements` feature (it gates `external-id`) in the frontend
validator;
- names the HIR `builtin.component` by its namespace path
(`miden::counter_contract::counter_contract`) instead of the
`namespace`/`name`/`version` attributes and removes `ComponentId`. A
component is a symbol keyed by its `::`-joined path; a segment that
itself contains `::` (which Miden Assembly spells quoted) is quoted in
the key, so `::"acme::pkg"::app` keeps its two segments through HIR and
back. The resolver matches the longest registered prefix, so a component
`miden::a::b` coexists with the kernel module tree rooted at `miden`. A
joined name resolves only to a symbol table (a linker stub function
named `intrinsics::felt::add` does not capture a path), a module name
cannot contain `::`, two component names must not nest, and parsed
`.hir` input is held to the same rules by the `World` and `Module`
verifiers. `SymbolPath` prints Miden style and `--emit=hir` output with
a component parses back;
- makes `[lib].namespace` in `miden-project.toml` a three-segment Miden
path and the single source of naming: the macros derive the WIT package
and interface ids, the export paths, the storage slot names
(`<namespace>::<field>`), the tx script entrypoint (`<namespace>::run`)
and the FPI procedure-root lookups from it. The component trait name no
longer takes part in naming;
- reads the paths in the frontend: exports must be exactly
`<namespace>::<leaf>`, imports use the dependency's paths verbatim, and
a function without `@external-id`, a path outside the namespace, exports
under different namespaces, two exports on one path, an import inside
the component's own namespace or nesting another import's parent, and a
world-level function export or import are compile errors. A WIT import
lowers to a cross-context `call`, so importing an `exec`-bindable
procedure of a linked library (the core library, a MASM dependency) is
rejected with a pointer to the native `#[link_name]` binding; role
procedures such as the standard account components shipped in the
`miden-standards` library stay importable. Of these errors only two hit
something `next` accepted: a function without `@external-id` (every
function qualified before, named by its CM id as one quoted segment plus
the kebab leaf) and exports from interfaces under different namespaces
(`next` compiled them, but named every function after the first exported
interface and silently lost the other). The rest replace a silent drop
or a panic (a world-level function export) or guard shapes that could
not be spelled before (an import inside the own namespace, an import of
a library procedure). `<namespace>::init` is reserved for the
initializer codegen emits, on both the SDK and the frontend side. A
component compiled without a target namespace (no `miden-project.toml`,
no `--name`) infers its namespace from its exports;
- names the core module functions behind lifted exports and lowered
imports at definition time: the leaf when it is free, else the leaf
prefixed by its parent segments (`read`, `api_read`, `second_api_read`),
never the Wasm name, so the emitted MASM carries no CM strings for
component procedures, and debug info links these functions by their HIR
names (the source name stays the display name). A core module named like
an export or like `init` (a crate `swap` exporting `swap`) and a program
crate named like a reserved library root (`intrinsics`, `std`) are
compile errors asking to rename the crate, instead of being renamed
behind the user's back;
- creates the synthetic wrapper component at the target namespace, so
codegen no longer re-roots it (the re-rooting stays for component-less
worlds only);
- makes the tx script export its own interface (the SDK
`transaction-script` interface and `base-world` are gone) and drops the
accidental `miden-` interface prefix from notes;
- spells Rust-derived names in the generated WIT in the explicit form
(`%get-url: func(%type: u32) -> u32;`, record fields and variant cases
too), names the guest methods the way `wit-bindgen` names them (`fn
getURL` implements `get_url`), and rejects at the macro what does not
round-trip through the bindings: two Rust names with one WIT name, a
method named like an interface type (`felt`), an exported type named
like a WIT keyword, a record field that is not the snake_case of its WIT
name, a variant case that is not the UpperCamelCase of its WIT name, and
identifiers `wit-bindgen` does not escape (`r#gen`). `bindings::export!`
receives an identifier through a local alias, so `impl Trait for
self::StorageAlias` works;
- hosts the naming rules the SDK macros and the compiler share in
`midenc-frontend-wasm-metadata`: the namespace segment rule (snake_case,
no WIT or Rust keyword including `gen`, `core_types` is not an interface
name, `miden::base`, `miden::protocol`, `miden::core`, `intrinsics` and
`std` are reserved), the procedure-path rule of `@external-id` (bare
MASM segments, so no quoting is ever needed), the FPI and
stored-procedure import prefixes (the macros reject exported WIT
function names starting with either, on every export site) and the
`core-types` interface id. `cargo miden new` validates the namespace it
derives with the same rule. The procedure names codegen generates
(`init`, `__init_function_table`, ...) live in
`midenc_hir::reserved_names`, and the frontend keeps user leaves off
them;
- derives the note storage schema's WIT package id from the namespace
(`<ns>:<pkg>-schema`) and spells its fields and cases in the same `%`
form;
- binds FPI calls to keyword-named dependency methods by `wit-bindgen`'s
spelling (`type_`), reads dependency `@external-id`s with the frontend's
rule, and reports unsupported component constructs (async canonical
functions, module sections out of range) as diagnostics instead of
panicking;
- asserts in the integration tests that a package manifest exports
exactly the WIT-declared procedures plus `init`, so leaked helpers fail
the build;
- updates the `cargo miden new` templates (namespace
`miden::<pkg>::<pkg>`), examples, fixtures, the docs (a `Naming` section
in `cargo-miden.md`) and `MIGRATION.md`, and regenerates the templates
bundle.

## Design decisions

### `[lib].namespace` is the single source of naming, with the interface
as the third segment

The namespace is already the library identity the assembler links
against, so deriving the WIT ids from it removes the three-way equality
check between the manifest, the crate name and the trait name, and the
hardcoded `miden:` package prefix. I kept three segments
(`ns::pkg::iface`) so a future multi-interface package has a slot, and
since it matches the CM id structure users already know. The first two
segments must be unique among the crates a consumer links (they become
the WIT package id) regardless of the version.

### Every function carries its full path, and there is no fallback

`@external-id` cannot annotate an `interface` definition, only its
items, so the value is per function and holds the full path. That makes
the compiler's lookup one string read with no world-level dependency,
and the WIT embedded in a package is self-describing when a consumer
inlines it. A function without the attribute is an error. Deriving a
path from the CM names as a fallback would mean two naming rules in the
compiler, so I did not add one. The five hand-written WIT files in the
tree got the attributes by hand.

### Exports sit exactly under the namespace

Every export path must be `<namespace>::<leaf>`. The rule is checked
only in the Wasm frontend, HIR, codegen and the package manifest do not
know it. Codegen places component functions as procedures of the root
module as before, a consumer gets a callee's component by dropping the
leaf, the namespace of a component compiled without a target namespace
is inferred from its exports, and the rule can be relaxed to nested
modules later without a format break.

### Core functions are named once, when the module is defined

The first version defined the core function behind an export under its
Wasm name and renamed it to the leaf afterwards, with a map of current
names and a fallback to the Wasm name when the leaf was taken, which let
a CM string reach MASM. Now the translator records the module instances,
the exports and the startup adapter during the initializer walk and
builds the HIR afterwards, when every export leaf is known. There is no
earlier point: `wit-component` lifts an interface's functions after the
core module is instantiated and after the exports of the preceding
interfaces. One rule names exports first, then imports; a function named
once keeps its name; when every candidate is taken the translation fails
instead of falling back.

### Component namespaces do not nest

Resolution prefers the longest registered name, so a stub component
`acme::app::app::main` next to a component `acme::app::app` would
capture the latter's `main` module. The HIR refuses nesting component
names in both directions, and the frontend refuses an import whose
parent lies inside the component's own namespace or nests another
import's parent. FPI imports and the stored-procedure imports
`#[component_storage]` generates for `StoredProcedure` slots
(`<namespace>::dyncall::<field>`) are exempt since they declare no stub
component. Hand-written WIT importing a module and its submodule
(`acme::math::add` and `acme::math::u64::add`) is rejected for now;
SDK-built dependencies always have three-segment namespaces, so they are
not affected.

### Generated WIT uses the explicit `%` form, and the macros reject what
the bindings cannot map back

The explicit identifier form is valid for keywords and ordinary names
alike, so any Rust identifier whose kebab-case form is a valid WIT name
can be exported without the macros carrying WIT's keyword list for
functions (the list is still needed for package, interface and type
names). `wit-bindgen` maps a WIT name back to Rust by its own rule, and
the generated code accesses the user's struct fields and enum cases by
that spelling, so the macros check the round trip up front and name the
spelling to use, instead of letting the generated bindings fail with an
error pointing into generated code.

### FPI keeps the `fpi-` marker

The frontend only needs to know that an import is a foreign procedure
invocation (the target root is a runtime argument computed by the macro
from the dependency manifest), so the `fpi-` function prefix and the
synthetic `fpi-v1-<pkg>` interface stay as macro-internal conventions.
The FPI functions carry a path under the consumer namespace
(`<consumer>::fpi::<dep ns>::<dep pkg>::<dep iface>::<fn>`), so the
strict rule has no exception, and the macro looks up the procedure root
by the dependency's own path read from its WIT.

### Clean break

Storage slot names become `<namespace>::<field>` with the field name as
written (was `<sanitized package>::<interface>::<field>`; raw
identifiers lose `r#`, a leading `_` is an error instead of an `x_`
rewrite), so deployed components are re-keyed, and packages built with
the previous naming do not link with the new ones. Since we are pre-1.0
with rc churn on the protocol side, I did not keep a compatibility path.
`MIGRATION.md` covers the namespace format and its reserved names, the
export paths, the slot names, the tx script interface, notes losing the
`miden-` prefix (and the consumer's binding module path with it), the
`%` form and the field and case rules, the reserved `init`, the nesting
rule, the hand-written and override WIT, and the `wit-bindgen` bump.

### Note

This is a breaking change for every project. The `[lib].namespace`
format, the export paths, the storage slot names and the
`FrontendMetadata` API of `midenc-frontend-wasm-metadata` change, and a
crate that calls `miden::generate!` over hand-written WIT must add
`@external-id` to every function. A consumer calls a keyword-named
dependency method by `wit-bindgen`'s spelling (`account.type_()`). The
templates bundle is regenerated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants