Repository navigation
Add external-id attribute - #672
Conversation
9e2219f to
eeb2ed0
Compare
ricochet
left a comment
There was a problem hiding this comment.
This looks good to me. I'd use it right away.
You may want to update Linking.md as well:
mutually known to later stages in the deployment pipeline (specifically with
the [depname] case ofimportnamein the text and binary format).
To something like:
mutually known to later stages in the deployment pipeline. (How an import
name is resolved to a particular registry artifact is left to the host or
build tooling; the optional [external-id] attribute can carry a
tooling-specific identifier such as a registry path or URL.)
|
I've got an initial implementation of this over at bytecodealliance/wasm-tools#2555 if you wouldn't mind skimming over the test I've added |
|
Awesome! Test looks good. |
…2555) * Initial work for `versionsuffix` and `external-id` This is intended to at least start laying groundwork and implementation for two features in the component model that this repository does not yet implement: * `(versionsuffix "...")` for components -- introduced in WebAssembly/component-model#536 this is intended to pave a path forward to canonicalizing versions in component imports/exports in the future. For example where today you'd import `a:b/c@1.0.0` tomorrow you'd import `a:b/c@1`. This is not yet fully integrated into `wit-component`, however, and further work will be necessary for bindings generators as well to take advantage of this. It's all encoded in the same place as `external-id`, though, so I figured I'd at least start to lay the groundwork. * `(external-id "...")` for components -- being introduced in WebAssembly/component-model#672 this is intended to attach arbitrary metadata to imports/exports in a component. This is surfaced in WIT as an `@external-id("...")` attribute. This should be hooked up throughout this implementation to WIT and such such that the implementation is intended to be complete. There's probably something I forgot, but all the major pieces should be there. The `versionsuffix` parsing/validation is gated by a new `cm-canon-names` feature, and the `external-id` parsing/validation is gated behind the preexisting `cm-implements` feature to match the specification's classification. * Fix msrv * Update expected stdout * Reduce debug mode stack size usage
|
Ok with bytecodealliance/wasm-tools#2558 that shoudl, I believe, implement all of this and have everything round-tripped in all the places |
|
That's great! I guess then I'll wait a bit to see if anyone else has comments and then merge, noting that the feature isn't yet released, so we can continue to iterate in subsequent issues/PRs. |
- task.return takes the functype resultlist, `0x00 t` or `0x01 0x00`; the AST keeps one optional result for functions and for task.return. - Accept only plain names and interface names as extern names: the dependency, url and integrity forms left the specification with the `external-id` attribute (WebAssembly/component-model#672), and their diagnostics go with them. - Name the canonical built-ins in the diagnostics as the specification spells them (thread.yield, stream.cancel-read, error-context.debug-message, thread.spawn-ref, thread.spawn-indirect, thread.available-parallelism). Assisted-by: Claude (Anthropic) Signed-off-by: YiYing He <yiying@secondstate.io>
The dependency, url and integrity forms left the extern-name grammar with the `external-id` attribute (WebAssembly/component-model#672): drop their parsing, their diagnostics and the tests that covered them. The wasm-core-20260922 wasm-tools suites still exercise those forms, so this change stays on the series, which pins wasm-dev/component. Assisted-by: Claude (Anthropic) Signed-off-by: YiYing He <yiying@secondstate.io>
The dependency, url and integrity forms left the extern-name grammar with the `external-id` attribute (WebAssembly/component-model#672): drop their parsing, their diagnostics and the tests that covered them. The wasm-core-20260922 wasm-tools suites still exercise those forms, so this change stays on the series, which pins wasm-dev/component. Assisted-by: Claude (Anthropic) Signed-off-by: YiYing He <yiying@secondstate.io>
The dependency, url and integrity forms left the extern-name grammar with the `external-id` attribute (WebAssembly/component-model#672): drop their parsing, their diagnostics and the tests that covered them. The wasm-core-20260922 wasm-tools suites still exercise those forms, so this change stays on the series, which pins wasm-dev/component. Assisted-by: Claude (Anthropic) Signed-off-by: YiYing He <yiying@secondstate.io>
The dependency, url and integrity forms left the extern-name grammar with the `external-id` attribute (WebAssembly/component-model#672): drop their parsing, their diagnostics and the tests that covered them. The wasm-core-20260922 wasm-tools suites still exercise those forms, so this change stays on the series, which pins wasm-dev/component. Assisted-by: Claude (Anthropic) Signed-off-by: YiYing He <yiying@secondstate.io>
This commit removes support for component names of the form `unlocked-dep=<...>`, `locked-dep=<...>`, `url=<...>` and `integrity=<...>`. These were removed upstream in WebAssembly/component-model#672 in favor of the `external-id` attribute/annotation. I'm not aware of anyone actually using these forms of imports anywhere in the ecosystem so the expected impact of this change in theory is negligible. If there are users, however, these can always be re-added with a more phased deletion plan.
…nal-id` (#1414) Close #1378 This PR makes Miden paths the only naming of component procedures below the Wasm frontend parser. The SDK macros write the full Miden path of every exported and imported function into the WIT `@external-id` attribute (WebAssembly/component-model#672), and the compiler reads only that attribute. The component-model names (`miden:counter-contract/counter-contract@0.1.0`, `get-count`) are used only to pair `canon lift`/`canon lower` with the core functions and in diagnostics. Until now the compiler did not convert names at all. The CM interface id became one quoted MASM path segment and the kebab function name stayed as the leaf, so a package exported `::"miden:counter-contract/counter-contract@0.1.0"::"get-count"`, a note called the wallet via `call ::"miden:basic-wallet/basic-wallet@0.1.0"::"receive-asset"`, and the tx script entrypoint was `miden:base/transaction-script@1.0.0::run`. These are now `::miden::counter_contract::counter_contract::get_count`, `call ::miden::basic_wallet::basic_wallet::receive_asset` and `::miden::basic_wallet_tx_script::basic_wallet_tx_script::run`. This PR: - bumps `wasmparser`, `wasmprinter`, `wat`, `wit-component`, `wit-parser` to `0.259` and `wit-bindgen` to `0.62`, and enables the `cm-implements` feature (it gates `external-id`) in the frontend validator; - names the HIR `builtin.component` by its namespace path (`miden::counter_contract::counter_contract`) instead of the `namespace`/`name`/`version` attributes and removes `ComponentId`. A component is a symbol keyed by its `::`-joined path; a segment that itself contains `::` (which Miden Assembly spells quoted) is quoted in the key, so `::"acme::pkg"::app` keeps its two segments through HIR and back. The resolver matches the longest registered prefix, so a component `miden::a::b` coexists with the kernel module tree rooted at `miden`. A joined name resolves only to a symbol table (a linker stub function named `intrinsics::felt::add` does not capture a path), a module name cannot contain `::`, two component names must not nest, and parsed `.hir` input is held to the same rules by the `World` and `Module` verifiers. `SymbolPath` prints Miden style and `--emit=hir` output with a component parses back; - makes `[lib].namespace` in `miden-project.toml` a three-segment Miden path and the single source of naming: the macros derive the WIT package and interface ids, the export paths, the storage slot names (`<namespace>::<field>`), the tx script entrypoint (`<namespace>::run`) and the FPI procedure-root lookups from it. The component trait name no longer takes part in naming; - reads the paths in the frontend: exports must be exactly `<namespace>::<leaf>`, imports use the dependency's paths verbatim, and a function without `@external-id`, a path outside the namespace, exports under different namespaces, two exports on one path, an import inside the component's own namespace or nesting another import's parent, and a world-level function export or import are compile errors. A WIT import lowers to a cross-context `call`, so importing an `exec`-bindable procedure of a linked library (the core library, a MASM dependency) is rejected with a pointer to the native `#[link_name]` binding; role procedures such as the standard account components shipped in the `miden-standards` library stay importable. Of these errors only two hit something `next` accepted: a function without `@external-id` (every function qualified before, named by its CM id as one quoted segment plus the kebab leaf) and exports from interfaces under different namespaces (`next` compiled them, but named every function after the first exported interface and silently lost the other). The rest replace a silent drop or a panic (a world-level function export) or guard shapes that could not be spelled before (an import inside the own namespace, an import of a library procedure). `<namespace>::init` is reserved for the initializer codegen emits, on both the SDK and the frontend side. A component compiled without a target namespace (no `miden-project.toml`, no `--name`) infers its namespace from its exports; - names the core module functions behind lifted exports and lowered imports at definition time: the leaf when it is free, else the leaf prefixed by its parent segments (`read`, `api_read`, `second_api_read`), never the Wasm name, so the emitted MASM carries no CM strings for component procedures, and debug info links these functions by their HIR names (the source name stays the display name). A core module named like an export or like `init` (a crate `swap` exporting `swap`) and a program crate named like a reserved library root (`intrinsics`, `std`) are compile errors asking to rename the crate, instead of being renamed behind the user's back; - creates the synthetic wrapper component at the target namespace, so codegen no longer re-roots it (the re-rooting stays for component-less worlds only); - makes the tx script export its own interface (the SDK `transaction-script` interface and `base-world` are gone) and drops the accidental `miden-` interface prefix from notes; - spells Rust-derived names in the generated WIT in the explicit form (`%get-url: func(%type: u32) -> u32;`, record fields and variant cases too), names the guest methods the way `wit-bindgen` names them (`fn getURL` implements `get_url`), and rejects at the macro what does not round-trip through the bindings: two Rust names with one WIT name, a method named like an interface type (`felt`), an exported type named like a WIT keyword, a record field that is not the snake_case of its WIT name, a variant case that is not the UpperCamelCase of its WIT name, and identifiers `wit-bindgen` does not escape (`r#gen`). `bindings::export!` receives an identifier through a local alias, so `impl Trait for self::StorageAlias` works; - hosts the naming rules the SDK macros and the compiler share in `midenc-frontend-wasm-metadata`: the namespace segment rule (snake_case, no WIT or Rust keyword including `gen`, `core_types` is not an interface name, `miden::base`, `miden::protocol`, `miden::core`, `intrinsics` and `std` are reserved), the procedure-path rule of `@external-id` (bare MASM segments, so no quoting is ever needed), the FPI and stored-procedure import prefixes (the macros reject exported WIT function names starting with either, on every export site) and the `core-types` interface id. `cargo miden new` validates the namespace it derives with the same rule. The procedure names codegen generates (`init`, `__init_function_table`, ...) live in `midenc_hir::reserved_names`, and the frontend keeps user leaves off them; - derives the note storage schema's WIT package id from the namespace (`<ns>:<pkg>-schema`) and spells its fields and cases in the same `%` form; - binds FPI calls to keyword-named dependency methods by `wit-bindgen`'s spelling (`type_`), reads dependency `@external-id`s with the frontend's rule, and reports unsupported component constructs (async canonical functions, module sections out of range) as diagnostics instead of panicking; - asserts in the integration tests that a package manifest exports exactly the WIT-declared procedures plus `init`, so leaked helpers fail the build; - updates the `cargo miden new` templates (namespace `miden::<pkg>::<pkg>`), examples, fixtures, the docs (a `Naming` section in `cargo-miden.md`) and `MIGRATION.md`, and regenerates the templates bundle. ## Design decisions ### `[lib].namespace` is the single source of naming, with the interface as the third segment The namespace is already the library identity the assembler links against, so deriving the WIT ids from it removes the three-way equality check between the manifest, the crate name and the trait name, and the hardcoded `miden:` package prefix. I kept three segments (`ns::pkg::iface`) so a future multi-interface package has a slot, and since it matches the CM id structure users already know. The first two segments must be unique among the crates a consumer links (they become the WIT package id) regardless of the version. ### Every function carries its full path, and there is no fallback `@external-id` cannot annotate an `interface` definition, only its items, so the value is per function and holds the full path. That makes the compiler's lookup one string read with no world-level dependency, and the WIT embedded in a package is self-describing when a consumer inlines it. A function without the attribute is an error. Deriving a path from the CM names as a fallback would mean two naming rules in the compiler, so I did not add one. The five hand-written WIT files in the tree got the attributes by hand. ### Exports sit exactly under the namespace Every export path must be `<namespace>::<leaf>`. The rule is checked only in the Wasm frontend, HIR, codegen and the package manifest do not know it. Codegen places component functions as procedures of the root module as before, a consumer gets a callee's component by dropping the leaf, the namespace of a component compiled without a target namespace is inferred from its exports, and the rule can be relaxed to nested modules later without a format break. ### Core functions are named once, when the module is defined The first version defined the core function behind an export under its Wasm name and renamed it to the leaf afterwards, with a map of current names and a fallback to the Wasm name when the leaf was taken, which let a CM string reach MASM. Now the translator records the module instances, the exports and the startup adapter during the initializer walk and builds the HIR afterwards, when every export leaf is known. There is no earlier point: `wit-component` lifts an interface's functions after the core module is instantiated and after the exports of the preceding interfaces. One rule names exports first, then imports; a function named once keeps its name; when every candidate is taken the translation fails instead of falling back. ### Component namespaces do not nest Resolution prefers the longest registered name, so a stub component `acme::app::app::main` next to a component `acme::app::app` would capture the latter's `main` module. The HIR refuses nesting component names in both directions, and the frontend refuses an import whose parent lies inside the component's own namespace or nests another import's parent. FPI imports and the stored-procedure imports `#[component_storage]` generates for `StoredProcedure` slots (`<namespace>::dyncall::<field>`) are exempt since they declare no stub component. Hand-written WIT importing a module and its submodule (`acme::math::add` and `acme::math::u64::add`) is rejected for now; SDK-built dependencies always have three-segment namespaces, so they are not affected. ### Generated WIT uses the explicit `%` form, and the macros reject what the bindings cannot map back The explicit identifier form is valid for keywords and ordinary names alike, so any Rust identifier whose kebab-case form is a valid WIT name can be exported without the macros carrying WIT's keyword list for functions (the list is still needed for package, interface and type names). `wit-bindgen` maps a WIT name back to Rust by its own rule, and the generated code accesses the user's struct fields and enum cases by that spelling, so the macros check the round trip up front and name the spelling to use, instead of letting the generated bindings fail with an error pointing into generated code. ### FPI keeps the `fpi-` marker The frontend only needs to know that an import is a foreign procedure invocation (the target root is a runtime argument computed by the macro from the dependency manifest), so the `fpi-` function prefix and the synthetic `fpi-v1-<pkg>` interface stay as macro-internal conventions. The FPI functions carry a path under the consumer namespace (`<consumer>::fpi::<dep ns>::<dep pkg>::<dep iface>::<fn>`), so the strict rule has no exception, and the macro looks up the procedure root by the dependency's own path read from its WIT. ### Clean break Storage slot names become `<namespace>::<field>` with the field name as written (was `<sanitized package>::<interface>::<field>`; raw identifiers lose `r#`, a leading `_` is an error instead of an `x_` rewrite), so deployed components are re-keyed, and packages built with the previous naming do not link with the new ones. Since we are pre-1.0 with rc churn on the protocol side, I did not keep a compatibility path. `MIGRATION.md` covers the namespace format and its reserved names, the export paths, the slot names, the tx script interface, notes losing the `miden-` prefix (and the consumer's binding module path with it), the `%` form and the field and case rules, the reserved `init`, the nesting rule, the hand-written and override WIT, and the `wit-bindgen` bump. ### Note This is a breaking change for every project. The `[lib].namespace` format, the export paths, the storage slot names and the `FrontendMetadata` API of `midenc-frontend-wasm-metadata` change, and a crate that calls `miden::generate!` over hand-written WIT must add `@external-id` to every function. A consumer calls a keyword-named dependency method by `wit-bindgen`'s spelling (`account.type_()`). The templates bundle is regenerated.
This PR extends #613, which added an
(implements <interfacename>)attribute to imports and exports, with an(external-id <string>)attribute. Theexternal-idattribute is useful when a component import or export name wants to match some external identifier, but the kebab-case syntax is too restrictive. One example is ESM-integration, where Module Specifiers can (now) be arbitrary Unicode strings (including URLs). Another is when importing stateful stores, e.g.:(as enabled by #613) and the platform-defined identifier of these stores isn't a valid
<plainname>, so you'd either have to add a whole separate name-mapping scheme (e.g. mappingredisandmemcachedto their host IDs) or mangle the host IDs into the<plainname>(say, via URL-encoding; ew). Instead, with this PR, you can use the@external-idattribute in WIT:and the WAT
(external-id <string>)is plumbed into the host, alongside(implements <interfacename>), so that the host can reflect on it and act accordingly.As mentioned above,
@external-idcan subsume the use cases forurl=<...>names for ESM-integration. Moreover, the reasoning in #613 for movingimplementsout of the name and into a separate attribute I think similarly suggests that the<urlname>,<depname>and<hashname>cases of<importname>aren't really the right place to solve their associated use cases. IIUC, these 3 name cases aren't currently being produced since there's no WIT syntax to emit them. Thus, this PR tentatively proposes removing these 3 name cases. The use cases are still important, but I think when we're ready to prioritize adding them to WIT/tooling, we should consider adding them as attributes instead. In the meantime, it lets us mergeimportnameandexportnameintoexternname, which simplifies nicely things.The PR also tidies up some sloppiness in the text-format grammar around string literals (e.g.,
"foo-bar") vs. the contents inside the quotes (e.g.foo-bar), using<Xstr>for the former and<X>for the latter.