Skip to content

Go components trap (and poison their instance) when the Go runtime reads the clock inside cabi_realloc #145

Open

Description

@lannbot

Observed at a2f84a5, driving the wosh composed client (Rust wosh-client + Go ssh-core + iroh endpoint, wac-plugged) in headless Chromium. Downstream report and mitigations: lann/wosh#71

Symptom, downstream

Under sustained output flood, every host call into the composed instance begins failing with

cannot enter component instance 8 (reentrance forbidden)

permanently — the embedder-visible face of instance poisoning. The page-level result was a dead SSH session mid-seq.

The first trap, captured

Instrumented the setOnInstancePoisoned seam (task/scheduler.ts) in the bundled runtime and reran the flood. The poison's recorded cause, consistently across ~10 reproductions against two separately-composed builds:

Trap: cannot leave component instance 1 (may_leave violation)
    at trap (dist/deltic.js:919)
    at trapIf (dist/deltic.js:922)
    ...
    at clock_time_get (wasm)
    at runtime.clock_time_get (Go runtime)
    at runtime.walltime1
    at time.now

Instance 1 is the Go (componentize-go) member. Mechanism:

  1. A cross-instance call lowers a list<u8> into the Go instance; the FACT adapter runs the callee's cabi_realloc with may_leave = false (reference semantics — fact/trampoline's set_may_leave_false, definitions.py's copy-window rule).
  2. Under allocation pressure, the Go runtime decides this allocation is the one that crosses a GC-pacing threshold and reads the wall clock — time.now → runtime.walltime1 → clock_time_get — from inside cabi_realloc.
  3. That is an import call during the copy window: cannot leave component instance (may_leave violation). The trap escapes the task, and per exec/boundary.ts's poisoning rules the instance is permanently un-enterable.
  4. Every later host entry then reports cannot enter component instance N (reentrance forbidden) — with no trace of the original cause.

Load-dependence explains the flakiness: the trap needs an allocation to cross a GC threshold inside the window, so it only shows under flood, and not on every run. wasmtime presumably has the same exposure with a different blast radius (store poisoning), but we have only observed it under deltic — the native gates simply never generated the same allocation pressure.

deltic's enforcement looks reference-faithful here; the composition is what's illegal. Filed against deltic anyway because this is where the practical decisions live:

Asks, in descending order of confidence

  1. Surface the poison cause on later entry refusals. The cause is already recorded (notifyInstancePoisoned(inst, cause)); the subsequent cannot enter component instance N (reentrance forbidden) message could carry — instance poisoned by: <original trap> (or an embedder API to query it). This is pure diagnosability, no semantic change, and would have cut a day off the downstream investigation: the reentrance message sent us chasing caller-side call overlap when the instance was simply dead.

  2. Consider whether WASI clock reads deserve a copy-window exemption. clock_time_get is a pure leaf: it re-enters nothing, observes no component state, and cannot re-enter the instance mid-copy. Exempting designated leaf imports from may_leave enforcement (or buffering the read) would make Go components viable under allocation pressure, at the cost of a documented divergence from the reference. Reasonable to refuse — but worth an explicit decision, because as it stands any composed component with a big-Go member is a time bomb: the Go runtime reserves the right to read the clock in any allocation.

  3. Track the componentize-go angle (their repo, not deltic's): cabi_realloc that never triggers GC pacing — pre-reserved arenas, or a GC hold during realloc — would fix this at the source for every runtime, wasmtime included.

Repro shape

Any composed component where host↔Go traffic crosses instance boundaries with list<u8> arguments at volume. With wosh: connect a session, seq 1 400000, and keep calling in (keystrokes suffice — xterm's automatic ESC[6n replies are enough). Poison typically lands within 60s. The downstream PR's browser-e2e leg L is a packaged reproduction.

Activity

  1. lannbot commented on Aug 15, 2026

    @lannbot
    CollaboratorAuthor

    Reviewed against the code and the spec authorities (definitions.py at the pinned submodule commit, CanonicalABI.md, wasmtime-environ 47). Verdict: the diagnosis is correct and well-evidenced, with one material sharpening below.

    Mechanism — verified end-to-end

    1. The copy window is reference-mandated. LiftLowerContext.reallocate (definitions.py:670) sets may_leave = False around the entire cabi_realloc call; canon_lower (definitions.py:2244) traps when it is false. Any import call from inside realloc traps, per the reference.
    2. deltic matches on the cross-instance path. FACT adapters toggle the callee's flags global — the WebAssembly.Global backing mayLeave (runtime/src/task/mod.ts:54) — and the lowered-import trampoline traps with exactly the reported message (runtime/src/exec/boundary.ts:1742).
    3. Poisoning and the blind refusal. Trap → callee poisoned (runtime/src/intrinsics/fact_calls.ts:674), entered set poisoned (boundary.ts:1296-1313); the cause flows only to the single-slot hook (runtime/src/task/scheduler.ts:144) — the instance lands in a WeakSet, the cause is not stored. Later entries refuse at boundary.ts:1161 with no trace of it. Exactly as described.
    4. wasmtime corroborates: the may-leave flag machinery in wasmtime-environ 47's fact/trampoline.rs, and the CM corpus asserts the analogous trap for the post-return window (test/values/post-return.wast).

    Sharpening: the host-entry path does not enforce the window

    The JS-side LiftLowerContext.reallocate (runtime/src/cabi/context.ts:92) is a documented v1 simplification: it calls realloc directly, without the may_leave toggle. So host→Go lowering of a list<u8> runs Go's realloc with may_leave = true, and a clock read there succeeds on deltic — while the reference traps. Consequences:

    • The "only under composition" precondition in the repro shape is a deltic artifact, not a property of the mechanism. Per the reference, plain host→Go with allocating arguments is exposed too.
    • deltic already carries an undocumented divergence here, in the lenient direction, on one of the two lowering paths. Whoever completes the deferred task machinery in context.ts would silently start trapping currently-working host→Go traffic under GC pressure.

    Tracked separately so the decision is made deliberately rather than inherited from a TODO.

    The asks

    Ask 1 (surface the cause) — agreed, implementing. poisonedInstances becomes a WeakMap keyed by instance holding the cause; the entry-refusal sites append — instance poisoned by: <cause> when the target is poisoned (and only then — a transient reentrance refusal stays unchanged, which is precisely the disambiguation this ask wants). Conformance-safe: the harness matches trap messages by substring (harness/src/runner.ts:319). One more data point in favor: instrumenting setOnInstancePoisoned — the only way to observe the cause today — replaces the stream-end retirement walk that task/streams.ts:831 registers on that single-slot seam (#66), so the downstream instrumentation disabled retirement while it ran. That seam is not a diagnostics API.

    Ask 2 (copy-window exemption for clock reads) — recommend refusing the exemption as stated.

    • The explainer text directly after reallocate (CanonicalABI.md, "Thus, in general, realloc must be called reentrantly…") makes the may_leave traps load-bearing: they are what license compiling realloc calls as plain synchronous calls instead of the specced fresh-thread semantics. This is not an incidental guard.
    • It is outside the parity policy's sanctioned-divergence space (docs/architecture.md §1): the reference unconditionally traps, so an exemption is a fork of trap semantics, not a choice within the spec's freedom.
    • Components validated against a lenient deltic break on wasmtime. The exemption moves the time bomb; it does not defuse it.

    The honest option set, given the host-path gap above: (1) close the gap — Go components get worse on deltic; (2) keep the current inconsistent state; (3) exempt on both paths as a documented opt-in, accepting ecosystem-fork risk. If leniency is wanted, the durable venue is an upstream spec conversation — a "pure leaf import" carve-out is at least coherent (a sync, non-reentrant import that never touches task/thread state cannot observe the difference between the fresh-thread and synchronous-call implementations), but that is the spec repo's call, not this runtime's.

    Ask 3 (componentize-go) — agreed; it is the real fix. A cabi_realloc that can trigger GC pacing (and hence clock reads) is non-conforming under any Component Model host. Recorded in upstream-consumer-findings.md; filing upstream is the operator's call.

    Two wording notes on the report: "the composition is what's illegal" should be "the Go component's cabi_realloc is non-conforming" — composition is merely where deltic happens to enforce today; and the wasmtime-exposure claim is appropriately hedged but unverified specifically for wasmtime's host-side lowering path.

  2. added a commit that references this issue on Aug 15, 2026
  3. added
    policyDecision owed on deltic-owned semantics
    upstream-trackingBlocked on or watching an external project (spec, wasmtime, engines, toolchains)
    p2Minor bugs; desirable lower-priority features
    on Aug 21, 2026
  4. added
    p1Correctness bugs likely to impact consumers; high-priority missing features
    and removed
    p2Minor bugs; desirable lower-priority features
    on Sep 12, 2026
  5. lannbot commented on Sep 12, 2026

    @lannbot
    CollaboratorAuthor

    Triage: p2 → p1, prioritizing observed behavior failures. This has the strongest downstream evidence in the p2 queue: repeated allocation-pressure reproductions caused a permanently dead wosh SSH session. The cause-diagnostic improvement has landed; the remaining priority is resolution of Go realloc compatibility (upstream fix or an explicit policy decision), not reimplementing diagnostics. High priority does not authorize a copy-window exemption: the report establishes that the current trap is reference-faithful. Reproduce against current consumer/compiler pins before choosing the remedy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    p1Correctness bugs likely to impact consumers; high-priority missing featurespolicyDecision owed on deltic-owned semanticsupstream-trackingBlocked on or watching an external project (spec, wasmtime, engines, toolchains)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions