Skip to content

Bound local model resources and engine ownership - #136

Closed
andersalm wants to merge 52 commits into
developfrom
feat/0.7.1-model-resources
Closed

andersalm wants to merge 52 commits into
developfrom
feat/0.7.1-model-resources

Conversation

@andersalm

@andersalm andersalm commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Runtime admits a local model against observed memory and disk, gives one engine owner per OS account, and keeps a bounded cancelable queue. Idle release, cancellation, crash recovery and shutdown preserve ownership until engine cleanup is confirmed. Launch and admission share batch128 bounds and disabled prompt/checkpoint caches.

The Linux CI journey uses fresh installed CPU fixtures in finite1GiB refusal and4GiB lifecycle services. It verifies the complete global cgroup hierarchy, numeric limits, the Runtime account, source/tree, built and installed Runtime/provider, test executable, both complete engine bundles and pinned GGUF. The refusal checks its error and recovery text before a guard starts; the lifecycle checks reply, shared busy, idle release, crash cleanup, reuse and final descendants before service cleanup. Safe receipts include measured step time; raw logs remain private.

Validation at07c6f4f7:249 unit tests,10 production-process tests,15 policy tests,10 refused-case harness tests and four basic gates passed. Two independent gpt-6.1-sol high reviews passed. Normal merges preserve verified PR153f6 and current developc7cc9804. Exact installed Linux CI follows the push.

Owning issue:#98. This draft retains installed human acceptance, #97/#133, reviewed#155 integration and Irzhy's current-candidate approval. Backend/Home peak-memory qualification, supported Qwen profiles and Jetson acceptance remain separate gates. Source proof and the retained Mac proof establish their recorded scope only.

Stage the accepted b10516 Jetson archive as a release input and select its CPU profile. Check host CPU features, installed ELF architecture and library loading before writing the engine receipt. Extend ARM64 admission and native proof coverage.

Verified: release-platform-input-test.py; prepare-release-platform-test.py; focused elastos-server setup/profile/startup tests; repository format and entropy gates.
Keep explicitly loopback-bound Carrier endpoints inside their local socket boundary by disabling Iroh port mapping. The default portmapper can start UPnP discovery through an auxiliary wildcard UDP socket.

Verified: required diff, Home entropy and formatting gates; cargo test -p elastos-server test_explicit_ (3 pass). Focused security review passes diff 0064a077522efea9f37e277a17d18fe0555476e4db9160df3dc60f64df710f91. Installed ARM64 proof remains open.
Share the strict Carrier address reader between Runtime startup and ordinary standalone component setup. Every trusted-source route uses the chosen address without an ephemeral fallback; fetch cleanup releases the owned endpoint before the next component.

Verified: required source gates; configured setup port ownership fixture (occupied refusal, two sequential fetches, immediate rebind); existing strict configuration test; focused security review of diff 0c645b36edfbe54a63bcb16b948f2cd3fb351dc76005d57a03d022cfd6f79bd0. Installed Assistant proof remains open.
Preserve the installed journey proof and package the ARM PR archive before its compatibility verifier. Verify event/platform package guards, missing-archive refusal, all 13 CI policy/fixture tests, basic source gates and YAML syntax.
Use the existing eight-token request limit for the installed greeting proof and bind the provider bytes in its safe receipt. Refuse missing or oversized request limits and missing provider files before launch. All 15 policy/fixture tests, Node syntax and basic gates pass; two independent reviews cover the exact correction.
@andersalm andersalm mentioned this pull request Oct 2, 2026
10 tasks
@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-02), head 77dd1c9.
VERDICT: FAIL

Blocking findings:

  1. elastos/crates/elastos-server/src/setup.rs:347-349: verify_arm64_model_host(&platform)? runs inside the "Components to install" loop. llama-server is in the default home profile (components.json:1568), and install.sh:762 runs plain setup. On ARM64 Linux hosts without FP16 and dot-product support, such as the Raspberry Pi 4 or other Cortex-A72 hosts, the whole Home setup now stops before it installs anything. Before this PR, linux-arm64 used source-build and setup skipped the engine. Setup should skip the model engine and still install Home. No test covers this refused host.
  2. capsules/model-provider/src/local_resources.rs:13-17,38-44: the account lease uses a predictable name in shared /tmp (/tmp/elastos-local-model-<euid>.lock). Any other local account can create that file first. The victim's open then fails, or the uid/mode check fails, and every local model run returns Failed, which the user sees as "model backend failed". The user has no recovery, because the sticky bit stops them deleting the file. This breaks the account boundary that docs/MODEL_PROVIDER.md promises ("Other OS accounts … have separate ownership"). The lease should live in a directory that only the account owns.

Non-blocking notes:

  1. Admission now fails closed on cgroup v1 or hybrid hosts, WSL2 and containers. Those hosts get "This model's resource profile is unavailable", which points at the model when the cause is the host.
  2. No test forces a disk refusal. host_observation_and_small_fixture_admission accepts Ok, memory refusal or disk refusal alike, so the 10% disk-floor refusal is never asserted.
  3. The ubuntu-22.04-arm job still reports under the ubuntu-24.04-arm check name, which misstates what the check ran on.

Not checked: I ran no tests or builds. I did not check the CI results for this commit, the installed Linux, Mac or Jetson runs, or whether the published release contains the linux-arm64 engine archive. I also did not check whether the Mac free+inactive estimate, which counts weights twice and reserves 10% for Home, refuses Qwen models that work today.

@andersalm

Copy link
Copy Markdown
Contributor Author

Superseded by #322 (alpha.13): free-memory admission, one engine, 60 s idle release, batch 128 and the ARM64 setup skip were ported; the account lease and cgroup CI services were dropped (one Runtime per OS account is the boundary, see MODEL_PROVIDER.md).

@andersalm andersalm closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant