G4 AI on your device: Local AI on seed, Mac and Jetson #84

Release: 0.8.0.

Owner: Codex, assigned by Anders on 11 Oct; Anders owns released Mac/Linux/Jetson acceptance after alpha.15.

Goal

SmolLM2 135M and Qwen2.5 1.5B work stably on Mac, Linux and Jetson: Get a signed model in Marketplace, use it in Assistant, switch and stop it, and keep it across restart and update. Prompts and replies stay on the device. Runtime decides which model fits a device and explains a refusal plainly.

Done means

Release acceptance: Each exposed feature works clearly from start to end, including recovery. Keep unfinished or confusing features hidden until their complete journey passes. Hiding a feature does not complete its task; source checks alone do not establish installed acceptance.

• Marketplace Get leads directly to Assistant use. Model switching and Stop work without a Home restart. Test: Get SmolLM2 → Open in Assistant → reply. Pick Qwen and reply. Press Stop once during a long reply and see 'Stopped.' Then Get and reply again without restarting Home.
• Runtime enforces memory limits and exclusive local execution within the documented installation boundary. Test: PR Local AI for 0.8.0: reliable Stop and switch, engine updates keep models, plain refusals, bounded journals (#84) #348 CI source-home-linux shows finite_1gib_refusal and finite_4gib_lifecycle passed.
• An established Home keeps its chats, draft, selection and retained models across offline use, restart and a supported update. Test: with a model kept and selected, a saved chat and an unsent draft: restart Home, reply with the network off, then update in System. Everything stays and a reply still works. alpha.22 PASS
• SmolLM2 and Qwen2.5 1.5B reply through Assistant on Mac, Linux and Jetson, or the device explains a capacity refusal. Test: fresh install on Mac, Linux x86, Linux ARM and Jetson. Get SmolLM2, then Qwen2.5 1.5B. Send 'Say hello in one sentence.' Record the reply or the exact refusal text.
• Final update acceptance comes from UP-07 (UP-07: Accept updates on Mac, Linux and Jetson #113).

Model packages are published separately from Runtime releases (#236, #242). Hosted models and remote inference are after 0.8.0.

Remaining

• alpha.15 runs on Linux x86, Linux ARM and Jetson → superseded by later installed runs: alpha.22 update keeps models, alpha.23 ARM64/amd64 11/11, alpha.24 Linux.
• LA-04: Bound memory and local execution #98 memory bound (page-cache fix in alpha.15) → moved to After 0.8.0 (LA-04: Bound memory and local execution #98 is in that milestone)
• LA-07: Make model state and recovery clear #101 plain refusal reasons → moved to After 0.8.0 (LA-07: Make model state and recovery clear #101 is in that milestone)
• LA-08: Preserve model state through catalogue and Runtime updates #102 model state across updates → moved to After 0.8.0 (LA-08: Preserve model state through catalogue and Runtime updates #102 is in that milestone)
• LA-06: Activate and switch without restarting Home #100: on alpha.15 ARM and Jetson, Stop leaves the run unsettled and the next Get says 'Model service unavailable' (on Jetson even after restart). Fix is Local AI for 0.8.0: reliable Stop and switch, engine updates keep models, plain refusals, bounded journals (#84) #348 item 1. → moved to After 0.8.0 (LA-06: Activate and switch without restarting Home #100 is in that milestone)
• Moved to 0.8.1: 0.8.1: Jetson model Get is slow and replies use the CPU only #365 (Jetson on alpha.15: Get runs at 0.44–0.55 MB/s, Qwen 43 min, and inference uses the CPU only). Known issue in the 0.8.0 release notes.
• Assistant: full screen hides Home, and a missing model gives no hint #331: on alpha.15 ARM, Assistant shows 'No model' with no hint during Get. The Mac black screen is not yet rechecked. → moved to After 0.8.0 (Assistant: full screen hides Home, and a missing model gives no hint #331 is in that milestone)
• 'Preparation failed.' after resuming an interrupted Get still happens on alpha.15 ARM and Jetson (Jetson restarts from 0 B); Local AI for 0.8.0: reliable Stop and switch, engine updates keep models, plain refusals, bounded journals (#84) #348 item 3 gives the plain cause.
• Jetson reports "Storage capacity is unavailable." with 16 GB free; the 11 Oct source audit identifies storage-service shutdown during download. Item 3 fixes the cause report. Not seen on Jetson alpha.15.
• Moved to 0.8.1: 0.8.1: Mac model Get fails on a slow link #366 (Mac on a slow link: Get fails with 'Model content could not be read'). Known issue in the 0.8.0 release notes.

Deferred to after 0.8.0: #85 (hosted AI), #99 (provider confinement), #104 (Qwen3.5 qualification), #105 (production model publication; scope is SmolLM2 and Qwen2.5 1.5B).

Code work left (audit of develop 291dd04, 11 Oct)

Causes found for two Remaining items: "Storage capacity is unavailable." with 16 GB free comes from stopping Home during a download (the storage service stops first and its failure is recorded as a capacity failure). "Preparation failed." after resume is restart cleanup of a killed download that keeps the part but gives no reason; Retry resumes correctly.

For 0.8.0, in order of user impact. Checked rows have source regression proof. Item 5 remains open for owner journal recovery after adversarial review. Item 7 has a tested CI harness and still needs its hosted installed run:

• Stop and activation: a local Stop confirms the engine stopped, so later Get, activation and Remove are not blocked by an "unknown" run; pending models re-activate after idle stop or run end; activation failure is reported (LA-06: Activate and switch without restarting Home #100). Tests: local_llama_stop_confirms_closure_and_unblocks_offer_refresh; pending_model_status_retries_activation_without_repreparing_content.
• Engine replacement: an update that changes the engine keeps the saved model working (rebuild activation, map the saved CID to the current offer), with a regression test (LA-08: Preserve model state through catalogue and Runtime updates #102). Tests: model_startup_binding_rebuilds_only_verified_engine_successors; restored CID uses its current engine offer and preserves historical runs, inactive chats and drafts.
• Plain refusal reasons: interrupted download, stopped storage service (not "capacity"), memory and cache budget refusals, activation failure (LA-07: Make model state and recovery clear #101). Tests: model_preparation_capacity_owner_distinguishes_space_and_service; model_preparation_resumes_after_a_restart_from_the_bytes_on_disk; model_preparation_budget_causes_refuse_before_effects_and_recover.
• Device fit before Get: Marketplace says whether the model fits this device's memory. Tests: model_catalog_device_fit_is_a_signed_row_fact_with_safe_refusals; model_get_device_refusal_starts_no_worker_or_storage_calls.
• Run journal bounds (count and size), useful owner recovery and fail-closed memory check on unreadable cgroup files (LA-04: Bound memory and local execution #98). Passing bounds tests: unreadable_root_and_required_ancestor_files_refuse; census_bounds_entries_and_aggregate_before_any_decode_or_prune. Owner recovery from retained uncertain history and legacy over-limit journals remains open under LA-07: Make model state and recovery clear #101 (lesson 195). → moved to After 0.8.0: owner recovery from old journals is not implemented (LA-07: Make model state and recovery clear #101); bounds tests already pass.
• Replayed old request IDs are refused safely (LA-07: Make model state and recovery clear #101). Tests: retained_request_precedes_freshness_floor_policy_and_unrelated_pruning; pruned_floor_survives_restart_and_refuses_clock_rollback_replay.
• Installed CI proof: 1 GiB refusal and 4 GiB lifecycle. Source fixture: test_finite_cleanup_needs_whole_exit_reap_empty_unit_and_finished_observer, plus 17 other finite-memory fault fixtures (18 total). Installed Linux x86-64 CI receipts now pass both finite_1gib_refusal and finite_4gib_lifecycle: validated journey and summary, receipt bundle. The raw core-summary.json and individual installed-journeys receipts were inspected. This is the installed CI fixture proof; final signed-candidate and Mac/Jetson acceptance remain separate.
• One engine per machine across Runtime data roots: document the boundary (LA-04: Bound memory and local execution #98). Test: model_host_lock_is_data_root_scoped. One provider owns one engine; separate data roots own separate locks. Supported operation uses one active Home per device; account-global coordination follows under LA-05: Enforce Linux model process isolation #99.

After 0.8.0: content-only Get/Keep beyond execution capacity (#100) and a catalogue path separate from Runtime releases (#102, #105).

Acceptance record

• Candidate: see 0.8.0: Release 0.8.0 #93.
• Exercised checks: focused CI/setup regressions passed 3/3; adjacent model-repair/timing fixtures passed 9/9 and source-upstream consumers passed 9/9. Product-data passed 99/99 with zero skips. Shell syntax, diff, workspace, chain-provider and model-provider formats pass. The new Rust tests cover hosted Unknown followed by local use, hosted/mixed addition refusal, 1.5-second polling with bounded retry and verified stamp reuse, a waiting Reserved Get, changed-file verification, reboot-reused PID recovery, held-writer recovery and oversized archive locking. The focused exact-4096 provider library check finished with exit 101 at 12:08 UTC13Oct: E0559 at state.rs:6269, where the new hosted-Unknown test supplies backend_report to DispatchResult::Terminal, which has no such field. Zero tests executed. The later process, server preparation and setup stages stayed unrun. The same fixture and enum exist in train 04016; lesson435 records this compile failure and the source review gap. The official exact-head 4096 precheck retry passed at 07:21 UTC13Oct, with actual PASS in its final verdict and the exact-head precheck record. The first attempt returned NO VERDICT due model capacity; lesson413 retains that separate check failure. Reuse the admitted PASS while HEAD remains unchanged. The premortem checks retained all baseline test names and original 93 policy checks, explicit platform cases, production callers and target sets, real module fixtures, and fd/device/inode ownership. Prior cc1 and a81d checks remain ancestor evidence.
• Reviews: Anders's independent Claude FAIL at cc1 identified five P2 findings, with Stop itself correct. The hosted Unknown correction is in c022ad04b; retry/cache/Reserved correction in e300ada67; abandoned transaction recovery in 4096a19f7; required workflow journey/summary in 511e18b66; early qualified-bundle setup guidance in 2b4d22616. Independent development source review passes for each corrected scope, carried to the prepared head through unchanged reviewed bytes. Provider peer review also fixed the overbroad hosted exemption and the oversized archive writer-lock gap in the same round. Review lessons395-399 and406-407 record these misses. Source review grants no Rust test or installed acceptance. OMP owns the current train adversarial review.
• Installed release evidence: the exact cc1 Ubuntu24/systemd model-repair command exited 0. The complete bounded raw packet passed independent custody, artifact and result checks: all 10 installed and 5 UI results passed. A visible reply followed by one Stop settled as known cancelled and closed the engine. The distinct real Qwen model became ready and replied before any Home restart; the old journal outcome stayed cancelled. After a forced Home exit during Get, Retry retained 15,138,816 weight bytes, began the remaining read at that offset and reached the pinned full Qwen hash. The four normal owner-stop/startup regressions provide separate source proof. The reviewers viewed all 11 model and 2 Home screenshots. Desktop Stop, Qwen reply and Retry media pass; Qwen phone capture has a sidebar overlay, so phone media acceptance stays open. Built, installed and actual process hashes agree. The earlier a81d navigation failure packet stays private. Earlier amd64 qualification gave a real reply with libgomp1 absent and all 8 engine/20 capsule ELF audits passed. Released Jetson and hosted finite evidence retain separate scope; item7 stays open.
• Remaining gate: box 5 (UP-07: Accept updates on Mac, Linux and Jetson #113); Jetson Get on the last alpha for the two open Remaining rows (status); Jetson Qwen re-measure (#93).
• Next owner: OMP owns train publication, correction, Mac gate, dispatched CI and integration. Anders froze feat/84-local-ai-080 and assigned Codex a separate model-picker investigation from train 04016. The integrator owns shared tools and reclaim; Anders owns released-device acceptance.

Released-device run procedure

Anders owns these runs on Mac, Linux x86-64 and the actual Jetson on the last alpha. Record results in this issue. Keep raw logs, private screenshots, operator paths and credentials in the existing private evidence store. Installed CI and released-device acceptance have separate results.

1. Bind the release and installation. Confirm that the published release includes the accepted Local AI changes. Record the release version, signed release/head CIDs, source commit/tree, platform, physical memory, effective memory limit and free disk space. Verify the installed and running Runtime, model-provider, Assistant and Marketplace hashes against the release. Use the actual installed Runtime for version and Home status. Run one active Home per device. A fresh installed Home must prove on-demand acquisition of the release's verified engine; CI starts with a verified engine prerequisite and supplies separate evidence.

2. Check the signed models in a fresh receiving Home. Open Assistant before Get: expect “Get a model to start. Models run on this Home.” and the Get a model action. Open Marketplace → Models → Technical details. Expect “Runtime verified the catalog publisher signature”, the publisher and complete Content ID. Record the actual signed catalogue/head and model CIDs. Current source packages are SmolLM2 135M Instruct Q8_0 (bafybeiew3vuq32fvuz2kmps7lmgl4rxpvwsvklxykx4covogbrkhgy5qky) and Qwen2.5 1.5B Instruct Q4_K_M (bafybeievyi6xp5b62qkuh65ywkd72medhx6uxrmik2fhnr3aameeqnxvwe). A published successor needs its verified identity recorded explicitly.

3. Get, time and reply with each model. Use normal Marketplace Get from the separate publisher/holder. Start a monotonic timer at the deliberate Get click; stop when “Available on this device” and enabled Open in Assistant appear. Record model payload bytes, elapsed seconds, progress, any read failure and separate download/setup intervals. Open in Assistant, confirm the model, send “Say hello in one sentence.” and retain the visible reply plus matching completed Runtime run. Repeat for Qwen. A capable device needs an actual Qwen reply; a capacity refusal proves only its refusal row. On Mac, open Assistant during Get and check usable content and a visible return to Home; record black-screen or navigation failures under Assistant: full screen hides Home, and a missing model gives no hint #331.

4. Compare the Jetson SmolLM2 Get with 430 seconds. Use a fresh isolated receiving Home with neither admitted model bytes nor cached package/Kubo blocks. Preserve existing Homes and caches. Record network/holder conditions, OS cache warmth and whether the engine was already installed. Report t seconds, t - 430 seconds and (t - 430) / 430 × 100%; keep resumed/cached and engine-acquisition measurements separate. Verify the alpha.14 receipt used the same timing boundaries before treating the comparison as a speed result. The 430-second baseline remains the reference; the new elapsed result is pending this real run.

5. Stop, switch and release the engine. Start a bounded long reply, select the other model, and verify that the current run keeps its original identity. Press Stop once; wait for the confirmed cancelled outcome and “Stopped.” Send with the other model and obtain its reply or measured capacity refusal. Also Get the second model while a reply is active, keep the active run intact, then confirm pending activation completes after settlement. Check a competing reply displays “Model is busy.” Record engine PID/birth identity, closure before replacement and one active local engine. After 60 seconds without requests, observe engine closure and confirm another reply starts it again.

6. Recover a partial Get. In the isolated test Home, stop Home normally during download and retain the completed-byte observation. Restart, inspect the cause, then Retry. Confirm verified saved bytes resume and Get completes. Test Cancel preparation separately: cancellation has a separate retention result. Exercise storage interruption, failed shutdown, corruption and imposed memory/cache limits only in the owner's isolated fixture. Verify the cause, next action, retained model/chat/draft and usable return to Home.

7. Restart with retained Qwen. Set and read back Keep on this device in System → Models. Save a harmless chat, an unsent draft and selected model. With both models retained, stop and cold-start Home. Observe Runtime and child memory for the first 60 seconds, including the prior 15-second failure window; record Linux OOM counter changes where available. Confirm stable startup, preserved model hashes/Keep/selection/chat/draft and another reply from each model or its measured capacity refusal.

8. Use retained content offline. Deny the receiving device's external routes while preserving localhost; keep the publisher online. Sign in, reply from retained models, cold-restart and reply again. Observe zero model-fetch and hosted-inference traffic at the Runtime/provider boundary. Record unavailable new-content Get as a separate result.

9. Prove the update that keeps models. Use an established alpha.15 Home with already downloaded models for the 15→16 hop. Record its signed head, model hashes, Keep choices, selection, chat and draft before update. Update through System with normal passkey approval. Verify the new installed/running hashes, then repeat restart, both replies, switch and Stop. Model bytes and saved state must survive. Link this row to LA-08: Preserve model state through catalogue and Runtime updates #102/LA-09: Verify the local model journey on all targets #103/UP-07: Accept updates on Mac, Linux and Jetson #113. Fresh install and update have separate receipts. UP-07: Accept updates on Mac, Linux and Jetson #113's two Mac update runs, Wi-Fi-off run and fresh-installer repetition remain open until Anders records them.

Refusal and recovery text to verify

Qwen's signed minimum is 2048 MiB; its description asks for about 3 GB free. Runtime separately checks weights, context, scratch and Home headroom at engine start. With each refusal, record the observed memory facts, cause code and engine-spawn evidence; Home remains usable and retained content, chats and drafts stay intact.

Receipt and retention requirements

Run the accepted source's scripts/installed-provider-verify.sh --require-verified model-provider with ELASTOS_DATA_DIR set to the intended installed data directory. Check the engine and libraries against their .elastos-engine.json entry hashes separately. Bind catalogue trust, complete model CID/payload hashes, UI/run identity, built/installed/running artifact hashes, memory/OOM evidence, restart/update state and final process cleanup. Stop each isolated Home and its owned children at test end; keep its data until the owner records its retention decision.

Results so far: alpha.15 x86, ARM, Jetson.