[spa5k]

fix(sessions): make host ingest and hook redrive converge instead of re-driving permanent failures

Fixes the non-converging retry family behind #4481 and #4564. On the reporting machine the daemon pinned ~180% CPU indefinitely: 68,402 admission-refusal warnings (100% observation_retrieval_anchor_alias_collision) across ~640 Pi sessions over two days, 176 discovery header failures per pass, and one hook redrive warning every 30 s — all from work that re-runs forever because per-item permanent failures are never memoized durably.

Root pattern

Several sweeps re-drive items whose failure is deterministic:

• The in-process SourceLedger forgets every converged source at each daemon restart, so the whole corpus (observed: 1,029 files / 713 MB) is re-admitted from scratch.
• The cover-past cursor advance is the only convergence mechanism for refused records; a non-retryable advance failure fails the whole file pass, which re-reads and re-refuses the same records on the next pass, forever.
• Owed hook replay work is redriven every sweep (~30 s) with no backoff and no cap.

Changes

1. Durable per-path convergence memos for Pi (hosts/pi.rs)

• Completion memo: a file admitted to EOF records (len, mtime) durably; a later sweep (including after a daemon restart, where the in-process ledger is empty) skips the unchanged file before opening it.
• Header-quarantine memo: a file whose header cannot be parsed is skipped until its (len, mtime) changes, instead of being re-opened and re-failed every pass.
• In-process discovery skip set: malformed headers are not re-read within a process either; the entry drops as soon as the file changes.

2. Refusal high-water memo for the shared JSONL admission (jsonl_observation_admission.rs)

• Every deterministic content refusal records, per (source, scope), a durable never-regressing refused_through mark under the file generation, via the existing parse-offset authority.
• When a later pass sees a durable cursor behind that mark for the same generation, admission resumes from the memo instead of re-reading and re-refusing the same records. A changed generation invalidates the memo naturally. This covers pi, codex, kimi, vibe, and cursor — every host on the shared JSONL seam.
• A non-retryable cover-past advance failure no longer fails the file pass: admission advances in-memory with one warning, and the durable memo bounds the next pass's re-work. Cancellation and retryable failures keep their existing typed-block semantics.
• Repeat refusals below the memo frontier log at debug instead of warn, so a stuck cursor no longer produces thousands of identical WARNs.

3. Hook pending-work redrive backoff and cap (hook_v2_replay_consumer.rs)

• Each owed envelope retries with exponential backoff (30 s doubling, capped at 1 h) instead of every sweep.
• After 16 attempts the in-process loop stops retrying and logs one terminal warning naming the envelope. The durable ledger is untouched, so a restart or a fixed dependency re-offers the work.

Deliberately out of scope (follow-ups)

• Idempotent re-recognition of retention-released anchors (sessions: admission refusal storm (observation_retrieval_anchor_alias_collision, ~68k WARNs, ~640 sessions) pins daemon at ~180% CPU #4564's deeper store question): the memo bounds the re-work but the alias-collision classification itself is unchanged — codex_sibling_over_a_retention_released_owner_is_a_typed_alias_collision documents the current semantics.
• Hermes state.db row admission has its own cover-past with the same intent; it does not share the JSONL seam and keeps its current behavior.
• Session-temporal refresh scheduler inner loop (daemon: high idle CPU from session refresh scheduler, store over size bound, repeated corruption retries #4481's registry.rs hot spawn sites): the supervisor already backs off panics and exits on completion; the load there was driven largely by the ingest storm this PR removes.
• Corruption-locked store healing: the retention tick is cadence-bounded and log-gated; the stuck store itself needs product attention.

Tests

New:

• converged_session_file_is_not_re_admitted_by_a_fresh_sweep — a fresh SourceSweep models a daemon restart; the durable completion memo must prevent re-admission.
• malformed_session_header_is_not_re_failed_by_a_fresh_sweep — unchanged malformed files are quarantined after their first failure.
• refusal_memo_floor_applies_only_within_its_generation — the memo floor is generation-scoped and never regresses.
• pending_work_redrive_backoff_doubles_then_caps — backoff math.

Updated (they asserted the old re-fail-forever semantics):

• discovery_summary_reports_changes_and_recovery_without_repeating_unchanged_failures — quarantined files stop counting; a rewrite (sub-second stamp change) retries them.
• repeated_discovery_summary_keeps_malformed_sources_and_coverage_refused — a later pass reports zero new failures while retaining coverage.

Full tracedecay-sessions lib suite: 635 passed, 0 failed (run with --features tracedecay-rusqlite-runtime/test-runtime; without the feature the cargo harness runs async tests with no process runtime and they fail on both this branch and master — CI exercises the Bazel targets instead).

Observed impact expected

On the reporting machine, after one full pass post-upgrade: no re-admission of the unchanged 1,029-file corpus on restart, the anchor-collision storm reduced to at most one refusal per record per generation, and the 30-second hook redrive warning gone after bounded retries.

---

[changeset-bot]

⚠️ No Changeset found

Latest commit: 39a49de

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[spa5k]

Review findings addressed in 5797d25:

• Rewritten transcript records go unseen — the memo floor now applies only when the memo carries a resume fingerprint; the scanner gets a JsonlResumeState at the floor, validates the skipped prefix, and a rewritten file diverges into the existing committed-checkpoint recovery. A memo without a fingerprint stays inert (unit-tested).
• Records after refusals cannot resume — when the floor applies, the in-memory expected cursor is re-baselined to the floor so subsequent frames advance contiguously; the cover-past fallback (NonDurableRecord) now warns once per pass instead of per frame. The durable cursor staying behind the floor is bounded by the memo and documented as residual in the PR description.
• Replaced Pi sessions stay invisible — path memos now bind the unix inode (dev/ino are already inside the JSONL file identity the ledger uses); an equal-length timestamp-preserving replacement changes the inode and re-enters admission.
• Temporary Pi read failures persist indefinitely — both the in-process skip set and the durable quarantine memo record only InvalidSessionHeader; SessionHeaderUnavailable stays retryable every sweep.

Full sessions suite: 635 passed, 0 failed. Re: the changeset bot — this repo releases via release-please from conventional commit titles (no .changeset directory exists), and both commits follow the convention.

[spa5k]

Both new findings addressed in 0380c4d:

• Refusal frontier stalls on smaller hashes — correct: the parse-offset writer gates on monotonic mtime and a prefix fingerprint is unordered. The memo now persists through replace_parse_offset (exact CAS, read-current-then-replace), which orders on nothing but the (generation, offset, fingerprint) pair. A lost CAS means a peer moved the same memo and the winner's row is at least as fresh. Full suite 635/635.
• Coverage fallback still warns every frame — the flag was inverted (mem::take reset it every call); now mem::replace(&mut …, true) as suggested.

[spa5k]

Addressed in the follow-up commit: Err from get_parse_offset no longer fabricates a zero expected value — it surfaces the read outcome at debug, keeps the in-memory mark, and defers the persist to the next pass; only a genuine Ok(None) CASes from zero. Suite stays 635/635.