Repository navigation
fix(storage): sync memory stores on every commit - #4119
Conversation
Records the architect run that redesigned TraceDecay storage after the 2026-10-16 incidents: per-repository class files (journal, memory, evidence, derived) on one bounded executor, fenced TRUNCATE checkpoints, per-stream dead letters, class budgets, repository identity through aliases, and an eleven-step cutover. The trail and the arena, judge and interrogate evidence ship beside it.
StoreRuntimeClientLease::dispatch_read and its attachment port are now async and take an Arc probe, matching dispatch_submit. The repository attachment runs the read as one job on the native blocking pool, so a native worker awaiting a fact, observation or external-source read no longer parks on reader admission or the reader's reply. The seven production callers await the read; the sync remote query port, which runs on the blocking pool, waits through block_on. A rusqlite_suite test awaits a typed read on a native worker and asserts the native reader-wait counter does not move. Refs #4085
The project store and the profile memory store hold facts and the anchors they cite, which are never rebuilt. Their writers now run at synchronous=FULL, so a power loss cannot drop a commit the writer acknowledged. Session and registry stores keep NORMAL until their durable rows move into classes of their own.
|
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| &self.path, | ||
| self._opened_database.as_deref(), | ||
| canonical_path, | ||
| connection::Durability::for_scope(&self.binding.shard_id.scope), |
There was a problem hiding this comment.
🟡 Durable commits remain unsynced on other platforms
On non-Unix, non-Windows targets, Worker::run still opens project and profile-memory writers at NORMAL. Its alternate branch calls open, which fixes durability to NORMAL, so acknowledged commits can vanish after power loss.
Learn more
The writer chooses its SQLite connection in platform-specific branches. Unix and Windows now pass the shard's durability to open_writer, but the alternate writer branch still uses open, which always configures NORMAL. On other targets, SQLite thus does not sync each project or profile-memory commit even though the new scope policy classifies them as FULL.
Example: On a target that is neither Unix nor Windows, a project writer acknowledges a fact commit at NORMAL. A power failure before a WAL checkpoint can erase that commit, whereas the same project writer on Linux syncs it at FULL.
Recommended fix: Pass Durability::for_scope(&self.binding.shard_id.scope) through the non-Unix/Windows writer opening path too, while preserving that path's existing identity checks.
Was this helpful? React with 👍 or 👎 to provide feedback.
Records the worktree-liveness fix as the first growth step and moves dead letters into the partition step, where corrupt derived authority can be rebuilt instead of skipped.
Summary
Step 2 of the storage redesign (
docs/plans/storage/README.md). Stacked on #4118; once that merges, this diff is the one commit below.The project store (
tracedecay.db) and the profile memory store (user-memory.db) hold facts and the retrieval anchors they cite. Those rows are never rebuilt, but every writer ran atsynchronous=NORMAL, which in WAL mode can drop the newest acknowledged commits on power loss. Their writers now run atsynchronous=FULL; the writer verifies the setting as it already did forNORMAL. Session stores andglobal.dbkeepNORMAL:global.dbstill takes per-hook analytics writes until telemetry splits out in step 8. The README's reconciliation records that, and that the memory migration rule ships with the first memory schema change instead of as machinery with nothing to run.Test plan
bazel test --config=clippy //crates/tracedecay-rusqlite-runtime:unit_test(401 passed).stores_holding_durable_memory_commit_at_full_syncopens a writer per shard scope and asserts FULL for the project and profile-memory stores and NORMAL for the rest.bazel test //crates/tracedecay-rusqlite-runtime:rusqlite_suite(146 passed)rustfmt_check_2024for the touched crate//crates/tracedecay:daemon_suite,:mcp_suite,:session_suite,:unit_testin CI