Skip to content

refactor(cache): restructure cache lookup and update around the task's fs story - #601

Closed
wan9chi wants to merge 1 commit into
claude/fs-fingerprint-recordsfrom
claude/task-fs-seam
Closed

wan9chi wants to merge 1 commit into
claude/fs-fingerprint-recordsfrom
claude/task-fs-seam

Conversation

@wan9chi

@wan9chi wan9chi commented Aug 9, 2026 •

Copy link
Copy Markdown
Member

Motivation

The filesystem side of a task run — snapshotting inputs, judging traced accesses, the read-write-overlap verdict — is spread across the lookup, update, and observation code, so the unsettled overlap policy can only be exercised by running a whole task through the engine.

This gives it one seam (stack: 2/3, on top of #600):

  • TaskFs::pre_run — before the task executes: validate the io configuration, snapshot the listed inputs, and report the first input changed since a previous run's fingerprints.
  • TaskFs::post_run — after it finished: judge the traced accesses and conclude InputModified (caching unsound) or Cacheable with the run's fingerprints and outputs.

The seam makes the overlap policy and its filters drivable with synthetic traces on temp directories — the new unit tests in task_fs/task_run.rs do exactly that, covering the verdict across all auto-tracking combinations, per-side gating, negative-glob/ignore filtering, access normalization, and fingerprint round-trips.

Lookup flips to fetch-first so pre_run can compare against the stored fingerprints; since that makes the entry fetch and the old-key miss classification adjacent, both now run in one deferred read transaction under a single lock hold (fetch_entry), closing the window where a concurrent run of the same task could break the classifier's the-key-just-missed assumption. The cache module shrinks to storage primitives; FspyTracking dissolves into TaskFs.

The task_fs module tree deliberately mirrors a standalone-crate layout: #594 moves it out of the engine wholesale.

🤖 Generated with Claude Code

…s fs story

Give the filesystem side of a task run one seam: TaskFs::pre_run (before the
task executes: validate io config, snapshot the listed inputs, and report the
first input changed since a previous run's fingerprints) and TaskFs::post_run
(after: judge the traced accesses and conclude InputModified or Cacheable with
the run's fingerprints and outputs). The seam makes the read-write-overlap
policy and its filters drivable with synthetic traces on temp directories —
the new unit tests in task_fs/task_run.rs do exactly that.

Lookup flips to fetch-first so pre_run can compare against the stored
fingerprints, and — since that makes the entry fetch and the old-key miss
classification adjacent — both now run in one deferred read transaction under
a single lock hold (fetch_entry), closing the window where a concurrent run
could break the classifier's the-key-just-missed assumption. The cache module
shrinks to storage primitives; FspyTracking dissolves into TaskFs.

The task_fs module tree mirrors a standalone-crate layout on purpose: the
follow-up moves it out of the engine wholesale.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.67%  [ -3.36% ..  +5.33%]  overhead   +57.17%
dynamic/access             change  -0.01%  [ -5.46% ..  +3.04%]  overhead    +5.67%
static/launch              change  +0.80%  [ -5.30% ..  +6.91%]  overhead  +178.34%
static/access              change  -0.20%  [ -2.44% ..  +1.00%]  overhead  +754.02%

macos

dynamic/launch             change  +0.34%  [ -1.77% ..  +2.63%]  overhead  +212.90%
dynamic/access             change  +0.00%  [ -2.44% ..  +1.96%]  overhead    +3.05%

windows

dynamic/launch             change  +0.79%  [ -2.35% ..  +5.09%]  overhead   +27.38%
dynamic/access             change  +0.35%  [ -1.28% ..  +1.97%]  overhead    +1.98%

wan9chi pushed a commit that referenced this pull request Oct 26, 2026
<!-- claude-projects-attribution: {"github_login":"Brooooooklyn"} -->
_Requested by **Brooklyn** · [project
thread](https://claude.ai/code/project/chan_018s9RxRdtH424j2EhXdzV3N?thread=cmsg_018s9RxRdtH424j2EhXdzV3NQxbeR8Pt7qaqSQwoq9rEFo)_

<!-- routine:prune-tests:redundant-unit-tests -->

Before: five unit tests could not fail on a regression that no other
test catches: a `f() == f()` tautology, a "hash twice and compare"
check, and three tests whose assertions are a subset of a neighbour's.
Two `fspy_shared_unix` tables and the labeled-writer tests were
hand-unrolled into one `#[test]` per row.

After: the five tests are gone, and the three tables are table-driven
tests that keep every row, its inputs and its expected value. Only
`#[cfg(test)]` code changes.

How: each removal was defended by mutating product code with the most
realistic regression it could catch, running the crate's tests, and
reverting.

### Totals

- Files touched: 8, test modules only.
- Cases deleted: **5**. Cases merged: **18** rows folded into 3
table-driven tests (21 tests become 3). Across the six crates run
(`fspy_shared_unix`, `vt`, `vt_glob`, `vt_powershell`, `vt_workspace`,
`vt_plan` lib tests): 282 → 259 cases. The whole workspace was not run
(see below), so there is no workspace-wide total.
- Net lines: **−169** (+107 / −276).
- Runtime saved: about 34 ms, the summed duration of the deleted cases
(`npm_workspace_with_exclusions` 28 ms,
`test_file_hashes_are_consistent` 6 ms, the rest under 1 ms).

### Removals

1. **`vt_powershell` `tests::is_stdin_terminal_is_idempotent`** —
tautology: `assert_eq!(is_stdin_terminal(), is_stdin_terminal())` on a
`LazyLock`. Mutation R1 (negate the cached `is_terminal()`): the test
passes, as does everything else in `vt_powershell` and `vt_plan`. The
behaviour that depends on it is covered by `vt_plan`
`ps1_shim::tests::skips_rewrite_when_not_interactive` (mutation R1b,
skipping the `!is_interactive` check, fails it).
2. **`vt`
`session::execute::glob::tests::test_file_hashes_are_consistent`** —
determinism check. Mutation R2 (seed the file hasher randomly per call)
fails it and also
`session::execute::hash::tests::other_files_hash_raw_content` and
`redacted_file_hashes_without_content`, which pin exact hash values.
3. **`vt` `session::reporter::grouped::writer::tests::buffers_output`**
— subset of `shared_buffer_interleaves`, which appends several chunks
from two writers to one shared buffer and checks the joined result.
Mutation R5 (replace instead of append) fails both.
4. **`vt_glob`
`path::tests::test_match_ignores_directory_only_patterns`** — `build/**`
with `!build/keep/**`; `test_match_ignores_directory_patterns` asserts
the same shapes (`build/**`, `dist/**` with `!dist/public/**`).
Mutations R4 (ignore negations) and R4b (treat negated globs as
positive) fail 8 and 9 other `path` tests.
5. **`vt_workspace`
`tests::test_get_package_graph_npm_workspace_with_exclusions`** —
despite its name it only asserts that the root and `packages/normal` are
found; its closing comment says exclusions "might not work" and asserts
nothing about them. Mutation R3 (drop negated workspace globs): it
passed (run at the earlier base), while
`test_get_package_graph_workspace_exclusions`,
`…_workspace_work_with_last_match_wins` and
`…_npm_workspace_with_root_relative_prefixes` fail. Mutation R3b (stop
adding the root package) fails 13 other graph tests,
`test_get_package_graph_npm_workspace` among them.

### Folds (every row kept)

| File | New test | Rows | Mutation that fails it |
|---|---|---|---|
| `fspy_shared_unix/src/exec/mod.rs` | `appends_value_to_the_env_list` |
7 | RF2: always prepend `:` |
| `fspy_shared_unix/src/exec/which.rs` | `test_which` | 8 | RF3:
`ENAMETOOLONG` → `ENOENT` |
| `vt/src/session/reporter/labeled/writer.rs` | `prefixes_each_line` | 6
| RF1: drop the prefix |

Each assertion carries the row's name, so a failure still names the
case.

### Kept after defense

- `vt_subprocess_test` `tests::test_command_for_fn` was nominated and
kept: its defense could not be completed safely (see "What ran").

### Case counts (lib tests)

| crate | before | after | delta |
|---|---|---|---|
| fspy_shared_unix | 30 | 17 | −13 (folds) |
| vt | 88 | 81 | −7 (2 deleted, 5 folded) |
| vt_glob | 23 | 22 | −1 |
| vt_powershell | 6 | 5 | −1 |
| vt_workspace | 85 | 84 | −1 |
| vt_plan | 50 | 50 | 0 |

0 failures before and after.

### What ran and what did not

- Ran on Linux at `0346c268` and on this branch: the lib tests of the
six crates above with `--report-time`; mutations R1–R5, R1b, R3b, R4b
and RF1–RF3, each reverted afterwards; `cargo fmt --check` on the
touched files; `cargo clippy -p fspy_shared_unix -p vt -p vt_glob -p
vt_powershell -p vt_workspace --tests -- -D warnings` (clean).
- Not run: the workspace-wide `just lint` and `just test`, macOS and
Windows, and the e2e/snapshot suites. The `vt_subprocess_test` and
`vt_pty_terminal` suites were deliberately not run: while defending
`test_command_for_fn`, a mutation that swapped the child's id and
payload arguments left children with an empty argument, which libtest
took as a filter matching every test, so each child re-ran the suite and
spawned without bound.

Open pull requests: of the 61 open, only #601 touches a file here, and
it renames `crates/vt/src/session/execute/glob.rs` without changing its
content.

Ages: all five deleted tests are older than a month (newest:
`is_stdin_terminal_is_idempotent`, 2026-06-28). None was added for a
bug, and none is in a permission or security suite.

To revert: `git revert 2c32410`.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01PxVsmvB9xfZKfVnUonY8kf)_

Co-authored-by: Claude <noreply@anthropic.com>
@wan9chi

wan9chi commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Stale; closing with #600.

@wan9chi wan9chi closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant