Repository navigation
feat(cache): let a hit rebuild empty folders the task made - #917
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
d7c8a30 to
78b8ee8
Compare
fspy benchmarklinuxmacoswindows |
78b8ee8 to
922a074
Compare
922a074 to
2166c58
Compare
2166c58 to
b81c7bb
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b81c7bb. Configure here.
b81c7bb to
e038c4c
Compare
e038c4c to
548a884
Compare
548a884 to
dedf87c
Compare
Three cases for a task whose output folder holds an empty folder it made. In `empty_folder_in_output`, the task makes `out/empty`, writes `out/a.txt` and stats `out`, through `vtt fs`; in `empty_folder_in_output_globs`, the same task has the explicit `output` glob `out/**`, which matches `out/empty`. In `astro_empty_collections`, `astro build` (Astro 6, added to `packages/tools`) on a one-page site makes `.astro/collections` and leaves it empty, next to the `.d.ts` files it writes in `.astro`, and stats both folders. Today a folder holding an empty folder isn't rebuildable, since fspy doesn't record a mkdir as one, so removing it, as on a clean checkout, is a cache miss. On Windows, where a mkdir is recorded as a read, the empty folder is an input too, and vt names whichever removed input its parallel validation finds first, so the synthetic cases redact the miss reason. Astro needs Vite 7, so a scoped `astro>vite` override keeps the global `vite` override from handing it the Vite+ fork. esbuild and sharp, which Astro depends on, get their prebuilt binaries as optional dependencies, so their build scripts are skipped. The Astro case removes `.astro`, `dist`, `node_modules/.astro` and `node_modules/.vite/deps`, not all of `node_modules/.vite`, which holds the task cache too, and recreates nothing, as on a clean checkout. Several inputs are then missing, and vt names whichever its parallel validation finds first, so the next step redacts its miss reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A task that makes a folder and leaves it empty, like `astro build` making `.astro/collections`, pinned the folder holding it as an input: fspy didn't record a mkdir as one (on Windows it was a read), so the empty folder wasn't rebuildable, and the archive held only files, so a hit couldn't recreate it. Removing the folder, as on a clean checkout, was a cache miss. fspy now records the path of a mkdir as a mkdir, a new access kind, before the call, so a failed call, like one finding the folder there already, is recorded too: - Unix preload: `mkdir` and `mkdirat`. - Linux seccomp supervisor, for static binaries: `mkdirat`, plus `mkdir` on x86_64. - Windows: `NtCreateFile` with `FILE_DIRECTORY_FILE` and `FILE_CREATE` or `FILE_OPEN_IF`, where `CreateDirectoryW` ends. These opens were recorded as reads; a mkdir isn't one on Unix either. `Rebuildable` takes the mkdirs that are empty folders after the run (`.DS_Store` aside) as rebuildable folders and climbs from them too, so a folder holding only such folders, rebuildable folders and files the task wrote is rebuildable. Each folder is listed once: a mkdir'd folder above a written path is decided while climbing, where an empty one counts as rebuildable, and only the other mkdirs are checked for being empty. A run without inferred reads skips the climb and checks every mkdir, for the archive. The archive gets directory entries for the empty folders, filtered by output exclusions and `ignoreOutput` like files: all of them with automatic output tracking, and otherwise those an `output` glob matches, since the glob walk yields files only. Restoring recreates them. A folder holding anything needs no entry, since what it holds recreates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dedf87c to
5cdc29e
Compare

Motivation
A task that makes a folder and leaves it empty can't be rebuilt by a hit today;
astro build, for example, makes.astro/collections. fspy records nothing formkdir, and the archive holds only files, so the folder holding the empty one stays pinned and a clean checkout misses.Changes
mkdiras a new access kind,Mkdir, before the call. This covers the Unix preload, the Linux seccomp supervisor and WindowsNtCreateFilecreating a directory; on Windows those calls were recorded as reads.ignoreOutputlike files: all of them with automatic output tracking, otherwise those anoutputglob matches. A restore recreates them.E2E cases:
empty_folder_in_outputandempty_folder_in_output_globs(synthetic; automatic outputs and explicitoutputglobs) andastro_empty_collections(astro buildwith Astro 6.4.8, added topackages/toolswith a scopedviteoverride; true clean checkout).Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.
🤖 Generated with Claude Code
Note
Medium Risk
Changes core cache fingerprinting, output archiving, and cross-platform fspy semantics; incorrect mkdir or empty-folder handling could cause false cache hits or incomplete restores.
Overview
Cache hits now recreate empty directories the task created (e.g. Astro’s
.astro/collections), instead of leaving them missing and forcing misses when a parent folder was deleted.fspy gains a
Mkdiraccess kind: Unixmkdir/mkdirat(preload + seccomp) and Windows directoryNtCreateFile(FILE_CREATE/FILE_OPEN_IF+FILE_DIRECTORY_FILE) are recorded as mkdir, not as reads. Failed mkdirs are still logged, matching rename-target behavior.Rebuild / fingerprint / archive pipeline: empty task-made folders are treated as rebuildable (skipped in input fingerprints like other output-only trees), collected into
empty_folders, written into the output tar.zst as directory entries, and restored on replay. With automatic outputs, all such folders are archived; with explicitoutputglobs, only paths the globs match. Parent folders that hold only rebuildable outputs can be absent on a clean checkout without invalidating the fingerprint.Adds e2e snapshots for a synthetic empty-folder task (with and without globs) and
astro build(Astro 6 inpackages/tools).Reviewed by Cursor Bugbot for commit 5cdc29e. Bugbot is set up for automated code reviews on this repo. Configure here.