Skip to content

fix(cache): stop skipping dist in directory listings - #919

Merged
wan9chi merged 2 commits into
feat/cache-input-negatives-in-listingsfrom
fix/cache-dist-in-listings
Nov 5, 2026
Merged

wan9chi merged 2 commits into
feat/cache-input-negatives-in-listingsfrom
fix/cache-dist-in-listings

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation

Every directory listing skips an entry named dist. #181 added the skip so that a lint task listing the package root stays a cache hit when another task's build creates dist there. It helps no other folder name and hides a dist that does matter. A dist folder appearing in a folder the task listed goes unseen, and the next run replays stale output: oxlint, for example, passes from the cache after a dist folder with lint errors appears.

Changes

Listings skip only ., .. and .DS_Store. A task that should ignore another task's dist gets that back with "!dist/**" in its input, which keeps the entry out of its listings since #941. A task's own dist doesn't need the skip, since #915 makes a rebuildable folder an ignored entry. The cache schema version is bumped, so entries recorded with the skip aren't reused. No other snapshot changes.

E2E cases: dist_appears_in_listed_folder (synthetic, vtt list-dir) and oxlint_dist (oxlint -D no-debugger, with oxlint's rule count redacted). Both are false hits today and correct misses after.

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code


Note

Medium Risk
Changes cache fingerprint semantics for any task that lists directories, which can increase cache misses but fixes incorrect hits; schema bump invalidates existing local cache entries for the new version directory.

Overview
Directory listings for cache fingerprints no longer always omit a dist entry. Only ., .., and .DS_Store are skipped when fingerprinting folder contents, so a dist folder appearing or disappearing in a listed directory now triggers a cache miss instead of replaying stale results (e.g. oxlint passing after bad code lands in a committed dist/).

To keep ignoring another task’s dist, tasks can use "!dist/**" in input (listing exclusions from #941). A task’s own output dist still doesn’t affect listings when it only holds rebuildable outputs (#915). CACHE_SCHEMA_VERSION is bumped 22 → 23 so old fingerprints aren’t reused.

E2E coverage adds dist_appears_in_listed_folder and oxlint_dist snapshots for the new miss behavior.

Reviewed by Cursor Bugbot for commit 625af30. Bugbot is set up for automated code reviews on this repo. Configure here.

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 7df8d21 to 0d60891 Compare October 30, 2026 03:19
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.00%  [ -3.35% ..  +4.82%]  overhead   +60.53%
dynamic/access             change  -0.82%  [ -3.98% ..  +5.12%]  overhead   +18.02%
dynamic/access-relative    change  +0.59%  [ -2.50% ..  +3.24%]  overhead   +70.72%
dynamic/access-contended   change  +5.54%  [ -3.49% ..  +9.68%]  overhead   +36.06%
static/launch              change  +0.16%  [ -4.03% ..  +5.14%]  overhead  +199.83%
static/access              change  -0.35%  [ -3.69% ..  +2.18%]  overhead +1459.71%
static/access-relative     change  -0.55%  [ -2.55% ..  +1.21%]  overhead +1946.87%
static/access-contended    change  +0.59%  [ -3.31% ..  +2.80%]  overhead +1658.63%

macos

dynamic/launch             change  -0.54%  [ -4.90% ..  +4.68%]  overhead  +247.72%
dynamic/access             change  +2.25%  [-10.22% .. +142.98%]  overhead    +6.34%
dynamic/access-relative    change  +0.49%  [-12.26% .. +16.00%]  overhead  +230.90%
dynamic/access-contended   change  +1.16%  [ -7.15% ..  +9.83%]  overhead    +4.53%

windows

dynamic/launch             change  -0.73%  [ -4.60% ..  +3.67%]  overhead   +26.66%
dynamic/access             change  +0.19%  [ -1.13% ..  +1.31%]  overhead    +0.93%
dynamic/access-relative    change  +0.19%  [ -0.55% ..  +0.76%]  overhead    +0.58%
dynamic/access-contended   change  +0.00%  [ -0.74% ..  +0.74%]  overhead    +1.28%

@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 0d60891 to dbd1644 Compare October 30, 2026 04:06
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from dbd1644 to 4e828d2 Compare October 30, 2026 06:31
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 4e828d2 to 992cd6a Compare October 30, 2026 07:08
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 992cd6a to 5b6b3b8 Compare November 1, 2026 11:55
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Nov 1, 2026
@wan9chi wan9chi added the read-write overlap Caching tasks that read and write the same paths label Nov 4, 2026
@wan9chi
wan9chi removed this pull request from stack #920 November 5, 2026 08:52
@wan9chi
wan9chi changed the base branch from feat/cache-created-symlinks to feat/cache-input-negatives-in-listings November 5, 2026 08:53
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 5b6b3b8 to 6d1f47f Compare November 5, 2026 08:54
@wan9chi
wan9chi added this pull request to stack #942 November 5, 2026 08:55
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 6d1f47f to 0c75118 Compare November 5, 2026 09:03
…unseen

Two cases for a task that lists a folder in which a `dist` folder it
didn't write appears later. In `dist_appears_in_listed_folder`, the task
lists the package root with `vtt list-dir .` and prints its entries. In
`oxlint_dist`, `oxlint -D no-debugger` lints a package whose `.gitignore`
lists `node_modules` but not `dist/`, like a GitHub Action that commits its
built `dist/`: it lists the package root, `src` and, once there, `dist`,
and reads the files in them.

Today every listing skips an entry named `dist`, so a new `dist` folder
in the listed root goes unseen and the next run is a cache hit: the
first case replays the old listing without `dist`, and the second passes
although `dist/bad.js` holds a `debugger` statement.

The oxlint fixture ignores `node_modules` as a real package would;
otherwise oxlint walks into vp's task cache in `node_modules/.vite`,
whose files change between runs. The oxlint steps redact oxlint's rule
count, which changes between oxlint releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every directory listing fingerprint skipped an entry named `dist`. It was
added in #181 so that a lint task listing a folder, like the package root,
stayed a cache hit when a `dist` folder it didn't write, such as another
task's build output, appeared there. It helped no other folder name and
hid a `dist` that does matter: a `dist` folder appearing in or
disappearing from a folder the task listed went unseen, and the next run
was a cache hit replaying stale output, like `oxlint` passing after a
`dist` folder with lint errors appeared in a package that commits its
built `dist/`.

Listings now skip only `.`, `..` and `.DS_Store`. A task that should
ignore another task's `dist` gets that back with `"!dist/**"` in its
`input`, since input negatives now keep matching entries out of listings.
A task's own `dist` output folder doesn't need the skip: a folder holding
only what the task can rebuild is an ignored entry of the listing, so it
may be missing. No other snapshot changes, including `typescript_out_dir`
and `astro_empty_collections`, whose tasks list the folder holding their
own `dist`.

The cache schema version is bumped: an entry recorded with the skip holds
listings without `dist`, which would otherwise be a hit after a `dist`
folder the task listed went away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the fix/cache-dist-in-listings branch from 0c75118 to 625af30 Compare November 5, 2026 09:17
@wan9chi
wan9chi merged commit cad0326 into main Nov 5, 2026
22 checks passed
@wan9chi
wan9chi deleted the fix/cache-dist-in-listings branch November 5, 2026 10:44
wan9chi added a commit that referenced this pull request Nov 5, 2026
…gs (#941)

## Motivation

An exclusion in a task's `input`, like `"!build/**"`, leaves out only
the paths the task reads. When the task lists a folder, every entry
still counts, so an excluded folder appearing there is a cache miss: a
lint task listing the package root misses once another task's build
output appears in it. That is the case the hard-coded `dist` skip was
added for (#181). #919 removes the skip, so users need a setting to get
it back, and today `"!dist/**"`, which `docs/inputs.md` recommends,
doesn't help.

## Changes

Listings leave out what the input config excludes, after the entries the
task can rebuild (#915):
- An entry whose path the input negative globs match. A folder entry is
matched by its own path; wax's `**` matches zero or more components, so
`build/**` leaves out `build`, while `build/*` keeps it. The globs are
part of the cache key, so validation applies the same ones to the
current listing, and an entry that appears after the run is left out
too.
- An entry that a tool-reported `ignoreInput` path names. Those paths
are known only after the run, so the names are recorded in the listing's
`ignored_entries`, including entries that aren't there yet.

The cache schema version is bumped. `docs/inputs.md` now says negatives
also keep entries out of listings.

E2E cases: `negative_glob_hides_listing_entry` (synthetic, `vtt
list-dir` with `"!out/**"`; an unrelated new file still misses) and
`oxlint_negative_build` (`oxlint -D no-debugger` with a gitignored
`build/` and `"!build/**"`). Both miss today and hit after. A unit test
covers `ignoreInput`.

Part of a stack that lets automatic tracking cache tasks which read
their own outputs and list the folders holding them, so they also hit on
a clean checkout.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes post-run fingerprint semantics and cache validation for any
task that lists directories with input negatives or ignoreInput;
incorrect glob matching could cause stale cache hits, mitigated by
schema bump and targeted tests.
> 
> **Overview**
> **Input exclusions now apply to directory listings**, not only to
paths the task reads. Tasks that list a folder (e.g. a linter scanning
the package root) no longer miss the cache when an excluded entry
appears—such as another task’s `build/` folder after `"!build/**"`—as
long as nothing else in the listing changed.
> 
> Post-run fingerprints **omit listing entries** matched by the task’s
input negative globs (same rules as reads: `build/**` drops the `build`
folder entry; `build/*` does not) and by tool **`ignoreInput`** paths
(names recorded in `ignored_entries` for validation). Cache hits
recompile those globs from the entry key and apply the same filters when
comparing listings.
> 
> **Cache schema version is bumped to v22** (fresh cache). Docs and
changelog describe the behavior; e2e covers `!out/**` with `list-dir`
and an oxlint + `!build/**` scenario, plus a unit test for globs and
`ignoreInput`.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
0655edb. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

read-write overlap Caching tasks that read and write the same paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant