Skip to content

Fix Cargo source replacement being ignored (#338, #455) - #1420

Merged
Mikola Lysenko (mikolalysenko) merged 12 commits into
mainfrom
agent/fix-cargo-source-replacement
Oct 12, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 12 commits into
mainfrom
agent/fix-cargo-source-replacement

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #338
Fixes #455

Summary

socket-patch never read Cargo's source replacement
([source.crates-io] replace-with = "…" → [source.<name>] directory = "…"),
so it guessed where a cargo vendor project builds from, and guessed wrong.

Behaviour change to note: a vendor/ that no replace-with wires is no
longer crawled, patched, rolled back or pruned. Cargo never built from it, so a
patch an earlier release wrote there had no effect. Rollback and prune now look
at the copy cargo does build. A stray edit an earlier release left in such an
unused vendor/ stays where it is; restore it from version control if needed.

The new pure module formats/cargo/source_replacement.rs holds the resolver
(crates_io_source) and the config file walk (config_layer_paths).
CLI_CONTRACT.md and docs/ecosystems.md are updated.

Per-issue checklist

Issue Regression tests (red before the fix → green)
#338 shape 1 (custom dir) crawler_cargo_e2e::custom_vendor_dir_named_by_source_replacement_is_crawled; real cargo: e2e_safety_cargo_build::apply_patches_a_custom_named_vendor_dir
#338 shape 2 (member) crawler_cargo_e2e::workspace_root_source_replacement_applies_to_a_member; real cargo: e2e_safety_cargo_build::apply_from_a_member_patches_the_workspace_vendor_dir
#338 shape 3 + non-cargo vendor/ crawler_cargo_e2e::unwired_vendor_dir_falls_back_to_the_registry_cache
#338 fresh checkout / missing dir crawler_cargo_e2e::missing_replacement_directory_crawls_nothing
#338 $CARGO_HOME config crawler_cargo_e2e::cargo_home_source_replacement_is_honoured
#455 hosted redirect patch::redirect::tests::cargo_directory_source_replacement_is_refused (both config spellings, CRLF, a multi-hop chain; mirror control still redirects)
#455 vex reads the wrong copy covered by the crawler change: vex discovers through the same crawler (vex_consumed.rs); the crawler tests above
resolver formats::cargo::source_replacement::tests (8 unit tests)

Red before the fix: all 5 new crawler_cargo_e2e tests failed on origin/main
code. The hosted test failed with the Socket registry pin written into
Cargo.toml, Cargo.lock and .cargo/config.toml.

Fixtures that staged a bare vendor/ and relied on the old guess now write the
.cargo/config.toml that cargo vendor asks for (6 crawler unit/oracle tests,
crawlers_empty_paths_e2e, e2e_cargo ×2, e2e_vendor_cargo_build's #336
"behind a cargo vendor dir" rollback). None of their assertions were loosened.

Local evidence (on 30aa879; 9083269 adds 3 fixture wirings, CI green)

/code-review high: findings not fixed here

  • Vendored mode's is_vendored (vendor/cargo.rs) still probes
    <root>/vendor for its already_vendored_in_tree refusal. That is a
    fail-closed refusal with its own contract tests, and neither issue
    reports it. Follow-up.
  • Upgrade from a release that patched an unwired vendor/. Cargo never
    built from that copy, so the patch had no effect. rollback now looks
    where cargo builds. The stray edit in the unused vendor/ copy stays.
  • Registry mirror / local-registry ordering. With a mirror or local-registry replacement, when both the
    crates.io and the mirror unpack dirs hold the crate, the crawler takes the
    first in listing order. That predates this PR: main crawls every
    registry/src/* dir the same way. Choosing the mirror's dir means
    reproducing cargo's SourceId hash. Follow-up.
  • Per-key (not per-table) merge of conflicting [source.x] definitions:
    cargo itself rejects those ("more than one source location"). A .. in
    --cwd is folded before the walk (ae078c1, b583db6 keeps the given cwd as the reported prefix). A symlinked cwd is walked
    logically rather than physically: absolute() is used rather than
    canonicalize() so reported paths keep matching the cwd the user gave
    (macOS /var → /private/var).
  • Env-var and --config source replacement (CARGO_SOURCE_*) is not
    read in either mode.
  • A third config-chain walker / no memo. vendor::cargo_config::read_config_chain
    is [patch]-specific (it drops files with no patch entries). Unifying the
    walkers is a refactor for its own PR. The extra reads are at most a handful
    of small files per crawl.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TUHnkhA7W5gHh21jSMkY7P


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
Agent-mode apply, scan and vex looked for a `cargo vendor` tree only at
`<cwd>/vendor`. A custom vendor directory, a workspace root's vendor
directory seen from a member, or a `vendor/` that no config wires all
made apply patch a copy cargo ignores while reporting success, and made
vex judge the wrong copy.

The crawler now follows the `[source.crates-io] replace-with` chain of
every cargo config cargo reads (each ancestor's `.cargo/config*`, then
`$CARGO_HOME`'s) and crawls the directory source it ends on, resolved
the way cargo resolves config paths. Without a directory replacement it
crawls the registry cache, and a configured directory that is missing
yields nothing rather than a copy cargo doesn't read.

Refs #338, #455

Assisted-by: Claude Code:claude-opus-5-5
A project whose .cargo/config replaces crates.io with a directory
source (`cargo vendor`) builds every crate from that directory and
never reads a registry. `scan --mode hosted` still pinned the crate
to a per-patch Socket registry and reported success, so every fresh
`cargo build --frozen --offline` failed with "no matching package".

The hosted cargo rewriter now follows the project config's
`[source.crates-io] replace-with` chain and, when it ends on a
directory source, skips every cargo dependency with nothing written
and a `redirect_cargo_directory_source` warning that points at agent
mode. A registry mirror replacement is still redirected.

Refs #455

Assisted-by: Claude Code:claude-opus-5-5
Adds real-cargo e2e coverage for a custom-named vendor directory and
for apply run from a workspace member, and documents how agent mode
finds the directory source and the new hosted refusal code.

Refs #338, #455

Assisted-by: Claude Code:claude-opus-5-5
Three fixtures staged a bare `vendor/` with no source replacement and
relied on the crawler treating it as cargo's source. They now write the
`.cargo/config.toml` that `cargo vendor` asks for, and the resolver
strips a BOM through the shared text helper.

Refs #338

Assisted-by: Claude Code:claude-opus-5-5
Rollback and prune look for an agent-mode patch left in the shared
registry cache when the project crawl searches only a `cargo vendor`
directory. That check still looked for `<cwd>/vendor`, so a custom or
workspace-root vendor directory skipped it, and an unwired `vendor/`
triggered it needlessly. It now asks the crawler for the directory
source the project's cargo configs name.

The hosted refusal's message now says why a Socket registry pin breaks
an offline vendored build, instead of claiming cargo never reads it.

Refs #338, #455

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 12, 2026 07:33
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

The dispatch, all-ecosystem rollback and scan-scope tests staged a
bare `vendor/` crate tree and relied on the crawler treating it as
cargo's source. They now write the source replacement `cargo vendor`
asks for, so they keep exercising the vendored-crate path.

Refs #338

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 12, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent)


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: socket-patch now reads Cargo's source replacement ([source.crates-io] replace-with → [source.X] directory), following the chain through every ancestor's .cargo/config* and then $CARGO_HOME. Agent mode patches the directory cargo actually builds from, not a hard-coded <cwd>/vendor, and rollback and prune use the same lookup. Hosted mode skips cargo pins with a new redirect_cargo_directory_source warning when crates.io is replaced by a directory source.

Risk: low to medium. The resolver is small, pure and well tested: cycles and malformed TOML fall back to the registry, with no panic or loop. The risk is the intended behaviour change: a vendor/ that no replace-with wires up is no longer crawled or patched, and a configured directory that is missing crawls nothing.

Look here:

Verified:

  • Read the full diff and checked it against Cargo's config rules: precedence, config-relative directory paths, cycles, CRLF/BOM, and CARGO_HOME unset.
  • Ran locally, all passing: crawler_cargo_e2e (39), crawlers_empty_paths_e2e (13), the source_replacement, crawler and hosted-refusal unit tests (44), and the two new real-cargo e2e_safety_cargo_build tests (2/2; cargo check --frozen --offline succeeds on the patched vendor copy).
  • CI ci-ok and clippy are green on 9083269, and Bugbot reports no open findings.
  • No unresolved threads, and CHANGELOG.md is untouched.

Changes I made: none.

Open questions (none block merge; they could be follow-ups):

  1. A cwd containing .. (e.g. --cwd ../proj) walks the configs of directories that aren't ancestors, because std::path::absolute keeps .. on Unix. A probe with T/b/../proj picked up T/b/.cargo/config.toml. A symlinked cwd differs from Cargo's physical-path walk in the same way. A lexical .. cleanup before the walk would fix it.
  2. Hosted mode refuses only directory replacements. A local-registry replacement is just as offline and is still pinned. Should it also be refused, or at least warned about?
  3. Hosted mode reads only the repo-root .cargo/config*, so a replacement set in $CARGO_HOME or an ancestor above the repo isn't seen. The env-var and --config forms (CARGO_SOURCE_CRATES_IO_REPLACE_WITH, …) are ignored in both modes. These may be worth a line in CLI_CONTRACT.md.
  4. The PR description could call out the behaviour change for an unwired vendor/, which earlier releases patched. Without a replace-with, rollback and prune no longer see those copies.
  5. Test nit: the crawler unit tests and some CLI e2e tests now read the developer's real $CARGO_HOME/config.toml. Pointing CARGO_HOME at a tempdir, as CargoHomeGuard already does, would make them hermetic.

Auto-merge is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

A --cwd such as `b/../app` walked `b`'s ancestors and could pick up a
sibling directory's `.cargo/config.toml`, so the crawl followed a
source replacement cargo never sees from `app`. The path is folded
lexically before the walk.

Refs #338

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] Thanks for the brief. On the open questions:

  1. .. in --cwd: fixed in ae078c1. The cwd is folded lexically before the config walk, and a new regression test, parent_dir_in_cwd_does_not_read_a_sibling_config, fails without the fold. A symlinked cwd is still walked logically. canonicalize() would make reported paths diverge from the user's cwd (macOS /var → /private/var), so I left that as is.
  2. local-registry in hosted mode: not refused here. Cargo unpacks local-registry crates into registry/src, so agent mode is correct. Whether a hosted pin breaks there depends on whether the Socket registry is reachable at build time, which is a separate decision. I'd rather track it as a follow-up than widen this PR.
  3. Hosted mode reads only the project config; env/--config forms are ignored: both are now listed in the PR description under the review notes. Wiring outer configs into the hosted engine, as yarn's OuterYarnMirror does, is a follow-up.
  4. Behaviour change for an unwired vendor/: now called out in the PR description.
  5. Test hermeticity: the new tests use CargoHomeGuard. The older crawler unit tests and CLI fixtures could read a developer's ~/.cargo/config.toml, but only a crates.io replacement there would change their result. Making them hermetic is a test-only follow-up.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-core/src/crawlers/cargo_crawler.rs Outdated
Folding `..` before the config walk also moved the reported vendor
directory onto the folded path, so path-scoped scan and rollback, which
strip the given cwd as a prefix, saw the vendored crates as outside the
project. The folded path now only picks the config files; a directory
under it is reported back under the cwd as given.

Refs #338

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-core/tests/crawler_cargo_e2e.rs Outdated
The new prefix test compared crawler output with a path that still
held `..`, but on Windows `std::path::absolute` folds it, so the
crawler (and path-scoped commands) report the folded path there. The
test now builds its expectation with `absolute` too.

Refs #338

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief (update for 747c561)

The earlier brief covered everything up to 9083269. This one reviews the three commits since then.

What it does: Before walking the ancestor .cargo/config* files, the cwd is now folded lexically to remove .. and .. So --cwd a/b/../proj reads a/proj's configs and no longer picks up the sibling a/b/.cargo; this was open question 1 in the earlier brief. A directory source that resolves under the folded cwd is still reported under the cwd exactly as the user gave it, so path-scoped commands can strip it as a prefix. The rest of the PR is unchanged from the earlier brief.

Risk: low. The change is small and pure: +30/-4 in one function plus two tests. .. above the root is a no-op, because PathBuf::pop stops at /, C:\ and UNC roots.

Look here:

  • cargo_crawler.rs#L305-L318:`` fold_parent_dirs
  • cargo_crawler.rs#L320-L357:``` the split between given and `folded`, and the `strip_prefix(&folded)` → `given.join(rel)` remap
  • crawler_cargo_e2e.rs#L1030-L1075:```` the two new regression tests

Verified:

  • Read the delta 9083269..747c561.
  • Local runs: crawler_cargo_e2e 41/41, cargo_crawler unit tests 35/35, source_replacement tests 9/9.
  • Both new tests fail without the fix. parent_dir_in_cwd_does_not_read_a_sibling_config fails against 9083269, and directory_source_keeps_the_given_cwd_prefix fails against ae078c1.
  • CI ci-ok and clippy are green on 747c561. Bugbot was requested on that head.
  • No unresolved threads, and CHANGELOG.md is untouched.

Changes I made: none.

Open questions (non-blocking):

  1. The fold is lexical. Cargo walks the physical cwd, so for link/../app, where link is a symlink, Cargo reads the configs above the link's target instead. The code before this PR also got this wrong, so it is not a regression.
  2. On Windows, std::path::absolute already folds ... directory_source_keeps_the_given_cwd_prefix therefore only guards the remap on Unix and macOS.

Auto-merge is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

Comment thread crates/socket-patch-core/src/patch/redirect/mod.rs Outdated
The hosted refusal for a `cargo vendor` project only read the project's
own `.cargo/config*`. A directory source set in a monorepo root's config
or in `$CARGO_HOME` was missed, and a `cargo local-registry` was never
recognised, so `scan --mode hosted` still pinned the Socket registry
and every `cargo build --frozen --offline` broke.

The hosted scan now resolves the source replacement from every config
cargo reads for a build there, read-only, and passes it into the cargo
rewriter. A directory source or local registry from any of them skips
every cargo dependency with `redirect_cargo_directory_source`. Agent mode
keeps crawling the registry cache for a local registry, which cargo
unpacks there. The process-level resolve now lives in
`source_replacement` so the crawler and the hosted scan share it.

Refs #455

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-core/src/formats/cargo/source_replacement.rs Outdated
The format models are pure by rule (formats::architecture_tests), but
the last commit put the file-reading resolve into
formats/cargo/source_replacement.rs. It now lives in the cargo crawler
as `CargoCrawler::crates_io_source_for`, which the hosted scan calls;
the format module keeps only the pure chain resolution.

Refs #455

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit d81161b. Configure here.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final reviewer: disarmed auto-merge. My earlier brief covered 747c561; two non-merge commits have landed since (4032716 "Refuse hosted cargo pins on outer offline sources", d81161b "Keep cargo config reads out of formats"). I'll write a fresh brief for d81161b on a later run once the head has settled, then re-arm.


Generated by Claude Code

Merged via the queue into main with commit 7236190 Oct 12, 2026
54 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-cargo-source-replacement branch October 12, 2026 17:58
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 12, 2026
Main's #1420 added a restored field to the e2e Shape struct; the
declared-cargo-floor test this branch adds predates it and no longer
compiled after merging main.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

3 participants