Skip to content

fix: Stop ambient CARGO_INCREMENTAL from suppressing compile cache injection - #13298

Merged
anthonyshew merged 1 commit into
mainfrom
shew/sccache-conflict-guard
Jul 7, 2026
Merged

anthonyshew merged 1 commit into
mainfrom
shew/sccache-conflict-guard

Conversation

@anthonyshew

Copy link
Copy Markdown
Contributor

Why

The compile cache engaged nowhere in CI — discovered while trying to produce GHA proof for #13292. The executor's conflict guard suppressed the entire injection when any injected key already existed in the task environment. This repository's own setup-environment action exports CARGO_INCREMENTAL=0, globalPassThroughEnv carries it into every task env, and so the guard tripped on every CI run: no wrapper, no proxy traffic, identical timings (bench runs 28838729601 and 28838893571 — proxy started, zero requests). The irony: the pre-existing value was 0, exactly what the injection would have set.

What

The composition decision moves from the executor into Toolchain::compile_cache_env, which now receives the task environment and returns exactly what to inject:

  • A pre-existing RUSTC_WRAPPER or any SCCACHE_* variable signals a competing compiler-cache configuration → the whole set stands down (injecting on top could hijack that setup's backend).
  • A pre-existing CARGO_INCREMENTAL is ambient CI hygiene, not a competing cache → injection proceeds, the explicit value is left alone.

The executor injects the returned variables verbatim, staying free of toolchain knowledge (per the toolchain design rules).

How

Only the toolchain can classify its own environment variables — the executor's generic "any overlap suppresses all" rule had no way to distinguish a user's wrapper from a CI image's CARGO_INCREMENTAL=0. Covered by three compile_cache_env tests: clean env (full set), competing config (RUSTC_WRAPPER / SCCACHE_GHA_ENABLED → empty), ambient CARGO_INCREMENTAL (injected minus the override).

After this ships in a canary, the bench workflow on shew/sccache-bench re-runs without its env -u CARGO_INCREMENTAL workaround.

…jection

The executor suppressed the entire compile-cache injection when any
injected key was already present in the task environment. That guard was
too blunt: this repository's own setup-environment action exports
CARGO_INCREMENTAL=0, which globalPassThroughEnv carries into every task
env, so the injection engaged nowhere in CI — the exact environment the
CI-only compile cache exists for.

Move the composition decision into the toolchain, which is the only
layer that knows which pre-existing variables signal a competing
configuration versus ambient hygiene: a user-supplied RUSTC_WRAPPER or
any SCCACHE_* variable stands the whole set down (injecting on top could
hijack that setup's backend), while a pre-set CARGO_INCREMENTAL is
tolerated — injection proceeds without overriding the explicit value.
The executor now injects whatever the toolchain returns, keeping generic
code free of toolchain knowledge.
@anthonyshew
anthonyshew requested a review from a team as a code owner July 7, 2026 03:47
@anthonyshew
anthonyshew requested review from tknickman and removed request for a team July 7, 2026 03:47
@vercel

vercel Bot commented Jul 7, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
examples-basic-web Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-designsystem-docs Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-gatsby-web Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-kitchensink-blog Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-nonmonorepo Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-svelte-web Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-tailwind-web Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
examples-vite-web Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
turbo-site Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am
turborepo-eve-agent Ready Ready Preview, Comment, Open in v0 Jul 7, 2026 3:49am

@anthonyshew
anthonyshew merged commit 784af75 into main Jul 7, 2026
42 checks passed
@anthonyshew
anthonyshew deleted the shew/sccache-conflict-guard branch July 7, 2026 04:01
anthonyshew added a commit that referenced this pull request Jul 7, 2026
> Stacked on #13298 (retargets to main when it merges).

## Why

Bench run
[28839285743](https://github.com/vercel/turborepo/actions/runs/28839285743)
exposed the compile cache's worst failure mode: with unauthorized
credentials, sccache's background server failed its storage self-check
at startup, refused to start, and every wrapper invocation exited 2 —
cargo saw failed rustc calls and **the build failed**. An optimization
must degrade to "slower", never to "broken".

## What

Two independent layers:

1. **Don't start when the run already knows the backend is doomed.** The
proxy gate now checks the resolved runtime `RemoteCacheStatus` (which
folds in the preflight: auth failure, unreachable API, team over
limit/paused) instead of configuration alone. The bench run had
literally printed `Remote caching unavailable (Authentication failed)`
before starting the proxy anyway.
2. **Don't die when it breaks mid-run.** The fork picks up a port of
mozilla/sccache#2756 (hand-applied to the v0.16.0 base — upstream it's
stacked on the unmerged client-side-mode PR): with
`SCCACHE_IGNORE_SERVER_IO_ERROR=1` — now injected — a client that cannot
connect to or start the server warns and runs the compiler directly.
Covered by two integration tests on the fork (forced startup failure →
fallback with the env var, hard error without).

Also tightens #13298's `CARGO_INCREMENTAL` tolerance: reading sccache's
wrapper source revealed it **hard-exits** on `CARGO_INCREMENTAL=1`, so
tolerating arbitrary pre-set values could itself fail builds. Only an
ambient `0` composes; any other explicit value stands the injection down
(incremental compilation was deliberately requested and is incompatible
with sccache).

## How

Layer 1 kills the predictable class before any wrapper runs; layer 2 is
the guarantee for everything unpredictable. Fork rev bumped to `52bfcd0`
(`vercel/main-from-args`). Verified: fork integration tests for the
fallback both ways; `compile_cache_env` matrix updated (`=0` tolerated,
`=1` stands down); workspace check/lint/fmt green.
anthonyshew pushed a commit that referenced this pull request Jul 7, 2026
## Release v2.10.5-canary.3

> [!CAUTION]
> Versioned docs aliasing FAILED. [View
logs](https://github.com/vercel/turborepo/actions/runs/28874392650)

### Changes

- release(turborepo): 2.10.5-canary.2 (#13297) (`e6e6fb6`)
- fix: Stop ambient CARGO_INCREMENTAL from suppressing compile cache
injection (#13298) (`784af75`)
- fix: Never let compile cache storage failures fail the build (#13299)
(`a6fc6c5`)
- test: Add outputs path-traversal negative-case regression tests
(#13300) (`2ff6df9`)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew pushed a commit that referenced this pull request Jul 13, 2026
## Release v2.10.5

> [!CAUTION]
> Versioned docs aliasing FAILED. [View
logs](https://github.com/vercel/turborepo/actions/runs/29267191097)

### Changes

- perf: Evaluate simple include globs without wax compilation (#13285)
(`189897a`)
- chore: Dogfood native Cargo support in this repository (#13283)
(`42f067b`)
- release(turborepo): 2.10.4 (#13286) (`f2fce38`)
- ci: Remove dead sccache configuration (#13289) (`558df3f`)
- feat: Serve the Remote Cache as an sccache backend for Cargo tasks
(#13288) (`0d9803f`)
- fix: Reject output path traversal (#13290) (`733ccca`)
- fix: Disable the sccache proxy when remote cache use is off (#13291)
(`3249e22`)
- feat: Embed sccache so the Cargo compile cache needs no installation
(#13293) (`b6d0035`)
- release(turborepo): 2.10.5-canary.1 (#13294) (`e6cd498`)
- fix: Pin a flag-aware turbo canary for the eve-agent deployment
(#13295) (`e60a4bd`)
- fix: Make the sccache compile cache actually cache (#13296)
(`2ac099c`)
- release(turborepo): 2.10.5-canary.2 (#13297) (`e6e6fb6`)
- fix: Stop ambient CARGO_INCREMENTAL from suppressing compile cache
injection (#13298) (`784af75`)
- fix: Never let compile cache storage failures fail the build (#13299)
(`a6fc6c5`)
- test: Add outputs path-traversal negative-case regression tests
(#13300) (`2ff6df9`)
- release(turborepo): 2.10.5-canary.3 (#13302) (`7e44a29`)
- ci: Authenticate to Remote Cache via OIDC token exchange (#13303)
(`a86839c`)
- fix: Keep pnpm patches with version range keys during prune (#13307)
(`3c27a89`)
- test: Scrub ambient turbo configuration from integration test children
(#13306) (`b03d0d6`)
- fix: Show toolchain tasks in the TUI and never run in silence (#13308)
(`d7622b6`)
- ci: Fix change detection on push events (#13304) (`eec3d61`)
- refactor: Rename the Cargo toolchain id to rust (#13311) (`a548b02`)
- fix: Remove extraneous bun.lock entries during prune (#13317)
(`382e9f5`)
- feat: Require a user-declared name for the Cargo workspace package
(#13312) (`5f01746`)
- feat: Parse and validate the task command field (#13313) (`389ea49`)
- chore: Harden turbo-vsc to invoke turbo without a shell (#13319)
(`4a19b6e`)
- feat: Resolve and execute task command overrides (#13315) (`a549baa`)
- release(turborepo): 2.10.5-canary.4 (#13325) (`6626261`)
- feat: Run the Rust workspace tests through nextest via command
override (#13316) (`6711bbc`)
- feat: Engage the Remote Cache and sccache compile cache for Rust CI
(#13292) (`1ae2065`)
- ci: Pin GitHub Actions to full commit SHAs (#13143) (`ad614d6`)
- fix: Make npm prune rehoisting deterministic and complete (#13323)
(`2dac737`)
- feat: Report incremental cache reuse in the run summary (#13327)
(`9f3d24a`)
- chore: Remove planning docs (#13329) (`5cd1d02`)
- docs: Migrate docs site to @vercel/geistdocs package (#13320)
(`5517bb2`)
- ci: Update Remote Cache action (#13330) (`4a39887`)
- release(turborepo): 2.10.5-canary.5 (#13331) (`f699719`)
- ci: Remove path-based workflow scheduling (#13332) (`84f2b2c`)
- fix: Resolve EADDRINUSE in kitchen-sink api dev script (#13328)
(`5886c71`)
- ci: Disable telemetry messages in workflows (#13334) (`2218dea`)
- fix: Prevent Cargo run tasks from being cached (#13335) (`68f449b`)
- fix: Isolate Cargo cache by host platform (#13337) (`c71f5cd`)
- fix: Continue TUI text selection beyond viewport (#13338) (`9cbfd90`)
- ci: Dogfood Cargo target restoration (#13336) (`8ad90a7`)
- fix: Require current Cargo lockfiles for caching (#13339) (`3a3d381`)
- fix: Reject unsupported Cargo local packages (#13340) (`708d656`)
- fix: Allow outputs outside package roots (#13342) (`de6a0d3`)
- fix: Resolve Cargo lock dependencies by source (#13343) (`2223a33`)
- docs: Update Cargo workspace support (#13349) (`241ada8`)
- fix: Build Ghostty for baseline CPUs (#13352) (`a2a04cc`)
- fix: Preserve watch rerun semantics for task inputs (#13351)
(`6ed5eab`)
- fix: Isolate Command Overrides From Toolchain Cache I/O (#13354)
(`e76f0b4`)
- release(turborepo): 2.10.5-canary.6 (#13355) (`f82e2c7`)
- fix: Include patched Ghostty crate in Cargo workspace (#13357)
(`b3cd7a1`)
- ci: Shard Rust tests across runners (#13356) (`ef122d9`)
- fix: Hash Cargo build environment inputs (#13348) (`d533266`)
- fix: Synchronize Cargo prune Docker lockfile (#13350) (`ed17249`)
- chore: Update agents app Eve dependency (#13364) (`38aa7d2`)
- fix: Preserve Yarn package extension ranges when pruning (#13363)
(`6ed2fb4`)
- refactor: Add `Toolchain` output availability (#13360) (`82bcfb6`)

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
Preview – turborepo-eve-agent — 4a63a574 Deployed Jul 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant